📄️ Index
In a database, users can create indexes on tables to accelerate queries. Indexing in RisingWave is similar to traditional databases and is designed to speed up random queries. Users can build indexes on tables and materialized views.
📄️ Join
In this article, we discuss how to perform join operations on data streams within RisingWave.
📄️ Time Windowing
Time windows can group and partition data on a time column according to certain rules.
📄️ Watermark
Watermark is one of the core concepts in a stream processing system. Data streams are infinite, and the watermark is used to impose limitations on these infinite data streams, guiding the system to handle data more efficiently. Watermark messages are interleaved within the data stream, and each watermark carries a timestamp indicating that all data up to that timestamp has arrived. RisingWave allows users to define watermark strategies on the time columns of source and append-only tables. RisingWave injects watermarks into the stream based on the defined expressions. For subsequent data that arrives later than the watermark, RisingWave filters and discards it.
📄️ EMIT ON WINDOW CLOSE
In stream processing, because the input stream is unbounded, stream processing engines can have different computation triggering strategies to determine when to compute and output results. Currently, RisingWave supports specifying two different triggering strategies when creating materialized views and sink tasks in stream processing:
📄️ Temporal Filter
The data streams processed by RisingWave are infinite. However, in many cases, users are only interested in data from a recent period (such as the past 1 hour, 1 day, or 1 week). Storing and processing unnecessary expired data is often unnecessary and a waste of resources. In some other systems, users are often allowed to define a Time-To-Live (TTL) to eliminate expired data, but the definitions can be vague, and the specific elimination time is unclear. Users also find it challenging to precisely specify the target states to eliminate. RisingWave provides a temporal filter syntax in SQL to help users eliminate expired data.
📄️ Schema Change
Schema changes in a database are particularly important. In a production environment, we may undergo table schema changes every few weeks or months. Since stream processing systems are often used as downstream systems for database systems, how to respond to upstream database schema changes is a common issue for stream processing systems.