Skip to content

Milestones

List view

  • **v1.0.0 Milestone Description** > **Objective**: Deliver the first production-ready release of the `pipelines` library, complete with essential streaming features such as **keyed processing** (`KeyBy`/`KeyedDataStream`), **watermarking**, **windowing** (tumbling and sliding windows), and **joins**. > **Scope**: > - **KeyedDataStream & Watermarking**: Introduce event-time semantics, watermark generation, and per-key transformations. > - **Windowing**: Implement time-based and count-based windows (tumbling, sliding), allowing for aggregations and incremental processing of batches. > - **Join**: Enable correlation of two keyed streams within a defined window, allowing for rich event correlation use cases. > - **Pipeline Functions**: Add new streamlined methods to `Pipeline` which abstract and build on KeyedDataStreams, Windowing, and Joining with an e2e pipeline level focus. > **Outcome**: A stable API that offers feature parity with fundamental concepts found in established streaming frameworks i.e., Flink, Spark Streaming. All while maintaining a Go-native approach with minimal overhead. After releasing **v1.0.0**, subsequent issues will focus on building higher-level features on top of the foundations laid here.

    No due date
    2/5 issues closed