Glassflow

GlassFlow is an open-source stream processing engine designed for high-volume data ingestion and data transformation from multiple sources to ClickHouse. GlassFlow comes with the following core functionalities- Stateless transformations: Powered by the expr expression engine, enabling flexible data transformations using helper functions and standard operators (e.g, removing null values or replacing missing timestamps)- Stateful transformations: A built-in state store allows deduplication logic and temporal joins over configurable time windows- Filtering: Drop events you don’t want to ingest into ClickHouse before they reach your tables- Ingest only: Direct data transfer from Kafka to ClickHouse without transformations- Metrics & OTEL: Built-in pipeline metrics with OpenTelemetry support- Dead-Letter-Queue: Keep pipelines running when faulty events occur. Inspect failed events and reprocess them later.