Stateless Filter + Enrich
Per-record filter keeping European regions (eu-*), then enrich each kept row with a derived latency_ratio and an ok/breach SLA decision. No state — measures the raw I/O + serde + per-record compute ceiling (fed above each consumer's capacity so CPU saturates).
Last updated: 2026-06-18 13:19 · same-machine, relative comparison.
Bench characteristics
In-process computation — Per record: filter region LIKE 'eu-%', then derive latency_ratio = latency_ms / 80 and an ok/breach SLA flag. Stateless.
| product | delivery | processing | engine | partition parallelism |
|---|---|---|---|---|
| turbine | at-least-once | batch (Arrow columnar) | in-process · Rust core + free-threaded CPython 3.14t | intra-partition (3 shards, hash-keyed) |
| faststream | at-least-once | batch I/O, per-row compute | in-process · pure-Python async (aiokafka; orjson JSON codec) | none (one async consumer) |
| flink | at-least-once | batch (pipelined, JVM code-gen SQL) | in-process · JVM (PyFlink local MiniCluster, parallelism=1) | none (one task slot / partition, parallelism=1) |
| quix | at-least-once | row (per-record StreamingDataFrame) | in-process · pure-Python over librdkafka (C client) | none (one consumer per partition) |
| risingwave | at-least-once | batch (vectorized SQL) | external server · Rust (single-node: meta+compute+frontend) | none (one source reader per partition) |
All products are benched at at-least-once (each one's default): Turbine, Quix, RisingWave and Flink can opt into exactly-once (Kafka transactions / a transactional sink) but the bench keeps the default, so the comparison is iso-semantics. Input is a single Kafka partition (lz4), so partition parallelism is what each product can extract from that one partition.
Throughput / resource usage
| product | offered_in/s | consumed_in/s | output_rate | cpu_avg | cpu_p95 | rss_avg_mb | rss_max_mb | run |
|---|---|---|---|---|---|---|---|---|
| turbine 0.1.0a5 | 750,000 | 501,558 | 131,551 | 103.9 | 106.0 | 1,319.8 | 1,554.2 | 2026-06-17 22:02 |
| faststream 0.7.1 | 110,000 | 95,399 | 31,384 | 98.6 | 99.1 | 134.0 | 136.2 | 2026-06-17 14:01 |
| flink 2.2.1 | 700,000 | 572,341 | 175,141 | 212.3 | 225.8 | 1,127.3 | 1,153.6 | 2026-06-18 13:19 |
| quix 3.23.6 | 300,000 | 178,860 | 58,963 | 99.9 | 103.0 | 198.4 | 200.9 | 2026-06-15 22:26 |
| risingwave v2.8.4 | 250,000 | 170,652 | 57,918 | 45.6 | 65.2 | 846.9 | 876.9 | 2026-06-16 13:04 |
faststream — Pure-Python async (aiokafka): CPU-bound at ~1 core (~99%), draining ~95k/s on the single partition — about 0.5× quix's per-core throughput (quix's librdkafka is a C extension). Fed 110k, just above that knee, so it saturates (the intended ceiling) while the backlog grows only modestly. Leanest footprint of the four (~134 MB RSS). JSON is decoded with orjson — FastStream auto-detects it once it is installed — which MATCHES quix, whose default JSON deserializer is also orjson; on stdlib json FastStream's knee was ~78k, so orjson both LEVELS the comparison (same C-speed JSON as quix) and lifts the ceiling ~22%. Benched with FastStream's batch=True subscriber (getmany) + fire-and-forget publish (no_confirm=True, so aiokafka batches the sends): the default per-record path awaits the broker ack on every message and is round-trip-bound (~8k/s here, CPU idle), not how FastStream is run at throughput. FastStream ships ONLY this stateless workload — it has no built-in windowing, so a windowed bench would measure hand-rolled timer/state code, not FastStream.
flink — PyFlink running a pure Table API / SQL job on a LOCAL MiniCluster, pinned to parallelism=1 — one task slot reading the single Kafka partition, the bench's KPI. Single-partition knee ≈ 550k/s (consume ≈ offered, lag ≈ 0); the consume ceiling is ~570k, so it is fed 700k to saturate and read that ceiling — far above quix (~180k) and RisingWave (~170k), a touch under turbine (~500k) per CORE since Flink needs ~2 cores for it. Measured in-process: CPU/RSS are summed across the process tree INCLUDING the JVM child, so the figure carries JVM overhead (GC, checkpoint, network/produce threads) and reads ~2 cores (~200% CPU) here even at parallelism=1 — the "one slot" is the WORK parallelism, but the Kafka-produce side (it writes ~31% of records downstream) spreads onto other cores, which is also why its I/O ceiling sits below the windowed-lowcard knee (650k, ~nothing written downstream). At-least-once checkpointing commits the consumed offset to the group each checkpoint, so consumed_in/s is a real reading like quix. Benched version from flink/uv.lock; the Kafka SQL connector JAR is fetched from Maven on first run.
risingwave — Single-partition I/O-bound: RisingWave runs one source reader per Kafka partition, so on the bench's single partition it drains only ~170k/s — well under the offered rate, so the backlog grows — while its CPU stays well below saturation (~0.5 core). It cannot trade its idle cores for more throughput; scaling needs more Kafka partitions (≈3× at 4), not more cores. consumed_in/s is read from RisingWave's own source_partition_input_count metric (it commits no Kafka group offset); CPU/RSS are the whole single-node container.
Sanity — are we benching the same thing?
| product | selectivity (out/in) | expected | breach_rate | enrichment | sample |
|---|---|---|---|---|---|
| turbine | 0.1773 | 0.3100 | 0.204 | ✓ | 500 |
| faststream | 0.2882 | 0.3100 | 0.184 | ✓ | 500 |
| flink | 0.2567 | 0.3100 | 0.184 | ✓ | 500 |
| quix | 0.1490 | 0.3100 | 0.184 | ✓ | 500 |
| risingwave | 0.2341 | 0.3100 | 0.218 | ✓ | 500 |
Verdict: ✓ same order of magnitude (selectivity & breach_rate within 2× across products).