CONCEPT Cited by 1 source

Two-way state buffering

Definition

A stateful stream join strategy where events from both input streams are buffered in state, as opposed to one-way buffering where only one stream's events are stored and the other must match immediately on arrival.

Why it matters

In event-time joins where two streams carry correlated events (e.g., auction bids and user interactions), the arrival order is not guaranteed. One-way buffering (store bids, match interactions on arrival) drops any interaction that arrives before its corresponding bid. Two-way buffering handles this out-of-order case by storing both sides:

• Interaction arrives first → stored in interaction state; when bid arrives later, match is made
• Bid arrives first → stored in bid state; when interaction arrives later, match is made

Seen in

• Zalando Ad Platform (2026-07): Switching from one-way (bid-only) to two-way buffering increased event match rate by 0.5% — directly impacting billing revenue. With business-critical match rates, every fraction of a percent represents unbilled advertiser actions (Source: sources/2026-07-23-zalando-from-homegrown-to-flink-migrating-a-stateful-ad-event-join).

Trade-offs

• Pro: Higher match rate, handles out-of-order delivery gracefully
• Con: ~2× state storage (both sides buffered), more complex expiry logic
• When justified: When match rate is a business-critical metric and the additional state cost is acceptable relative to the revenue from recovered matches