Skip to content

PATTERN Cited by 1 source

Weekly batch forecast + daily batch optimise cadence

Problem

In a two-stage forecast + optimise pipeline, the forecast is the expensive ML workload (model training + batch inference over all entities) and the optimiser is the cheaper downstream layer. Running both at the same cadence means:

  • If both are weekly: daily input changes (inventory levels, new orders, price changes) aren't reflected in recommendations for up to 7 days.
  • If both are daily: the forecast tax is paid every day even though the forecast signal doesn't change much intra-week.

The cadence of the two workloads should not be coupled.

Pattern

Choose independent cadences based on the natural refresh rate of the underlying signal:

  • Forecast pipeline cadence — tuned to how fast the demand signal evolves at the SKU level + how long it takes to retrain at scale. Weekly is a common choice for e-commerce demand forecasting.
  • Optimise pipeline cadence — tuned to how often actionable inputs change (inventory state, prices, lead times, stock in transit). Daily batch for the majority case + real-time online for ad-hoc parameter changes.

The optimiser always consumes the latest available forecast from the forecast pipeline; forecasts are cached and reused across optimiser runs.

Why it works

  • Expensive workload runs less often. In Zalando's case, the forecast is < 2h for 5M SKUs — still expensive to run daily, and the marginal forecast change per day is small compared to the weekly retrain.
  • Cheap workload runs when actionable state changes. Inventory state + settings change daily; the optimiser runs daily so recommendations reflect that.
  • No cadence conflict on shared state. Forecasts are written once per week, optimisation reads them all week — simple pub-sub contract.

Canonical instance (Zalando ZEOS)

  • Forecast: weekly.
  • Optimisation: daily batch + online interactive.

Verbatim:

"The Demand Forecast pipeline is a batch prediction pipeline that produces probabilistic forecasts for articles at a weekly cadence. The Inventory Optimisation pipeline offers daily batch predictions, as well as a real-time inference endpoints to enable our B2B partners to interactively plan inventory settings."

Variations

  • Weekly forecast + hourly optimise — for price-sensitive commodities or fast-turnover inventory.
  • Monthly forecast + weekly optimise — for slow-moving industrial supply chains.
  • Daily forecast + realtime optimise — for news-cycle-driven workloads where demand shifts hourly.

The load-bearing idea is the two cadences should be independently tunable, not the specific choice of weekly + daily.

Seen in

Last updated · 501 distilled / 1,218 read