Skip to content

CONCEPT Cited by 1 source

Workflow aggregated view

A workflow aggregated view reconstructs the terminal status of every step across multiple runs of a single workflow instance — essential when a workflow is restarted partway through and some steps ran in the original attempt, others ran in the restart.

Canonical wiki instance is Netflix Maestro's aggregated view (Source: sources/2024-07-22-netflix-maestro-netflixs-workflow-orchestrator):

"Aggregated view is computed by merging base aggregated view with current runs instance step statuses."

Worked example

From the article:

  1. Run 1: step1 succeeded, step2 succeeded, step3 failed, step4 + step5 not started.
  2. User investigates step3 failure, fixes the underlying issue, and restarts the workflow.
  3. Run 2 starts at step3 (step1 + step2 are marked "skipped because already successful"). After step3 succeeds, step4 + step5 run.
  4. Aggregated view at workflow completion: every step shows its successful run state — step1/2 from run 1, step3/4/5 from run 2.

Why this is non-trivial

A naive "look at the latest run's step statuses" approach would show step1 + step2 as skipped (because they didn't execute in run 2), losing the information that they previously succeeded. Conversely, a naive "concatenate all runs" approach would show step3 as having both failed (run 1) + succeeded (run 2), without the semantic that the successful outcome is the authoritative one.

The aggregated view explicitly tracks:

  • Base aggregated view — the accumulated terminal statuses of steps that completed successfully in prior runs (which are skipped in subsequent runs).
  • Current run step statuses — the statuses of steps that are executing or terminal in the current run.

Merge resolves each step to its most recent successful terminal state across runs, or its current-run state if still active.

Why users need this view

Workflow restart is a first-class operation in Maestro — users manually restart failed runs after fixing an underlying issue. If the orchestrator only exposes per-run state, users would have to reconstruct "what actually ended up succeeding" by hand across the run history.

The aggregated view answers "did my workflow instance ultimately succeed?" directly, without users having to chase the workflow across runs.

Relationship to rollup

The aggregated view is per-step, per-instance — the terminal status of each step in the workflow. The rollup is a summary of the aggregated view across the whole workflow instance, recursing into subworkflows + foreach iterations. Both are "eventually consistent" against the underlying run state.

Seen in

Last updated · 319 distilled / 1,201 read