Skip to content

CONCEPT Cited by 1 source

Gradient-free black-box optimisation

Definition

Gradient-free black-box optimisation is the class of optimisation methods that treat the objective f(θ) as an opaque function — you can call f(θ) and get a value, but you cannot inspect its analytic form, compute gradients, or exploit problem structure. The methods work by sampling f at strategically chosen points and adapting the search based on observed outputs.

Canonical families:

  • Evolutionary strategies — CMA-ES, NES; maintain and adapt a sampling distribution.
  • Bayesian optimisation — fit a surrogate model (Gaussian process, tree Parzen estimator) to observed samples; pick next evaluation via an acquisition function.
  • Direct search — Nelder-Mead simplex, Powell's method, pattern search.
  • Simulated annealing — accept worsening moves with decreasing probability.
  • Random search / grid search — baseline.

When gradient-free is the natural choice

  • Non-differentiable cost function — e.g. the output of a discrete-event simulator, a Monte Carlo expectation over a branching simulator, a cost involving thresholds or if-else branches on uncertain inputs.
  • Expensive evaluations — each f(θ) call involves many Monte Carlo samples, full-system simulation, or physical experimentation; smart sampling beats wasteful gradient estimation.
  • Noisy evaluations — f(θ) returns a noisy estimate; stochastic-gradient methods can destabilise.

Canonical instance (Zalando ZEOS)

Zalando ZEOS's replenishment optimiser minimises the inventory cost objective by Monte Carlo simulation over probabilistic demand + lead-time forecasts (see concepts/monte-carlo-simulation-under-uncertainty). The cost function is non-differentiable — stockout cost is a step at zero inventory, storage cost has shelf-capacity breakpoints, lost-sales cost depends on branching inventory depletion dynamics. Zalando frames this verbatim:

"All inputs are fed into a recommendation engine that leverages Monte Carlo simulations and black-box gradient-free optimisers for optimisation under uncertainty."

The specific black-box optimiser family is not disclosed in the canonical source.

Tradeoffs

  • Evaluation budget is typically the binding constraint — you get a fixed compute budget and have to extract the best θ you can within it.
  • Dimensionality — gradient-free methods scale poorly to high-dimensional decision spaces; work fine up to ~50 decision variables, degrade beyond that.
  • Parallelism — most methods support embarrassing parallelism within an iteration (many f(θ) evaluations in parallel), which is how Zalando exploits multi-threading in the online Lambda path.

Seen in

Last updated · 501 distilled / 1,218 read