Skip to content

SYSTEM Cited by 2 sources

Sidekiq

Sidekiq is a Ruby background-job framework by Mike Perham (mperham/sidekiq) using Redis as the durable job store. Dominant async-job substrate for Ruby on Rails applications.

Architecture in one paragraph

Applications enqueue jobs by calling Worker.perform_async(args) (or .perform_in / .perform_at for scheduled). The job is serialised as a hash and pushed to a Redis list under a queue key. Sidekiq server processes pop from these lists, deserialise, and invoke Worker#perform in a thread pool. Failed jobs go through a retry queue with exponential backoff and ultimately a dead-letter set.

Middleware chains

Sidekiq exposes two middleware chains as composition points:

  • Client middleware — runs in the calling process at perform_async time, before the job is written to Redis. Receives (worker_class, job_hash, queue, redis_pool). A middleware that yields passes the job through; one that returns false / nil / falsy short-circuits the enqueue — nothing ends up in Redis. This is the hook feature-flagged job-enqueue rejection rides on.
  • Server middleware — runs in the worker process around perform. Wraps execution; can retry / skip / log / trace.

Both chains are configured in config/initializers/sidekiq.rb:

Sidekiq.configure_server do |config|
  config.client_middleware do |chain|
    chain.add(SidekiqMiddleware::SidekiqJobsFlipper)
  end
end

Enqueue APIs

  • Worker.perform_async(args) — enqueue immediately. One Redis round-trip per call.
  • Worker.perform_in(delay, args) / .perform_at(time, args) — enqueue with a fixed delay or absolute run-at.
  • Worker.perform_bulk([args1, args2, ...]) (Sidekiq 6+) — enqueue N jobs in a single Redis command. The amortisation lever for scheduler bulk-enqueue. PlanetScale uses batches of 1,000.
  • Worker.set(wait: duration).perform_later(args) (ActiveJob) — delayed enqueue, composable with jitter for jittered scheduling.

Sidekiq Enterprise unique jobs

Sidekiq Enterprise adds unique_for: + unique_until: worker options (concepts/sidekiq-unique-jobs):

sidekiq_options unique_for: 1.minute, unique_until: :start

Framework-level enqueue deduplication: a duplicate perform_async for the same (class, args) within the window is rejected silently. Complements application-level idempotence defences (state re-check, DB locks) as the third layer of defence against duplicate enqueues — particularly useful when paired-scheduler architectures deliberately produce duplicate enqueues.

Retry semantics

Default: 25 retries with exponential backoff, jitter, ultimately a dead-letter set. Retry happens on any unhandled exception. Combined with scheduler re-enqueue and user enqueue, this is why idempotent job design is load-bearing — the same logical work can arrive via three independent paths.

Seen in

Last updated · 378 distilled / 1,213 read