Skip to content

SYSTEM Cited by 1 source

Tron (Yelp)

Tron is Yelp's in-house batch processing and job scheduler, open-sourced at github.com/Yelp/Tron. Used across Yelp to run scheduled jobs and workflows, including the daily S3 Server Access Logs compaction pipeline and related data-engineering batch jobs.

Role on the wiki

Tron is the orchestration substrate underneath Yelp's batch data jobs. Named role in the 2025-09-26 post: "Tron, Yelp's in-house batch processing system, runs daily and converts the previous day's logs to parquet format via Athena insertion queries."

In the same SAL pipeline:

  • Drives the daily compaction job that runs Athena INSERT queries across all monitored buckets.
  • Drives the weekly access-based-retention job that joins S3 Inventory with SAL.
  • Coordinates the S3 Batch Operations manifest build and direct per-object tagging for lifecycle expiration.

Seen in

  • sources/2025-09-26-yelp-s3-server-access-logs-at-scale — Tron named as the daily batch runner for the SAL compaction pipeline; responsible for issuing parallel Athena queries, orchestrating the queue-processing flow (location query → idempotent insertion → verification → tagging), and handling TooManyRequestsException retries.
Last updated · 476 distilled / 1,218 read