Skip to content

SYSTEM Cited by 2 sources

Datadog Agent

The Datadog Agent is the on-host telemetry collector + forwarder shipped to customers. A single product to users, but internally a family of Go binaries built from one codebase with build tags + dependency injection to select features per target (OS × arch × distribution).

Binary family (Linux amd64 example)

  • Core Agent — main metric / check collection loop.
  • Trace Agent — APM trace ingestion + processing.
  • Process Agent — process / container discovery + live-process.
  • Security Agent — CSM / Workload Protection user-space rule engine (see systems/datadog-workload-protection).
  • System Probe — eBPF-based kernel telemetry (network, CSM).

Scale of build matrix

"Dozens of builds that vary based on operating system, architecture, and specific distribution targets." Targets include Docker, Kubernetes, Heroku, IoT, Linux distros, cloud providers. Hundreds of dependencies: cloud SDKs, container runtimes, security scanners.

Binary-size reduction program (2024-12 → 2025-07)

Canonical wiki instance of systematic Go binary pruning. Over 6 months, across versions 7.60.0 → 7.68.0, Datadog cut sizes by up to 77 % without removing any feature:

Binary Before After Reduction
Core Agent 236 MiB 103 MiB −56 %
Process Agent 128 MiB 34 MiB −74 %
Trace Agent 90 MiB 23 MiB −74 %
Security Agent 152 MiB 35 MiB −77 %
System Probe 180 MiB 54 MiB −70 %

Compressed .deb: 265 → 149 MiB (−44 %); uncompressed 1.22 GiB → 688 MiB (−44 %). Three-part recipe:

  1. Dependency audit via goda + go-size-analyzer + go list, splitting packages whose single function pulls a whole dep stack. Trace Agent: 570 k8s packages / 36 MiB removed by moving one function.
  2. Re-enable method dead-code elimination — patch every reflect.MethodByName offender across the codebase + dependencies, fork stdlib text/template + html/template into pkg/template/ with method calls disabled. 16-25 % per binary, ~100 MiB total. (concepts/reflect-methodbyname-linker-pessimism)
  3. Remove the containerd plugin import. Importing stdlib plugin at all forces the linker into dynamic-link mode, disabling method-DCE AND keeping every unexported method. containerd/containerd#11203 added a build tag to gate it; the Agent applied the tag. 245 MiB (~20 %) reduction, ~75 % of users affected. (concepts/go-plugin-dynamic-linking-implication)

See sources/2026-02-18-datadog-how-we-reduced-agent-go-binaries-up-to-77-percent for the full walkthrough + upstream PR list.

Prior-art growth trajectory

From v7.16.0 (2019) to v7.60.0 (Dec 2024): - .deb compressed: 126 MiB → 265 MiB (+110 %). - .deb uncompressed: 428 MiB → 1,248 MiB (+192 %).

Feature + dependency accumulation without corresponding prune discipline — canonical concepts/binary-size-bloat datum.

Go runtime operation

Separately covered in sources/2025-07-17-datadog-go-124-memory-regression: the Go 1.24 mallocgc refactor silently removed a "skip zeroing for OS-fresh memory" optimization for >32 KiB pointer-bearing allocations, producing ~20 % RSS uplift invisible to runtime/ metrics. Upstream fix shipped in Go 1.25.

Seen in

Last updated · 200 distilled / 1,178 read