Skip to content

CONCEPT Cited by 1 source

FactoryBot object explosion

A test-fixture-library failure mode where declaring an association on a factory implicitly creates a much larger object graph than the test author realised — each test that uses the factory then pays the cost of building the full graph, not just the named object.

The mechanism

FactoryBot (the dominant Rails test-data library) makes declaring an association on a factory one line:

factory :database do
  association :organization
  association :owner
  # ... more fields
end

Each association :x tells FactoryBot "when you build a :database, also build an :x". Those factories in turn declare their own associations, which declare theirs, and so on — a transitive closure of create(...) calls executes every time a test does create(:database). The chain runs quietly: there's no warning, no cycle-detection, no count-aware assertion.

Production failure shape

PlanetScale's canonical datum (How our Rails test suite runs in 1 minute on Buildkite, 2022-01-18, Mike Coutermarsh):

After a bit of digging, we noticed most of our test time was spent setting up test data. We use FactoryBot for this.

We began investigating this by putting a debugger in our tests to stop execution right after the test setup. We used pry here to look around at all the objects created and see if they matched our expectations. We found a few surprising places where we were creating up to 8× as many objects as we thought we were.

Up to 8× more rows than the author expected for a single create(:database) call. Multiply by hundreds of tests and you have a test suite dominated by fixture-setup cost. At 64- way parallelism each worker pays the amplified setup cost simultaneously, so the absolute CPU budget spent on fixtures per build is the per-test excess × test-count — wasted work even when wall-clock looks acceptable.

Why the failure is hard to see

  • No warning: FactoryBot silently builds whatever the association declarations specify; there is no debug mode that logs the transitive object graph count.
  • Expectation drift: factories evolve incrementally over years — a new teammate adds association :audit_log to :database, not realising this cascades through :audit_log → :event → :actor → :user → :organization each of which has its own associations, etc.
  • Cost lands in downstream tests, not the factory: whoever introduces the extra association pays no test-speed penalty in CI because their new test was (probably) fast; the cost lands diffusely on every downstream test that happens to use :database.
  • Survives review: PR diffs show a one-line association added; reviewers can't see the transitive graph either.

Diagnosis technique

PlanetScale's canonical diagnosis (verbatim):

We began investigating this by putting a debugger in our tests to stop execution right after the test setup. We used pry here to look around at all the objects created and see if they matched our expectations.

  • Drop into pry (or binding.pry) right after the setup block / factory call.
  • Query Model.count for each candidate table.
  • Cross-check against the author's expectation for the test.
  • If the model count is higher than expected, walk the factory's association chain and identify which association is cascading further than necessary.

Remediation shape

Once the excess association is found, the fix is usually local: replace association :x with a lighter trait-scoped or build-only equivalent, or remove the association entirely and stub the dependency at test time.

Lock the fix in place with an assertion on factory object count — the new assert-factory-object- count pattern (canonical worked example):

test "factory doesn't create tons of databases" do
  create(:database)
  assert_equal 1, Database.count
end

The assertion is a property of the factory itself, not of any downstream test. One broken factory now fails one dedicated test rather than silently slowing dozens of unrelated tests. "We keep these tests in our models, protecting us from any regressions when making changes to our factories."

Why this matters beyond Rails / FactoryBot

The shape "fixture-library associations implicitly build larger graphs than the author thinks" generalises to every test-data library that supports declarative associations:

  • Python's factory_boy — same SubFactory cascade.
  • Go's testfixtures — implicit foreign-key inserts.
  • Laravel's factory helpers — same for() chain behaviour.
  • Any ORM with eager-building fixture helpers.

The PlanetScale datum is specifically about FactoryBot + Rails but the pattern fixture-helper-explosion is the general class.

Seen in

Last updated · 347 distilled / 1,201 read