Skip to content

perf(relations): use single-pass group-by for eager-load matching#1187

Merged
thetutlage merged 3 commits into
22.xfrom
fix/eager-load-quadratic-matching
Jul 23, 2026
Merged

perf(relations): use single-pass group-by for eager-load matching#1187
thetutlage merged 3 commits into
22.xfrom
fix/eager-load-quadratic-matching

Conversation

@thetutlage

@thetutlage thetutlage commented Jul 23, 2026

Copy link
Copy Markdown
Member

Problem

setRelatedForMany re-scans the entire related array once per parent (related.filter(...) / related.find(...)), making eager-load (preload) matching O(parents × related) — quadratic. Every iteration also goes through the Lucid model Proxy, amplifying the constant factor.

Reproduced against this codebase (better-sqlite, models with only id + name), timing a real User.query().preload('skills'):

dataset current fixed speedup
500 users × 5,500 pivot 246 ms 28 ms 8.9x
1000 users × 11,000 pivot ~1000 ms 51 ms ~19x
2000 users × 22,000 pivot 4,195 ms 101 ms 41x

Scaling confirms the complexity: current is ~4x per 2x data (quadratic), fixed is ~2x per 2x data (linear).

Fix

Group the related rows into a Map in a single pass, then match each parent with an O(1) lookup — O(parents + related). Applied across every relation implementing setRelatedForMany:

  • Multi-match: manyToMany, hasMany, hasManyThrough
  • Single-match: belongsTo, hasOne (first-wins + skip undefined keys, to mirror the previous find + guard semantics)

Associations produced are byte-for-byte identical — the value !== undefined parent guard and related-row ordering are preserved. Removed the now-unused isRelatedRow helpers in hasMany/belongsTo.

Tests

Regression tests covering both behavior classes:

  • Multi-match (manyToMany, hasMany): per-parent isolation, empty array for an unmatched parent, order preservation, no row duplicated across parents.
  • Single-match (belongsTo, hasOne): correct keying across parents, null for unmatched (orphan FK), and first-wins on duplicate keys.

All existing relation suites pass (415 tests), lint + typecheck clean.

Closes #1185

Summary by CodeRabbit

  • Performance Improvements

    • Optimized relationship preloading for BelongsTo, HasMany, HasOne, HasManyThrough, and ManyToMany by indexing related rows once to avoid repeated per-parent matching work.
  • Bug Fixes

    • Preserved “first match wins” behavior when multiple related rows share the same key.
    • Prevented cross-contamination of related rows between parents and ensured missing relations resolve to null or [].
  • Tests

    • Added preload tests for missing related records, duplicate matches, deterministic ordering, and multi-parent isolation.
    • Updated schema builder test fixtures (syntax-only).

setRelatedForMany re-scanned the entire related array once per parent
(related.filter/find), making preload O(parents × related) — quadratic.
On 1000 parents × 11000 pivot rows a manyToMany preload took ~1s.

Group the related rows into a Map once, then match each parent with an
O(1) lookup (O(parents + related)). Applied across manyToMany, hasMany,
hasManyThrough (multi-match) and belongsTo, hasOne (single-match,
first-wins to mirror the previous find). Associations are unchanged;
the ~19x speedup at 1000 parents is now backed by regression tests
covering per-parent isolation, empty/null results and first-wins.

Closes #1185
@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 75259a42-42cf-4e86-a36f-b953a8b9b328

📥 Commits

Reviewing files that changed from the base of the PR and between c217a30 and 77f4f8f.

📒 Files selected for processing (1)
  • test/orm/orm_schema_builder.spec.ts

📝 Walkthrough

Walkthrough

Relation preloading now indexes related rows with Map instances before assigning them to parents across five relationship types. New tests cover missing relations, duplicate matches, ordering, empty collections, row counts, and cross-parent isolation.

Changes

Relation preloading

Layer / File(s) Summary
Map-based relation matching
src/orm/relations/{belongs_to,has_one,has_many,has_many_through,many_to_many}/index.ts
setRelatedForMany implementations now group related rows in one pass and assign matches by parent keys, preserving null, empty-array, and first-match behavior.
Preload association coverage
test/orm/model_{belongs_to,has_one,has_many,many_to_many}.spec.ts, test/orm/orm_schema_builder.spec.ts
Preload tests verify missing relations, duplicate selection, ordering, empty results, row counts, and isolation between parents; schema fixtures use unquoted property keys without changing assertions.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Poem

I’m a bunny with a map in my paws,
Sorting rows by keys and laws.
No more searching hop by hop,
First-match treasures never stop.
Empty nests stay empty too—
Preloads now know what to do!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Out of Scope Changes check ⚠️ Warning The schema-builder test file has unrelated syntax-only fixture edits that do not support the eager-load performance work. Remove or justify the schema-builder fixture rewrites, or split them into a separate formatting-only PR.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: optimizing eager-load relation matching with single-pass grouping.
Linked Issues check ✅ Passed The relation changes address #1185 by replacing repeated scans with Map-based grouping in manyToMany and hasMany while preserving matching behavior.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/eager-load-quadratic-matching

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/orm/model_has_one.spec.ts`:
- Around line 1011-1026: Make eager-relation ordering deterministic in all three
tests: in test/orm/model_has_one.spec.ts#L1011-L1026, order the related profiles
before asserting the first duplicate; in
test/orm/model_has_many.spec.ts#L1138-L1158, order posts by the column matching
the expected sequence; and in test/orm/model_many_to_many.spec.ts#L1464-L1484,
order the related or pivot query by the column producing [3, 1].
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: f6007bf3-f404-4cfa-8937-16a526aac089

📥 Commits

Reviewing files that changed from the base of the PR and between b23d814 and 5853cbb.

📒 Files selected for processing (9)
  • src/orm/relations/belongs_to/index.ts
  • src/orm/relations/has_many/index.ts
  • src/orm/relations/has_many_through/index.ts
  • src/orm/relations/has_one/index.ts
  • src/orm/relations/many_to_many/index.ts
  • test/orm/model_belongs_to.spec.ts
  • test/orm/model_has_many.spec.ts
  • test/orm/model_has_one.spec.ts
  • test/orm/model_many_to_many.spec.ts

Comment thread test/orm/model_has_one.spec.ts
…rtions

The grouping tests asserted bucket order and first-match selection while
relying on insertion order, but the related queries were unordered and
SQL result order is unspecified. Order each related query explicitly so
the assertions are deterministic across adapters and query plans.
@thetutlage
thetutlage merged commit e617683 into 22.x Jul 23, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ManyToMany preloading is O(parents × pivot rows) — up to 380x slower than equivalent SQL

1 participant