perf(relations): use single-pass group-by for eager-load matching#1187
Conversation
setRelatedForMany re-scanned the entire related array once per parent (related.filter/find), making preload O(parents × related) — quadratic. On 1000 parents × 11000 pivot rows a manyToMany preload took ~1s. Group the related rows into a Map once, then match each parent with an O(1) lookup (O(parents + related)). Applied across manyToMany, hasMany, hasManyThrough (multi-match) and belongsTo, hasOne (single-match, first-wins to mirror the previous find). Associations are unchanged; the ~19x speedup at 1000 parents is now backed by regression tests covering per-parent isolation, empty/null results and first-wins. Closes #1185
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughRelation preloading now indexes related rows with ChangesRelation preloading
Estimated code review effort: 3 (Moderate) | ~20 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/orm/model_has_one.spec.ts`:
- Around line 1011-1026: Make eager-relation ordering deterministic in all three
tests: in test/orm/model_has_one.spec.ts#L1011-L1026, order the related profiles
before asserting the first duplicate; in
test/orm/model_has_many.spec.ts#L1138-L1158, order posts by the column matching
the expected sequence; and in test/orm/model_many_to_many.spec.ts#L1464-L1484,
order the related or pivot query by the column producing [3, 1].
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: f6007bf3-f404-4cfa-8937-16a526aac089
📒 Files selected for processing (9)
src/orm/relations/belongs_to/index.tssrc/orm/relations/has_many/index.tssrc/orm/relations/has_many_through/index.tssrc/orm/relations/has_one/index.tssrc/orm/relations/many_to_many/index.tstest/orm/model_belongs_to.spec.tstest/orm/model_has_many.spec.tstest/orm/model_has_one.spec.tstest/orm/model_many_to_many.spec.ts
…rtions The grouping tests asserted bucket order and first-match selection while relying on insertion order, but the related queries were unordered and SQL result order is unspecified. Order each related query explicitly so the assertions are deterministic across adapters and query plans.
Problem
setRelatedForManyre-scans the entire related array once per parent (related.filter(...)/related.find(...)), making eager-load (preload) matchingO(parents × related)— quadratic. Every iteration also goes through the Lucid modelProxy, amplifying the constant factor.Reproduced against this codebase (better-sqlite, models with only
id+name), timing a realUser.query().preload('skills'):Scaling confirms the complexity: current is ~4x per 2x data (quadratic), fixed is ~2x per 2x data (linear).
Fix
Group the related rows into a
Mapin a single pass, then match each parent with an O(1) lookup —O(parents + related). Applied across every relation implementingsetRelatedForMany:manyToMany,hasMany,hasManyThroughbelongsTo,hasOne(first-wins + skip undefined keys, to mirror the previousfind+ guard semantics)Associations produced are byte-for-byte identical — the
value !== undefinedparent guard and related-row ordering are preserved. Removed the now-unusedisRelatedRowhelpers inhasMany/belongsTo.Tests
Regression tests covering both behavior classes:
manyToMany,hasMany): per-parent isolation, empty array for an unmatched parent, order preservation, no row duplicated across parents.belongsTo,hasOne): correct keying across parents,nullfor unmatched (orphan FK), and first-wins on duplicate keys.All existing relation suites pass (415 tests), lint + typecheck clean.
Closes #1185
Summary by CodeRabbit
Performance Improvements
Bug Fixes
nullor[].Tests