Skip to content

feat: bulkStore batch write + SmartExtractor optimization (Issue #665, #666)#669

Merged
rwmjhb merged 7 commits intoCortexReach:masterfrom
jlin53882:fix/auto-capture-batch-write
Apr 20, 2026
Merged

feat: bulkStore batch write + SmartExtractor optimization (Issue #665, #666)#669
rwmjhb merged 7 commits intoCortexReach:masterfrom
jlin53882:fix/auto-capture-batch-write

Conversation

@jlin53882
Copy link
Copy Markdown
Contributor

Summary

Implements batch write optimization for auto-capture to reduce lock contention:

  1. bulkStore() - Single lock acquisition for N entries (Issue feat: batch write to reduce lock contention from auto-capture (extends Issue #643) #665)
  2. SmartExtractor batch - Collect CREATE decisions, batch write at end (Issue SmartExtractor should use bulkStore() to reduce lock acquisitions #666)

Changes

  • src/store.ts - Added �ulkStore() method
  • src/smart-extractor.ts - Batch collection for CREATE decisions

Test Results

Test Suite Tests Pass
smart-extractor-bulk-store.test.mjs 9
smart-extractor-bulk-store-edge-cases.test.mjs 17
bulk-store.test.mjs + edge-cases 8

Lock Reduction

Scenario Before After
5 CREATE entries 5 locks 1 lock
10 CREATE entries 10 locks 1 lock
20 concurrent writes ~20 locks ~2-4 locks

Related Issues

- Add bulkStore() method that acquires lock once for multiple entries
- Reduces lock contention for auto-capture writes
- Fixes Issue CortexReach#665
- Filter out undefined/null entries
- Skip empty text
- Early return for empty validEntries (skip lock)
- Add edge case tests

Fixes potential crashes with invalid input
…ce lock contention (Issue CortexReach#666)

- Add buildStoreEntry() helper to construct entries without writing
- Add createEntries array to collect all CREATE decisions
- Pass createEntries to all handler methods (handleProfileMerge, handleSupersede, handleContextualize, handleContradict, handleMerge)
- Wrap direct store.store() calls with if(createEntries) guards
- Final bulkStore() call at end of extractAndPersist() reduces N locks to 1

Changes:
- src/smart-extractor.ts: batch creation logic
- test/smart-extractor-bulk-store.test.mjs: 9 tests
- test/smart-extractor-bulk-store-edge-cases.test.mjs: 17 tests

Note: SUPERSEDE/CONFLICT still require 2 locks per decision (cannot batch due to ID dependency)
@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants