Skip to content

feat(indexer): add historical event backfill catch-up mode#302

Open
0xBernny wants to merge 1 commit into
Invoice-Liquidity-Network:mainfrom
0xBernny:feat/indexer-catchup-mode
Open

feat(indexer): add historical event backfill catch-up mode#302
0xBernny wants to merge 1 commit into
Invoice-Liquidity-Network:mainfrom
0xBernny:feat/indexer-catchup-mode

Conversation

@0xBernny

Copy link
Copy Markdown

closes #256

Summary

Related Issue

Closes #256

Complexity

  • Trivial (typo, docs, small refactor)
  • [ ✔️] Medium (new feature, bug fix)
  • High (major refactor, breaking change, security-critical)

Changes Made

  • Created a new ingestion utility class inside indexer/src/ingestion/catchup.ts to check ledger gaps on startup
  • Implemented historical event fetching logic in batches of 200 ledgers using Horizon's paginated transactions endpoint.
  • Added descriptive log outputs for tracking backfill status and remaining progress estimation.

Test Coverage

TypeScript / Jest framework test structure verified

Mocked transaction arrays successfully validate non-streaming pagination limits

# Example:
# cargo test
# running 15 tests
# test result: ok. 15 passed; 0 failed
Details Test output

Security Considerations

N/A

Checklist

  • [ ✔️] Code follows project conventions (see CONTRIBUTING.md)
  • [✔️ ] Tests added/updated and passing
  • [✔️ ] Documentation updated (if needed)
  • [✔️ ] Commit message follows Conventional Commits
  • [ ✔️] No secrets or sensitive data in code
  • Breaking changes documented (if applicable)

@drips-wave

drips-wave Bot commented Jun 28, 2026

Copy link
Copy Markdown

@0xBernny Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add indexer catch-up mode for historical event backfill

1 participant