Skip to content

cache gateway lookup and async checkpoint I/O#86

Merged
cchinchilla-dev merged 3 commits intomainfrom
fix/tech-debt-infra-9-14
Apr 3, 2026
Merged

cache gateway lookup and async checkpoint I/O#86
cchinchilla-dev merged 3 commits intomainfrom
fix/tech-debt-infra-9-14

Conversation

@cchinchilla-dev
Copy link
Copy Markdown
Owner

Summary

  • Cache resolved provider candidates in ProviderGateway._get_candidates() to avoid rebuilding the fallback list on every complete()/stream() call; cache invalidates on register()
  • Wrap save_checkpoint and from_checkpoint blocking I/O with anyio.to_thread.run_sync so disk operations don't block the event loop during concurrent step execution

Test plan

  • uv run pytest — 690 pass, 7 e2e deselected
  • uv run mypy src/ — no issues
  • uv run ruff check src/ tests/ — all checks passed

Closes #9
Closes #14

Copilot AI review requested due to automatic review settings April 3, 2026 16:44
@github-actions github-actions bot added providers Provider gateway and adapters core Core engine, DAG, state labels Apr 3, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 3, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves runtime performance and async responsiveness by (1) caching provider candidate resolution in ProviderGateway and (2) moving checkpoint file I/O off the event loop via anyio.to_thread.run_sync.

Changes:

  • Cache _get_candidates() results per model in ProviderGateway, invalidating the cache on register().
  • Wrap checkpoint read/write operations in StateManager.save_checkpoint() / StateManager.from_checkpoint() with anyio.to_thread.run_sync.
  • Update state checkpoint tests to await the now-async StateManager.from_checkpoint().

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
src/agentloom/providers/gateway.py Adds per-model candidate caching to avoid rebuilding fallback/provider lists on every request.
src/agentloom/core/state.py Offloads checkpoint JSON read/write to a worker thread; makes from_checkpoint async.
tests/core/test_state.py Updates checkpoint restore tests to await from_checkpoint.

@cchinchilla-dev cchinchilla-dev merged commit 3a84341 into main Apr 3, 2026
9 checks passed
@cchinchilla-dev cchinchilla-dev deleted the fix/tech-debt-infra-9-14 branch April 3, 2026 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core engine, DAG, state providers Provider gateway and adapters

Projects

None yet

Development

Successfully merging this pull request may close these issues.

checkpoint save uses blocking I/O in async context cache provider lookup in gateway

2 participants