Skip to content

#447: make off_quota? aware of GitHub Search API quota#448

Open
igor-belousov wants to merge 2 commits intozerocracy:masterfrom
igor-belousov:447
Open

#447: make off_quota? aware of GitHub Search API quota#448
igor-belousov wants to merge 2 commits intozerocracy:masterfrom
igor-belousov:447

Conversation

@igor-belousov
Copy link
Copy Markdown
Contributor

Closes #447. Downstream consumer: zerocracy/judges-action#589.

GitHub has two independent rate limits: core (5000/hr) and search (30/min).
off_quota? previously only checked core, so search endpoints proceeded
even when search quota was exhausted, then crashed mid-judge with 403.

Changes:

  • Fbe::SEARCH_METHODS lists the 6 Octokit search methods.
  • off_quota? accepts resource: :core | :search and threshold:
    (defaults: 50 for core, 5 for search). For :search it reads
    resources.search.remaining from last_response, falls back to core
    with a warning if the search payload is unavailable.
  • The intercepted block routes SEARCH_METHODS calls through the
    search-quota check.
  • Fbe::Middleware::RateLimit now tracks @searchleft separately and
    decrements it for /search/* paths, so the cached /rate_limit
    response reflects accurate search-remaining counts within the
    100-call cache window.
  • The cache-hit path no longer mutates the cached response body. It
    builds a fresh body via patched_body (deep-dup via JSON round-trip)
    and a response_env that dups the original Faraday env, preserving
    invariants like finished?.

Tests: 11 new tests across test/fbe/test_octo.rb and
test/fbe/middleware/test_rate_limit.rb covering quota routing,
threshold defaults, fallback paths, cached-stale scenario across the
100-request window, the cache-body-leak regression, and the
JSON-string body decrement path. 268 tests total, all green, rubocop
clean.

Known follow-ups (separate issues to be filed):

  • .send / .public_send on the Fbe.octo wrapper bypasses the
    quota guard. The wrapper extends BasicObject, so :send lands
    in method_missing with m = :send. Affects both core and
    search paths, predates this change.
  • Source-of-truth: middleware tracks @searchleft while off_quota?
    reads last_response. Should consolidate so middleware is the
    single source. Current design works but is fragile (depends on
    call ordering).
  • Concurrency hardening: middleware state is not thread-safe.
    judges-action runs sequentially today, so not an immediate issue,
    but should be guarded with a mutex if concurrent callers are ever
    introduced. Marked with an inline NOT thread-safe comment.

@igor-belousov
Copy link
Copy Markdown
Contributor Author

@yegor256 review please

Comment thread lib/fbe/middleware/rate_limit.rb Outdated
stringed = body.is_a?(String)
return body unless body.is_a?(String)
JSON.parse(body)
rescue JSON::ParserError
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@igor-belousov two terrible design ideas in one line: exception swallowing and returning NULL

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yegor256 the rescue is there because test_ignores_non_hash_response_body on master feeds 'invalid json' to /rate_limit and expects the middleware to pass it through. Without rescue, that test crashes. Want me to delete the test, or keep one rescue at the call site that logs and raises Fbe::Error?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@igor-belousov

feeds 'invalid json' to /rate_limit and expects the middleware to pass it through

why should it pass an invalid JSON through?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point, removing the test and inlining JSON.parse without rescue

@igor-belousov
Copy link
Copy Markdown
Contributor Author

@yegor256 pushed cleanup in 82fe71f, dropped the rescue and the test. Ready for another look.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fbe.octo.off_quota? ignores GitHub Search API quota — search_issues hits 403 even with core quota available

2 participants