Skip to content

Updating retry logic for API requests#600

Open
scorreard wants to merge 4 commits intomainfrom
scorreard_citation_API_issue
Open

Updating retry logic for API requests#600
scorreard wants to merge 4 commits intomainfrom
scorreard_citation_API_issue

Conversation

@scorreard
Copy link
Collaborator

The weekly update crashed because of an API connection error, I tested this new configuration and it worked.

Crashed weekly update :
https://github.com/galaxyproject/galaxy_codex/actions/runs/21859224353/job/63083600802

Increased the default retry attempts from 3 to 7 and modified the delay to increase exponentially with each retry attempt.
Update retry delay to increase exponentially with attempts.
@scorreard
Copy link
Collaborator Author

@bebatut or @paulzierep could you check this PR?

We had another API connection error this week, this should give more flexibility

Error in this weekly update

Connection error on attempt 1/3. Retrying in 2.0 seconds...
Connection error on attempt 1/3. Retrying in 2.0 seconds...
Connection error on attempt 1/3. Retrying in 2.0 seconds...
Connection error on attempt 1/3. Retrying in 2.0 seconds...
Connection error on attempt 1/3. Retrying in 2.0 seconds...
Connection error on attempt 2/3. Retrying in 2.0 seconds...
  File "/home/runner/work/galaxy_codex/galaxy_codex/sources/bin/extract_galaxy_workflows.py", line 309, in add_workflows_from_public_servers
    self.add_workflows_from_a_server(url)
  File "/home/runner/work/galaxy_codex/galaxy_codex/sources/bin/extract_galaxy_workflows.py", line 285, in add_workflows_from_a_server
    server_wf = shared.get_request_json(
                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/galaxy_codex/galaxy_codex/sources/bin/shared.py", line 136, in get_request_json
    raise ConnectionError(
requests.exceptions.ConnectionError: Connection aborted after multiple retries: Remote end closed connection without response

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant