Skip to content

fix: improve async job retry exhaustion error#1025

Draft
devin-ai-integration[bot] wants to merge 1 commit into
mainfrom
devin/1778767758-async-job-error-message
Draft

fix: improve async job retry exhaustion error#1025
devin-ai-integration[bot] wants to merge 1 commit into
mainfrom
devin/1778767758-async-job-error-message

Conversation

@devin-ai-integration
Copy link
Copy Markdown
Contributor

Summary

  • Updates the low-code async job orchestrator's final user-facing error from One or more async jobs failed after exhausting all retry attempts. to Async job partition did not complete successfully.
  • Classifies terminal vendor-side async job failures as transient_error while preserving system_error for job creation failures and unexpected internal paths.
  • Adds unit coverage for the exhausted retry path and records the change in the CDK changelog.

Resolves https://github.com/airbytehq/airbyte-internal-issues/issues/16437
Related to https://github.com/airbytehq/oncall/issues/12605

Review & Testing Checklist for Human

  • Confirm transient_error is appropriate for vendor-side terminal async statuses such as Amazon SP-API FATAL after retry exhaustion.
  • Confirm preserving system_error for job creation failures matches expected CDK behavior.
  • Verify downstream consumers do not rely on the previous final async job error string.

Notes

Not breaking: this only changes error handling and FailureType for exhausted vendor-side async job statuses; it does not change schema, state, spec, or stream behavior. Local checks run:

  • poetry run pytest unit_tests/sources/declarative/async_job/test_job_orchestrator.py -q
  • poetry run ruff check airbyte_cdk/sources/declarative/async_job/job_orchestrator.py unit_tests/sources/declarative/async_job/test_job_orchestrator.py
  • poetry run ruff format --check airbyte_cdk/sources/declarative/async_job/job_orchestrator.py unit_tests/sources/declarative/async_job/test_job_orchestrator.py

Link to Devin session: https://app.devin.ai/sessions/908cbb8f9e1e495eafa702f360cf4df6

@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@github-actions
Copy link
Copy Markdown

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1778767758-async-job-error-message#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1778767758-async-job-error-message

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /autofix - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test - Runs connector tests with the updated CDK
  • /prerelease - Triggers a prerelease publish with default arguments
  • /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
  • /poe <command> - Runs any poe command in the CDK environment
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

@github-actions
Copy link
Copy Markdown

PyTest Results (Fast)

4 067 tests  +3 474   4 056 ✅ +3 474   7m 43s ⏱️ + 4m 12s
    1 suites ±    0      11 💤 +    1 
    1 files   ±    0       0 ❌  -     1 

Results for commit a3018cf. ± Comparison against base commit 19a7083.

@github-actions
Copy link
Copy Markdown

PyTest Results (Full)

4 070 tests  +1   4 058 ✅ +1   11m 0s ⏱️ +10s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit a3018cf. ± Comparison against base commit 19a7083.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants