New Workflow: Bedrock Batch with SLA#418
Open
subhaviv wants to merge 15 commits into
Open
Conversation
Implements Bedrock batch inference with automatic on-demand SLA fallback using the callback (task token) and saga patterns. Includes CDK (Python), 6 Lambda functions, Distributed Map redrive, and integration tests for all three fallback triggers: timeout, stuck job alarm, and failed job.
Derive ACCOUNT_ID at runtime via aws sts get-caller-identity so the script works across accounts without leaking credentials in source.
Add PartiallyCompleted to the EventBridge rule filter and explicitly route it to FallbackEntry in CheckOutcome — without this, a partially completed job would hang at WaitForBatchCompletion until the SLA timeout.
…tch-with-sla New workflow: bedrock-batch-with-sla
bfreiberg
suggested changes
Jul 17, 2026
bfreiberg
suggested changes
Jul 17, 2026
Co-authored-by: Ben <9841563+bfreiberg@users.noreply.github.com>
Co-authored-by: Ben <9841563+bfreiberg@users.noreply.github.com>
bfreiberg
reviewed
Jul 21, 2026
bfreiberg
suggested changes
Jul 21, 2026
bfreiberg
reviewed
Jul 21, 2026
bfreiberg
reviewed
Jul 21, 2026
Co-authored-by: Ben <9841563+bfreiberg@users.noreply.github.com>
…record minimum, fix lambda comment
bfreiberg
suggested changes
Jul 22, 2026
…imports, ModelIdIndex, CDK version
subhaviv
marked this pull request as draft
July 23, 2026 04:09
Contributor
|
Looks good now, thank you. Can you please mark the PR as ready for review again. |
…tion
IAM:
- Scope InvokeModel to inference-profile ARN, StopModelInvocationJob/
GetModelInvocationJob to model-invocation-job/* instead of *
- Keep CreateModelInvocationJob on * — AWS batch inference permissions
guide prescribes this; cross-region profiles route to whichever region
has capacity so the target ARN cannot be predicted at deploy time
(docs.aws.amazon.com/bedrock/latest/userguide/batch-inference-permissions.html)
- Fix Bedrock service role: add InvokeModel on foundation-model ARNs in all
destination regions using bedrock:InferenceProfileArn condition to avoid
hardcoding regions (docs.aws.amazon.com/bedrock/latest/userguide/geographic-cross-region-inference.html)
Inference profile detection:
- Fix model_id.split(".")[0] in ("us","eu","ap") instead of "." in model_id,
which incorrectly matched base model IDs like anthropic.claude-sonnet-4-6
S3 helpers:
- Move 4 duplicate S3 helper functions from merge/index.py and reconcile/index.py
into lambdas/common/s3_utils.py; helpers accept s3 client as first param so
callers control the client (testable via mock)
Other:
- Fix missing model_id argument in BedrockServiceConstruct call in stack.py
- Add AWS_PROFILE_FLAG support to integration test script so no profile names
or account numbers are hardcoded in the public repo
subhaviv
marked this pull request as ready for review
July 23, 2026 15:58
bfreiberg
approved these changes
Jul 23, 2026
bfreiberg
left a comment
bfreiberg
left a comment
Contributor
There was a problem hiding this comment.
Thanks for your contribution. Your workflow sample will be merged soon
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
bedrock-batch-with-sla— a Step Functions workflow that runs Amazon Bedrock batch inference as the primary cost-effective path andautomatically falls back to on-demand inference when an SLA deadline is at risk
stored in DynamoDB
(terminal job state — Completed, Failed, Expired, PartiallyCompleted)
webhook-provider/example-workflow.jsonwith updated author bio and image URLHow it works
S3 input (.jsonl) → Registrar λ → Step Functions
├─ Happy path: batch completes → MergeResults → Success
└─ Fallback path (timeout / alarm / terminal state):
StopBatchJob → ReconcileRecords
└─ RedriveOnDemand (Distributed Map) → MergeResults → Success
Test plan
cd test/lambdas && python -m pytest— 5/5 passingcdk deploy)cdk destroy)