fix: poll through CANCELING state to deliver terminal callback#331
Merged
mwiebe merged 1 commit intoJul 21, 2026
Merged
Conversation
seant-aws
force-pushed
the
fix/cancel-poller-canceling-state
branch
from
July 16, 2026 22:03
f19bf23 to
fa3037b
Compare
leongdl
approved these changes
Jul 21, 2026
seant-aws
marked this pull request as ready for review
July 21, 2026 18:56
mwiebe
approved these changes
Jul 21, 2026
The _poll_for_completion thread exited the polling loop as soon as the session state transitioned away from RUNNING. When cancel_action() is called, the pyo3 binding immediately sets the snapshot state to CANCELING — causing the poller to exit before the subprocess has been killed and before the Rust callback delivers the terminal CANCELED ActionStatus. The result: the worker agent never receives the CANCELED callback, leaving the action stuck in CANCELING forever from the service's perspective. Fix: treat CANCELING as an in-progress state (like RUNNING) and continue polling. The poller now exits only when the state reaches READY or READY_ENDING, at which point action_status contains the correct terminal state. Signed-off-by: Sean Tang <seant-aws@users.noreply.github.com> Signed-off-by: Sean Tang <171081544+seant-aws@users.noreply.github.com>
mwiebe
force-pushed
the
fix/cancel-poller-canceling-state
branch
from
July 21, 2026 19:56
fa3037b to
5dde728
Compare
mwiebe
enabled auto-merge (squash)
July 21, 2026 19:57
godobyte
reviewed
Jul 21, 2026
| state = self._rust_session.state | ||
| status = self._rust_session.action_status | ||
| if state == SessionState.RUNNING: | ||
| if state in (SessionState.RUNNING, SessionState.CANCELING): |
Contributor
There was a problem hiding this comment.
Validated end-to-end via the AWS Deadline Cloud worker agent e2e suite (cancel scenarios all passing; full Linux suite 48/49 with the one failure unrelated to this change).
Were there any test failing due to the lack of this change?
Contributor
Author
There was a problem hiding this comment.
yea, all the cancel tests in the E2E when hardcoding to use the rust runtime were not working due to this miss
godobyte
approved these changes
Jul 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was the problem/requirement? (What/Why)
The
_poll_for_completionthread exited the polling loop as soon as the session state transitioned away from RUNNING. Whencancel_action()is called, the pyo3 binding immediately sets the snapshot state to CANCELING — causing the poller to exit before the subprocess has been killed and before the Rust callback delivers the terminal CANCELED ActionStatus.The result: the worker agent never receives the CANCELED callback, leaving the action stuck in CANCELING forever from the service perspective.
What was the solution? (How)
Treat CANCELING as an in-progress state (like RUNNING) and continue polling. The poller now exits only when the state reaches READY or READY_ENDING, at which point
action_statuscontains the correct terminal state.What is the impact of this change?
Actions that are cancelled mid-execution now correctly transition through CANCELING → CANCELED instead of hanging in CANCELING forever.
How was this change tested?
Was this change documented?
Is this a breaking change?
No. This is a bugfix to internal polling behavior; the public API is unchanged.
Does this change impact security?
No.
Cross-port to openjd-rs
_v1polling layer that bridges the Rust session to the Python callback. The Rust session itself already handles cancellation correctly — the bug was only in the Python polling shim that sits between the two.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.