feat(taskbroker): add TaskError to SetTaskStatusRequest

[s-starostin]

Summary

Add a bounded TaskError envelope to the taskbroker status update contract.

This extends SetTaskStatusRequest with an optional error field so workers can attach exception context when reporting task failures/retries.

Downstream PRs

Consumed by:

• feat(taskbroker): propagate TaskError from worker and log failure context on broker taskbroker#607
• feat(taskworker): capture task exception context via TaskErrorCaptureHook sentry#114009

Changes

• add TaskError message:
  • exception_type
  • exception_message
  • traceback
• add optional error field to SetTaskStatusRequest

Why

This is the wire-format foundation for end-to-end failure observability across:

• taskworker
• taskbroker
• Sentry

Without this, taskbroker cannot log worker-observed exception details.

Compatibility

• field is additive
• old workers can continue sending requests without error
• new brokers must tolerate error being absent

Validation

• updated taskbroker.proto
• regenerated Rust bindings
• verified that:
  • TaskError is generated with the expected fields
  • SetTaskStatusRequest includes optional error = 4
  • existing field numbers remain unchanged

Follow-up

This is consumed by downstream changes in:

• getsentry/taskbroker
• getsentry/sentry

[markstory]

Without this, taskbroker cannot log worker-observed exception details.

This isn't something that we need though. Workers capture their own errors.

[s-starostin]

@markstory, thanks for the response.

I think we may be talking past each other a bit.

I agree the worker already captures errors locally. The problem I ran into was different: when debugging starts from taskbroker-side failure symptoms, the worker-local exception context is often not available in the place the operator is looking. In practice, that can mean manually reproducing the task just to discover the actual error.

So the motivation here is observability across the async boundary, not changing task semantics.

That said, I understand the concern about adding complexity to the protocol/path. A smaller change would solve most of the pain for me:

• no protobuf changes
• no broker payload changes
• a guaranteed structured single-line worker failure log in the existing exception path, including task id, task name, namespace, exception type, and exception message

That alone would have been enough to avoid the debugging loop that motivated this proposal.

Could you point me to the worker log line or Sentry capture call site in taskbroker_client that runs when a task exception is caught? I want to make sure I’m not missing an existing path before proposing a narrower change.

[markstory]

The problem I ran into was different: when debugging starts from taskbroker-side failure symptoms, the worker-local exception context is often not available in the place the operator is looking.

This isn't a usecase/scenario we had ever planned for. When a task fails the context is best captured in the worker, as the broker has no context on what the task logic or parameter payload is. Even within the worker, tasks failing is expected and normal.

Could you point me to the worker log line or Sentry capture call site in taskbroker_client that runs when a task exception is caught?

All non-retriable failures from tasks are caught here:

https://github.com/getsentry/taskbroker/blob/1c7b62d14ed6494cb84d0995b4369da643e69c55/clients/python/src/taskbroker_client/worker/workerchild.py#L296-L297

I want to make sure I’m not missing an existing path before proposing a narrower change.

Ok. In the future, before spending your tokens generating pull requests, open an issue and get alignment on the path to take.