fix(workflows): route unlisted tasks through unknown parameter/output types#2606
fix(workflows): route unlisted tasks through unknown parameter/output types#2606JustAnotherNormalDev wants to merge 29 commits into
Conversation
Parse functionApp parameters and execution output as first-class types. Route unrecognized task types through Unknown parameters/output so listing workflow versions no longer fails when the API adds new task kinds. Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
Code Review
This pull request introduces support for the functionApp task type and adds UnknownWorkflowTaskParameters and UnknownWorkflowTaskOutput classes to handle unrecognized task types gracefully. Feedback was provided to improve the robustness of these new classes by handling non-dictionary data in their dump methods and ensuring consistent task_type tracking.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…-dict output Co-authored-by: Cursor <cursoragent@cursor.com>
…omplete normalize Co-authored-by: Cursor <cursoragent@cursor.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #2606 +/- ##
==========================================
- Coverage 93.05% 93.02% -0.04%
==========================================
Files 486 486
Lines 49481 49528 +47
==========================================
+ Hits 46047 46072 +25
- Misses 3434 3456 +22
🚀 New features to boost your workflow:
|
…r functionApp Co-authored-by: Cursor <cursoragent@cursor.com>
| return cls(task_type, output if isinstance(output, dict) else {}) | ||
|
|
||
| @property | ||
| def task_type(self) -> str: # type: ignore[override] |
There was a problem hiding this comment.
This should pretend to return a ValidTaskType
There was a problem hiding this comment.
Let me know if this works
haakonvt
left a comment
There was a problem hiding this comment.
A few comments, otherwise looks good!
… on unknown dumps Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…com/cognitedata/cognite-sdk-python into fix/workflows-functionapp-task-type
Co-authored-by: Cursor <cursoragent@cursor.com>
| UserWarning, | ||
| stacklevel=2, | ||
| ) | ||
| params = self._parameters if isinstance(self._parameters, dict) else {} |
There was a problem hiding this comment.
You ignore parameters if it is not a dict. That means your init should say dict rather than Any?
There was a problem hiding this comment.
agree. Its dict now. dict[str, Any]:
| UserWarning, | ||
| stacklevel=2, | ||
| ) | ||
| out = self._output if isinstance(self._output, dict) else {} |
There was a problem hiding this comment.
I also dont think we should have any kind of such logic? The payload is unkown, so we should return what we got instantiated with
There was a problem hiding this comment.
i changed it to dict. But do you mean , remove the warn message as well?
There was a problem hiding this comment.
No the warning is great! I mean, if self._output is not dict we ignore it in the output. I don't think we should
There was a problem hiding this comment.
Reason: The unknown class should just accept what gets passed to it - maybe in the future you want this to be a list or whatever, then this would break
Co-authored-by: Cursor <cursoragent@cursor.com>
…eters Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…com/cognitedata/cognite-sdk-python into fix/workflows-functionapp-task-type
There was a problem hiding this comment.
If you are exempting your data classes from automatic testing, you need to add a better explanation please
There was a problem hiding this comment.
please ignore this one. i reverted it back. Pushed it by mistake . I had added it for testing.
Merge origin/master. Align UnknownWorkflowTaskOutput typing with raw load. Add TestUnknownWorkflowTaskParametersCogniteResourceParity for envelope round-trips. Shorten test_base exclude comment for UnknownWorkflowTaskParameters. Co-authored-by: Cursor <cursoragent@cursor.com>
…est; trim workflow tests Co-authored-by: Cursor <cursoragent@cursor.com>
… wiring Wrap non-dict payloads in an opaque dict so CogniteResource.load round-trips. Build unknown parameters from task type plus parameters blob, not full task dict. Narrow dynamic_task_type to str | None; document opaque wrap/unwrap in comments. Keep UnknownWorkflowTaskParameters in generic CogniteResource base tests. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…com/cognitedata/cognite-sdk-python into fix/workflows-functionapp-task-type
UnknownWorkflowTaskParameters round-trips are covered by workflows opaque wrapper. Co-authored-by: Cursor <cursoragent@cursor.com>
Problem
Listing workflow versions and loading executions could fail when the API returns task types that are not first-class in the SDK (for example new or beta task kinds in responses).
Solution
Route unlisted task types through UnknownWorkflowTaskParameters and UnknownWorkflowTaskOutput so the wire JSON is preserved and load no longer throws. Keep known task types on their existing dedicated classes.
UnknownWorkflowTaskParameters is constructed from the task type / taskType plus the parameters / input blob (not the full task envelope), so WorkflowTask.dump() stays aligned with the API shape.
For generic CogniteResource JSON/YAML round-trips, dump() must always yield a top-level dict. When _parameters is not a dict (for example a string from the fake resource generator in unit tests), dump() wraps the value in a single-key dict using _OPAQUE_WRAPPER_KEY ("cogniteSdkUnknownWorkflowTaskParametersOpaque"). That satisfies CogniteResource.load → load_resource_to_dict, which only accepts parsed JSON/YAML that resolves to a dict. _load detects that exact one-key shape and unwraps the inner value so the in-memory payload matches what was stored before serialization. Normal API dict payloads are returned unchanged and never use this wrapper.
Files
cognite/client/data_classes/workflows.py — unknown task parameters/output handling, load_parameters wiring, _OPAQUE_WRAPPER_KEY wrap/unwrap for non-dict unknown parameter bodies.
cognite/client/data_classes/init.py — export surface for updated workflow types.
tests/tests_unit/test_data_classes/test_workflows.py — coverage for unknown task serialization and related behavior.
tests/tests_unit/test_base.py — UnknownWorkflowTaskParameters included in generic CogniteResource round-trip tests where applicable.
cognite/client/_cognite_client.py — generated/synced client updates.
cognite/client/_sync_cognite_client.py — generated/synced sync client updates.
scripts/sync_client_codegen/sync_client_template.txt — codegen template updates matching generated client changes.