Hey, I made this PR to fix the issue that I currently have, it's related with this Slack Thread.
Currently have an issue, all enrichments listed in apply meta are not replayed. We added this processor to replay all enrichment, even skipped ones because, some of them can be skipped because of a dependencies issue with the failed one. We added this processor before the workflow:
- label: replay_skipped
switch:
- check: this.meta.enrichments.exists("skipped") && this.meta.enrichments.skipped.length() > 0
processors:
- label: replay_skipped_enrichments
bloblang: |-
root = this
root.meta.enrichments.apply = this.meta.enrichments.failed.keys().merge(this.meta.enrichments.skipped)
Our workflow looks like that
---
processor_resources:
- label: enrichments
workflow:
meta_path: meta.enrichments
order:
...
But, in the document resulted, the enrichments listed in apply are not replayed, only the one previously in failed is replayed. They are not listed in succeeded, they are in skipped and the field added by enrichment are not present.
I'm not expert in Go but in my PR, if I well understand, in function skipFromMeta, the if skipped section should be before the if apply one because the if apply will remove the enrichment from the list skipList filled by the if skipped section.
I wrote test to validate if we have the expected result and it's works 🍾
I run tests for others functions TestWorkflows, TestWorkflowsDeps, TestWorkflowsParallel and TestWorkflowsWithOrderResources
Hey, I made this PR to fix the issue that I currently have, it's related with this Slack Thread.
Currently have an issue, all enrichments listed in
applymeta are not replayed. We added this processor to replay all enrichment, even skipped ones because, some of them can be skipped because of a dependencies issue with the failed one. We added this processor before the workflow:Our workflow looks like that
But, in the document resulted, the enrichments listed in
applyare not replayed, only the one previously infailedis replayed. They are not listed insucceeded, they are inskippedand the field added by enrichment are not present.I'm not expert in Go but in my PR, if I well understand, in function skipFromMeta, the
if skippedsection should be before theif applyone because theif applywill remove the enrichment from the listskipListfilled by theif skippedsection.I wrote test to validate if we have the expected result and it's works 🍾
I run tests for others functions
TestWorkflows,TestWorkflowsDeps,TestWorkflowsParallelandTestWorkflowsWithOrderResources