Dynamic filters must be collected from workers and propagated up to the coordinator for displaying. This is a good way to help test that they are working. It's also a good way to test that collection works.
We need some sort of primitive to collect dynamic filters from plans (for displaying but also for when we need to propagate changes across the network).
See apache/datafusion#21207 (comment). apply_expressions was useful for this but it got reverted. We can try re-adding something like that OR have some sort of solution involving downcasting in datafusion-distributed. The downcasting option is less ideal because it hurts extensibility. If we downcast, we will probably need a way for users to register their custom ExecutionPlan nodes into the downcast chain.
Dynamic filters must be collected from workers and propagated up to the coordinator for displaying. This is a good way to help test that they are working. It's also a good way to test that collection works.
We need some sort of primitive to collect dynamic filters from plans (for displaying but also for when we need to propagate changes across the network).
See apache/datafusion#21207 (comment).
apply_expressionswas useful for this but it got reverted. We can try re-adding something like that OR have some sort of solution involving downcasting in datafusion-distributed. The downcasting option is less ideal because it hurts extensibility. If we downcast, we will probably need a way for users to register their customExecutionPlannodes into the downcast chain.