There's one architectural decision that has worked really well in this project:
While injecting network boundaries, the single-node plan is already fully formed, good for a single-node execution, and with all the appropriate ExecutionPlans already in place.
This rule is broken by the way we currently inject the ChildrenIsolatorUnionExec nodes, which are inserted in one of the top-down recursive passes of the distributed planner.
It would be nice to refactor this, and do something similar that:
- Injects the
ChildrenIsolatorUnionExec before the distributed planner injects any network boundary (similar to how we inject BroadcastExec beforehand).
- Make
inject_network_boundaries react to the presence of a ChildrenIsolatorUnionExec and do the per-task children assignment there.
There's one architectural decision that has worked really well in this project:
While injecting network boundaries, the single-node plan is already fully formed, good for a single-node execution, and with all the appropriate
ExecutionPlans already in place.This rule is broken by the way we currently inject the
ChildrenIsolatorUnionExecnodes, which are inserted in one of the top-down recursive passes of the distributed planner.It would be nice to refactor this, and do something similar that:
ChildrenIsolatorUnionExecbefore the distributed planner injects any network boundary (similar to how we injectBroadcastExecbeforehand).inject_network_boundariesreact to the presence of aChildrenIsolatorUnionExecand do the per-task children assignment there.