Skip to content

[QNN EP] Add support for If Op#409

Open
qti-niscmami wants to merge 2 commits into
mainfrom
dev/qti-niscmami/add-support-for-if-op
Open

[QNN EP] Add support for If Op#409
qti-niscmami wants to merge 2 commits into
mainfrom
dev/qti-niscmami/add-support-for-if-op

Conversation

@qti-niscmami
Copy link
Copy Markdown
Collaborator

[QNN EP] Add support for If Op

Description

Adds ONNX If operator support in the QNN Execution Provider through inline-both-branches translation that lowers If into a single ElementWiseSelect.

Motivation & Context

If is a control-flow op that selects between two subgraphs based on a runtime boolean condition. It appears in models with conditional execution paths (input-dependent processing branches, optional post-processing, fallback paths). Previously unsupported in QNN EP.

https://onnx.ai/onnx/operators/onnx__If.html

Opset Type Constraint V (output types)
1, 11, 13, 16, 19, 21, 23, 24, 25 tensor (sequence/optional rejected)

Translation

Both branches are inlined into the parent QNN graph and a single ElementWiseSelect picks between their outputs at runtime:

ONNX (control flow — one branch runs):

                 ┌─── If node ────────────────┐
                 │                            │
  outer ────────►│  then_branch: ... ► then_o │
                 │                            │ ──► if_out
  outer ────────►│  else_branch: ... ► else_o │
                 │                            │
                 └────────────────▲───────────┘
                                  │
  cond ───────────────────────────┘

QNN (dataflow — both branches inlined, select picks):

                 ┌──► [then_branch ops inlined] ──► then_out ───┐
                 │                                              │
  outer ─────────┤                                              │
                 │                                              ├──► ElementWiseSelect ──► if_out
                 │                                              │             ▲
                 └──► [else_branch ops inlined] ──► else_out ───┘             │
                                                                              │
  cond ───────────────────────────────────────────────────────────────────────┘

For each branch, op-builders are dispatched recursively on the branch's nodes (OrtNodeUnit per node). Both branches always execute on-device — this is the trade-off vs. native QNN_OP_IF's real conditional dispatch.

Restrictions enforced in IsOpSupported

  • Condition is rank-0 or rank-1 with exactly 1 element
  • Each branch produces exactly 1 output
  • Branch outputs have identical shape and dtype
  • Branch outputs are plain tensors (sequence / optional rejected)
  • Every op inside both branches has a registered QNN op-builder
  • Branch terminus name does not collide with the If output name

Files Changed

File Change
core/providers/qnn/builder/opbuilder/if_op_builder.cc New IfOpBuilder inline-both-branches translation
core/providers/qnn/builder/op_builder_factory.h Added CreateIfOpBuilder declaration
core/providers/qnn/builder/op_builder_factory.cc Registered CreateIfOpBuilder("If", *this)
core/providers/qnn/builder/qnn_model_wrapper.h Added Push/PopBranchGraphScope and scope-aware FindInitializer so branch initializers resolve as QNN_TENSOR_TYPE_STATIC
test/providers/qnn/if_test.cc New test file with CPU, HTP, GPU sections

Implementation Notes

  • Branch graph scope stack. ORT folds Constant ops inside subgraphs into branch-graph initializers. FindInitializer now walks a stack of branch graphs (pushed/popped during branch translation) before falling back to the parent graph. Without this, IsConstantInput returns false for folded constants and the resulting tensor is classified as NATIVE, expecting a producer node that no longer exists (QNN error 6007: Input tensor doesn't have any producers).

  • Implicit inputs. Outer-scope tensors consumed inside branch subgraphs are not surfaced as If node inputs by ORT's partitioner. ProcessInputs calls Node_GetImplicitInputs and registers each as a QNN tensor before branch translation, so branch op-builders can resolve them.

  • Pure-constant branches. When ORT folds an entire branch away (0 compute nodes, 1 initializer), EnsureBranchOutputRegistered explicitly registers the branch output from its initializer table after TranslateBranch. Covers the Then=Constant, Else=Constant, and Both=Constant test cases.

  • Tensor name collision. ONNX permits a branch terminus to share its name with the parent If output. QNN cannot represent two tensors with the same name. The check rejects this case in IsOpSupported; the If falls to CPU EP. Renaming is deferred.

  • Single registration covers all opsets. The op-builder claims "If" by op name; ORT routes opset variants 1, 11, 13, 16, 19, 21, 23, 24, 25 through it. The structural spec is unchanged across versions; recent opsets only add new dtypes which are filtered by the per-op-builder dtype allowlist.

Test coverage

Backend Tests
CPU BasicBranches, ShapeMismatch_DeclinesFusion, NameCollision_DeclinesFusion, ThenConstant_ElseDynamic, ThenDynamic_ElseConstant, BothBranchesConstant
HTP (FP32-as-FP16) BasicBranches, ThenConstant_ElseDynamic, ThenDynamic_ElseConstant, BothBranchesConstant
GPU Same four positive cases as HTP, gated by _M_ARM64

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant