[QNN EP] Add support for If Op#409
Open
qti-niscmami wants to merge 2 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[QNN EP] Add support for If Op
Description
Adds ONNX
Ifoperator support in the QNN Execution Provider through inline-both-branches translation that lowersIfinto a singleElementWiseSelect.Motivation & Context
Ifis a control-flow op that selects between two subgraphs based on a runtime boolean condition. It appears in models with conditional execution paths (input-dependent processing branches, optional post-processing, fallback paths). Previously unsupported in QNN EP.https://onnx.ai/onnx/operators/onnx__If.html
Translation
Both branches are inlined into the parent QNN graph and a single
ElementWiseSelectpicks between their outputs at runtime:ONNX (control flow — one branch runs):
QNN (dataflow — both branches inlined, select picks):
For each branch, op-builders are dispatched recursively on the branch's nodes (
OrtNodeUnitper node). Both branches always execute on-device — this is the trade-off vs. nativeQNN_OP_IF's real conditional dispatch.Restrictions enforced in
IsOpSupportedIfoutput nameFiles Changed
core/providers/qnn/builder/opbuilder/if_op_builder.ccIfOpBuilderinline-both-branches translationcore/providers/qnn/builder/op_builder_factory.hCreateIfOpBuilderdeclarationcore/providers/qnn/builder/op_builder_factory.ccCreateIfOpBuilder("If", *this)core/providers/qnn/builder/qnn_model_wrapper.hPush/PopBranchGraphScopeand scope-awareFindInitializerso branch initializers resolve asQNN_TENSOR_TYPE_STATICtest/providers/qnn/if_test.ccImplementation Notes
Branch graph scope stack. ORT folds
Constantops inside subgraphs into branch-graph initializers.FindInitializernow walks a stack of branch graphs (pushed/popped during branch translation) before falling back to the parent graph. Without this,IsConstantInputreturns false for folded constants and the resulting tensor is classified asNATIVE, expecting a producer node that no longer exists (QNN error 6007:Input tensor doesn't have any producers).Implicit inputs. Outer-scope tensors consumed inside branch subgraphs are not surfaced as
Ifnode inputs by ORT's partitioner.ProcessInputscallsNode_GetImplicitInputsand registers each as a QNN tensor before branch translation, so branch op-builders can resolve them.Pure-constant branches. When ORT folds an entire branch away (0 compute nodes, 1 initializer),
EnsureBranchOutputRegisteredexplicitly registers the branch output from its initializer table afterTranslateBranch. Covers theThen=Constant,Else=Constant, andBoth=Constanttest cases.Tensor name collision. ONNX permits a branch terminus to share its name with the parent
Ifoutput. QNN cannot represent two tensors with the same name. The check rejects this case inIsOpSupported; theIffalls to CPU EP. Renaming is deferred.Single registration covers all opsets. The op-builder claims
"If"by op name; ORT routes opset variants 1, 11, 13, 16, 19, 21, 23, 24, 25 through it. The structural spec is unchanged across versions; recent opsets only add new dtypes which are filtered by the per-op-builder dtype allowlist.Test coverage
BasicBranches,ShapeMismatch_DeclinesFusion,NameCollision_DeclinesFusion,ThenConstant_ElseDynamic,ThenDynamic_ElseConstant,BothBranchesConstantBasicBranches,ThenConstant_ElseDynamic,ThenDynamic_ElseConstant,BothBranchesConstant_M_ARM64