Skip to content

[BUG] The pass "eliminate_nop_monotone_argmax" causes output mismatch #281

@C1ri007

Description

@C1ri007

[BUG] The pass "eliminate_nop_monotone_argmax" causes output mismatch

Issue
Running the single pass eliminate_nop_monotone_argmax with onnxoptimizer 0.4.2 changes numerical outputs.

Environment

  • Ubuntu 20.04
  • Python 3.10
  • onnx==1.19.0
  • onnxruntime==1.23.2
  • onnxoptimizer==0.4.2(latest)

Repro steps (run from this folder)

eliminate_nop_monotone_argmax_repro.tar.gz

  1. Create a Python environment (Python 3.10) and install dependencies:
python3 -m venv .venv
source .venv/bin/activate
pip install -U pip
pip install -r requirements.txt
  1. Optimize a case with only eliminate_nop_monotone_argmax (writes model.opt.onnx next to model.onnx):
  • python optimize_model.py --case ./case_03140_seed45116679
  1. Differential test original and optimized models with the same oracle inputs (ORT graph optimization disabled):
  • python diff_test.py --case ./case_03140_seed45116679

Expected
eliminate_nop_monotone_argmax should be semantics-preserving. Applying this pass alone should not change any output values.

Differential Test Output Details

  1. case_03140_seed45116679 — mismatch; output[14] max_abs=4.022e+04 max_rel=4.022e+12, output[15] max_abs=5.190e+02 max_rel=5.120e+10; overall mismatch.
  2. case_00324_seed22816775 — mismatch; output[3] max_abs=5.022e+18 max_rel=1.000e+00; overall mismatch.
  3. case_09334_seed94166965 - mismatch; 1 / 3 of its output nodes are mismatch.
Case: case_03140_seed45116679 (ORT graph optimizations disabled)
  output[0]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1, 21)
  output[1]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(2, 1, 24)
  output[2]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(42336,)
  output[3]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1, 21, 24)
  output[4]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(21, 1)
  output[5]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(21, 1)
  output[6]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(21, 1)
  output[7]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(2, 24)
  output[8]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1, 24)
  output[9]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1, 24)
  output[10]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1, 24)
  output[11]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1, 24)
  output[12]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(21, 24)
  output[13]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(24, 21, 1)
  output[14]: max_abs=4.022e+04, max_rel=4.022e+12, shape=(1, 21, 24)
  output[15]: max_abs=5.190e+02, max_rel=5.120e+10, shape=(32, 21, 24)
Overall: max_abs=4.022e+04, max_rel=4.022e+12


Case: case_00324_seed22816775 (ORT graph optimizations disabled)
  output[0]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1, 1, 1, 1)
  output[1]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1, 2, 1, 1)
  output[2]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1, 1, 1, 51)
  output[3]: max_abs=5.022e+18, max_rel=1.000e+00, shape=(51, 1, 1, 1)
  output[4]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1,)
  output[5]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1, 2, 1, 1, 51)
  output[6]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(255,)
  output[7]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1, 2, 2, 51)
  output[8]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1, 2, 2, 1)
  output[9]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1, 2, 1, 1, 51)
  output[10]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1, 2, 1, 1)
  output[11]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1, 2, 51)
  output[12]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1, 30, 29, 51)
  output[13]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(51, 29, 30)
Overall: max_abs=5.022e+18, max_rel=1.000e+00


Case: case_09334_seed94166965 (ORT graph optimizations disabled)
  output[0]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1, 2, 1, 1, 1)
  output[1]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(5, 1, 1, 1, 1)
  output[2]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(2, 1, 62, 1, 1)
  output[3]: max_abs=6.211e-01, max_rel=2.543e-01, shape=(2, 1, 1, 1, 62)
  output[4]: max_abs=1.000e+00, max_rel=1.000e+00, shape=(2, 1, 1, 1, 62)
  output[5]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(58, 1, 1, 1)
  output[6]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(58, 1, 1, 1)
  output[7]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(58, 1, 1, 1)
  output[8]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1, 58, 1, 62)
  output[9]: max_abs=2.009e-01, max_rel=5.905e-01, shape=(2, 58, 1, 62)
  output[10]: max_abs=3.920e-01, max_rel=3.029e+02, shape=(58, 2, 62, 1)
  output[11]: max_abs=1.792e-01, max_rel=5.003e-01, shape=(2, 58, 1, 62)
  output[12]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(58, 1, 62)
  output[13]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(4, 62)
  output[14]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1, 58, 1, 62)
  output[15]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(21576,)
Overall: max_abs=1.000e+00, max_rel=3.029e+02

Attachments

  • requirements.txt
  • optimize_model.py
  • diff_test.py (runs original and optimized with same oracle inputs; ORT graph optimizations disabled)
  • run_optimized_model.py (executes optimized model with oracle inputs)
  • case_03140_seed45116679/,case_00324_seed22816775/,case_09334_seed94166965/ (original models and oracle inputs)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions