The log output indicate of cpu&npu percentage looks strange

[heweiwill]

Hi ,
I run this command: examples/arm/run.sh --model_name=mv2 --target=ethos-u55-128
And it runs successfully, but the log output indicate CPU ratio looks strange, suppose the mv2 model could 100% running on npu.
Any idea on this?

I [executorch:arm_perf_monitor.cpp:133 StopMeasurements()] NPU Inferences : 1
I [executorch:arm_perf_monitor.cpp:134 StopMeasurements()] Profiler report, CPU cycles per operator:
I [executorch:arm_perf_monitor.cpp:138 StopMeasurements()] ethos-u : cycle_cnt : 1859660 cycles
I [executorch:arm_perf_monitor.cpp:145 StopMeasurements()] Operator(s) total: 1859660 CPU cycles
I [executorch:arm_perf_monitor.cpp:151 StopMeasurements()] Inference runtime: 2219663 CPU cycles total
I [executorch:arm_perf_monitor.cpp:153 StopMeasurements()] NOTE: CPU cycle values and ratio calculations require FPGA and identical CPU/NPU frequency
I [executorch:arm_perf_monitor.cpp:162 StopMeasurements()] Inference CPU ratio: 99.98 %
I [executorch:arm_perf_monitor.cpp:166 StopMeasurements()] Inference NPU ratio: 0.02 %
I [executorch:arm_perf_monitor.cpp:175 StopMeasurements()] cpu_wait_for_npu_cntr : 336 CPU cycles
I [executorch:arm_perf_monitor.cpp:180 StopMeasurements()] Ethos-U PMU report:
I [executorch:arm_perf_monitor.cpp:181 StopMeasurements()] ethosu_pmu_cycle_cntr : 1085680
I [executorch:arm_perf_monitor.cpp:184 StopMeasurements()] ethosu_pmu_cntr0 : 499818
I [executorch:arm_perf_monitor.cpp:184 StopMeasurements()] ethosu_pmu_cntr1 : 378668
I [executorch:arm_perf_monitor.cpp:184 StopMeasurements()] ethosu_pmu_cntr2 : 299694
I [executorch:arm_perf_monitor.cpp:184 StopMeasurements()] ethosu_pmu_cntr3 : 47887
I [executorch:arm_perf_monitor.cpp:184 StopMeasurements()] ethosu_pmu_cntr4 : 388

[GregoryComer]

@digantdesai Do you have any suggestions, or is there anyone from ARM that can help answer?

[digantdesai]

Math looks right here -

executorch/examples/arm/executor_runner/arm_perf_monitor.cpp

Lines 176 to 184 in aefdc8d

	ET_LOG(
	Info,
	"Inference CPU ratio: %.2f %%",
	100.0 * (cycle_count - ethosu_ArmWhenNPURunCycleCount) / cycle_count);
	ET_LOG(
	Info,
	"Inference NPU ratio: %.2f %%",
	100.0 * ethosu_ArmWhenNPURunCycleCount / cycle_count);
	}

[gggekov]

Hello @heweiwill ,
Sorry for the late reply - we are used to monitor the issues on github, not so much the discussions.

Note that on FVP, the CPU cycle count is not accurate. When you run on FVP, you should only trust the numbers coming from the Ethos-U. We should probably also disable the log for printing the CPU ratio because it is misleading unless you run on FPGA. In the meantime, can I please ask you to only look at the NPU cycle counters.

Also, I suspect you have modified the Timing Adapters because your NPU cycle count is 1M NPU cycles when actually mv2 runs in a little over 7M NPU cycles. Here is the performance I get on u55-128 from the FVP:

I [executorch:arm_perf_monitor.cpp:196 StopMeasurements()] ethosu_pmu_cycle_cntr : 7159754 (3579877.00 per inference)
I [executorch:arm_perf_monitor.cpp:203 StopMeasurements()] ethosu_pmu_cntr0 : 2626426 (1313213.00 per inference)
I [executorch:arm_perf_monitor.cpp:203 StopMeasurements()] ethosu_pmu_cntr1 : 420944 (210472.00 per inference)
I [executorch:arm_perf_monitor.cpp:203 StopMeasurements()] ethosu_pmu_cntr2 : 1290155 (645077.50 per inference)
I [executorch:arm_perf_monitor.cpp:203 StopMeasurements()] ethosu_pmu_cntr3 : 300 (150.00 per inference)
I [executorch:arm_perf_monitor.cpp:211 StopMeasurements()] Ethos-U PMU Events:[ETHOSU_PMU_AXI0_RD_DATA_BEAT_RECEIVED, ETHOSU_PMU_AXI1_RD_DATA_BEAT_RECEIVED, ETHOSU_PMU_AXI0_WR_DATA_BEAT_WRITTEN, ETHOSU_PMU_NPU_IDLE]

These PMU counters are the data you can trust when running on FVP.