Record LLM model execution latency

[yenhao-huang]

Summary

Fixes #11771.

model_execution_start_ms and model_execution_end_ms were exposed in the LLM runner stats but stayed at zero because the runner never recorded the model execution boundary. This change shares the generation stats with TextDecoderRunner, measures calls to Module::execute(), resets the values for each generation, and emits them in the observer JSON.

Test plan

• Reproduced on current main with a Qwen3.5-0.8B PTE: both model execution timestamps were 0.
• test_runner --gtest_filter='StatsTest.*' (2 passed)
• test_runner --gtest_filter='RunnerTest.*' (12 passed)
• test_runner --gtest_brief=1 (116 passed, 2 skipped because optional PTE test fixtures were not supplied)
• Built llama_main and ran the same Qwen3.5-0.8B PTE twice. Both runs reported nonzero model execution timestamps ordered within the inference interval; for example, 1784555623119 <= 1784555624620.

cc @larryliu0820 @mergennachin @cccclai @helunwencser @jackzhxng @digantdesai

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21040

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 6f8928d with merge base f8b7f62 ():

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• pull / test-models-linux (ic4, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[meta-cla]

Hi @yenhao-huang!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: yenhao-huang / name: yenhao (3cb8bf2, b3ebdd5)

[yenhao-huang]

@JakeStevens, the issue still reproduces on current main, and I have opened this fix with unit and real-model validation. Could you please review it when you have a chance? Thanks!

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[meta-cla]

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

[JakeStevens]

Thanks for the PR!
Please make sure you run the linter as described in the contributing guide.

As written, we are capturing the very first model inference and then keep grabbing the last for "end".

This means we are covering not just model execution, but the entire generate window including sampling etc.

I think it would be better to drop the == 0 guard so on_begin overwrites each step → the pair becomes the latest forward window.

Then we can also add aggregate_model_execution_time_ms, mirroring aggregate_sampling_time_ms exactly

And document this better in stats.h, that the pair is the most recent forward window and aggregate_model_execution_time_ms is the running total

Would have to wire it up into the json and reset like you already did.

[yenhao-huang]

Thanks for the feedback! I’ve updated the PR so on_model_execution_begin() overwrites the timestamp on every forward call. The start/end pair now represents the most recent forward window.

I also added aggregate_model_execution_time_ms to JSON and reset, with matching Python bindings and documentation. Tests cover the latest forward window, aggregation, serialization, and reset behavior.

Local validation passed:

• lintrunner --skip MYPY
• test_runner --gtest_filter='StatsTest.*' (2 passed)

GitHub CI is still running. Thanks again!

[JakeStevens]

nit: all of these edited comments (line 35, 37, 49) have an extra slash, please remove

[JakeStevens]

Once we clean up the comments, I think we will be good to go!