Skip to content

fix(local-dev): rebuild a source edited in the stamp's timestamp tick - #7079

Merged
Yicong-Huang merged 1 commit into
apache:mainfrom
Yicong-Huang:fix/local-dev-dirty-mtime-tick
Jul 30, 2026
Merged

fix(local-dev): rebuild a source edited in the stamp's timestamp tick#7079
Yicong-Huang merged 1 commit into
apache:mainfrom
Yicong-Huang:fix/local-dev-dirty-mtime-tick

Conversation

@Yicong-Huang

@Yicong-Huang Yicong-Huang commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

The dirty-source fast filter treated "source mtime equal to the build stamp's" as clean, so an edit landing in the same filesystem timestamp tick as the stamp write was never rebuilt — auto reported everything up-to-date and bounced nothing.

The window is one tick wide, which sounds negligible but isn't: the stamp is written at the end of a build, and the natural next action is editing the file you were just building. How wide the tick is depends on the filesystem and kernel clock granularity, not on how fast anyone types.

Both implementations had it, and the shell one is the consequential half — it gates the rebuild, while tui.py only colours the SRC column:

Where Was Now
main.sh svc_src_changed find … -newer "$stamp" (strictly newer) compares against a throwaway marker one second behind the stamp
tui.py _newest_mtime_after st_mtime > stamp_mtime >=
Before:  build -> edit within the same tick -> auto: "everything up-to-date"
After:   build -> edit within the same tick -> hash compared -> rebuilt

The two fixes differ because the two languages can express different things: find has no portable "not older than", so the shell borrows one second of slack via a marker; Python can say exactly what it means, so it uses >=. The marker is a throwaway — the stamp itself keeps its real mtime, so the mtime refresh at the end of the slow path converges exactly as before.

Both directions stay conservative rather than wrong. Widening the filter only enlarges the candidate set; the content hash underneath — which was always correct — still makes the decision. And it self-heals: the first tick after a build takes the hash path once, finds the content unchanged, and bumps the stamp past the sources, so later ticks are cheap again. A source that is genuinely older than the stamp is not dragged in by the slack (asserted below).

Any related issues, documentation, discussions?

Closes #7075

How was this PR tested?

The bug's own symptom was an existing test that failed only on some filesystems. This PR adds a deterministic version of it — os.utime forces the colliding mtime instead of racing for it — so it holds on every platform:

$ python -m pytest bin/local-dev/tests/ -q
44 passed

That count matters: before this change the suite was 1 failed, 42 passed on this machine, the failure being test_is_dirty_after_seed_then_edit, which had been hitting the same bug by accident wherever the filesystem granularity was coarser than its two consecutive writes. It now passes for the right reason rather than by platform luck.

The shell side is covered by the mechanism it actually uses, with the colliding mtime forced by touch -r:

$ bash bin/local-dev/tests/test_local_dev_sh.sh
...
  ✓ stamp backdate: bare `-newer $stamp` misses an equal mtime (the bug)
  ✓ stamp backdate: backdated marker sees the equal-mtime edit
  ✓ stamp backdate: a clearly older source stays clean
  ✓ stamp backdate: missing file is a quiet no-op
  ✓ svc_src_changed compares against the backdated marker

55 passed, 0 failed

The first of those characterises the bug itself, so it will start failing if a future find learns to include equal mtimes — at which point the marker can go.

Reproduction outside the suite, for the record:

$ D=$(mktemp -d); mkdir -p $D/src; : > $D/stamp; : > $D/src/A.scala
$ touch -r $D/stamp $D/src/A.scala          # identical mtimes
$ find $D/src -name '*.scala' -newer $D/stamp -print
                                            # empty — A.scala is invisible

Not covered: an end-to-end auto that rebuilds off a same-tick edit. Forcing that on the real stack means winning the same race the test now sidesteps, so the suite's deterministic version is the check that carries the weight here.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 5)

The dirty-source fast filter treated "source mtime equal to the build stamp's"
as clean, so an edit landing in the same filesystem timestamp tick as the stamp
write was never rebuilt: `auto` reported `everything up-to-date` and bounced
nothing. The window is one tick wide, but the stamp is written at the end of a
build and the natural next action is editing the file you were just building,
so it is hit in normal use — how wide the tick is depends on the filesystem and
kernel clock granularity, not on how fast anyone types.

Both implementations had it. `svc_src_changed` used `find -newer "$stamp"`,
where `-newer` is strictly newer; `tui.py`'s `_newest_mtime_after` used a
strict `>`.

The shell side is the consequential half — it is what gates the rebuild, while
tui.py only colours the SRC column. It now compares against a throwaway marker
one second behind the stamp, because `find` has no portable "not older than".
The stamp itself keeps its real mtime, so the mtime refresh at the end of the
slow path still converges. tui.py can say what it means, so it uses `>=`.

Both directions stay conservative rather than wrong: widening the filter only
enlarges the candidate set, and the content hash underneath — which was always
correct — makes the actual decision. It also self-heals, because the first tick
after a build takes the hash path once, finds the content unchanged, and bumps
the stamp past the sources.

This also fixes `test_is_dirty_after_seed_then_edit`, which had been failing on
any filesystem whose granularity is coarser than its two consecutive writes;
the new test forces the colliding mtime with `os.utime` so it no longer depends
on the platform.

Closes apache#7075
@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Backport auto-label report

This fix: PR was checked against each actively-supported release branch. release/* labels drive the post-merge backport, so add or remove one to change where this fix lands.

Release branch Analysis
⚠️ release/v1.2 Not labeled automatically — none of the files this PR modifies exist on this branch (bin/local-dev/main.sh, bin/local-dev/tests/test_local_dev_sh.sh, bin/local-dev/tests/test_local_dev_tui.py, bin/local-dev/tui.py). The fix may target code that isn't on this release, or the files were moved/renamed after the branch was cut. Please check and add release/v1.2 by hand if this fix should be backported here.

Auto-label run.

@github-actions

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @aglinxinyuan
    You can notify them by mentioning @aglinxinyuan in a comment.

@Yicong-Huang
Yicong-Huang requested a review from mengw15 July 29, 2026 23:18
@Yicong-Huang
Yicong-Huang added this pull request to the merge queue Jul 30, 2026
Merged via the queue into apache:main with commit ebc8c64 Jul 30, 2026
28 checks passed
@Yicong-Huang
Yicong-Huang deleted the fix/local-dev-dirty-mtime-tick branch July 30, 2026 00:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

local-dev misses a source edit that lands in the same timestamp tick as the build stamp

2 participants