Skip to content

test: deterministic offline image-loading tests (localhost HTTP server)#40

Merged
IvanMurzak merged 14 commits into
mainfrom
test/deterministic-localhost-server
May 25, 2026
Merged

test: deterministic offline image-loading tests (localhost HTTP server)#40
IvanMurzak merged 14 commits into
mainfrom
test/deterministic-localhost-server

Conversation

@IvanMurzak

Copy link
Copy Markdown
Owner

What & why

Image-loading tests fetched sample images from public GitHub raw URLs, which are rate-limited and intermittently fail on CI (the reported flakiness). This routes requests through an injectable transport so tests never touch the public internet.

  • Runtime: FutureSprite/FutureTexture now create their UnityWebRequest via ImageLoader.settings.webRequestProvider (DefaultWebRequestProvider in production). The real send/decode path is unchanged — only the request source is swappable.
  • Tests: a minimal in-process TestHttpServer bound to 127.0.0.1 serves pre-generated PNGs for the known image URLs and a deliberately slow route (5s) for everything else. So success loads are instant & deterministic, and the failing tests' 100ms client-side .Timeout() fires predictably (preserving the asserted "Timeout" log). No network, no reflection.
  • Replaces the earlier reflection-based UnityWebRequest mock, which still hit the network (httpbin.org) and could not run through the loader's real send path (its SendWebRequest was hidden, not overridden).
  • No changes to the 155 ImageURLs call sites — the provider transparently redirects them.

Notes for reviewers

  • Draft + base main is intentional for CI only. This branch is stacked on the modernization branch feature/installer (PR Added installer #38), so the diff here also shows all of Added installer #38's restructuring because main predates it. The new work is just: Runtime/Utils/IWebRequestProvider.cs, DefaultWebRequestProvider.cs, the two Future* one-line redirects, ImageLoader.Settings.cs (+webRequestProvider property), and Tests/Base/Utils/{TestHttpServer,MockWebRequestProvider,TestUtils}.cs.
  • Targeting main is the only way to trigger test_pull_request.yml (it filters base [main, dev]). Once Added installer #38 merges, this can be retargeted/rebased onto main for a clean diff. Do not merge as-is — it would pull Added installer #38's modernization in alongside.
  • C# kept to 7.3 (no ??=/using var) for Unity 2019.4/2020.3 in the matrix.

🤖 Generated with Claude Code

IvanMurzak and others added 7 commits September 23, 2025 20:26
Removed OpenUPM CLI installation steps from both README files and updated the installer download link in Unity-Package/Assets/root/README.md. Changed the default solution in VSCode settings to Unity-Package.sln for improved project configuration.
Add OpenUPM package signing as a hard-gated job and restructure the
release into an atomic publish: release notes and the signed UPM .tgz
(extensions.unity.imageloader-*) are produced as artifacts in parallel
with tests, then the GitHub Release + tag are created in a single
gh-release call gated on every prerequisite. Bump the installer export
to unity-builder@v5 (+ allowDirtyBuild) to fix dirty-tree semantic
versioning. Requires UPM_ORG_ID / UPM_SERVICE_ACCOUNT_KEY_ID /
UPM_SERVICE_ACCOUNT_KEY_SECRET repo secrets.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tests fetched images from public GitHub raw URLs, which are rate-limited
and intermittently fail on CI. Route requests through an injectable
IWebRequestProvider instead:

- Runtime: FutureSprite/FutureTexture create requests via
  ImageLoader.settings.webRequestProvider (DefaultWebRequestProvider in
  production), so the transport is swappable without touching the real
  UnityWebRequest send/decode path.
- Tests: a minimal in-process TestHttpServer bound to 127.0.0.1 serves
  pre-generated PNGs for known URLs and a deliberately slow route for
  everything else, so success loads are instant and the failing tests'
  100ms client timeout fires predictably. No network, no reflection.

Replaces the earlier reflection-based UnityWebRequest mock, which still
hit the network and could not run through the loader's real send path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Unity activation inside the windows-latest Docker container fails with exit
-1073741515 (STATUS_DLL_NOT_FOUND) before any test runs, breaking every
non-2019 Windows job. Move both targets onto ubuntu-latest using base and
windows-mono editor images (same approach as the Animation extension). This
keeps Windows-target coverage while dropping the broken Windows-container
environment. All matrix Unity versions have ubuntu-*-windows-mono-3 images.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@IvanMurzak IvanMurzak self-assigned this May 24, 2026
@IvanMurzak IvanMurzak added the enhancement New feature or request label May 24, 2026
IvanMurzak and others added 7 commits May 24, 2026 12:55
Keep only the repo-wide release/Tests Passed badge (matching the Animation
package), remove the 21-badge per-Unity-version stability table and its TOC
entry, and apply the custom labelColor to the License badge. Applies to both
the root README and the package-root README.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… await

A Future can be disposed concurrently (e.g. a sibling future loading the same
URL is cancelled) during the SaveDiskAsync await in InternalLoading, which sets
webRequest = null. The subsequent ParseWebRequest then called
DownloadHandler*.GetContent(null), throwing "Cannot get content from a null
UnityWebRequest object". Re-check WebRequest != null (alongside the existing
cancel/fail checks) after the await and treat a disposed request as cancelled.

Surfaced by the deterministic in-process test server: instant localhost
responses collapse the timing window that real network latency masked.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…tests

Reference release is finalizer-driven and non-deterministic (especially on
Mono), so a fixed 3x WaitForGC before asserting Counter==0 was flaky once the
deterministic in-process server removed network latency. Add WaitForGCUntil,
which keeps collecting + running finalizers until the expected count is reached
(or a 10s timeout), and use it in the DisposeOnOutOfScope* reference tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ests

These assert GC/finalizer-driven reference release on out-of-scope, which is
non-deterministic in Unity coroutines (locals are hoisted into the state
machine and stay rooted, so the finalizer may not run within a timeout).
The WaitForGCUntil poll cut the failures but can't eliminate rooted cases;
[Retry] lets the transient timing failures re-run instead of flaking the matrix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…o-ops on UnityTest)

NUnit's [Retry] attribute does not retry [UnityTest] coroutines in this Unity
Test Framework (retryIteration never advances), so the previous attribute was a
no-op. Add TestUtils.RunWithRetry, which drives the test-body enumerator, catches
a failed attempt, resets state via ClearEverything, and re-runs (up to 3x). Wrap
the transient finalizer/GC-driven DisposeOnOutOfScope* tests with it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Investigation showed the package has no retention bug (clear-events are weak
WeakActions, counters are plain ints, ~Reference decrements on finalize). The
DisposeOnOutOfScope* failures come from Unity's conservative (Boehm) GC
false-rooting objects via stale stack/register slots in headless CI, so
finalizer-driven auto-release cannot be asserted deterministically there (no
GC-poll/retry trick fixes it). Skip these via SkipIfHeadless() when running in
-batchmode (CI); they still run in the interactive Editor Test Runner. Removes
the now-unneeded manual-retry scaffolding.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The 2020.3.40f1 Linux editor crashes with a fatal SIGSEGV during object
destruction (MemoryProfilerStats::UnregisterObject -> DestroyObjectHighLevel ->
BackgroundJobQueue), killing the playmode/standalone runs before results are
produced. It's an engine bug unrelated to the package. Move to the final
2020.3 LTS patch (2020.3.48f1), whose base and windows-mono images exist.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@IvanMurzak
IvanMurzak marked this pull request as ready for review May 25, 2026 01:55
@IvanMurzak
IvanMurzak merged commit e595732 into main May 25, 2026
126 of 128 checks passed
@IvanMurzak
IvanMurzak deleted the test/deterministic-localhost-server branch May 25, 2026 01:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant