Always use spawn for data loader workers

[jder]

Removes fork/spawn selection based on data source location and always use spawn when data loader workers are enabled.

Why?

• We originally used fork as a local-data performance optimization. PR Fix performance regression by moving towards a more buildy world with DataContainer #295 introduced supports_fork so local Zarr paths could use fork while S3-backed paths used spawn, since s3fs was known not to be fork-safe.
• That storage-based distinction does not cover the failure mode we are seeing now: the parent process can already be multithreaded before DataLoader workers start, for example under pytest-xdist, xarray/zarr, CUDA-adjacent setup, or internal executor threads.
• Python 3.12 now warns explicitly that calling fork() from a multithreaded process can deadlock. A failing GPU CI job showed that warning repeatedly before timing out in a Zarr read. I also see similar behavior locally on my mac for tests sometimes. It is hard to prove this is the problem, of course.
• Real training runs should amortize the extra startup cost through persistent DataLoader workers, so defaulting to spawn is the safer policy.

Testing

• uv run pytest -m "not manual and not cuda" tests/test_trainer.py tests/test_config.py -q

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5e7da987db

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Avoid spawning transient inference workers

When inference_epochs is enabled with num_workers > 0, this context is also passed to the inline inference DataLoader in init_inference_stores, but that loader does not set persistent_workers and is re-iterated in inference_one_epoch. Unlike the train/val loaders, each inference pass now has to spawn workers and serialize the xarray-backed InferenceDatasets, so the startup cost is not amortized and small/default CPU configs with num_workers: 4 can become extremely slow or appear frozen; keep inline inference single-process or make those workers persistent separately.

Useful? React with 👍 / 👎.

[jder]

This behavior isn't actually changed, I don't think? And we're not using inference at training time anyway. We can fix as part of #772 or after that lands.

[alxmrs]

Brilliant, always love to see a lot of deletions / reduction in complexity. Thank you.