Merged
Conversation
Builders and the queue-runner must agree on the Nix store directory (typically `/nix/store`) or else store paths exchanged over the protocol wil be unactionable (store dirs are included in store path hash calcualtion). The version check handshake already runs before any builds are dispatched, so this is the natural place to catch the mismatch early with a clear error. Add a `store_dir` field to `VersionCheckRequest` in the proto, send it from the builder, and verify it on the queue-runner side — returning `failed_precondition` on mismatch. Also tighten up `nix_utils::get_store_dir()` to return `StoreDir` directly instead of `String`, to be more informative.
Validate store dir in gRPC version check
…dOne` It's generally better to use fewer languages. The bash script was the only non-Perl script in the test suite and was only invoked from `CliRunners::runBuilds`, meaning Perl was shelling out to bash just in one spot, for one purpose. A concrete benefit of the rewrite is we can reuse `getHydraConfig` instead of an ad-hoc `sed` script.
Replace `queue-runner-build-one.sh` with Perl module `QueueRunnerBuil…
…ydra_setup` Ever since `HydraTestContext` was introduced (eca09bc, 2021) to fix teardown errors, its `db()` had created the root user on first access — making the older `hydra_setup($db)` idiom redundant. But not every test was converted from the `hydra_setup` way of doing things, and also subsequent test files continued copying the older pattern of `Hydra::Model::DB->new` + `hydra_setup($db)` from neighboring tests. Also `createBaseJobset` internally constructed its own `Hydra::Model::DB->new` handle. Now, Migrate all tests to the `$ctx->db()` / `$ctx{context}->db()` pattern and pass the `$db` handle explicitly to `createBaseJobset` and `createJobsetWithOneInput` instead of letting them create their own.
…::Model::DB` In general, it is better not to use the app to test the app. Relying on just the database framework but not also the web framework is a strict improvement. `Hydra::Model::DB` is a `Catalyst::Model::DBIC::Schema` subclass — a Catalyst adapter around DBIC. Tests never needed Catalyst; they just needed a database handle. The original tests (890a786, 2011) connected directly via `openHydraDB`. The Catalyst dependency was introduced in fa62c8b (2012) when that was replaced with `Hydra::Model::DB->new`, and it stuck. Now that all test files obtain their `$db` from `HydraTestContext::db()`, we can switch that single callsite from `Hydra::Model::DB->new()` to `Hydra::Schema->connect(...)`. The `DESTROY` method is adjusted accordingly: `->storage->disconnect()` instead of `->schema->storage->disconnect()`, since we now hold the schema object directly rather than a Catalyst model wrapper.
No catalyst for db
This should always be locally (dynamically, cause Perl) scoped to just the command we want to run, so it doesn't also affect the rest of the program.
Clean up env var setting in a few places
Each bzr test now sets `BZR_HOME` to its own temp subdir via
`local $ENV{BZR_HOME}`, rather than relying on the meson-level
testsuite environment. This isolates bzr's config directory per test.
Set `BZR_HOME` per-test instead of testsuite-wide
Stop assigning to `$ENV` at `HydraTestContext` construction time. Instead,
collect env vars for central services (evaluator, `hydra-init`,
`hydra-notify`, Catalyst) in a `central_env` hash and apply them via
scoped `%ENV` overrides at each subprocess spawn point.
This is preparation for giving the builder its own Nix store, separate
from the central store used by the evaluator and web app. Even if we
weren't doing that, it is better to keep using environment variable
ambient authority / spooky action at a distance to a minimum.
- All structured config (paths, DB URLs, store URIs) lives in
`$ctx->{central}`. `central_env` is derived from `central` and
treated as an opaque blob — only applied wholesale to `%ENV`,
never cherry-picked for individual values.
- `evalSucceeds`, `evalFails`, `sendNotifications` (in `CliRunners.pm`),
`runBuild`, and `runBuilds` (in `QueueRunnerBuildOne.pm`) now take
`$ctx` as their first argument. This allows them to access
`$ctx->{central_env}` and apply the env vars locally.
- `QueueRunnerBuildOne.pm` reads config from `$ctx->{central}`
and uses scoped `local %ENV` blocks for queue runner and
builder subprocess startup
- New `setup_catalyst_test()` helper wraps `Catalyst::Test` exports so
`central_env` is applied around each in-process Catalyst request
Per the comments how this is implemented is rather gross, but the
result (less env var pollution) is I believe worth it.
- `HydraTestContext` gains `run_cmd`/`capture_cmd` methods that apply
`central_env` before running commands
- Remove `HYDRA_DATA`, `HYDRA_DBI` from meson-level
test env (each test has its own)
Avoid setting `$ENV` globally in tests as much as possible.
It was broken since moving hydra to subprojects. Doesn't entirely fix static files in hydra, but basic functionality works for testing.
Fix foreman development
Pump the queue-runner `IPC::Run` harness alongside the builder and flush accumulated stderr on every poll iteration, so when a test hangs we actually see both processes' logs in real time instead of only getting them after the harness is killed.
hydra-tests: stream queue-runner/builder logs while polling
Add new CA→CA→CA test
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated PR to keep
nix-nextin sync withmaster.