Skip to content

[FEATURE] CI: smoke-test ./bootstrap.sh end-to-end (closes the gap that let #2 ship) #4

@Defilan

Description

@Defilan

Feature Description

CI does not exercise ./bootstrap.sh end-to-end. The two macOS jobs
(macos-check and macos-homebrew) invoke ansible-playbook
directly with hard-coded arguments, bypassing the bash entrypoint
that users actually run. Add a smoke-test step that runs
./bootstrap.sh against a stubbed ansible-playbook so future bugs
in the wrapper (argument parsing, tag composition, variable
expansion) are caught at PR time.

Problem Statement

Issue #2 was a bug in bootstrap.sh's EXTRA_ARGS expansion: when
the user invoked ./bootstrap.sh with no ---passed args, the
script appended an empty-string positional to the ansible-playbook
call, which failed with the unhelpful unrecognized arguments:
(with empty space after the colon).

CI passed despite the bug because:

  • yamllint / ansible-lint scan YAML and roles, not bash scripts
  • shellcheck scans bootstrap.sh but "${arr[@]:-}" is
    syntactically valid bash; the semantic problem (the expansion
    produces unwanted empty positionals) is invisible to shellcheck
  • macos-check and macos-homebrew invoke ansible-playbook
    directly, never going through the bash wrapper

The wrapper is the documented user entrypoint, and it has no CI
coverage. Every future fix-then-regress cycle in bootstrap.sh
will follow the same shape unless we close the gap.

Proposed Solution

Add a smoke-test job (or step inside an existing macOS job) that
runs ./bootstrap.sh against a stubbed ansible-playbook which
asserts no empty positional args are passed:

- name: ./bootstrap.sh entrypoint smoke (empty EXTRA_ARGS path)
  run: |
    mkdir -p /tmp/stubs
    cat > /tmp/stubs/ansible-playbook <<'STUB'
    #!/usr/bin/env bash
    # Assert no empty positional args (regression guard for #2).
    for a in "$@"; do
      if [[ -z "$a" ]]; then
        echo "REGRESSION: empty positional arg passed to ansible-playbook" >&2
        printf '  arg: [%s]\n' "$@" >&2
        exit 1
      fi
    done
    # Echo the full argv for the log, so a maintainer can see exactly
    # what bootstrap.sh would have invoked.
    printf 'argv:\n'
    printf '  [%s]\n' "$@"
    STUB
    chmod +x /tmp/stubs/ansible-playbook
    export PATH="/tmp/stubs:$PATH"
    ./bootstrap.sh

- name: ./bootstrap.sh entrypoint smoke (non-empty EXTRA_ARGS path)
  run: |
    # Same stub on PATH from previous step; verify the passthrough
    # path also stays clean.
    ./bootstrap.sh -- --check --diff

The stub does nothing real — it just scans its own argv for empty
strings and bails if it finds any. Total wall-clock cost is
sub-second per invocation. Both the empty-EXTRA_ARGS path (the
exact path that broke in #2) and the non-empty pass-through path
get covered.

Could optionally also add --with-foreman and --with-carnice
invocations against the same stub to lock in the tag composition.

Alternatives Considered

Additional Context

Priority

  • Medium - Nice to have

(Bumps to High once anyone except the maintainer is consuming the
bootstrap.)

Willingness to Contribute

  • Yes, I can submit a PR

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions