Readme validator#49
Merged
Merged
Conversation
… terminal output from console with the one in readme file
add alternative commands
…onary.yaml in terminaloutput of command - relative path not present in console log
…us message was incorrect
…on, fenced windows command with
…ments-request.yaml) + WIP counts
…a (test_find_available_products_book_200.json) + WIP counts
Add a run-level preflight that checks Docker, Docker Compose, Docker daemon access, Specmatic license presence, Specmatic license validity, and remote labs-contracts reachability before executing labs. Use the real specmatic/enterprise show-license command with the repo-root license mount, switch the mount to a repo-relative path, and harden license validation so expired licenses and fallback-to-default-trial cases fail even when show-license exits zero. Refine preflight output so dependent checks are reported as SKIP instead of FAIL when an upstream dependency is missing, and rename the license checks to separate file existence from license validation. Improve the multi-lab summary to include per-lab status, duration, validated command counts, and skipped command counts, and give dry-run its own DRY RUN summary so it does not imply execution success. Add Docker image warmup with docker compose pull --ignore-buildable on a per-lab just-in-time basis before each lab runs, using a longer timeout for cold image downloads, and treat warmup failures as lab failures without aborting later labs. Expand the unit test coverage to cover preflight discovery, Docker and license failure cases, expired-license fallback behavior, dry-run summary behavior, and per-lab Docker warmup flow.
Refine the README validator UX so setup progress is visible immediately and the final summary is easier to scan. Print the preflight header before checks begin and emit each preflight result as soon as it completes, including PASS results. Do the same for per-lab Docker warmup so docker compose pull progress is no longer silent before lab execution starts. Keep the existing aggregate print helpers as a fallback for mocked and non-streaming test paths, but use callback-based progressive output in real runs. Change the multi-lab summary from repeated inline records to a fixed-width table for both normal runs and dry-run mode, while preserving the same underlying metrics. Retain the per-lab just-in-time docker compose pull warmup behavior and update the unit tests to cover progressive output compatibility, table-format summaries, and warmup flow.
Extend the README validator with CI-oriented modes for shared preflight, per-lab JSON result output, and consolidated report generation. Only run the shared preflight automatically when validating the full lab suite, while single-lab invocations skip preflight by default so matrix jobs do not repeat common checks. Add --preflight-only, --result-json, and --report-from to support a GitHub Actions workflow that runs common checks once, fans out one lab per matrix runner, uploads machine-readable results, and renders a final consolidated summary. Add a repository workflow at .github/workflows/validate-readmes.yaml that triggers on pull requests, pushes to main, and manual dispatch, runs shared preflight first, validates each lab in parallel with fail-fast disabled, and aggregates all per-lab results into a final report step. Update the Python test suite to cover single-lab preflight skipping, preflight-only mode, result JSON generation, and report aggregation from a directory.
…ults on cleanup failure
…artup from CI path
…r as host user to avoid Git dubious ownership on mounted repo
… README command blocks by introducing a single _readme_command_specs() helper and using it in preflight and Docker warmup detection, collapsed the repeated append/callback logic in both warm_docker_images() and run_preflight() with local _emit() helpers, and simplified run_single_readme() by printing the command mapping once and consolidating the skip-count handling.
…ec file copies we can just use sed to do the necessary changes instead of copies whole files
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.