docs: add CPAN compatibility random tester and report#492
Merged
Conversation
Add dev/tools/cpan_random_tester.pl which: - Randomly selects modules from the full CPAN index (~14K distributions) - Tests each with jcpan -t, parsing results for all modules in output (target + dependencies) - Records PASS results with git commit hash for regression bisecting - Upgrades FAIL->PASS when re-tested modules now pass - Generates a Markdown report at dev/reports/cpan-compatibility.md - Saves results incrementally to .dat files (crash-safe, never discarded) Also add: - dev/prompts/cpan-compatibility-testing.md (reusable workflow instructions) - dev/reports/ with initial test data (3 modules tested) Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
- Add WNOHANG to POSIX imports - Prefer external timeout/gtimeout (from coreutils) over manual fork+signal handling — borrowed from perl_test_runner.pl - Cache timeout command lookup via _find_timeout_cmd() - Fix FD leak: close pipe on all paths including timeout - Increase reap retries from 5 to 10 before SIGKILL - Keep fork+setpgrp+kill(-pid) as fallback for systems without coreutils Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
- Add $| = 1 for autoflush so progress is visible when piped to file - Regenerate .md report after each target module (not just at end) - 171 modules tested: 37 pass (21.6%), 134 fail - Notable passes: Algorithm::Diff, CGI::Application, DateTime::Format::Strptime, HTML::Form, Lingua::EN::Inflect, Parse::RecDescent, Test::Output, Test::Most Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
dev/tools/cpan_random_tester.pl— a script that randomly selects CPAN modules and tests them withjcpan -t, building a compatibility report over timedev/prompts/cpan-compatibility-testing.md— reusable workflow instructionsdev/reports/with initial test data and Markdown reportKey features of the tester
jcpanoutput (target + deps), so one target yields multiple data pointsgit bisectwhen regressions are detected.datfiles are saved after each target moduleUsage
Test plan
perl -c)Generated with Devin