diff --git a/.github/WORKFLOWS.md b/.github/WORKFLOWS.md index 8d219e2..ae22067 100644 --- a/.github/WORKFLOWS.md +++ b/.github/WORKFLOWS.md @@ -4,40 +4,11 @@ This document describes the GitHub Actions workflows implemented in this reposit ## Overview -The repository includes a comprehensive suite of CI/CD workflows that ensure code quality, security, and automated releases. All workflows follow security best practices and are designed to be modular and maintainable. +The repository includes CI/CD workflows that ensure code quality and automated releases. All workflows follow security best practices and are designed to be modular and maintainable. ## Workflows -### 1. Test Action (`test-action.yml`) - -**Purpose**: Validates all features of the Python Linting GitHub Action. - -**Triggers**: - -- Pull requests to `main` (when action files change) -- Pushes to `main` (when action files change) -- Manual workflow dispatch - -**Jobs**: - -- **test-basic-linting**: Tests basic linting functionality -- **test-custom-options**: Tests custom linting options -- **test-requirements-file**: Tests with requirements file -- **test-badge-generation**: Tests SVG badge generation -- **test-readme-update**: Tests automatic README updates -- **test-update-badges-script**: Tests `update_badges.py` directly -- **test-python-versions**: Tests multiple Python versions (3.9-3.12) -- **test-summary**: Provides a summary of all tests - -**Key Features**: - -- Creates test Python files dynamically -- Validates badge generation -- Verifies README update functionality -- Tests across multiple Python versions -- Comprehensive validation of all action features - -### 2. Changelog Check (`changelog-check.yml`) +### 1. Changelog Check (`changelog-check.yml`) **Purpose**: Ensures CHANGELOG.md is updated for substantive changes. @@ -62,9 +33,9 @@ The repository includes a comprehensive suite of CI/CD workflows that ensure cod - Unreleased section must have entries - Follows Keep a Changelog format -### 3. Lint and Test (`lint-test.yml`) +### 2. Lint and Test (`lint-test.yml`) -**Purpose**: Runs comprehensive linting and testing on all code changes. +**Purpose**: Validates YAML syntax and formatting for workflow files and the action.yml using yamllint with strict rules. **Triggers**: @@ -74,19 +45,20 @@ The repository includes a comprehensive suite of CI/CD workflows that ensure cod **Jobs**: -- **lint-yaml**: Validates YAML syntax -- **shellcheck**: Validates shell script syntax -- **test-summary**: Provides completion summary +- **lint-yaml**: Validates YAML syntax using yamllint + - Lints all workflow files in `.github/workflows/` + - Validates `action.yml` syntax and structure + - Enforces formatting standards with custom rules **Quality Checks**: -- Code formatting (Black) -- Code quality (Pylint, Flake8) -- Type checking (MyPy) -- YAML validation -- Security vulnerabilities (Bandit, Safety) +- YAML syntax validation (ensures valid YAML structure) +- Line length enforcement (max 120 characters) +- Consistent indentation (2 or 4 spaces) +- Comment spacing (minimum 1 space from content) +- Proper YAML formatting standards -### 4. Release and Marketplace (`release.yml`) +### 3. Release and Marketplace (`release.yml`) **Purpose**: Automates semantic versioning, tagging, and release creation. @@ -126,57 +98,6 @@ The repository includes a comprehensive suite of CI/CD workflows that ensure cod - Must follow Keep a Changelog format - Semantic versioning (MAJOR.MINOR.PATCH) -### 5. Security Audit (`security-audit.yml`) - -**Purpose**: Performs comprehensive security scanning and audits. - -**Triggers**: - -- Schedule (daily at 2 AM UTC) -- Pushes to `main` -- Pull requests to `main` -- Manual workflow dispatch - -**Jobs**: - -- **dependency-review**: Reviews dependency changes in PRs -- **python-security-scan**: Runs Bandit, pip-audit, and Safety -- **codeql-analysis**: Performs CodeQL security analysis -- **secret-scanning**: Scans for leaked secrets with TruffleHog -- **workflow-security**: Validates workflow security practices -- **security-summary**: Provides audit summary - -**Security Checks**: - -- Dependency vulnerabilities (Dependency Review, pip-audit, Safety) -- Code security issues (CodeQL) -- Secret leakage (TruffleHog) -- Workflow permission review -- Action.yml security validation - -**Artifacts**: - -- Bandit security report (JSON) -- Safety security report (JSON) -- CodeQL analysis results - -### 6. Dependabot Configuration (`dependabot.yml`) - -**Purpose**: Automated dependency update management. - -**Configuration**: - -- **GitHub Actions**: Weekly updates on Mondays -- **Python packages**: Weekly updates on Mondays -- Auto-assigns reviewers -- Limits open PRs to 5 per ecosystem -- Ignores major version updates by default - -**Labels**: - -- `dependencies` -- `github-actions` or `python` (ecosystem-specific) - ## Workflow Permissions All workflows follow the principle of least privilege: @@ -189,17 +110,12 @@ All workflows follow the principle of least privilege: 1. **Pinned Actions**: All third-party actions use specific versions 2. **Minimal Permissions**: Workflows request only necessary permissions -3. **Secret Handling**: No secrets exposed in logs or artifacts -4. **Input Validation**: All workflow inputs are validated -5. **Dependency Scanning**: Automated vulnerability detection -6. **Code Analysis**: Static and dynamic security analysis +3. **Input Validation**: All workflow inputs are validated ## Workflow Dependencies ```text -test-action.yml (validates action) - ↓ -lint-test.yml (validates code quality) +lint-test.yml (validates YAML syntax) ↓ changelog-check.yml (validates documentation) ↓ @@ -208,17 +124,14 @@ release.yml (creates releases) marketplace submission (manual) ``` -Security workflows run independently and continuously. - ## Usage Guidelines ### For Contributors 1. **Making Changes**: - Update CHANGELOG.md under `[Unreleased]` section - - Ensure all tests pass in `test-action.yml` - - Address linting issues from `lint-test.yml` - - Review security scan results + - Ensure YAML files pass `lint-test.yml` validation + - Follow changelog format requirements 2. **Creating Releases**: - Merge changes to `main` with updated CHANGELOG.md @@ -234,10 +147,9 @@ Security workflows run independently and continuously. ### For Maintainers -1. **Review Dependabot PRs**: Check and merge dependency updates -2. **Monitor Security Scans**: Review daily security audit results -3. **Release Approval**: Verify changelog before merging to main -4. **Marketplace**: Manually publish new versions to marketplace +1. **Release Approval**: Verify changelog before merging to main +2. **Marketplace**: Manually publish new versions to marketplace +3. **Version Tags**: Ensure major version tags (e.g., v1) point to latest release ## Troubleshooting @@ -247,26 +159,20 @@ Security workflows run independently and continuously. - Verify changelog follows Keep a Changelog format - Check workflow logs for errors -### Tests Failing - -- Review test logs in `test-action.yml` -- Ensure all action features work correctly -- Validate badge generation and README updates - -### Security Alerts +### YAML Linting Fails -- Review security audit job outputs -- Check uploaded security report artifacts -- Address vulnerabilities before merging +- Review lint-test.yml workflow logs +- Check for YAML syntax errors in workflow files +- Ensure action.yml is valid YAML +- Verify line length limits (max 120 characters) ## Maintenance ### Regular Tasks -- Weekly: Review and merge Dependabot PRs -- Daily: Monitor security audit results - Per release: Update CHANGELOG.md - As needed: Review and update workflow configurations +- Monitor workflow runs for failures ### Workflow Updates diff --git a/.github/WORKFLOW_QUICK_START.md b/.github/WORKFLOW_QUICK_START.md index 68b5066..cc610fe 100644 --- a/.github/WORKFLOW_QUICK_START.md +++ b/.github/WORKFLOW_QUICK_START.md @@ -40,10 +40,8 @@ This guide provides a quick reference for using the GitHub workflows in this rep 4. **Create Pull Request**: - Go to GitHub and create a PR to `main` - Wait for automated checks to complete: - - ✅ Test Action - validates all action features - - ✅ Lint & Test - checks code quality + - ✅ Lint & Test - validates YAML syntax, formatting, and action.yml structure - ✅ Changelog Check - verifies changelog update - - ✅ Security Audit - scans for vulnerabilities 5. **Address any failures**: - Review workflow logs for errors @@ -126,39 +124,16 @@ The action.yml already includes marketplace metadata: - Branding (icon, color) - All required fields -### Managing Dependabot +### Monitoring Action Usage -Dependabot automatically creates PRs for dependency updates: +1. **Check Workflow Runs**: + - Monitor the **Actions** tab for workflow results + - Review any failures promptly -1. **Review Dependabot PRs** weekly: - - Check for breaking changes - - Review release notes of updated dependencies - - Ensure all tests pass - -2. **Merge safe updates**: - - Patch updates (e.g., 1.0.1 → 1.0.2) - usually safe - - Minor updates (e.g., 1.0.0 → 1.1.0) - review changes - - Major updates are ignored by default - -3. **Batch merge**: - - Can merge multiple Dependabot PRs at once - - Update changelog with "Updated dependencies" entry - -### Monitoring Security - -1. **Daily Security Audits**: - - Runs automatically at 2 AM UTC - - Check **Actions** tab for results - - Review any security findings - -2. **Security Alerts**: - - Check **Security** tab regularly - - Review Dependabot alerts - - Review CodeQL findings - -3. **Artifacts**: - - Security workflows upload detailed reports - - Download artifacts from workflow runs for analysis +2. **Review Logs**: + - Click on failed workflow runs to see detailed logs + - Address YAML syntax errors + - Fix changelog format issues ## Workflow Triggers @@ -166,11 +141,9 @@ Dependabot automatically creates PRs for dependency updates: | Workflow | PR to main | Push to main | Schedule | Manual | |----------|-----------|--------------|----------|--------| -| test-action.yml | ✅ (if action files change) | ✅ (if action files change) | ❌ | ✅ | | changelog-check.yml | ✅ | ❌ | ❌ | ❌ | | lint-test.yml | ✅ | ✅ | ❌ | ✅ | | release.yml | ❌ | ✅ | ❌ | ✅ | -| security-audit.yml | ✅ | ✅ | ✅ (daily) | ✅ | ### Manual Workflow Dispatch @@ -193,9 +166,7 @@ To manually run any workflow: 1. Click "Details" to view logs 2. Common issues: - **Changelog Check fails**: Update CHANGELOG.md - - **Lint fails**: Fix code formatting with Black - - **Tests fail**: Review test logs, fix code - - **Security scan fails**: Review and fix security issues + - **YAML Lint fails**: Fix YAML syntax errors ### Release Not Created @@ -220,24 +191,24 @@ To manually run any workflow: 2. Push to main or run release workflow manually -### Test Action Workflow Fails +### YAML Linting Fails -**Symptom**: test-action.yml fails +**Symptom**: lint-test.yml fails **Common causes**: -1. **Action.yml syntax error**: Validate YAML -2. **Script error**: Check update_badges.py -3. **Missing dependencies**: Check action setup steps +1. **Action.yml syntax error**: Validate YAML syntax +2. **Workflow file errors**: Check workflow YAML files +3. **Line length issues**: Lines exceeding 120 characters **Debug**: -1. View workflow logs -2. Test action locally with act (if available) -3. Test update_badges.py manually: +1. View workflow logs for specific errors +2. Validate YAML locally: ```bash - python3 update_badges.py --help + python -c "import yaml; yaml.safe_load(open('action.yml'))" + yamllint .github/workflows/ ``` ## Best Practices diff --git a/README.md b/README.md index ee5cf0a..3760c89 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![GitHub Marketplace](https://img.shields.io/badge/Marketplace-Python%20Linting-blue.svg?colorA=24292e&colorB=0366d6&style=flat&longCache=true&logo=github)](https://github.com/marketplace/actions/python-linting) -GitHub Action to perform Python linting with Black, Pylint, and MyPy. +A comprehensive GitHub Action for Python code quality enforcement. Runs **Pylint**, **Black**, and **MyPy** with automatic badge generation and detailed reporting. ## Features @@ -11,39 +11,90 @@ GitHub Action to perform Python linting with Black, Pylint, and MyPy. - 📦 **Custom requirements** - Install additional dependencies from a requirements file - 🔍 **Comprehensive linting** - Run Pylint, Black, and MyPy in a single action - 📊 **Detailed reporting** - View results in GitHub Actions summary +- 🏅 **Automatic badge generation** - Creates and commits SVG badges for each linter +- ⚙️ **Highly configurable** - Customize options for each linting tool + +## Prerequisites + +- Your repository must contain Python code +- For badge commits: the workflow needs `contents: write` permission ## Usage ### Basic Example ```yaml -- name: Python Linting - uses: thoughtparametersllc/python-linting@v1 - permissions: - contents: write +name: Python Linting +on: [push, pull_request] + +jobs: + lint: + runs-on: ubuntu-latest + permissions: + contents: write # Required for badge commits + steps: + - uses: actions/checkout@v4 + + - name: Python Linting + uses: thoughtparametersllc/python-linting@v1 ``` -### Advanced Example +### Advanced Example with Custom Options + +```yaml +name: Python Linting +on: [push, pull_request] + +jobs: + lint: + runs-on: ubuntu-latest + permissions: + contents: write + steps: + - uses: actions/checkout@v4 + + - name: Python Linting + uses: thoughtparametersllc/python-linting@v1 + with: + python-version: '3.11' + requirements-file: 'requirements.txt' + pylint-options: '--max-line-length=120 --disable=C0111' + black-options: '--line-length=120' + mypy-options: '--strict --ignore-missing-imports' + commit-badges: 'true' + badge-directory: '.github/badges' +``` + +### Example without Badge Commits ```yaml - name: Python Linting uses: thoughtparametersllc/python-linting@v1 - permissions: - contents: write with: - python-version: '3.11' - requirements-file: 'requirements.txt' - pylint_options: '--max-line-length=120' - black_options: '--line-length=120' - mypy_options: '--strict' + python-version: '3.10' # Always quote version numbers + commit-badges: 'false' # Disable automatic badge commits ``` -**Manual Badge References**: +### Example with Multiple Python Versions -```markdown -![Pylint](.github/badges/pylint.svg) -![Black](.github/badges/black.svg) -![MyPy](.github/badges/mypy.svg) +```yaml +name: Python Linting +on: [push, pull_request] + +jobs: + lint: + runs-on: ubuntu-latest + strategy: + matrix: + python-version: ['3.9', '3.10', '3.11', '3.12'] + steps: + - uses: actions/checkout@v4 + + - name: Python Linting + uses: thoughtparametersllc/python-linting@v1 + with: + python-version: ${{ matrix.python-version }} + commit-badges: ${{ matrix.python-version == '3.11' && 'true' || 'false' }} # Only the Python 3.11 job commits badges ``` ## Inputs @@ -51,10 +102,257 @@ GitHub Action to perform Python linting with Black, Pylint, and MyPy. | Input | Description | Required | Default | |-------|-------------|----------|---------| | `python-version` | Python version to use for linting | No | `3.x` | -| `requirements-file` | Path to requirements file for additional dependencies | No | `''` | -| `pylint_options` | Additional options to pass to pylint | No | `''` | -| `black_options` | Additional options to pass to black | No | `''` | -| `mypy_options` | Additional options to pass to mypy | No | `''` | +| `requirements-file` | Path to requirements file for additional dependencies | No | `requirements.txt` | +| `pylint-options` | Additional options to pass to pylint | No | `''` | +| `black-options` | Additional options to pass to black | No | `''` | +| `mypy-options` | Additional options to pass to mypy | No | `''` | +| `commit-badges` | Whether to commit generated SVG badges back to the repository | No | `true` | +| `badge-directory` | Directory to save the generated SVG badges | No | `.github/badges` | + +## Outputs + +This action does not produce any outputs. Results are displayed in the GitHub Actions summary and as SVG badges (if enabled). + +## Badges + +When `commit-badges` is set to `true` (default), the action automatically generates and commits three SVG badge files to your repository: + +- `{badge-directory}/pylint.svg` - Pylint status badge +- `{badge-directory}/black.svg` - Black status badge +- `{badge-directory}/mypy.svg` - MyPy status badge + +### Displaying Badges in README + +Add these badges to your README.md to show linting status: + +```markdown +![Pylint](.github/badges/pylint.svg) +![Black](.github/badges/black.svg) +![MyPy](.github/badges/mypy.svg) +``` + +### Badge Colors + +- 🟢 Green (`success`) - No issues found +- 🔴 Red (`failure`) - Issues detected + +## Permissions + +This action requires the following permissions: + +```yaml +permissions: + contents: write # Required only if commit-badges is 'true' +``` + +If you set `commit-badges: 'false'`, you can use: + +```yaml +permissions: + contents: read +``` + +## Troubleshooting + +### Badge Commits Failing + +**Issue**: The action fails to commit badges back to the repository. + +**Solutions**: +- Ensure `contents: write` permission is set in your workflow +- Verify that branch protection rules allow the `github-actions[bot]` to push +- Check that the badge directory exists or can be created + +### Linting Tools Not Found + +**Issue**: Pylint, Black, or MyPy commands fail with "command not found". + +**Solutions**: +- Ensure Python is properly set up (the action handles this automatically) +- Check that the Python version you specified is available +- Review the "Install Linting Tools" step logs for errors + +### Requirements Installation Fails + +**Issue**: Additional requirements cannot be installed. + +**Solutions**: +- Verify the `requirements-file` path is correct relative to repository root +- Ensure the requirements file exists and is readable +- Check for syntax errors in the requirements file +- Review dependency conflicts in the workflow logs + +### False Positive Linting Errors + +**Issue**: Linters report issues that you want to ignore. + +**Solutions**: +- Use linter-specific options to customize behavior: + - Pylint: `pylint-options: '--disable=C0111,W0212'` + - Black: `black-options: '--line-length=120'` + - MyPy: `mypy-options: '--ignore-missing-imports'` +- Create configuration files (`.pylintrc`, `pyproject.toml`) in your repository + +### Action Fails on Pull Requests from Forks + +**Issue**: Badge commits fail on pull requests from forked repositories. + +**Solution**: This is expected behavior for security reasons. Forks don't have write access. Consider: +- Setting `commit-badges: 'false'` for PRs from forks +- Using a conditional in your workflow: + ```yaml + commit-badges: ${{ github.event.pull_request.head.repo.full_name == github.repository && 'true' || 'false' }} + ``` + +## How It Works + +1. **Setup Python**: Installs specified Python version +2. **Install Tools**: Installs Pylint, Black, and MyPy +3. **Install Requirements**: (Optional) Installs dependencies from your requirements file +4. **Run Linters**: Executes all three linting tools sequentially +5. **Generate Badges**: Creates SVG badges showing pass/fail status +6. **Commit Badges**: (Optional) Commits badges back to your repository +7. **Generate Summary**: Creates detailed GitHub Actions summary with results + +## Best Practices + +### When to Use This Action + +✅ **Good use cases:** +- CI/CD pipelines for Python projects +- Pre-merge checks on pull requests +- Automated code quality enforcement +- Ensuring consistent code style across teams + +❌ **Not recommended for:** +- Local development (use pre-commit hooks instead) +- Projects with custom linting workflows requiring specific tool versions +- Repositories with complex multi-language setups + +### Recommended Workflow Configuration + +```yaml +name: Code Quality +on: + push: + branches: [main] + pull_request: + branches: [main] + +jobs: + lint: + runs-on: ubuntu-latest + permissions: + contents: write + steps: + - uses: actions/checkout@v4 + + - name: Python Linting + uses: thoughtparametersllc/python-linting@v1 + with: + python-version: '3.11' + requirements-file: 'requirements.txt' +``` + +### Configuration Files + +You can customize linter behavior using configuration files in your repository: + +- **Pylint**: `.pylintrc` or `pyproject.toml` +- **Black**: `pyproject.toml` +- **MyPy**: `mypy.ini` or `pyproject.toml` + +Example `pyproject.toml`: + +```toml +[tool.black] +line-length = 120 +target-version = ['py311'] + +[tool.mypy] +python_version = "3.11" +warn_return_any = true +warn_unused_configs = true + +[tool.pylint.format] +max-line-length = 120 + +[tool.pylint.messages_control] +disable = ["C0111", "C0103"] +``` + +## Examples + +### Example: Python Package with Tests + +```yaml +name: Test and Lint +on: [push, pull_request] + +jobs: + test: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: '3.11' + - run: pip install -r requirements.txt + - run: pytest + + lint: + runs-on: ubuntu-latest + needs: test # Run after tests pass + permissions: + contents: write + steps: + - uses: actions/checkout@v4 + - uses: thoughtparametersllc/python-linting@v1 + with: + requirements-file: 'requirements.txt' +``` + +### Example: Monorepo with Python Subdirectory + +```yaml +name: Lint Python Code +on: [push] + +jobs: + lint: + runs-on: ubuntu-latest + permissions: + contents: write + steps: + - uses: actions/checkout@v4 + + # Note: The action runs from repository root regardless of working-directory settings + # Specify full paths from root for requirements-file and Python code location + - name: Python Linting + uses: thoughtparametersllc/python-linting@v1 + with: + requirements-file: './python-service/requirements.txt' +``` + +### Example: Read-Only Mode (No Badge Commits) + +```yaml +name: PR Linting Check +on: pull_request + +jobs: + lint: + runs-on: ubuntu-latest + permissions: + contents: read # Read-only for PR checks + steps: + - uses: actions/checkout@v4 + + - name: Python Linting + uses: thoughtparametersllc/python-linting@v1 + with: + commit-badges: 'false' +``` ## Development @@ -67,6 +365,10 @@ This repository includes comprehensive GitHub workflows for CI/CD: For detailed workflow documentation, see [.github/WORKFLOWS.md](.github/WORKFLOWS.md). +### Quick Start for Contributors + +See [.github/WORKFLOW_QUICK_START.md](.github/WORKFLOW_QUICK_START.md) for a quick reference guide. + ### Releasing The release workflow supports both automatic semantic versioning and manual version specification: @@ -85,6 +387,10 @@ The release workflow supports both automatic semantic versioning and manual vers ### Contributing +We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines. + +**Quick steps:** + 1. Fork the repository 2. Create a feature branch 3. Make your changes @@ -92,6 +398,20 @@ The release workflow supports both automatic semantic versioning and manual vers 5. Ensure all tests pass 6. Submit a pull request +## Support + +- 📚 [Documentation](.github/WORKFLOWS.md) +- 🐛 [Report a Bug](https://github.com/thoughtparametersllc/python-linting/issues/new?template=bug_report.yml) +- 💡 [Request a Feature](https://github.com/thoughtparametersllc/python-linting/issues/new?template=feature_request.yml) +- 🔒 [Security Policy](SECURITY.md) + ## License This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. + +## Acknowledgments + +Built with: +- [Pylint](https://pylint.pycqa.org/) - Python code analysis +- [Black](https://github.com/psf/black) - Python code formatter +- [MyPy](https://mypy-lang.org/) - Static type checker