Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 76 additions & 3 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Cargo Build & Test
name: Build & Test

on:
pull_request:
Expand All @@ -7,8 +7,8 @@ env:
CARGO_TERM_COLOR: always

jobs:
build_and_test:
name: Rust project - latest
rust_tests:
name: Rust Tests
runs-on: ubuntu-latest
strategy:
matrix:
Expand Down Expand Up @@ -37,3 +37,76 @@ jobs:

- name: Run all tests in workspace
run: cargo test --workspace --verbose

cli_tests:
name: CLI Tests
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']
exclude:
# Reduce matrix size by testing fewer Python versions on non-Linux
- os: windows-latest
python-version: '3.8'
- os: windows-latest
python-version: '3.9'
- os: macos-latest
python-version: '3.8'
- os: macos-latest
python-version: '3.9'

steps:
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}

- name: Set up Rust
uses: dtolnay/rust-toolchain@stable

- name: Cache Rust dependencies
uses: actions/cache@v4
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
./target/
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}

- name: Cache Python dependencies
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ matrix.python-version }}-${{ hashFiles('**/pyproject.toml') }}

- name: Install system dependencies (Ubuntu)
if: matrix.os == 'ubuntu-latest'
run: |
sudo apt-get update
sudo apt-get install -y libwebkit2gtk-4.1-dev libappindicator3-dev librsvg2-dev patchelf

- name: Install maturin
run: pip install maturin

- name: Build and install Python package
working-directory: ./crates/python_api
run: |
maturin develop --release

- name: Install test dependencies
working-directory: ./crates/python_api
run: pip install -e ".[test]"

- name: Run CLI tests
working-directory: ./crates/python_api
run: pytest tests/ -v --tb=short

- name: Run CLI tests in parallel (Linux only)
if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.11'
working-directory: ./crates/python_api
run: pytest tests/ -n auto -v
172 changes: 172 additions & 0 deletions crates/python_api/README_TESTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
# CLI Testing Guide

This document explains how to run and develop tests for the OpenFire CLI tool.

## Test Structure

```
crates/python_api/
├── tests/
│ ├── __init__.py
│ ├── conftest.py # Test fixtures and configuration
│ └── test_cli.py # CLI test cases
└── pyproject.toml # Test dependencies and pytest config
```

## Running Tests Locally

### Prerequisites

1. **Install the package in development mode:**
```bash
cd crates/python_api
maturin develop --release
```

2. **Install test dependencies:**
```bash
pip install -e ".[test]"
```

### Running Tests

**Run all tests:**
```bash
cd crates/python_api
pytest tests/ -v
```

**Run specific test classes:**
```bash
pytest tests/test_cli.py::TestVersionCommand -v
pytest tests/test_cli.py::TestNewCommand -v
```

**Run tests in parallel:**
```bash
pytest tests/ -n auto -v
```

**Run tests with coverage:**
```bash
pytest tests/ --cov=ofire --cov-report=html
```

## Test Categories

### 1. CLI Availability Tests
- Verify `ofire` command is accessible
- Test module importability
- Basic command execution

### 2. Command-Specific Tests
- **Version Command:** Output format and content
- **Help Command:** Subcommand listing and help text
- **New Command:** Project creation, file structure, various names
- **Run Command:** Basic functionality and error handling
- **Docs Command:** Execution without errors

### 3. Cross-Platform Tests
- Path handling (Windows vs Unix)
- File permissions (Unix-specific)
- Platform-specific behaviors

### 4. Error Handling Tests
- Invalid commands
- Missing arguments
- Graceful failure scenarios

## CI/CD Testing

The GitHub Actions workflow tests across:
- **Operating Systems:** Ubuntu, Windows, macOS
- **Python Versions:** 3.8, 3.9, 3.10, 3.11, 3.12
- **Test Modes:** Sequential and parallel execution

### Matrix Strategy
- Full Python version matrix on Ubuntu
- Reduced matrix on Windows/macOS for efficiency
- Separate Rust and Python test jobs

## Writing New Tests

### Test Fixtures Available

**`cli_runner`** - Execute CLI commands:
```python
def test_my_command(cli_runner):
result = cli_runner("new", "my_project", "-d", "/tmp")
assert result.returncode == 0
```

**`temp_dir`** - Temporary directory:
```python
def test_file_creation(temp_dir):
test_file = temp_dir / "test.txt"
test_file.write_text("content")
assert test_file.exists()
```

**`ofire_available`** - Skip if CLI not installed:
```python
def test_cli_function(ofire_available, cli_runner):
# Test only runs if ofire is properly installed
result = cli_runner("version")
assert result.returncode == 0
```

### Test Naming Conventions
- Test files: `test_*.py`
- Test classes: `Test*`
- Test functions: `test_*`

### Platform-Specific Tests
```python
@pytest.mark.skipif(os.name == 'nt', reason="Unix-specific test")
def test_unix_feature():
# Unix-only test code

@pytest.mark.skipif(os.name != 'nt', reason="Windows-specific test")
def test_windows_feature():
# Windows-only test code
```

## Test Configuration

The `pyproject.toml` includes pytest configuration:
- Test discovery patterns
- Verbose output by default
- Custom markers for slow/integration tests
- Short traceback format

## Debugging Tests

**Run with detailed output:**
```bash
pytest tests/ -v --tb=long
```

**Run specific test with debugging:**
```bash
pytest tests/test_cli.py::test_version_command -v -s
```

**Stop on first failure:**
```bash
pytest tests/ -x
```

## Common Issues

1. **"ofire CLI not available"** - Run `maturin develop` first
2. **Import errors** - Ensure you're in the `python_api` directory
3. **Permission errors** - Check temp directory permissions
4. **Path issues** - Use `pathlib.Path` for cross-platform compatibility

## Contributing

When adding new CLI functionality:
1. Add corresponding tests in `test_cli.py`
2. Update this README if new test patterns are introduced
3. Ensure tests pass on all platforms via CI/CD
4. Consider edge cases and error conditions
Loading
Loading