Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -158,3 +158,9 @@ cython_debug/
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/

# Harness execution logs
pytest_out.txt
ruff_out.txt
ruff_errors.json
ruff_errors_utf8.json
20 changes: 20 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Use a lightweight Python image
FROM python:3.11-alpine

# Set the working directory
WORKDIR /usr/src/harness

# Prevent Python from writing pyc files and keep stdout unbuffered
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

# Install dependencies first (for optimal Docker layer caching)
COPY pyproject.toml HARNESS_README.md ./
RUN pip install --no-cache-dir .

# Copy your harness script and any local suite files
COPY annotation_harness.py .
COPY annotations/ annotations/

# Define the entry point
ENTRYPOINT ["python", "annotation_harness.py"]
27 changes: 27 additions & 0 deletions HARNESS_README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Annotation Test Suite Harness

**Objective:** A standalone test harness for the JSON Schema Annotation Test Suite, utilizing the `jschon` implementation natively.

**Architecture:** Separated the loader, filter, compiler, evaluator, normalizer, asserter, and reporter into distinct pipeline stages enforcing SOLID design principles. The normalization stage handles the conversion of jschon logical keyword paths into specification-compliant physical schema fragments.

**Current Results:** Out of the 84 assertions, 78 Passed, 6 Failed. The 6 failures are due to specific edge-case keyword evaluations native to `jschon` (e.g., `contains` over-annotating negative items, `propertyNames` evaluating actual values, and `$dynamicRef` cross-resource bugs), not harness parsing errors.

**Quickstart (Local):**

```bash
python -m venv .venv
source .venv/bin/activate
pip install -e .[dev]
python annotation_harness.py
pytest
```

**Quickstart (Docker):**

```bash
docker build -t gsoc-annotation-harness .
docker run --rm gsoc-annotation-harness
```

## Normalization Logic Note
Jschon outputs absolute locations (e.g., `urn:uuid:xxx#/$defs/bar/title`). To meet test suite structural expectations (`{"#/$defs/bar": ...}`), the pipeline resolves these references locally, trims trailing keywords, and injects `$schema` dynamically into fragments missing it on compilation.
Loading
Loading