[codex] Add certificate index endpoint

[ethanolivertroy]

Summary

Adds a generated api/certificates/index.json endpoint that lets consumers discover every per-certificate detail file without guessing paths or loading each detail payload individually.

The endpoint includes shared metadata, a certificate_paths lookup, and compact certificate summary rows with dataset, status, standard, vendor/module names, URLs, validation dates, algorithm counts, and detail availability.

Validation

• venv/bin/python -m py_compile scraper.py test_scraper.py validate_api.py
• venv/bin/python test_scraper.py
• venv/bin/python validate_api.py
• git diff --check

Notes

The current checked-in data still predates the full post-merge refresh, so algorithm extraction provenance fields remain nullable in this generated index until the long-running update workflow regenerates the data.

[qodo-code-review]

Review Summary by Qodo

Add certificate index endpoint for efficient discovery

✨ Enhancement

Walkthroughs

Description

• Adds /api/certificates/index.json endpoint for certificate discovery
• Includes compact summary rows with certificate metadata and algorithm counts
• Provides certificate_paths lookup for efficient path resolution
• Generates JSON Schema and OpenAPI specifications for the new endpoint
• Updates documentation, validation, and API reference artifacts

Diagram

flowchart LR
  A["Certificate Data"] -->|build_certificate_index_entry| B["Index Entries"]
  B -->|build_certificate_index_payload| C["Index Payload"]
  C -->|save_json| D["api/certificates/index.json"]
  D -->|validate_certificate_index| E["Validation"]
  C -->|generate_json_schema_artifacts| F["certificate-index.schema.json"]
  C -->|generate_openapi_spec| G["OpenAPI Spec"]

Loading

File Changes

1. scraper.py ✨ Enhancement +295/-1

Add certificate index generation and schema functions

scraper.py

---

2. test_scraper.py 🧪 Tests +102/-0

Add comprehensive certificate index payload tests

test_scraper.py

---

3. validate_api.py ✨ Enhancement +63/-3

Add certificate index validation logic

validate_api.py

---

View more (18)

4. index.html 📝 Documentation +3/-1

Update homepage with certificate index endpoint link

index.html

---

5. openapi.json 📝 Documentation +182/-0

Add certificate index path and schema definitions

openapi.json

---

6. api/schemas/certificate-index.schema.json 📝 Documentation +170/-0

Create JSON Schema for certificate index response

api/schemas/certificate-index.schema.json

---

7. api/docs.md 📝 Documentation +14/-5

Document certificate index endpoint and usage examples

api/docs.md

---

8. llms-full.txt 📝 Documentation +14/-5

Add certificate index to agent documentation

llms-full.txt

---

9. api/index.json ⚙️ Configuration changes +20/-2

Register certificate index endpoint and schema

api/index.json

---

10. README.md 📝 Documentation +5/-0

Add certificate index to API reference table

README.md

---

11. llms.txt 📝 Documentation +3/-1

Add certificate index to quick reference

llms.txt

---

12. api/schemas/index.schema.json ⚙️ Configuration changes +2/-1

Register certificate index schema in discovery

api/schemas/index.schema.json

---

13. api/schemas/modules-in-process.schema.json Formatting +1/-1

Fix trailing newline formatting

api/schemas/modules-in-process.schema.json

---

14. api/schemas/historical-modules.schema.json Formatting +1/-1

Fix trailing newline formatting

api/schemas/historical-modules.schema.json

---

15. api/schemas/metadata.schema.json Formatting +1/-1

Fix trailing newline formatting

api/schemas/metadata.schema.json

---

16. api/schemas/modules.schema.json Formatting +1/-1

Fix trailing newline formatting

api/schemas/modules.schema.json

---

17. api/schemas/module-in-process.schema.json Formatting +1/-1

Fix trailing newline formatting

api/schemas/module-in-process.schema.json

---

18. api/schemas/module.schema.json Formatting +1/-1

Fix trailing newline formatting

api/schemas/module.schema.json

---

19. api/schemas/algorithms.schema.json Formatting +1/-1

Fix trailing newline formatting

api/schemas/algorithms.schema.json

---

20. api/schemas/certificate-detail.schema.json Formatting +1/-1

Fix trailing newline formatting

api/schemas/certificate-detail.schema.json

---

21. api/certificates/index.json Additional files +152859/-0

...

api/certificates/index.json

---

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (2) 📘 Rule violations (0)

1. Docs check coverage reduced 🐞 Bug ☼ Reliability

Description

validate_api.validate_docs_and_index() now only asserts that llms-full.txt/api/docs.md/index.html
mention the new certificates/index endpoint, but it no longer asserts those same docs still mention
the per-certificate detail endpoint (certificates/{certificate}.json), allowing accidental removal
of key endpoint documentation to go unnoticed by validate_api.py.

Code

validate_api.py[R418-420]

+        ("llms-full.txt", "GET api/certificates/index.json"),
+        ("api/docs.md", "GET api/certificates/index.json"),
+        ("index.html", "api/certificates/index.json"),

Evidence

The required-text assertions in validate_docs_and_index() now check for the index endpoint text in
llms-full.txt and api/docs.md, but do not check for the certificate detail endpoint text in those
same docs (even though the detail endpoint is still present there today).

validate_api.py[415-432]
api/docs.md[109-114]
llms-full.txt[7-23]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`validate_api.py`'s `validate_docs_and_index()` reduced its documentation assertions to only check for `GET api/certificates/index.json` (and `api/certificates/index.json` in `index.html`). This drops coverage for ensuring the per-certificate detail endpoint remains documented (`GET api/certificates/{certificate}.json`) in the same key docs, so regressions can slip through `validate_api.py` even if `test_scraper.py` would catch them.

### Issue Context
This PR adds the new certificate index endpoint and updates docs accordingly, but the validation should be additive: keep the new checks **and** preserve checks for the detail endpoint.

### Fix Focus Areas
- validate_api.py[415-432]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Metadata docs mismatch 🐞 Bug ≡ Correctness

Description

The updated docs now state that api/metadata.json includes extraction metrics, but the committed
api/metadata.json (and the metadata embedded in api/certificates/index.json) in this PR branch does
not contain an extraction_metrics field, so consumers following the docs/examples won’t find the
advertised data.

Code

api/docs.md[20]

+`GET api/metadata.json` — Generation timestamp, source URLs, dataset counts, extraction metrics, and algorithm extraction status.

Evidence

api/docs.md/llms.txt/README.md explicitly advertise and/or query extraction metrics, but the
committed api/metadata.json and the embedded metadata inside api/certificates/index.json do not
include an extraction_metrics key. scraper.py shows that the generator intends to emit these fields,
implying the committed artifacts are out of sync with the documented contract.

api/docs.md[16-27]
llms.txt[9-16]
README.md[171-181]
api/metadata.json[2-13]
api/certificates/index.json[2-15]
scraper.py[3767-3782]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
Documentation updates claim `api/metadata.json` includes `extraction_metrics` (and README examples query `.extraction_metrics.combined`), but the committed `api/metadata.json` payload in this branch does not include `extraction_metrics` (nor does `api/certificates/index.json` embed it). This is an API contract/documentation mismatch for the current published artifacts.

### Issue Context
`scraper.py` already builds metadata including `extraction_metrics` and `algorithm_extraction_schema_version`, so the most consistent fix is to regenerate and commit the generated artifacts (`api/metadata.json` and any other artifacts embedding metadata, including `api/certificates/index.json`) using the current scraper, or roll back/qualify the docs/examples until the artifacts include those fields.

### Fix Focus Areas
- api/docs.md[16-27]
- README.md[171-181]
- llms.txt[7-16]
- api/metadata.json[1-13]
- api/certificates/index.json[2-15]
- scraper.py[3767-3782]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---