feat(packaging): add DOMjudge problem packager#598
Conversation
`rbx package domjudge` builds a DOMjudge-importable zip following the ICPC problem package format plus DOMjudge extensions, mirroring the layout pol2dom produces: domjudge-problem.ini + problem.yaml metadata, the already-built statement PDF as problem.pdf, data/sample + data/secret testcases, jury solutions under submissions/, and custom checkers flattened into output_validators/ with a testlib.h patched to speak the Kattis validator protocol (exit 42/43, team output on stdin, feedback dir). Builtin wcmp/ncmp/yesno/dcmp checkers map to DOMjudge's default validator instead. BATCH problems only for now. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
| 'yesno.cpp': None, | ||
| 'dcmp.cpp': 'float_tolerance 1e-6', | ||
| } | ||
|
|
There was a problem hiding this comment.
please let's not resort to domjudge default output validators. let's always honor the rbx checker.
There was a problem hiding this comment.
Done in 087f800. Dropped the default-validator mapping entirely — the rbx checker is now always shipped as a custom output validator (validation: custom), so DOMjudge judges with exactly the checker rbx uses locally. This applies to the builtins (wcmp, ncmp, …) too. Verified locally that a builtin checker compiles and exits 42/43 under the DOMjudge calling convention after the testlib patch.
| ExpectedOutcome.MEMORY_LIMIT_EXCEEDED: 'run_time_error', | ||
| } | ||
|
|
||
| _CPP_SUFFIXES = {'.cpp', '.cc', '.cxx', '.c++'} |
There was a problem hiding this comment.
please use the language kinds for the checker to identify this
There was a problem hiding this comment.
Done in 087f800. Now uses LanguageKind.CPP in environment.language_kinds(code.find_language(checker)) instead of the hardcoded suffix set, so detection derives from the actual toolchain and is robust to custom language names. Added test_output_validators_reject_non_cpp_checker to cover the rejection path.
| } | ||
|
|
||
| # DOMjudge reports MLE as RTE by default, hence the MLE mapping. | ||
| _SUBMISSION_DIRS = { |
There was a problem hiding this comment.
could you explain this better? i'm not sure i like the idea of solutions with certain outcomes just vanishing from the upload. please research how these submissions are used, when they matter, which ones we can map to their outcomes, etc. please specifically validate that domjudge (not only problem package format) supports what you're stating.
There was a problem hiding this comment.
You're right — dropping solutions was wrong. I researched how DOMjudge actually handles jury submissions (verified against the DOMjudge 9.0 source, not just the problem-package spec) and reworked it in 087f800 so no solution is ever dropped.
How DOMjudge uses these: on import it auto-judges every solution in submissions/ and records an expected verdict per submission, then surfaces any mismatch on the jury Judging verifier page (it never blocks the import). The expected verdict comes from one of two places:
- Directory name, when it normalizes to a real verdict via DOMjudge's
PROBLEM_RESULT_REMAPtable (SubmissionService.php). That's 7 dirs, not 4:accepted,wrong_answer,time_limit_exceeded,run_time_error,output_limit,compiler_error,no_output. - An
@EXPECTED_RESULTS@:source annotation, which supports multiple acceptable verdicts (e.g.CORRECT, TIMELIMIT). Crucially, in a verdict-named dir DOMjudge collapses the annotation to the dir's single verdict; in a non-verdict dir (e.g.mixed/) it keeps the full list. So multi-verdict expectations must live in a non-verdict dir.
New mapping (every outcome shipped):
| rbx outcome | placement |
|---|---|
| ACCEPTED / WRONG_ANSWER / TIME_LIMIT_EXCEEDED / RUNTIME_ERROR | matching standard dir, no annotation |
| OUTPUT_LIMIT_EXCEEDED | output_limit/ (DOMjudge extension dir) |
| ACCEPTED_OR_TLE | mixed/ + @EXPECTED_RESULTS@: CORRECT, TIMELIMIT |
| TLE_OR_RTE | mixed/ + TIMELIMIT, RUN-ERROR |
| INCORRECT | mixed/ + all non-CORRECT verdicts |
| ANY | mixed/ + all verdicts |
| MEMORY_LIMIT_EXCEEDED | mixed/ + RUN-ERROR, TIMELIMIT |
The one thing DOMjudge genuinely can't represent: there is no memory-limit verdict and no @EXPECTED_RESULTS@ token for it — an over-memory run surfaces as RTE (sometimes TLE). So MLE is the single lossy mapping; I encode it as RUN-ERROR, TIMELIMIT and documented the caveat. Everything else is exact.
Annotation comment prefix is # for Python, // otherwise. Docs/CLAUDE.md updated with the table and a note that mixed/ solutions trigger a harmless "result does not match directory" message on import. Covered by test_submissions_single_verdict_use_standard_dirs and test_submissions_ambiguous_outcomes_use_mixed_dir_with_annotation.
- Always ship the rbx checker as a custom output validator (validation: custom); never fall back to DOMjudge's default validators, so DOMjudge judges with the same checker rbx uses locally. Drops the wcmp/ncmp/yesno/dcmp -> default mapping. - Detect the C++ checker via language kinds (LanguageKind.CPP in environment.language_kinds(find_language)) instead of hardcoded file suffixes. - Ship every solution; none are dropped. Single-verdict outcomes go to the matching standard submission directory (incl. output_limit for OLE); multi-verdict outcomes (MLE, ACCEPTED_OR_TLE, TLE_OR_RTE, INCORRECT, ANY) go to submissions/mixed/ with an @EXPECTED_RESULTS@ annotation listing every acceptable DOMjudge verdict. MLE is the one lossy mapping (DOMjudge has no memory-limit verdict). Updates docs, CLAUDE.md and the design doc to match the corrected DOMjudge submission semantics (directory name -> verdict via remap table; annotation honored verbatim in non-verdict dirs). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Summary
Adds
rbx package domjudge, building a DOMjudge-importable problem zip. The layout follows the ICPC problem package format plus DOMjudge extensions, mirroring what pol2dom produces (known to import cleanly):Design doc:
docs/plans/2026-06-12-domjudge-packager-design.md.Key decisions
wcmp/ncmp/yesno→ no flags,dcmp→float_tolerance 1e-6). Anything else — including a same-named local file, which may have been user-edited — ships as a custom validator, flattened viaflattening.build_flat_namespace(cross-dir includes rewritten).domjudge/testlib_patch.py): DOMjudge validators speak the Kattis protocol (exit 42/43, team output on stdin, feedback dir), which vanilla testlib doesn't. Ports pol2dom's patch (from cn-xcpc-tools/testlib-for-domjudge), applied to the bundled testlib at package time; raises if an anchor is missing so a testlib upgrade fails tests, not packages. Verified locally: patchedwcmpcompiles and exits 42/43 withjudgemessage.txtunder the DOMjudge calling convention.domjudgelimits profile when saved, else package limits (not required, unlike BOCA — DOMjudge has a single per-problem TL).Testing
tests/rbx/box/packaging/test_domjudge.py(testlib patch against bundled testlib, ini/yaml content, builtin-vs-custom checker resolution, flattening, submissions mapping, sample/secret routing, fullpackage()smoke).tests/rbx/box/packaging+tests/rbx/box/completion: 1139 passed (3 docker-compose BOCA e2e errors are pre-existing local-env failures).mise run gen-completion-spec); drift test green.docs/setters/packaging/domjudge.md+ index table row + mkdocs nav.🤖 Generated with Claude Code