APPS env added

[SIMONLQY]

• Add APPS in envs
• Interfaces alignment with MATH env
• reason part add initial adaptation for APPS

[YanSong97]

Thanks for your contribution!

Can you also include a testing dataset to run a demo? See MATH.

[JibanKumar-cloud]

Hi! I would like to take on this issue and add code generation reasoning support to OpenR.

Proposed approach:

1. Code execution environment (envs/CODE/) — A safe subprocess-based executor that runs generated code against test cases (with timeout + sandboxing), producing binary pass/fail rewards. This replaces the PRM for the code domain since ground truth is executable.

2. Dataset loaders — HumanEval (164 problems) and MBPP (974 problems), with prompt formatting for step-by-step reasoning.

3. Integration with existing search — A CodeVerifier that produces [0, 1] rewards matching OpenR's PRM interface, so Best-of-N, Beam Search, and MCTS work out of the box. Plus a CodeSearchConfig adapter for MCTS.

4. Evaluation scripts — scripts/eval/code_greedy.sh, code_best_of_n.sh, code_mcts.sh, following the same pattern as the math evaluation scripts.

5. Benchmarks — I'll include pass@1 results on HumanEval using Qwen2.5-Coder-1.5B-Instruct across greedy, Best-of-N, and MCTS.

I have GPU access (T4 16GB) and can run the full evaluation pipeline. Estimated timeline: ~3 weeks.

Before I start coding, does this approach align with what you had in mind? Any feedback on the architecture or scope is welcome. Happy to adjust.

Thanks for building OpenR!