OpenCDA-MARL extends the original OpenCDA framework with Multi-Agent Reinforcement Learning (MARL) capabilities for cooperative driving automation research. Building upon OpenCDA's co-simulation-based framework integrated with prototype cooperative driving automation (CDA; see SAE J3216) pipelines, OpenCDA-MARL adds advanced MARL algorithms and training infrastructure for developing intelligent multi-agent driving policies in complex traffic scenarios.
OpenCDA-MARL combines the robust simulation capabilities of the original OpenCDA with state-of-the-art MARL algorithms to enable researchers to train and evaluate cooperative multi-agent driving policies. This extension supports various MARL paradigms including centralized training with decentralized execution (CTDE), communication-based cooperation, and emergent behaviors in mixed autonomy traffic.
The key features of OpenCDA-MARL are:
- MARL Integration: Native support for popular MARL algorithms (PPO, SAC, QMIX, MADDPG) with distributed training capabilities.
- Research Pipeline: Rich research pipelines including both rule-based CDA modules and learning-based MARL agents for platooning, cooperative perception, and traffic coordination.
- Integration: Seamless integration with CARLA and SUMO, plus compatibility with RLlib and other RL frameworks.
- Full-stack Simulation: Complete automated and cooperative driving platform in Python with perception, localization, planning, control, V2X communication, and MARL decision-making modules.
- Modularity: Highly modularized architecture allowing easy swapping between rule-based and learning-based components.
- Benchmark: MARL-specific benchmarks, training scenarios, and evaluation metrics for multi-agent driving tasks.
- Scalability: Distributed training infrastructure supporting large-scale multi-agent scenarios with hundreds of vehicles.
- Mixed Autonomy: Support for mixed traffic with human-driven vehicles, rule-based AVs, and learning-based agents.
Users can refer to our documentation for detailed guides on MARL integration, training procedures, and API references. For the original OpenCDA documentation, visit OpenCDA documentation.
- MARL Framework Integration: Core Multi-Agent Reinforcement Learning framework with implemented algorithms including Q-learning, DQN, and TD3 for intersection management and cooperative driving tasks.
- Training Infrastructure: Single-agent training capabilities with experience replay and checkpoint management, with distributed Ray/RLlib training planned for future releases.
- Mixed Autonomy Support: Seamless integration of learning-based MARL agents with rule-based vehicles, vanilla behavior agents, and human-driven traffic.
OpenCDA-MARL extends the original four components with MARL-specific modules:
- MARL Training Framework: Core training infrastructure with Q-learning, DQN, and TD3 algorithms for single-agent and multi-agent scenarios
- Cooperative Driving System: Enhanced with learning-based decision making for cooperative driving tasks
- Data Manager and Repository: Training data collection and replay buffer management
- Scenario Manager: MARL-specific training and evaluation scenarios
Check our documentation for detailed architecture and MARL integration.
Note: We continuously improve the performance of OpenCDA-MARL. Currently, it is mainly tested in our customized maps. However, we DO NOT guarantee the same level of robustness in other maps. We will update the documentation and the maps in the future.
We welcome contributions to OpenCDA-MARL! Please see our Contributing Guide for detailed instructions on:
- How to fork and clone the repository
- Creating feature branches
- Submitting pull requests
- Coding standards and guidelines
For quick reference:
- Report bugs and improvements by submitting issues
- Submit contributions via pull requests using our PR template
If you are using OpenCDA-MARL for your research, please cite both the MARL extension and the original OpenCDA paper:
@ARTICLE{11396022,
author={Guo, Lihao and Liu, Louis and Tang, Jiahao and Liu, Bo and Cao, Siyang},
journal={IEEE Robotics and Automation Letters},
title={OpenCDA-MARL: A Unified Benchmarking Framework for Cooperative Autonomous Intersection Management With Multi-Agent Reinforcement Learning},
year={2026},
pages={1-8},
keywords={Vehicle dynamics;Training;Robot sensing systems;Heuristic algorithms;Vehicle-to-everything;Benchmark testing;Real-time systems;Monitoring;Laser radar;Graphical user interfaces},
doi={10.1109/LRA.2026.3664656}
}
@misc{xu2021opencdaanopencooperativedriving,
title={OpenCDA: An Open Cooperative Driving Automation Framework Integrated with Co-Simulation},
author={Runsheng Xu and Yi Guo and Xu Han and Xin Xia and Hao Xiang and Jiaqi Ma},
year={2021},
eprint={2107.06260},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2107.06260},
}The arxiv link to the paper: https://arxiv.org/abs/2107.06260
Also, under this LICENSE, OpenCDA is for non-commercial research only. Researchers can modify the source code for their own research only. Contracted work that generates corporate revenues and other general commercial use are prohibited under this LICENSE. See the LICENSE file for details and possible opportunities for commercial use.
The MARL extension is developed and maintained by researchers focusing on multi-agent reinforcement learning for autonomous driving.
- Project Lead: Lihao Guo
- Core Team: Louis Liu, Jiahao Tang
- Advisors: Dr. Liu Bo, Dr. Siyang Cao
OpenCDA is originally developed by the UCLA Mobility Lab:
- Principal Investigator: Dr. Jiaqi Ma (UCLA Samueli)
- Project Lead: Runsheng Xu (github)
- Core Team: Xu Han, Hao Xiang, Zhaoliang Zheng, Zonglin Meng, Dr. Xin Xia
- UA Radar lab
- UCLA Mobility Lab
- UC Davis Professor Junshan Zhang's group for openScenario integration
- @GoodarzMehr for Docker support
- All contributors to the OpenCDA and OpenCDA-MARL projects