CAPE: Corrective Actions from Precondition Errors using Large Language Models

[Project Page] [Paper] [ArXiV]

Shreyas S Raman¹, Vanya Cohen², Ifrah Idrees¹, Eric Rosen¹, Raymond J Mooney², David Paulius¹, Stefanie Tellex¹

¹Brown University, ²The University of Texas Austin

This is the official code for our CAPE: Corrective Actions from Precondition Errors using Large Language Models paper. Our explicit prompting approach enables embodied LLM agents to generate plans for for complex tasks (e.g. "make breakfast") that are more semantically correct and executable whilst minimizing re-prompting -- without requiring any fine-tuning. The code can be used with any available language models from OpenAI API and Huggingface Transformers with a common interface.

If you find this work useful in your research, please cite using the following BibTeX:

@inproceedings{raman2024cape,
      title={{CAPE: Corrective Actions from Precondition Errors using Large Language Models}},
      author={Sundara Raman, Shreyas and Cohen, Vanya and Paulius, David and Idrees, Ifrah and Rosen, Eric and Mooney, Ray and Tellex, Stefanie},
      booktitle={2024 IEEE International Conference on Robotics and Automation (ICRA)},
      year={2024},
      note={(In Review)}
    }

Setup Instructions

git clone git@github.com:Shreyas-S-Raman/corrective_actions_precondition_errors.git
cd language-planner/
conda create --name cape-env python=3.6.13
conda activate cape-env
pip install --upgrade pip
pip install -r requirements.txt

Note:

• It is observed that best results can be obtained with larger language models. If you cannot run Huggingface Transformers models locally or on Google Colab due to memory constraint, it is recommended to register an OpenAI API account and use GPT-3 or Codex (As of 01/2022, $18 free credits are awarded to new accounts and Codex series are free after admitted from the waitlist).
• Due to language models' high sensitivity to sampling hyperparameters, you may need to tune sampling hyperparameters for different models to obtain the best results.
• The code uses the list of available actions supported in VirtualHome 1.0's Evolving Graph Simulator. The available actions are stored in available_actions.json. The actions should support a large variety of household tasks. However, you may modify or replace this file if you're interested in a different set of actions or a different domain of tasks (beyond household domain).
• A subset of the manually-annotated examples originally collected by the VirtualHome paper is used as available examples in the prompt. They are transformed to natural language format and stored in available_examples.json. Feel free to change this file for a different set of available examples.