AI-driven Robotic Chemists

Workshop Context

This project was developed as part of the Winter School: Robotics and AI for Materials Chemistry 2025 organized by AIchemy.

Project: Project 2 - AI-driven Robotic Chemists

Team

Team Lead: Laura Jones

Team Members:

• Aritra Roy
• Fanjin Wang
• Matthew Pratley
• Kefeng Huang
• Naneena Suresh

Project Overview

This project develops an intuitive framework for teaching robotic chemists through vision-guided learning, implemented on a UR5e robotic arm for automated vial handling workflows in chemistry laboratories.

Objective

Design and implement a system that enables robots to learn experimental procedures, eliminating the need for manual programming of complex manipulation tasks in chemistry labs.

Approach

• Hardware: UR5e collaborative robotic arm
• Learning Method: CNN-based classification for position guidance
• Perception: Computer vision for real-time vial tray hole detection and localization
• Task: Automated vial pick-and-place operations

Current Implementation

• Vision System: CNN classifier to identify hole numbers in vial trays
• Robot Guidance: Vision-guided positioning for precise vial pickup and placement
• Workflow: Automated transfer of vials from storage tray to target positions
• Real-time Processing: Computer vision pipeline for hole detection and classification

Future Work

• Imitation Learning: Enable the robot to learn pick-and-place operations from human demonstrations
• Flexible Positioning: Allow vial pickup from arbitrary positions, not limited to predefined tray holes
• Multi-step Workflows: Extend to complex chemistry procedures with multiple manipulation steps
• Adaptive Grasping: Improve robustness for different vial sizes and orientations

Acknowledgments

Special thanks to AIchemy for organizing this winter school and providing the opportunity to work on cutting-edge robotics and AI for materials chemistry.