This project investigates the automation of tomato flower pollination using a robotic arm. The system combines computer vision, clustering algorithms, and a precise control system to identify and pollinate tomato flowers effectively. By leveraging the Intel RealSense camera for depth and color imagery, the project uses machine learning models to predict flower locations and control a Universal Robotics Cobot Arm to perform the pollination task.
-
Set up the prerequisites:
-
Open Command Prompt: Press
Win, typecmdand pressEnter -
Clone the Repository:
- Download the repository by pasting the following into the terminal:
git clone https://github.com/Airc-Uwindsor/Pollinator cd Pollinator -
Install Required Libraries:
- Install the necessary Python packages by pasting the following into the terminal:
pip install -r requirements.txt
-
Configuring the Universal Robotics Cobot Arm:
- Configure the robot’s network settings following the steps in the Configuring a Universal Robotics Cobot Arm guide.
Ensure both the camera and the robot are connected to your computer then paste the following into the terminal to start the pollinator:
python pollinator.py- This script integrates the camera input, clustering algorithm, and robot control to execute the pollination.
The system relies on the Intel RealSense camera to provide both color and depth images. From these inputs, vectors from the camera to the flowers are calculated. These vectors are initially relative to the camera’s coordinate system and must be transformed to align with the robot’s base coordinate system.
- Vector Calculation: Vectors from the camera to the flowers are calculated using the color and depth images provided by the camera.
- Offset Adjustment: These vectors are offset to be relative to the Tool Center Point (TCP) of the robotic arm.
- Coordinate System Rotation: The vectors are then rotated to match the coordinate system of the robot’s base.
- Position Summation: Each vector is summed with the current TCP position to determine the exact position where the robot needs to move its TCP for successful flower pollination.
This system has been tested in a controlled environment. Future work could focus on adapting the system for more variable, real-world agricultural settings, which would involve handling different lighting conditions, occlusions, and varying flower densities.
Currently, the system is optimized for small-scale environments. Further development would be required to make it suitable for large-scale commercial operations.
- Lead Developer: Ario Barin Ostovary
- Contributors: Jongbin Lee