Poolinator

Project Sequence

• The game begins with the robot arm moving to the side to avoid obscuring the table.
• The camera detects two AprilTags to determine the transformations of the table, the cue, the end-effector, and the camera, all relative to the base of the Franka.
• The locations of the cue ball (red) and other balls (blue) are updated, and these coordinates are fed into the pool algorithm.
• The robot selects a striking position that enables it to hit the cue ball and pocket another ball.
• After striking, the robot arm moves to the side again to update the new positions of the balls.
• The game continues until all the blue balls are pocketed.
• Then the robot hits the red ball into a pocket to end the game.

Nodes and Python Modules

The project is divided into several subsystems:

AprilTags (Transform Node)

This node uses the table tag and the cue tag to establish the relationship between various coordinate frames (camera, end-effector, table, etc.) to build a TF tree.

Computer Vision (Image Processing Node)

This node processes data from the depth camera and camera info topics to locate the cue ball (red ball) and other balls on the pool table. It integrates OpenCV and ROS2 for image processing and publishes the detected ball positions as TF frames for use in robot control and planning.

Planning & Execution (Control Node)

This node manages the robot's movements through various states, such as striking, standby, and home positions.

Pool Algorithm

This Python module uses ball coordinates to plan feasible striking positions for the robot.

World

This Python module represents the environment, including the table, Franka's platform, and the ceiling-mounted camera. It ensures the robot's path planning avoids obstacles by maintaining spatial relationships and tracking key elements like table corners, pockets, and ball positions using TF transforms.