Clash of Hands

We programmed the Allegro Hand to play rock-paper-scissors using machine learning strategies that help it get smarter with every game.

Team Member: Pushkar Dave

DEMO

OVERVIEW

  • Real-time Gesture Recognition: Uses MediaPipe to detect rock, paper, and scissors from live webcam input.
  • AI Agents: Markov Model, Decision Tree
  • Robotic Hand Integration: Controls an Allegro robotic hand via CAN bus to perform AI moves.
  • Game History Tracking: Logs game data for ongoing learning and behavior prediction.
block diagram
Block Diagram of the Interactions Between Subsystems

AI AGENTS

Markov Transition Matrix

  • Tracks transition probabilities between player moves.
  • Predicts the next move based on the last move’s transition matrix.

Decision Tree

  • Uses features like previous move frequency and patterns.
  • Trains a decision tree to predict the player’s next move and respond optimally.

GESTURE RECOGNITION

The system classifies hand gestures using MediaPipe Hands:

  • Closed Fist → Rock
  • Open Palm → Paper
  • Victory Sign → Scissors

HARDWARE COMPONENTS

  • Allegro Hand V4
  • CAN interface and controller