Final sketch: https://editor.p5js.org/cp3636/sketches/uXJ5I9EbA
Presentation mode: https://editor.p5js.org/cp3636/full/uXJ5I9EbA
My project is an interactive cooking game that uses ml5’s handpose model to let players control the game through hand gestures. The goal is to complete a recipe by "grabbing" virtual ingredients with pinch gestures and dropping them into a bowl. It’s designed to feel intuitive and fun, translating real hand movements into in-game.
The game has a built-in countdown timer to add some pressure, and audio of Gordon Ramsay yelling at you to make things feel more intense as time runs out. If you finish the recipe on time and get everything right, you’ll hear a positive sound celebrating your success with a completed cake. But if you mess up by adding the wrong ingredients or running out of time, you’ll see a trash can with a negative sound and an option to restart.
One of the key mechanics is a recipe validation system that checks if the ingredients you’ve added match the correct recipe.
This project blends hand-tracking technology with a playful, stress-inducing game format. Along the way, I’ve had to tackle challenges like making the hand tracking accurate, translating gestures into smooth interactions, and syncing the audio and visual feedback in real time. It’s been a great way to explore how emerging tech like machine learning can create more immersive and accessible gaming experiences.
I used the following code provided as reference to help me complete the project. Also, honorable mentions to Dan, Alan, Shawn, and Ruby for help!
https://editor.p5js.org/icm/sketches/BkRHbimhm