Emotion Aura
Your webcam reads your mood, and 5,000 particles show it
Overview
Emotion Aura watches your face and hands through the webcam and draws what it sees as a glowing particle aura. MediaPipe extracts face landmarks and 52 blendshapes, and a classifier built on FACS rules maps them to one of 5 emotions. A Three.js particle system with custom GLSL shaders then draws the aura around your face, in the colors of the current emotion. An open palm pushes the particles away, a closed fist pulls them in. There's no backend, everything runs on your machine.
Why I built this
I'd wanted to play with computer vision for a long time, especially emotion detection and hand tracking. This project was the way in.
Features
- Emotion detection. 5 emotions classified from 52 facial blendshapes with weighted FACS rules. EMA smoothing keeps the transitions fluid.
- GPU particles. 5,000 particles with custom GLSL spark shaders. Each emotion has its own palette and way of moving, and the particles spawn from your face.
- Both hands. Tracks two hands at once. An open palm pushes the particles away, a closed fist pulls them in, and the aura shows the force field.
- Blink fireworks. Blink your left or right eye to fire a burst of particles on that side of the screen.
- Staggered inference. Face and hand detection take turns, one frame each. Each model runs at 30fps while rendering stays at 60fps.
- No backend. Everything runs in the browser, and the ML models load from a CDN.