Automations you build with your hands. Literally.

A drag-and-drop flow editor powered by hand tracking. Connect gesture triggers to keypresses, mouse control, shell commands and macros — then run your Mac hands-free.

editor drag & drop tracking MediaPipe hotkey ⌘⇧G global Electron + React 19 MIT
$ git clone https://github.com/L-ubu/Skeleton_Studio.git && cd Skeleton_Studio && npm install && npm run dev

GitHub· The flow· Features· Architecture· Why

01Drag a trigger onto an action. Done.

The flow editor turns "when I do this gesture, do that thing" into a visual chain. Changes sync live to the running engine — no restarts.

flow editor 
  ┌──────────────┐      ┌──────────────┐      ┌──────────────┐
  │  OPEN_PALM    │─────▶│  KEYPRESS     │─────▶│  SLACK: away  │
  │  trigger      │      │  cmd+tab      │      │  shell cmd    │
  └──────────────┘      └──────────────┘      └──────────────┘

  ● live  confidence 0.95 · 30 fps · skeleton overlay on

  gesture detected: OPEN_PALM → chain fired in 12ms ✓

02What you get

◆ Flow editor

Drag-and-drop chains: gesture triggers wired to action blocks — keypress, mouse click, shell command, macro.

◆ Live camera feed

Real-time hand skeleton overlay with gesture detection and per-frame confidence tracking.

◆ Skeleton-only mode

Black out the camera feed and show only the hand skeleton with S. Privacy-friendly demos.

◆ Mouse mode

Control your cursor with your hand via M. Yes, it feels like the future.

◆ Global hotkey

Cmd+Shift+G toggles gesture recognition system-wide — on for the demo, off for the typing.

◆ Save & load projects

Export gesture configurations, share them, import someone else's setup in one click.

03Architecture

Electron on top, Python underneath, WebSocket between.

LayerWhat it does
React UIFlow editor, block palette, properties panel, webcam viewer
Electron mainSpawns the engine, hosts the WebSocket server on :9150
Python engineMediaPipe + OpenCV — camera, tracking, action dispatch
Skeleton_EngineThe standalone CLI core this studio is built on

04Why it exists

The decisions worth knowing about, written down.

Gesture control shouldn't need code

The engine is powerful, but "edit a YAML and restart" kills the magic. The studio makes it a drag, a drop, and a wave of the hand.

Live sync, no restarts

Changes in the UI push instantly to the running engine over WebSocket. Tweak a threshold and test it in the same second.

Built on a real engine

Everything runs on Skeleton_Engine — the CLI core. The studio is the friendly face, not a rewrite.