TinyML hand-gesture detection

Problem

Recognizing hand gestures directly on a microcontroller requires a model that can learn from noisy accelerometer and gyroscope sequences while respecting tight memory and compute limits. The goal was to capture four movement families from left- and right-hand recordings and run gesture inference without sending sensor data to a remote service.

Approach

IMU samples were recorded with CoolTerm and organized as labeled CSV files for the Wakanda, clap, down, and up gestures. A notebook prepares the signals and trains a compact Keras multilayer perceptron with two hidden layers. The trained model is converted to TensorFlow Lite and exported as C headers for embedded inference. Two Arduino sketches also demonstrate BLE central-peripheral communication between boards.

Technical implementation

The inference sketch waits until total acceleration crosses a 2.5 g threshold, then captures 119 synchronized accelerometer and gyroscope samples. Six input channels are normalized into the model tensor before invoking TensorFlow Lite Micro. An 8 KB statically allocated tensor arena avoids dynamic allocation on the device. The included firmware selects a hand-specific model header and demonstrates a two-class output head, while the collected dataset covers the broader four-movement experiment.

if (aSum >= accelerationThreshold) {
    samplesRead = 0;
}

tflInputTensor->data.f[index++] = (aX + 4.0f) / 8.0f;
tflInputTensor->data.f[index++] = (gX + 2000.0f) / 4000.0f;
tflInterpreter->Invoke();