58.35 ± 0.14 FPS
Average throughput, within 3% of the 60 FPS target in every run.
📺 FLEX web interface with a skeleton overlay, live FPS and exercise feedback
FLEX is a virtual personal trainer with FPGA-accelerated pose estimation on an AMD Kria KV260. It analyses squat posture from a camera feed and uses a remote language model to generate feedback. I built it for the HPPS course at PoliMi and presented it at NECST Lab in July 2026.
In testing, the vision pipeline ran at 58 FPS with board power consumption of about 6 W. Only posture labels are sent to the remote language model.
Source code
📺 FLEX web interface with a skeleton overlay, live FPS and exercise feedback
I wanted to build a camera-based exercise coach that could analyse posture in real time on the KV260. The user wouldn't need wearable sensors, and the video processing would run locally.
The main constraint was fitting person detection, pose estimation and posture analysis into the board's compute and power budget. I kept those tasks on the FPGA and ARM cores, with language generation on a separate server.
The board runs person detection, pose estimation and posture analysis. A remote server receives the posture labels and generates coaching feedback. A browser displays the annotated video stream and reads the feedback aloud.
Keeping the language model on a separate server leaves the board's resources available for the real-time vision pipeline.
📺 System architecture: KV260, remote language model server and browser interface
The KV260 is a Zynq UltraScale+ board with a quad-core ARM Cortex processor and an FPGA. Both neural networks run on AMD's Deep Learning Processor Unit in the FPGA. The ARM cores handle the application logic, video I/O and communication.
I structured the application around an event loop capped at 60 Hz, with separate worker threads for capture, rendering and language model requests. These workers exchange data with the loop without blocking it.
The capture worker reads from a V4L2 camera or a video file and writes the latest frame to a single-slot buffer. The loop uses the most recent available frame, so it can continue processing even when capture is slower.
YOLOv5-nano detects the person in a 640×640 letterboxed frame. MoveNet INT8 takes a 192×192 crop of the bounding box and returns 17 keypoints in COCO format. An exponential moving average smooths the bounding box between detections.
The render worker draws the skeleton and status overlay with OpenCV, encodes the frame as JPEG and serves it to the browser over MJPEG. Rendering runs independently of the main loop.
Motion analysis converts the keypoints into labels for squat depth, shoulder symmetry and stance width. The remote server receives these labels; video frames and keypoint coordinates stay out of the language model requests.
A Python bridge forwards requests to Gemma 3 1B running through Ollama. The board connects through a reverse SSH tunnel and accesses the model as a local HTTP endpoint on port 8080.
I set the request interval to 7 seconds to allow time for the model response and the browser's spoken feedback.
I wrote the event loop, threading, synchronization and data flow in Zig. The application uses OpenCV and GStreamer for video, and Vitis AI and VART to load the models and run inference on the DPU.
C++ bridges expose these libraries through Zig's C interface. This keeps library-specific code separate from the application logic and compiles the components into a single binary.
I tested the pipeline over ten 30-minute runs, with 5 minutes of cooldown between runs. Each run used the same looped 90-second squat video to keep the input consistent. In total, the pipeline processed 1,050,295 frames over five hours.
Average throughput, within 3% of the 60 FPS target in every run.
Median latency from frame availability to posture analysis. The 99th percentile is 20.71 ms.
About 6.03 W for the whole board, with a peak temperature of 44.34 °C.
MoveNet takes 13.56 ms per frame, roughly 79% of the main loop's processing time. Capture takes 1.67 ms, and handing a frame to the renderer takes 1.81 ms.
The median loop latency of 17.30 ms is slightly above the 16.67 ms budget for 60 FPS, giving a measured throughput of about 58 FPS.
In this benchmark, YOLO runs once on the first frame and takes about 50 ms. The subject stays within the initial bounding box throughout the squat video. This initial detection accounts for the worst-case latency of 66–69 ms and has little effect on average throughput.
Rendering takes about 32 ms per frame on its worker thread. The main loop only spends 1.81 ms handing the frame over, so it can continue analysing posture while rendering finishes.
FLEX currently supports squats. Exercises where the subject moves outside the initial bounding box would require more frequent person detection.
The evaluation covers throughput, latency and power consumption. Coaching accuracy has not been evaluated against feedback from a trainer.
The source code is on GitHub 
.
The project report includes the full measurements, plots and references.