Real-time edge deployment
End-to-end inference on RPi CM4/CM5, Jetson Nano, and Luxonis OAK — power, memory, and performance optimized so the full oxygenation pipeline runs on-device.
Problem
A model that only exists on a workstation does not help a clinician at the bedside. The full pipeline — capture, detection and segmentation, optical-property inversion, oxygenation output — has to run on the device itself, on compute modules with a fraction of the memory and power budget of the machines the models were trained on.
That constraint is unforgiving. There is no room to stream frames to a server, and no thermal or battery headroom to run an unoptimized network. The whole chain from raw capture to clinician-ready output has to fit on the edge and keep up with the operator in real time.
Approach
I took the CV/ML pipelines end-to-end onto embedded targets — Raspberry Pi CM4 and CM5, Jetson Nano, and Luxonis OAK — and optimized them for power, memory, and performance rather than for benchmark accuracy on a desktop GPU. Quantization and edge-specific tuning shrink the models to fit the hardware while holding the outputs the clinical readout depends on.
Getting there meant treating deployment as a first-class engineering problem, not an afterthought: profiling where the time and memory actually go on each target, and reshaping the pipeline so capture-to-output stays inside the real-time budget across a range of modules.
What shipped
The oxygenation pipeline runs fully on-device, from capture to clinician-ready output, across RPi CM4/CM5, Jetson Nano, and Luxonis OAK — the same models the research side produced, made to live inside the power and memory envelope of a handheld device.
Supporting multiple compute modules kept the deployment portable rather than pinned to one board, which matters when the hardware roadmap is still moving.