External vision · Edge AI · Arm
The room becomes
the drone's eyes.
DroneVision turns four fixed camera feeds into a live 3D position that PX4 can fly on—without GPS and without putting perception hardware on the aircraft.
A real control workload—not a microbenchmark¶
DroneVision was built for the Physical AI track of the Arm Create: AI Optimization Challenge 2026. It combines a complete GPS-denied flight loop with a measurement framework that explains where the time goes on real Arm boards.
Four fixed cameras
JPEG frames arrive as synchronized sets. Stale views are rejected instead of silently freezing the output.
Switchable perception
A reliable colour baseline and a learned YOLO26n detector share the same geometry and state-estimation pipeline.
Consensus triangulation
Multi-view DLT rejects gross outliers, degrades gracefully with occlusion and emits an honest 3D estimate.
PX4 external vision
The companion simulator injects the estimate into EKF2 as external vision and keeps the X500 airborne with GPS disabled.
The optimization result¶
The profiler made the decision for us: detection dominates; geometry does not. On the NXP i.MX93, the final submitted Ethos-U65 path reduced the full four-camera YOLO loop from 977.7 ms on FP32 CPU to 159.91 ms per fix—a measured 6.11× end-to-end speed-up. The accelerated path localizes 336 of 337 frame sets, while the model shrinks from 9.31 MB to 2.42 MB.
One codebase, three kinds of proof¶
| Proof | What ships | Why it matters |
|---|---|---|
| Flight proof | Gazebo warehouse, PX4 X500, four cameras and GPS-denied control | Shows that the estimate is useful, not merely numerically plausible |
| Accuracy proof | A 337-set corpus with time-aligned ground truth | Makes every detector and runtime comparison repeatable |
| Hardware proof | Pi 4, Pi 5 and i.MX93 measurements with artifact hashes | Attributes speed-ups to the architecture and runtime actually used |
Current state
The system is functional and measured. After auditing how the marker offset interacts with bundle adjustment, the current nominal factory geometry reproduces 337/337 sets at 6.28 mm mean error. The learned i.MX93 NPU path reaches 6.25 four-camera fixes/s with 100% graph delegation. The hover corpus does not yet establish performance across varied real rooms or closed-loop point-to-point flight. These limits are documented, not hidden.