Bringing Robotics AI to Embedded Platforms: Dataset Recording, VLA Fine‑Tuning, and On‑Device Optimizations

Recent advances in Large Language Models have enabled the transition from text-only reasoning to multimodal systems . First, with the integration of visual perception in Vision–Language Models (VLMs) , and more recently with the generation of robot actions in Vision–Language–Action (VLA) models . Deploying these models on embedded robotic platforms remains a challenge due to tight constraints in terms of compute, memory, and power, as well as real-time control requirements.

In synchronous control pipelines, while the VLA is running inference, the arm is idle awaiting commands leading to oscillatory behavior and delayed corrections. To tackle that, asynchronous Inference can enable smooth and continuous motion by dissociating generation from execution. However, to be effective, the end-to-end inference latency must remain shorter than the action execution duration . This temporal constraint therefore sets an upper limit on the model's throughput.

Bringing VLA models to embedded platforms is not a matter of model compression, but a complex systems engineering problem requiring architectural decomposition , latency-aware scheduling , and hardware-aligned execution . Addressing these challenges is essential to translate recent advances in multimodal foundation models into practical and deployable embedded robotic systems.

The i.MX 95 SoC integrates 6× Arm Cortex‑A55, a Cortex‑M7 and a Cortex M33 MCU, a Mali GPU, a new NXP ISP, and the eIQ® Neutron NPU , targeting efficient, secure edge inference with multi‑camera support and strong I/O. [nxp.com]

This guide presents NXP’s hands‑on best practices for recording reliable robotic datasets, fine‑tuning VLA policies ( ACT and SmolVLA ), and hightlights the real-time performance that NXP i.MX 95 SoC achieves after optimization.

High‑quality, consistent data beats “more but messy” data. This section turns hard‑earned lessons into concrete checklists and schemas.

Moving from scene‑only views to mixed viewpoints increases the global accuracy, but the more cameras you have the more the latency is impacted. Therefore, you must choose right compromise. In our case that balance was reached with 3 cameras: