From language models to embodied AI

Reasoning about a task and performing it are very different problems. A robot has to turn perception, language and planning into smooth physical motion, in real time, with safety guarantees.

That is why we design perception, language and control as one loop rather than separate modules bolted together.

In upcoming posts we will share how we approach data collection, simulation and on-robot evaluation.