Telling a robot where. Point at a block and say put that here. deixis works out which block that means and which spot here means, and hands the arm the millimetres — no names, no coordinates.

Three things have to work for that sentence: something to see the table and find what is on it, something to work out which thing and which place were meant, and a body that can move. deixis is the middle one. We wrote each robot’s own code as well, but that part is different for every robot — deixis is the piece all three of our demos share.

In this demo the seeing is switched off. Every object and every position in this virtual world is already known, so no AI model works out what is on the table or where it is. The body is an off-the-shelf SO-101 arm.

The seeing runs for real in our other two demos — a pen plotter and a floor vacuum, each with a camera and NVIDIA Cosmos finding things in the room, and deixis still saying where.

Try saying:

Or a whole job at once:

made-up hand

Loading the arm…

deixis

Pick a block, then a spot, and say put that … here.

Loading the arm…

Your video stays in your browser. Only your fingertip’s position and your voice are sent. What runs this demo

What each part said to the next

One row for every message, in the order it happened. Amber is you, blue the two small models we run ourselves, red Nemotron deciding what to do, green the deixis core measuring where, and purple the arm. Press JSON on a row to see the real message.

1 What the robot heard and saw

2 Then every message between the parts

what it said

Say or type a command: every message between the parts shows up here as it happens.