Startups & product teams
Bring a shipping model and see where your product actually stands against the field – an objective, third-party read on identical tasks, not a self-reported number.
- Submit a shipping model as an endpoint or container
- Benchmark against baselines and peers on identical tasks
- Advance to the real-robot final on merit
Labs & individual researchers
A common yardstick does the most here – identical tasks let otherwise incomparable submissions rank directly against each other.
- Enter a lab model or a fine-tune of a public checkpoint
- Iterate against a live public leaderboard through the qualifier
- Finalists run on real Franka + Robotiq hardware in November
What you submit
Your model runs as a server that speaks one open protocol. Our simulator – and, in the final, the real robot – connect to it as a client: they stream observations to your model and execute the action trajectory it returns. One interface, every target.
Hosted inference endpoint
A webserver that exposes the WebSocket endpoint. Well suited to models that load fast and serve over the wire.
Docker container
Package your model with its dependencies. Handy for heavy models – it streams a loading status so the connection never times out while weights load.
One open WebSocket protocol, msgpack with numpy. The client streams an observation each timestep – camera images, robot state, the language task – and your model replies with a short action trajectory. Use the keys your model needs; ignore the rest.
Standardized manipulation tasks
The qualifier runs across two independent simulators. The real-robot final broadens onto DROID and PhAIL pick-and-place on Franka + Robotiq hardware.
Qualifier · simulation
Real-robot final · Franka + Robotiq hardware
How it stays un-gameable
- 1The public board runs through the summer as teams iterate – seeded from day one with baseline models.
- 2Finalists are selected on a held-out evaluation set – unseen initial states, scene variations and task variants – scored once when the qualifier closes.
- 3Models are frozen one month before the final deadline.
- 4The final's decisive ranking is scored on held-out conditions the teams never saw.
- 5Baselines run in both phases – the bar to beat, and a check that the field is really improving.
Concept preview – registration and exact dates are set with Nebius when the competition is announced.