Despite the stunning advances in AI in recent years, robots remain stubbornly dumb and limited. Those found in factories and warehouses usually go through precisely choreographed routines without much ability to perceive their surroundings or adapt on the fly. The few industrial robots that can see and grasp objects can only do a limited number of things with minimal dexterity due to a lack of general physical intelligence.
More generally capable robots could take on a much wider range of industrial tasks, perhaps after minimal demonstrations. Robots will also need more general capabilities to deal with the vast variability and messiness of human homes.
The general excitement about AI advances has already turned into optimism for big new leaps in robotics. Elon Musk’s car company, Tesla, is developing a humanoid robot called Optimus, and Musk recently suggested that it will be widely available for $20,000 to $25,000 and able to perform most tasks by 2040.
Previous attempts to teach robots to perform challenging tasks focused on training a single machine for a single task, as the learning appeared to be non-transferable. Some recent academic research shows that with sufficient scale and fine-tuning, learning can be transferred between different tasks and robots. A 2023 Google project called Open X-Embodiment involved sharing robot learning between 22 different robots in 21 different research labs.
A key challenge with the strategy that Physical Intelligence is pursuing is that there is not the same scale of robot data available for training as there is for large text-based language models. So the company has to generate its own data and come up with techniques to improve learning from a more limited data set. To develop π0, the company combined so-called visual language models, which are trained on images as well as text, with diffusion modeling, a technique borrowed from AI image generation, to enable a more general kind of learning.
For robots to take on any robotic job a human asks them to do, such training would need to increase significantly. “There’s still a long way to go, but we have what you can think of as a scaffold that illustrates things to come,” says Levine.
