When we built digital AI systems, we chose the right dataset by only using some parts of the internet. Chatbots were built using mostly social media data; Coding Agents were built using mostly open source code data.
Physical AI also requires special data for real world use cases. The world is extremely complex, and the sensors that “see” it can also be complex. But, this time, we can’t just look on the Internet to “find” it. The sensors we use, the environments in which robots move, and the special situations we have to prepare for have simply not been recorded.
Unlike digital AI, physical AI lacks a rich historical dataset like the Internet. There’s no ready made data set for how humans move through environments, how objects behave under force, or how tasks are physically completed. All the physical data must be created, not just collected.
The same generative tools that you use in digital AI can be applied within a simulated world, using real-world, grounded physics, and the physical AI hardware (robots and sensors) can be built alongside the AI. We don’t always have to wait for a robot to go out into the world, record the data, and bring it back to us – the robot isn’t ready yet! Companies like NVIDIA, Tesla, and Amazon are accelerating this by combining compute power, simulation, and real-world data collection.
At Torc, our autonomous driving system, TorcDrive, is being trained to work in the real world using both real-world data (recorded camera and lidar images of on-road driving) as well as complex simulated images created from those same on-road recordings, simultaneously.
