Humanoid Robotics Technology
NVIDIA announces NVIDIA Isaac GR00T N1.5, the first update to NVIDIA’s open, generalized, fully customizable foundation model for humanoid reasoning and skills; NVIDIA Isaac GR00T-Dreams, a blueprint for generating synthetic motion data; and NVIDIA Blackwell systems to accelerate humanoid robot development.
Humanoid and robotics developers Agility Robotics, Boston Dynamics, Fourier, Foxlink, Galbot, Mentee Robotics, NEURA Robotics, General Robotics, Skild AI and XPENG Robotics are adopting NVIDIA Isaac™ platform technologies to advance humanoid robot development and deployment.
“Physical AI and robotics will bring about the next industrial revolution,” said Jensen Huang, founder and CEO of NVIDIA. “From AI brains for robots to simulated worlds to practice in or AI supercomputers for training foundation models, NVIDIA provides building blocks for every stage of the robotics development journey.”
New Isaac GR00T Data Generation Blueprint Closes the Data Gap
Showcased in Huang’s COMPUTEX keynote address, NVIDIA Isaac GR00T-Dreams is a blueprint that helps generate vast amounts of synthetic motion data — aka neural trajectories — that physical AI developers can use to teach robots new behaviors, including how to adapt to changing environments.
Developers can first post-train Cosmos Predict world foundation models (WFMs) for their robot. Then, using a single image as the input, GR00T-Dreams generates videos of the robot performing new tasks in new environments. The blueprint then extracts action tokens — compressed, digestible pieces of data — that are used to teach robots how to perform these new tasks.
The GR00T-Dreams blueprint complements the Isaac GR00T-Mimic blueprint, which was released at the NVIDIA GTC conference in March. While GR00T-Mimic uses the NVIDIA Omniverse™ and NVIDIA Cosmos™ platforms to augment existing data, GR00T-Dreams uses Cosmos to generate entirely new data.
New Isaac GR00T Models Advance Humanoid Robot Development
NVIDIA Research used the GR00T-Dreams blueprint to generate synthetic training data to develop GR00T N1.5 — an update to GR00T N1 — in just 36 hours, compared with what would have taken nearly three months of manual human data collection.
GR00T N1.5 can better adapt to new environments and workspace configurations, as well as recognize objects through user instructions. This update significantly improves the model’s success rate for common material handling and manufacturing tasks like sorting or putting away objects.
Early adopters of GR00T N models include AeiRobot, Foxlink, Lightwheel and NEURA Robotics. AeiRobot employs the models to enable ALICE4 to understand natural language instructions and execute complex pick-and-place workflows in industrial settings. Foxlink Group is using them to improve industrial robot manipulator flexibility and efficiency, while Lightwheel is harnessing them to validate synthetic data for faster humanoid robot deployment in factories. NEURA Robotics is evaluating the models to accelerate its development of household automation systems.
New Robot Simulation and Data Generation Frameworks Accelerate Training Pipelines
Developing highly skilled humanoid robots requires a massive amount of diverse data, which is costly to capture and process. Robots need to be tested in the physical world, which can present costs and risk.
To help close the data and testing gap, NVIDIA unveiled the following simulation technologies:
- NVIDIA Cosmos Reason, a new WFM that uses chain-of-thought reasoning to help curate accurate, higher-quality synthetic data for physical AI model training, is now available on Hugging Face.
- Cosmos Predict 2, used in GR00T-Dreams, is coming soon to Hugging Face, featuring performance enhancements for high-quality world generation and reduced hallucination.
- NVIDIA Isaac GR00T-Mimic, a blueprint for generating exponentially large quantities of synthetic motion trajectories for robot manipulation, using just a few human demonstrations.
- Open-Source Physical AI Dataset, which now includes 24,000 high-quality humanoid robot motion trajectories used to develop GR00T N models.
- NVIDIA Isaac Sim™ 5.0, a simulation and synthetic data generation framework, will soon be openly available on GitHub.
- NVIDIA Isaac Lab 2.2, an open-source robot learning framework, which will support new evaluation environments to help developers test GR00T N models.
Foxconn and Foxlink are using the GR00T-Mimic blueprint for synthetic motion manipulation generation to accelerate their robotics training pipelines. Agility Robotics, Boston Dynamics, Fourier, Mentee Robotics, NEURA Robotics and XPENG Robotics are simulating and training their humanoid robots using NVIDIA Isaac Sim and Isaac Lab. Skild AI is using the simulation frameworks to develop general robot intelligence, and General Robotics is integrating them into its robot intelligence platform.
Source: NVIDIA
