Industry 4.0 is on the way and real-time artificial intelligence is becoming a key part of industrial automation.

At the GTC 2024 conference, NVIDIA CEO Jensen Huang presented how real-time AI is transforming manufacturing, factory logistics, and robotics.

A simulation-based approach

In manufacturing, factory logistics, and robotics, processes often involve large objects, expensive equipment, and logistically complex facilities. To improve the efficiency of these processes, NVIDIA has developed a simulation-based approach that allows developers to create, test and refine their AI in real time and at scale in a virtual environment before deploying it in industrial infrastructures. This approach reduces development costs and time while improving process safety and reliability.

“AI gyms”

To help robots and humans navigate unpredictable or complex situations, NVIDIA has developed “AI gyms” where developers can train successful AI agents. These gyms use NVIDIA’s Omniverse, Metropolis, Isaac, and cuOpt platforms to create virtual environments where AI agents can learn to interact with their environment. Developers can thus train AI agents capable of responding to the specific challenges of their sector of activity.

A warehouse of, 9000 m² managed by an AI

To illustrate the capabilities of real-time AI in industrial automation, NVIDIA presented an impressive demonstration of a 9,000-square-meter warehouse run entirely by real-time AI. The digital twin (a virtual representation) of the warehouse was created using Omniverse and served as a simulation environment to test and refine the AI. Dozens of digital workers, autonomous mobile robots (AMR) equipped with NVIDIA Isaac multi-sensor stack, vision AI agents and sensors were used to simulate warehouse operations.

Using Metropolis, a centralized occupancy map was created by merging data from 100 simulated camera streams. This map allowed optimal AMR routes to be calculated using cuOpt’s complex routing optimization AI. All of this occurred in real time as Isaac Mission Control harmonized the AMR fleet using cuOpt’s planning and routing data. When an incident blocked the path of an AMR, Metropolis updated the occupancy grid, cuOpt planned a new optimal route, and the AMR responded accordingly to minimize downtime.

Finally, NVIDIA presented how developers can use Metropolis’ vision models to build AI agents that can extract actionable insights from videos. These visual AI agents can help operations answer questions like “What happened in aisle three? “with information from video analysis. This allows industries to extract valuable information from their video data using natural language.