AI-powered robots get to do VR training in the virtual world of NVIDIA's Omniverse, robot trains on "digital twin" then applies training in real world

[ggx2ac]

You are probably going to think of VR training from the Metal Gear Solid series with what these robots are doing.

Sources -
Omniverse (NVIDIA Omniverse™ is a platform of APIs, SDKs, and services that enable developers to easily integrate Universal Scene Description (OpenUSD) and RTX rendering technologies into existing software tools and simulation workflows for building AI systems.): https://www.nvidia.com/en-au/omniverse/

Project GR00T (GR00T is a general-purpose foundation model that promises to transform humanoid robot learning in simulation and the real world): https://developer.nvidia.com/project-GR00T

Isaac Perceptor (NVIDIA Isaac™ Perceptor is a collection of hardware-accelerated packages for visual AI, tailored for Autonomous Mobile Robot (AMR) to perceive, localize, and operate robustly in unstructured environments.): https://developer.nvidia.com/isaac/perceptor

Isaac Sim (NVIDIA Isaac Sim™ is an extensible robotics simulation platform that gives you a faster, better way to design, test, and train AI-based robots. It’s powered by Omniverse™ to deliver scalable, photorealistic, and physically accurate virtual environments for building high-fidelity simulations.): https://developer.nvidia.com/isaac-sim

Omniverse cloud API press release: https://nvidianews.nvidia.com/news/omniverse-cloud-apis-industrial-digital-twin
Digital twins warehouse blog: https://blogs.nvidia.com/blog/ai-digital-twins-industrial-automation-demo/

NVIDIA had their GTC (GPU Technology conference) very recently, the 2-hour keynote video has over 7.5M views currently.

At the 1:30:00 mark of the video, Jensen Huang talks about the Omniverse and robotics. One of the uses that NVIDIA found for the Omniverse is creating simulations (with physics) which are then applied in the real world hence their term of creating a "digital twin", they claim this saves on costs and planning, etc. Obviously, this isn't entirely new to most engineers when they design things virtually before doing manufacturing.

Let's skip to the robotics part, here are a few short videos that were in that keynote (except for the video from 2022).

Here's their promotional video titled, "NVIDIA Robotics: A Journey From AVs to Humanoids":



Video title: "Fusing Real-Time AI With Digital Twins"
Video description:

Discover the AI that'll drive the next phase of industrial automation—how it'll be developed, refined, and first deployed in simulation in digital twins. Complex AI is being tested in real time inside an Omniverse digital twin of a warehouse, showcasing AI that's been developed inside this digital twin. It’s a workflow that developers can use to build AI gyms to train and evaluate complex AI, all in real time within the digital twin. This is something that otherwise would be incredibly costly or impossible to run in the real world—particularly for heavy industry, factories, and supply chains. This demo leverages NVIDIA Metropolis, Omniverse, CuOpt, and Isaac for robot perception to create an end-to-end concept of how to fully automate logistically complex co-bot spaces.

Video title: "NVIDIA Isaac Perceptor 3D Surround Vision"
Video description:

NVIDIA Isaac Perceptor, optimized on Jetson Orin, uses multiple cameras for 3D surround perception to detect obstacles like low-lying hazards or overhangs, which are invisible to standard 2D lidar.

Using robust AI-based depth estimation, GPU-Accelerated 3D reconstruction, and semantic segmentation, the mobile robot can work more safely alongside humans.

Key Points:

- Isaac Perceptor is a live multi-camera, surround visual perception running on Jetson.

- Isaac Perceptor is ‌GPU-accelerated and optimized on Orin, leaving headroom for adding other SW such as navigation stack.

- The traditional approaches have predominantly utilized 2D lidars, which offer limited functionality, or 3D lidars, known for their prohibitive costs. Isaac Perceptor revolutionizes this by offering an affordable, camera-based solution that doesn't compromise on capability. Also, brings visual AI semantics for autonomy.

Here's a video from March 2022 of Nvidia using their Omniverse platform and Isaac Sim to train robots in VR and then applying that in the real world titled, "Narrowing the Sim2Real Gap with NVIDIA Isaac Sim":





That's all the related videos I could find, now robots will get more skills and training then you ever will in your lifetime. In the future, companies can finally stop posting entry-level jobs that require a minimum of five years of experience. Instead, they'll train robots on the Omniverse in a simulated environment of the workplace and then they get the robot to do those entry-level jobs.

I'm being facetious since I already made a thread talking about the estimated costs of manufacturing a humanoid robot: https://www.installbaseforum.com/fo...k-per-unit-lets-look-at-the-competition.2505/

More realistically, what I could see happening is a company that sells services where they go to your workplace, scan the environment, put it on the Omniverse, find out what kind of robot you need and what it needs to be trained on. You as a business owner won't have to know how to get this all to work since you pay someone else to provide it. Then you just replace all the fleshy humans with your new AI-powered overlords.

 

[Sky2High]

I figured blue collar jobs would have a little more time (at least a decade) over white collar workers, probably not with the rate of AI progression. On the fully digital side, we have AI autonomous agents roaming the internet like the software developer Devin. I'm excited for AI tech, but golly, I still want people to be able to put bread on the table and not have to rely on charity or the government to do so.

So I suppose it's still a good time to buy Nvidia stocks?

 

[ggx2ac]

I figured blue collar jobs would have a little more time (at least a decade) over white collar workers, probably not with the rate of AI progression.

As mentioned in my other thread from the OP, the estimated cost of manufacturing a humanoid robot is $30k to $150k that doesn't include how much profit the company wants and the costs for training your robot on the Omniverse, etc.

Only the biggest companies are expected to use this at the beginning. If manufacturers don't figure out a way to produce humanoid robots with cheaper materials, then I don't expect costs to go down that easily.

I'd rather have a humanoid robot made of less dense materials.

If you have two humanoid robots with the exact same dimensions and volume, one made of iron will be much heavier than one made of carbon due to density.

I don't expect them to make robots out of carbon nanotubes but that's the future tech where you have carbon that is as strong as steel but much lighter. What does that do for robots? It saves on battery performance because less power is needed to move their body.

Anyway, the point is that humanoid robots have a long way to go before it changes the job market because they're not even in mass production now.

Goldman Sachs estimates: https://www.goldmansachs.com/intell...or-robots-could-reach-38-billion-by-2035.html

The team’s base case is for more than 250,000 humanoid robot shipments in 2030, almost all of which would be for industrial use. Our analysts’ base case is for consumer robot sales to ramp up quickly over the next decade, exceeding a million units annually in just over a decade.

And they have a graph where their base case says 1.4M units shipped in 2035, bull case = 6.5M units in 2035, "blue sky" = 11.6M units in 2035. Even with the highest prediction, that's possibly 11.6M people without a job globally by 2035, but we don't know.

So I suppose it's still a good time to buy Nvidia stocks?

The best time to buy is always in hindsight. You first have to find out what the share price is in the future, then you build a time machine and go to the past to purchase the shares. Anyway, that's a joke. I have no real advice on stocks.

line break

I want to point out the insane view rate on the GTC March 2024 Keynote video which I mentioned in the OP, when I made the thread, it was over 7.5M views. It is currently over 14.74M views.

To understand why it's so high and how it compares to other videos, take a look at this: https://socialblade.com/youtube/user/nvidia/videos/mostviewed

Date	Video Title	Views
2023-03-21	GTC 2023 Keynote with NVIDIA CEO Jensen Huang	22.5M
2022-09-20	GTC Sept 2022 Keynote with NVIDIA CEO Jensen Huang	19.3M
2024-03-18	GTC March 2024 Keynote with NVIDIA CEO Jensen Huang	14.7M
2022-03-22	GTC 2022 Spring Keynote with NVIDIA CEO Jensen Huang	10.4M
2022-08-09	NVIDIA Special Address at SIGGRAPH 2022	8.0M
2023-01-03	NVIDIA Special Address at CES 2023	5.6M
2009-12-04	Mythbusters Demo GPU versus CPU	5.3M
2022-01-04	NVIDIA CES 2022 Special Address	3.4M
2022-07-01	The Future of AI-Accelerated Industrial Automation with Siemens and NVIDIA	3.3M
2021-04-12	GTC Spring 2021 Keynote with NVIDIA CEO Jensen Huang	2.3M

Can you see that? The GTC 2021 Spring Keynote has only 2.3M views. Then the GTC 2022 Spring Keynote jumped to 10.4M views.

They still talked about AI in 2021, but the views really jumped in 2022. It is only very recent that NVIDIA is now... this popular.

Their channel only has 1.37M subs but the views from their keynote videos are very high.

 

[ggx2ac]

I want to point out the insane view rate on the GTC March 2024 Keynote video which I mentioned in the OP, when I made the thread, it was over 7.5M views. It is currently over 14.74M views.

To understand why it's so high and how it compares to other videos, take a look at this: https://socialblade.com/youtube/user/nvidia/videos/mostviewed

Date	Video Title	Views
2023-03-21	GTC 2023 Keynote with NVIDIA CEO Jensen Huang	22.5M
2022-09-20	GTC Sept 2022 Keynote with NVIDIA CEO Jensen Huang	19.3M
2024-03-18	GTC March 2024 Keynote with NVIDIA CEO Jensen Huang	14.7M
2022-03-22	GTC 2022 Spring Keynote with NVIDIA CEO Jensen Huang	10.4M
2022-08-09	NVIDIA Special Address at SIGGRAPH 2022	8.0M
2023-01-03	NVIDIA Special Address at CES 2023	5.6M
2009-12-04	Mythbusters Demo GPU versus CPU	5.3M
2022-01-04	NVIDIA CES 2022 Special Address	3.4M
2022-07-01	The Future of AI-Accelerated Industrial Automation with Siemens and NVIDIA	3.3M
2021-04-12	GTC Spring 2021 Keynote with NVIDIA CEO Jensen Huang	2.3M

Can you see that? The GTC 2021 Spring Keynote has only 2.3M views. Then the GTC 2022 Spring Keynote jumped to 10.4M views.

They still talked about AI in 2021, but the views really jumped in 2022. It is only very recent that NVIDIA is now... this popular.

Their channel only has 1.37M subs but the views from their keynote videos are very high.

The GTC March 2024 Keynote video is now at 21.46M views. The video has only been up for a few days by the way, it won't be long before it's the most watched video on the channel.

But in more important news, it's the year 2035, your boss calls you into his office, he tells you, "If you can do what my new robot does, then you can keep your job", you accept the challenge for some reason.

Your boss brings out the robot and it does this:



That was the video titled, "Unitree H1 The World's First Full-size Motor Drive Humanoid Robot Flips on Ground"

From the video description

Unitree H1
Deep Reinforcement Learning
In-place Flipping Parameters:
Weight: about 50kg
Height: about 1.8m
Actuator: electric motor

The point they emphasize is that this is a robot that can backflip without hydraulics unlike others.

Which they're most likely referring to this video:

 

[Kenka]

The GTC March 2024 Keynote video is now at 21.46M views. The video has only been up for a few days by the way, it won't be long before it's the most watched video on the channel.

But in more important news, it's the year 2035, your boss calls you into his office, he tells you, "If you can do what my new robot does, then you can keep your job", you accept the challenge for some reason.

Your boss brings out the robot and it does this:



That was the video titled, "Unitree H1 The World's First Full-size Motor Drive Humanoid Robot Flips on Ground"

From the video description


The point they emphasize is that this is a robot that can backflip without hydraulics unlike others.

Which they're most likely referring to this video:

I think you meant 2026.

 

[Bernd Lauert]

I expect the Nvidia GTC keynote in 2027 to be held by Huang's digital twin.

 

[ggx2ac]

The GTC March 2024 Keynote video is now at 21.46M views. The video has only been up for a few days by the way, it won't be long before it's the most watched video on the channel.

25.46M views currently, now it is the most viewed video on their channel. However, you would have noticed from my posts that the updates were daily and went up by 7M views each time, now it is slowing down to approximately 4M views.

Here's a video from October 2023 on the CNBC International YouTube channel titled, "Factories are heading for a ‘dark’ future — and it’s not what you think":


In that video, it mentions the concept of a "dark factory", what it means is that the factory will not have systems that humans need to work in. If the factory is completely automated, then you don't need lights, you don't need heating systems (during winter), you don't need a cooling system according to the video, those robots also won't need toilet breaks... however, there's still the matter that if humanoid robots are used, we know they are not stationary and have to charge their 5 to 6-hour battery when it runs low, I have no clue if they have fast charging.

Some companies have already implemented dark factories as early as the beginning of the 21st century according to the video. Now, there's a couple of problems with this video, they interviewed some suit from Accenture and the suit told the interviewer that:

"Factories in the future will be much more automated than they currently are, if not autonomous. Our manufacturing needs to get much more productive; to actually make the promise of net zero and sustainable practices work, we need to save on costs."

Notice what he's saying? Factories need to be more productive but also save on costs and, fulfill net zero.

Net zero sounds more like net zero humans, not net zero emissions because, firing all the fleshy humans and replacing them with robots that can work 24/7 with no lights on sounds like the solution to more productivity and less costs.

Later in the video, the narrator mentions that the World Economic Forum in 2023 said that over the next five years, 83 million jobs will be lost to automation and 69 million jobs will be added over that period of time, a net loss of 14 million jobs.

During the video, the interviewer had also interviewed the CEO of Wootzano (they make fruit-packing robots). The part I want to highlight comes at the end of the video where the CEO said this:

"Those roles with effectively low paid, in relatively insecure jobs, taking those jobs and actually taking them to be more secure and more skillful means higher quality of life, and much better performance and happiness of the workers themselves."

What he says is nothing new, because I responded to something that was more direct in the other humanoid robotics thread I mentioned in the OP, search for "Will EVE replace human labor?" in that thread if you want to read what I wrote in response to the answer to their own question because it's detailed.

You can already tell what's wrong in the quote by that CEO. There's a reason that everyone in the world can't get six-figure jobs, the top comment in the video summarized it well:

Basically how to replace 200 "unhappy" workers with 2 "happy" workers