The Race For AI Robots Just Got Real (OpenAI, NVIDIA and more)

Fascinating, crazy, maybe a little scary. I mean, it sounds like we're talking about not

just

using something to help humans do their jobs, but replacing it entirely. Could this be a new normal on a large scale? Hello, welcome to another episode of Cold. Fusion, this scene is from the popular American animated comedy Jetson. This episode titled Rosie the Robot first aired on September 23, 1962. The main character in the scene is introduced to a few variations of a robot, one that responds to commands, will do housework, and basically work. as a personal assistant 61 years later and we are one step closer to seeing that

real

ity this week, a tech startup called Figure launched a humanoid robot called figure1 with which you can have conversations thanks to its integration with open AI.

Now at this point, you can think that we have already seen

robots

like this over the years, whether from Boston Dynamics or Tesla, how is this different? Well, let's first take a look at the robot demo before we get into the weeds. Please note that this demo is claimed by the company to be unaltered and in

real

time, but for the sake of brevity, I have shortened some parts, so let's see okay, can I eat something? Sure, great, can you explain why you did what you

just

did while choosing? I put this trash on top, so I gave you the apple because it's the only edible item I can give you from the table, so based on the current scene, where do you think the plates in front of you go after the plates on the table? like that, the plate and cup will likely go on the drying rack next.

Great, can you put them there of course after seeing how well it worked for those tasks and human style conversation? Some of you will be impressed, some excited, and some even a little. Scared according to the company, this robot is not being operated by anyone, it is capable of carrying out full conversations with people and responding to external stimuli and that is one of the main differences between the figure 01 and the other Android

robots

we have seen . So far, but let's back up for a second, how did a startup founded in 2022 achieve such a Headway? Who is supporting the figure and what are their intentions, given what Nvidia just announced last week?

Are we moving towards an aid-driven robotics

race

and, most importantly? How do these advances in robotics and artificial intelligence impact all of us? We'll take a look at all of that and have a rundown of the latest robots in this episode, so sit back, relax at your docking station as we discover the future of robotics. are you watching cold fusion television it all started in the wake of Co at this time robotics sales began to increase reaching a record in 2022 as the workforce retired these robots were the type used in warehouses and assembly lines the exact type of predictable environment that was the low-hanging fruit for training and deploying robots and in this context is Figure AI founded in 2022.

Figure AI Inc is a California-based tech startup that focuses on humanoid robots of general use. In just 1 year since its inception, the company raised $70 million. but now they have recently raised a whopping 675 million and now have a valuation of 2.6 billion officially a unicorn and you know it's a serious business when the likes of Microsoft, Nvidia, Jeff Bezos, Intel capital and many others have made it their mission to the figure of investors according to their website. is, to quote, expanding human capabilities through advanced AI, it aims to do so by specifically targeting the workforce, based on the figure that the workforce is shrinking.

They mentioned that there are 10 million unfilled jobs in the United States, 7 million of those unfilled positions are for essential roles like warehouse transportation and retail and there are only 6 million people available to fill these vacant positions, while suppliers warehouse key predict they will run out of people to hire by 2024. Quote as we reach the upper limits of our production capacity humanoids will join. the workforce with the ability to think, learn and interact with their environment safely alongside us. figur founder Brett Adcock is no stranger to the tech industry. He previously founded two other companies, vety and Archer Aviation veter, is an online talent marketplace and in 2018 they were acquired for 100 million Archer Aviation are now publicly traded and are working on building electric vertical take-off and landing aircraft, for which, with his experience, Adcock has created a team of industry veterans, including professionals who have worked at Boston Dynamics, Google Deep Mind and Tesla, perhaps his greatest breakthrough. has come in the form of its partnership with open AI In a February 2024 press release, Figer announced its collaboration and will help accelerate Figure's business timeline by enhancing its robots' capabilities to process and reason from natural language. using open AI research and large multimodal models in simple terms, they allow you to have full conversations with the robot and I suspect it will also improve computer vision, so ultimately the idea is to simply build autonomous humanoid robots to address the global labor shortage and do just that.

The first robot r is here, let's start by seeing how the robot actually works. First, every action and behavior in the video was learned and not teleoperated. The robot is equipped with cameras and microphones, so when someone speaks and asks it to do something, the transcribed texts and images from the cameras are captured in a large multimodal model that is trained by open AI to understand both the text and the images. images. The model processes the entire conversation, including previously shared images. Then you decide which specifically learned Clos Loop action to take by selecting and By loading the relevant neural network configurations to the GPU and then executing them, the mechanical action in Figure 01 is fast at 200HZ, which basically means that after knowing what to do, It sends instructions 200 times per second, that is, 5 milliseconds of reaction time for all those micro adjustments.

Humans are on the order of 250 milliseconds and, as a curious fact, the reaction time of a cat is around 30 milliseconds, for which are essentially using a combination of different neural networks to classify audio and visual inputs, understand them, and then take appropriate actions. It's actually amazing, how do you think you did it? I think I did pretty well, the apple found its new owner, the trash is gone and the clothes on the table are right where they belong. Did you notice the hesitation when speaking? They are the small human imperfections. that allows us as viewers to break the illusion that it is an innate robot we have already seen it in the GPT voice chat hey GPT chat can you describe a nice kitchen scene but make it very informal and breathe and say um and things that sound ? very natural while you are describing something for sure, imagine this, you are in this cozy kitchen, just the sun is peeking through the curtains giving that warm glow, you can almost smell the aroma of freshly brewed coffee, um, there is this soft hum from the refrigerator in the background.

It's kind of calming, you know, and maybe there's a little bit of chatting while someone is flipping pancakes on the stove, it's just this relaxed atmosphere, like time taking a chill pill to see it in a real robot, just It was the next frontier, as mentioned in one episode. I did it at the end of 2021 and talked about how it was only a matter of time before a company cracked the code of mixing dexterity, mobility and natural language in a single robot and 3 years later I think we're getting close, but all that . He said that, as you can see, there is still some latency between speaking and getting a response and I think that is mainly due to the open AI side, although we don't know the intricate details of the integration.

The expressed instructions probably still have to go to a server and return. I've noticed the same lag issue when playing with GPT voice chat, so there's still work to be done if these bots are going to be used in large-scale commercial applications. Imagine an emergency response robot standing there for 10 seconds while someone has a heart attack others just aren't impressed months ago the figure showed a demo of the robot making coffee some people back then asked if robots were also learning bad habits by From the training data the robot in the figure is not the best at walking either, but we can.

Don't be too harsh, it's still an early stage product as I always say criticizing is easy but building is difficult with such an incredible investment and very few demo videos to show, some have even made a comparison with a certain Trevor Milton, if you wish. If you're a regular viewer of cold fusion, you'll know what this is about, but it's obviously too early for that conversation. Generally speaking, it's an exciting time for the robotics industry, so the playing field for figure AI won't be barren by any means. means let's take a quick look at the latest in the industry, first up to this digit which is made by Agility Robotics and backed by Amazon.

Until now, robots could only replace part of a human job, enough to make coffee or an omelet, but this is a robot. Its manufacturer says it's ready to do what human warehouse workers do and can lift about 35 pounds. You know that the robot weighs approximately what a person would weigh. It can reach about the size of what a person can reach. You know we're really going. For those kinds of regulations and OSHA requirements for robot agility, robotics is funded in part by Amazon, which is investing a billion dollars in new industrial technologies, the fact that it can enter places that were designed for humans and solve those challenges that exist without having to rebuild the environment for automation and it doesn't stop and it doesn't stop, there's about $250,000, so they're not cheap, but you know in a few years they're worth it, digital robots They will be deployed in Amazon warehouses and the company aims to produce 10,000 units per year, so Jake is crazy fascinating, maybe a little scary.

I mean, it seems like we're talking about not just using something to help humans do their jobs, but replacing it entirely. Could this be a new normal on a large scale, I mean, this is the right thing to do as a technology correspondent for years. I've had companies tell me that you know well that this technology is to enhance a human being or make a human being's job a little bit easier, in this case. It is really suitable to replace a human worker. Amazon warehouse workers. You know it's one of the largest employers in the United States.

People really look for that job. So you know we're really, I think, at this kind of historical turning point where all of a sudden. It's no longer the case of, you know, when could technology replace a set of Javas, but this is really something that is designed to do the following? Abtronic is designed to move products in a warehouse and can run for 22 hours a day, all you need. What you have to do is change the battery every few hours. Deliveries will begin in 20125. The company is also working with NASA on a robot called Valkyrie that aims to work in hazardous environments and perhaps even on future spacewalks.

Toyota and Hunde are also currently working on robots of their own, but I would say less serious than the others compared to other robots, as mentioned above. The key difference between one is the ability to converse directly in natural language to perform tasks for you. It's not a new concept for about a year now. Back in Cold Fusion we covered a Google project that did the same thing at the time we saw that the concept was still in its infancy and the robot was quite slow earlier this year here we saw Elon Musk sharing a video of Tesla's Optimus robot folding some clothing at first glance was impressive, but the curtain fell later when Elon added that Optimus couldn't do it autonomously, but since then there have been images of Tesla's robot carrying boxes, watering plants, walking crouching, manipulating delicate objects like eggs and performing others Interestingly, these abilities come from the knowledge and benefits of Tesla's self-driving efforts, which is pretty cool, but as far as we know, it lacks that natural language aspect, perhaps Boston Dynamics' Atlas robot be the most agile robot we have ever known.

We've seen it organize objects and even perform backflips, but are this and the robot in Figure 1 really the same thing? The Boston Dynamics Atlas is definitely

more

focused on mobility, while the figure seems to be suitable for complex tasks,there is

more

focus on finger dexterity. alluded to by the five fingers with three joints and for factory deployment this might make more sense in many cases assembly line work is done by sitting or being stationary in a specific location for long periods of time and as we were working on this video Nvidia announced Project Groot, a general-purpose fundamental model for humanoid robots that is intended to help humanoid robots understand natural language and emulate movements by observing human actions, quickly learning coordination skills and other navigation skills , adapt and interact with reality. world about the same size the intersection of the computer Graphics physics artificial intelligence everything came up at this time well I think we have some special guests we little Jetsons robotics computers inside learn to walk in Isaac Sim let's go five things where vas I'm sitting here, the system eliminates some of the legwork of training robots in the real world through simulation, the lessons and skills learned can be put into a knowledge base and transferred to other parts of Robots and the video wants the Groot machines They can understand humans. sense of language and navigate the world around them, uh, and perform arbitrary tasks, the training part of the stack starts with a video so that the model can understand the physics intuitively and then accurately model the robots in the virtual space to teach the model how to move. and interact with terrain objects or other robots, Jensen described this as a gym, here the model can learn the basics much faster than in the real world, as we covered in another episode about the history and size of Nvidia, which is already attracting attention with its shares growing at an unprecedented rate. levels this year, I'll leave a link to that episode in the description and the company also mentioned that they're working with Boston Dynamics agility Robotics abtronic and of course Ai figure and just thinking about this I can already see an industry standard.

Open AI training provides the fundamental models. Nvidia provides some of the processing hardware and software packages and eventually there will be a large number of different robot bodies from different vendors. So why is all this happening now? Well, the reason why a lot of these things happened. What didn't work in previous decades was because instructions for robots had to be laboriously coded by hand, now with generative AI and the latest generations of robot models they are starting to be able to dynamically understand their environment and adjust seamlessly without problems. encode the amount of change. we will see it in the next 5 years 10 years will eclipse everything that happened in the last 30 we went from automating pencil and paper directly with spreadsheets to connecting PCS to the loc network to connecting networks to get Internet and get the network effect of communicating to people globally, but now we are introducing artificial intelligence, you know, artificial intelligence, machine learning, deep learning, neural networks, etc., and what that allows is the automation of automation, so the people who were writing software, particularly at the lower end, unless you're doing these advanced things, they're gone, the people who are writing are doing well, it's just a math programmer in the past, so that when I was writing code, it was the algorithms that were if this or that, ex or whatever. right, but you had to get it right, you had to use your best instance, then it got smarter and smarter and the libraries to do bigger and better things, all of that is being automated, the progress of automation has been a fun story to watch if it develops.

I've been following the tech world closely for long enough to remember all those discussions from over a decade ago about how manual jobs would eventually be taken over by robots, but as seen since the rise of AI a couple of years ago years, it was cognitive jobs that were the first to be affected in 2017, only one in five companies used AI according to McKenzie, now half of all companies we have already seen many companies laying off thousands of employees, in part due to advances technologies such as AI. Contrary to what most people thought would happen, Goldman Sachs reported last year that AI could lose or diminish more than 300 million jobs. that the price of robots and their components will drop drastically and they have already dropped sharply since last year, unlike humans, robots don't need to take a break, they don't need to sleep, go on holiday strike and not complain.

As soon as these robots are feasible and cheap enough, companies will be eager to install them, so what do we do with all this? Well, it's your lucky day because I already did an episode on life after automation, it's a huge topic. and it's going to be too long to get into this episode, so I'll leave another link below. Well, let's go back to figure one at this point, you might think okay, the figure demo looks great, but it's sure to be a long time coming. Time until these human robots are in production, well, it seems Figure didn't want to waste any time even before the demo earlier in the year.

Figer signed an agreement with BMW manufacturing. They are going to test the robot in figure 01 on car manufacturers. plant in Spartanburg, South Carolina, but we haven't received any reports yet on how the 01 is performing and exactly what tasks it will perform, but I still think they are pretty interesting humanoid robots like the one in figure one. They are a future conundrum: will they replace humans completely or just relieve us of menu tasks by leaving the most important things for humans, but on the other hand, of course, the military will probably want to sniff out humanoid robots in the future and also look at consumer products, although we are quite far from getting a highly functional robot that is cheap and that you can buy at your local store, but the day will probably come.

Figure 01 is the worst robot from this company, so there will only be improvements. Other than that, the competition is heating up, so if it's not them, it's going to go away. to be another build that will crack the code from the broadest perspective possible. I'm sure all of you's opinions will be divided for engineering types. This is an excellent example of human ingenuity. For the skeptics, they will say that everything is useless and we should focus our efforts elsewhere, the optimists will see this as an exciting New Age for Humanity free of monotony, regardless of whether that future becomes Jetson, Terminator or Utopia, we will simply have than to let Humanity collectively discover it.

All I know is that I will be watching this space closely and that is where we are today with Figure One and Robotics. I hope you enjoyed it or found it interesting, and if you do, feel free to share it with someone who finds it interesting too. Okay, so that's it. For me, my name is toogo and you've been watching Cold Fusion and if you're new, don't hesitate to subscribe if you like it, it's free, cheers guys, have a good time. Cold Fusion, it's a new thought.