NVIDIA Is On a Different Planet

there are so many companies that would like to build they are sitting on gold mines gold mine gold mine is a gold mine gold mine gold mine gold mine and we have the pickaxes Nvidia's GTC event saw the unveiling of its Blackwell GPU and, Generally speaking, as far as Nvidia presentations go, this one was pretty well organized, there were still some memorable quotes. God, I love Nvidia, if you take all my friends, okay, Hopper, you're so good, good boy, good, good girl. PCI Express on the bottom on uh which one is M and the left one of them doesn't matter but M as side Jensen was absolutely right when he said that Nvidia is no longer a gaming company and it's clear why to companies like open Ai and Google and Amazon get a little nervous considering that they have functionally a single source of GPUs for some of their biggest companies that they're working on right now and that Nvidia at this point has grown to really unfathomable heights, it's a crazy to think this was basically a big part.

The gaming company until more recent years always had professional workstations and data centers that were growing, but games were the majority of

nvidia

's revenue for a long time and that has changed and it is clear that its performance in the market of AI will affect how it behaves in the consumer and gaming markets, but at this point, yes, we knew from the numbers that Nvidia was gigantic, but it didn't really dawn on me until I forced myself to see this in the coverage main news. Nvidia is still a center of the universe, uh, big performance improvement now and I had to Google what a pedophile failure it was, but please stop, it will eventually democratize Computing and give code for Java, Python or any other thing that the vast majority of us never learned, which makes us all hostages to the autocratic IT class, is cracking that clue, but I think what a lot of people want to hear is the debut of What's called b1000, that's not That's not the name, it's not even the technical part, but it is expected to be the first, what is called a multi-dye product, basically larger technological designs put on very small ones, they are called chiplets, it sounds really nice on in a way, what you said software, there are also, yes, talking about digital Enterprise, there was more.

Besides Blackwell University, that new technology that was introduced was not there. What else is correct? And actually, the guy's name is David Harold Blackwell, a mathematician, it wasn't Richard Blackwell the fashionista, but just shut up. Please shut up like a host to a parasite. Gaming has finally done something productive in the eyes of the massive non-technical media conglomerate as they rush to tell everyone that the bigger the number, the better and, uh, try to understand literally anything about the stocks they're pumping. They don't understand what it is or why it exists, but they know that money comes and goes, so you can talk to it in English and it would generate dollars directly.

However, you have to wonder if the engineers are seeing this. Those who designed and developed all the advances are disappointed because their job is reduced to making investors more money now please, but the cause of all this, as expected, was AI, so today we will talk about something of that, uh, the technology is really interesting that Nvidia has discussed some of the most important takeaways for us: advances in the modules and components of multiple chip-to-chip communication chips like Envy Link or Envy Link Switch, where the Actual communication link between

different

pieces of silicon starts to become the biggest limiting factor.

It already was, but that's going to be one of the main areas and we're also going to spend a good amount of time just on commentary because we live in an absurd world right now and at least in the technology sphere. It deserves a bit of discussion, some comments on that as well, so we'll space it out throughout and in the conclusion we'll go more into our thoughts on it, okay, let's get started before this video is brought to you by Thermal Take and the 300 tower. The Tower 300 is a complete display PC case designed to present the computer directly with its angled tempered glass windows or on a dedicated mounting bracket to display the build in new ways.

The Tower 300 has a design that places the GPU fans against the mesh panel with ventilation on the opposite side for liquid and CPU coolers. Also included are 2 140mm fans on top. The panels use a tool-less quick access system to quickly get in and out for maintenance. You can learn more at the link in the description below, so whether or not you're interested in the whole AI discussion, this is still an interesting set of technological advances or at least just technologies to talk about because some of them will reach the consumer multichip modules that are definitely the future. of large silicon, which makes it have higher yields for manufacturing, hopefully a lower cost that is at least partially passed on to consumers, but I have some thoughts on that, which we'll talk about later, but in general terms , we talked about AI, that's what created all the hype and despite it being a relatively technical conference and a relatively technically dense keynote when it comes to Nvidia Keynotes, they knew that a lot of financial and investment firms and their eyes were on observing this one, so there was a certain appeal to some of the chaos that those organizations like to observe. making us all hostages to the autocratic computer science class, let's move on, let's start with a recap of the two hours of Nvidia announcements, this time there is a lot less nonsense than they have usually had, there was still some nonsense, okay, so 3000 pounds per ton and a half.

It's not exactly an elephant four elephants a GPU and for our European viewers that's actually an imperial measurement. It's quite common here we use the weight of elephants to compare things, for example one of our mod mats weighs about 0.00 4165 of an adult African savanna elephant as fast as possible pretending that Blackwell Nvidia made these ads the ad Blackwell's architectural design took most of the attention Nvidia discussed its two largest possible models acting as a single GPU some time to the various interconnections and communications. The hardware necessary to facilitate chip-to-chip transfer. The nvlink switch is probably one of the most important in its presentation.

Its Quantum infin band switch was another communications announcement that many also spent time on various black configurations, as well as multiple GPU deployments and servers of

different

sizes for data centers. dgx was another one discussed outside of these announcements. Nvidia showed off some humanoid robotics shows like Project Groot giving us some Tony Stark vibes, including showing off these awfully cute robots that were definitely meant to calm all worries about the future downfall of society due to Terminators five things where are you going I sit here , don't be afraid, come here, green, hurry up, what are you saying?

As always the quickest way to get people to accept something that scares them is to make it cute and now after seeing that my biggest worry is will they let green keep his job, was it stage fright, is it new, is it still learning, you don't deserve to lose your job and your livelihood over that little mistake in the scenario and wait a minute, it's working and during this whole robotics discussion they also spent time talking about various partnerships and digital twin software is a big aspect of this . Companies are using the digital representation of their real workspaces to train robotic solutions and a modernized vision of automation and the general theme was that Nvidia is promoting itself differently now, so we are effectively an artificial intelligence foundry, but our estimate is that this technology will still make its way into consumer parts in some form. or other multi-solutions are clearly the future for high-end silicon.

AMD has done well to get the first one in a big way, but Nvidia published its own whitepaper on multichip modules many years ago and has been working on this for about as long. While AMD opted for multi-chip use with its consumer GP products, the RX 7000 series, Nvidia has now done the same with Blackwell, but with a more impressive execution of inter-chip communication, which will perhaps be facilitated due to the fact that companies spend millions of dollars. In these, looking at the Blackwell silicon held by Juan during the keynote despite the obviously limited quality of the footage in this view, we believe we can see the centrally located division as described with additional divisions for the hbm, you can see those dividing lines in In this shot, we're now going to recap the hardware side of Blackwell.

Blackwell combines two of what Nvidia calls the largest dies possible into basically a single GPU solution or a single package solution, at least in combination with a single memory, and Juan described Blackwell. like not knowing your connection between the two separate dyes and this is the most important aspect of this because as described at least in the scenario, this would imply that the silicon would behave like normal monolithic silicon where it doesn't need special programming considerations. made by software developers by those who use the chip to solve things like chip-to-chip communication, chip-to-chip latency, as seen for example in the consumer world on Ryzen CPUs, where crossing from a CCX imposes all kinds of new challenges on another. deal with them and there really hasn't been a good way to deal with them if you're badly affected, other than trying to isolate the work on a single piece of silicon in that specific example, but Nvidia says it has fixed this problem. the combined total solution is a 208 billion transistor GPU or more accurately a combination of two pieces of silicon that each have 104 billion transistors for contacts the h100 has 80 billion transistors that's the one above, but they are still selling it, they still have backorders To comply, Nvidia had several claims about how many it's related to power uh or whatever, but at the end of the day Blackwell seems to be a big leap for AI workloads, there was no real mention of games, but games will likely get a derivative somewhere;

Series 50 is likely to see Blackwell unless Nvidia retires Volta and skips it, but it seems that way. unlikely given current rumors, Blackwell supports up to 192GB of HPM 3E memory or depending on which slide the hmb 3E memory is referencing, the good news is that bug means the slides are still made by a human at Nvidia, the bad news is that we I don't know how much longer a human will do them on Nvidia Blackwell has 8 terabs per second of memory bandwidth as defined in this image and as for derivative or alternative configurations of this, the Grace Blackwell combination uses a Blackwell GPU.

The previously released solution and Grace CPUs, which is a collaboration with Nvidia, these combinations create a complete CPU and GPU product with different gpus counts and CPUs present and the video noted that the configuration of two Blackwell gpus and one Grace CPU It would house 384 GB of HPM 3E 72 ARM Neoverse V2 CPU cores and has 900 GB per second of chip-to-chip nvlink bandwidth. Nvidia says the so-called GB200 super chip has 16 terabytes per second of high-bandwidth memory, 3.6 terabytes per second of NV link bandwidth, and 40 AI pedop flops. Performance depends on how charitably you define it and turning it into a Blackwell compute node bumps up to 1.7 terabytes of hbm 3E, which is an obscene amount of memory, uh 32 TB per second of memory bandwidth and liquid cooling.

Most of the discussion beyond this point focused on various communications hardware solutions, including inip and intranode or data center solutions. We previously met with Sam Nafziger from AMD, who is an excellent engineer, excellent communicator and presenter. Basically the highest technical range you can get there, eh, but anyway we talked to him earlier about AMD moving to multichip for RX 7000, although it's a different production product, it was a different era, different target market , many of the key challenges are the same. what Nvidia faced and what AMD faced and ultimately those challenges largely center around the principle that if you run multiple pieces ofsilicon obviously can only be as fast as the literal link connecting them, although the reason for mentioning this is probably because there are a lot of you have forgotten that discussion or never seen it, and it's very relevant here, so this will be a short snippet from our previous interview with AMD, talking about some of the chip-to-chip communications and interconnect and chiplet structure limitations so we all have a good context on what Nvidia is facing and the bandwidth requirements. they are much higher with GBU because we are distributing all this work, you know, terabytes of data.

We loved the chiplet concept. We knew that cable counts. We had too many graphics to reproduce what we did on CPUs, so we were scratching our heads, you know, how can we make a significant profit?, and we were aware of those scaling curves that I showed and the observation was, you know. There's actually a pretty clear boundary between the infinite cache and the outer cache and we recognized that these things weren't necessary at 5 nanometers and they were fine for the product and at six we were barely wasting any energy, you know? the and the G gddr6 itself doesn't benefit from the technology at all, so that's where we came up with the idea, you know, we already have these gddr6 interfaces in bug tech, like I talked about the cost of proper portability and all the engineers and we already had that and we could just split it into its own little die and um I mean, you can see the results right, so we were spending 520 square millimeters here, we increased our compute unit count by 20%, we added a bunch of new capabilities, but this would be like you know, it's pushing. 600 550 square millimeters or something like that, but we reduced it to 300.

The rest of that discussion was great too. You should check it out if you haven't seen it. We'll link to it below. It's on our engineering discussions playlist where we are. In fact, we recently had Nvidia to discuss latency and Intel to talk about how drivers work, driver optimization, all that, but that's all linked below, so the key challenge is getting the number of small cables that connect the chips fit without losing performance using that space for them it is difficult to ensure that, although the performances benefit, they are not causing new problems and then only the speed itself is the number one problem and for AMD it was able to solve these problems quite well when segmenting the components. of the GPU into mcds and gcds, so it didn't actually split the compute, it split parts of the memory subsystem and that's very different from what Nvidia is doing.

Nvidia goes to the next step with Blackwell, which is a much more expensive part. Different use case than what we see with the RX 7000, although AMD also has its own Mi Instinct cards that we have talked about with Wend in the past, but either way, this is something where NVIDIA goes a step further and, actually it seems to be just two literal complete Blackwell models side by side that behave like a GPU if what Jensen Juan says is correct there is a small line between two models this is the first time two have Abed so together in such a way that the two chips, the two dies think it's one chip, there's 10 terabytes of data between them, 10 terabytes per second, so these two, these two sides of the Blackwell chip have no idea Which side they're on, there's no memory locality issues, no cash issues, it's just a giant chip, so that's the big difference between AMD's consumer design that we saw earlier and what

nvidia

is doing here.

From what nvidia's Katanzaro said on Twitter, we understand that the 10 terab per second bond theoretically makes all silicon appear uniform. software and we are not experts in this field or programming, but if this means that there is no need to write special code here to manage and schedule work beyond what would normally be done for a GPU, then that is a critical step to encourage the functional socket upgrades for data centers faster adoption things like that, the next biggest challenge after this is getting each individual GPU to communicate with the other GPUs in the same rack or data center, this is handled by a large number of components, including only the actual literal physical cables. which are connecting things as you scale to the actual size of the data center, but for us again, as people who are not part of the data center world, the most seemingly important thing is the Envy link and the envy link change, uh, improvements that Nvidia also announced on Nvidia's Envy.

The Link 5 generation solution supports up to 576 GPUs simultaneously and in a white paper Nvidia said this about its new Envy Link switch quote, while the new Envy Link on black GPUs also uses two high-speed differential pairs in each direction to form a single bond as in the hopper. Nvidia Blackwell GPUs double the effective bandwidth per link to 50 GB per second in each direction Blackwell GPUs include 18 fifth-generation NV link links to provide 1.8 terabytes per second of total bandwidth 900 GB per second in each direction and the via whitepaper also noted that this is more than 14 times the bandwidth of PCI Gen 5 quickly without wasting much time here.

Nvidia also highlighted a Diagnostics component to all of this which I thought was really cool, it's actually the Reliability, Availability and Serviceability Engine they call it Ras. to monitor the health of the hardware and identify potential downtime before it happens, so this is one of the things we talk about a lot internally, which is that we produce a lot of data logs from all the tests that we're running, but one of the biggest challenges is that it's hard to do anything with that data, uh, and we have systems in place for the graphs that we make, but we can do a lot more with it, it's just that you need a system like on a computer to basically start identifying those things for you. to really take advantage of it and Ras does it from a serviceability, maintainability and uptime standpoint, but they also talked about it with their Nims, so Nvidia talks about their new Nvidia inference, Nim Tool, uh, it's much less flashy than the latest generation gpus-powered humanoid robots. but probably one of the most useful from a business point of view and is intended to be a selection of pre-trained AI models for businesses that they can use to perform a range of tasks including data processing training, retrieval, augmented generation or is a great acronym, but Nims are based on Cuda, so they can run basically anywhere.

Nvidia gpus live On-premise cloud servers, local workstations, any business will also be able to retain full ownership and control over their intellectual property, something that is very important when considering adding AI to any workflow, a great example This is either Nvidia's own Nemo chip or an LLM that the company uses internally to work with its own vast, unpublishable, and incredibly useful chip design documentation. Nims will be able to interact with various platforms. like service now and crowd strike or custom internal systems, there are many companies generating oceans of data that could help them identify problems, optimizations or trends in general, but they may not have had a good idea of ​​what to do with the data or how draw them. patterns of this and Jensen Juan said this during the GTC presentation: the enterprise IT industry is sitting on a gold mine, it's a gold mine because they have a great understanding of the way work is done, they have all these Incredible tools that have been created over the years and have a lot of data, if you could take that gold mine and turn it into co-pilots, these co-pilots could help us do things and again we come back to whether the data is the gold mine. gold. then in the gold rush Nvidia is selling spikes, maybe like the Bagger 288 digger or something in the case of the dgx.

Are you impressed with my excavation knowledge? Nvidia also announced its Groot project with zeros, which is a legally significant distinction as described. as a general purpose Foundation model for humanoid robots, your quote is like that, it's a robot that's coming soon, not the vacuum cleaner type but the Will Smith type and it's going to hit us in the face. Autonomous or AI-driven robotics extends to many practical applications in vehicles. industrial machines and warehouse jobs, just to name a few; However, the sci-fi appeal of pseudo-intelligent humanoid robots is part of our collective culture that even the mainstream media has been capturing or trying to capture, announced at the end of the show what they call Groot, which is basically a new type of AI architecture, a basic model for humanoid-like robots, so you can think, not exactly, Terminator or Bender from Futurama, but it's the classic study of a false dichotomy, your two options are. a killing machine or an inappropriate alcoholic robot that has a pension for gambling, yeah well I'm going to build my own theme park with Blackjack and hook Groot as a project that is both software and hardware and represents the generalist technology of the 0000 robot as addition. of parts and videos wants Groot machines to be able to understand the meaning of human language and navigate the world around them, uh, and perform arbitrary tasks like taking GPUs out of the oven, which is obviously an extremely common occurrence at Nvidia these days , the training part of the stack starts with a video so that the model can intuitively understand physics and then accurately models the robots in the Nvidia Omniverse digital twin Tech virtual space to teach the model how to move and interact with objects in the terrain or other robots that will eventually rise. to rule us all, Jensen described this as a gym.

Here the model can learn the basics much faster than in the real world. A blue robot is really good at knocking others down stairs. We realized we're not sure if that's included in the training data, but yes. does, it's just the first step and there are personal methods to dispatch humans. I just saw a lot of what all the mainstream media was talking about about how the world was going to end and I wanted to feel left out. the hardware is nvidia's Thor, so it has a Blackwell based GPU with 800 Tera flops of fp8 capacity on board.

The Jets and Thor will run multi-modal AI models like Groot and have a built-in functional safety processor that will definitely never malfunction, so I'll mostly cover G Hardware here, back to the comments, part of the fun, just kind of discussion and thought experiment, and in this case it seems that the main advances that, at least to me, are the most interesting personally are those in multi-chip design, which We have a lot of evidence that, when it works, it works phenomenally well for Value . It's a question of whether NVIDIA will be the company that feels it is in a position to need to offer good value like AMD was when faced with Intel being slaughtered by the old Skylake silicon, repeated year after year, the answer is probably no, Nvidia is not in that position huh so it's a different world for them but we're not sure if this is going to happen for the RTX 5000 Series, no real news yet, there are some rumors huh but no news real.

Nvidia is clearly moving in the direction of multichip modules again, they published that whitepaper or something that was a long time ago, it could be more than five years ago, so this was known, it was just a question of if they can make the implementation work in a competitive, high performance way, where they feel it's worth flipping the switch away from monolithic for the consumer, now AMD has shown this in less success in technical capability for their RX 7000 series uh, they are relatively competitive, especially in rasterization. R sometimes tracks a different story, depends on the game, but they get slaughtered in, say, cyberpunk with really high RT workloads, but either way the point is that particular problem may have existed monolithic or multichip and they have shown that the multichip can work for the consumer, so either way, Nvidia right now, whether AMD has its Zen moment for gpus, also Nvidia is still a scary beast, it's very powerful. a massive company and that quote from the summary above is actually somewhat true when you said this is nvidia's world and you're not entirely wrong, so it's almost certain that intel and amd are motivated to stay on gpus for purposes of artificial intelligence, like Nvidia.

They chase money, that's the job of corporations that are publicly funded and gigantic like these. AI is money, they are going to chase it and the consequences of that are consumer GPUs, so the difference maybe is that Nvidia has less meaningto provide a cheap entry. aims at the gaming market as we've seen they haven't really done anything less than a 4060 in the modern line uh that's affordable and they don't have as many reasons there's not really much reason to chase the smaller dollar. amount when you then get $1,600 $2,000 for 490 uh at least the retail assembly cost and then tens and hundreds and over thousands of dollars in Ai and data center parts, then that's where AMD and Intel will continue to be at least immediately critics. that part of the market healthy and alive that makes it possible to build PCS at a reasonable price without it getting out of control, where eventually everyone is left snowy and GID thinking that this is just what GPUs should cost now that GPUs have obtained such a high sales average. price and have held up a little better from the company's perspective than before, it seems unlikely that they will simply lower it, in other words, if people are used to spending $1,000 on a GPU and even if the cost may come down to some sense, with advances and things like in the future we will have Ryzen style chiplets that work at the GPU level, even though those costs maybe more controllable for the manufacturer and better performances, they may still try to find a way to sell you a $1,000 GPU if you buy them, if you show interest in them, that's how they'll respond, unless there's significant competition where they start undercutting each other, which is obviously ideal for everyone here, but that's not the world in which we live.

Right now the market share is largely Nvidia's, no matter what market you look at in the GPU world and no matter how you define GPUs on the positive side, our hope is that we'll see some of what they're doing with Blackwell with multichip means there's a path to bringing multichip to the consumer in bigger ways than there is now for GPUs, which, again, Ryzen just states is this huge success story of being able to make relatively affordable parts without losing money on them and that's been the critical component of it's just that we're missing another key aspect of Ryzen's success, which was that AMD didn't have any other options when it launched Ryzen, it didn't have the option to be expensive whereas Nvidia does. quite different, one thing is clear, although Nvidia is a giant, it again operates on a completely different

planet

than most other companies in this industry in general, if you reallocate any amount of the assets it is getting from AI to the games, that will further widen the There is a gulf between them and their competitors in all aspects of gaming GPUs and part of this takes the form of a fair market share in which Nvidia can drive real gaming development and trends. and the features that are included in the games because it has captured a lot of the market, uh, it can convince these developers that if you put this statistically, the majority of people who play your game will be able to use it on our GPU and that forces Intel and AMD into a position where they're basically trying to perpetually keep up, uh, and it's hard to fight that position when the one they're trying to keep up with has incredible revenue from segments outside of the battle they're fighting in the games, so anyway, AMD and then Intel are not far behind, so despite whatever both companies are having problems with their GPUs, we have seen that they are also making progress.

They clearly have trained engineers and are also looking at artificial intelligence, so some of that technology will also reach the consumer and AMD. Once again it is presented as a good example to show that although a company can have an almost complete monopoly in a segment, talking about Intel here many years ago, if it becomes complacent, or if it simply has a series of stumbles like the ones Did Intel have 10 nanometers, no. by being able to ship it forever they can lose that position faster than they gained it, that's a little scary for companies too, it just seems like Nvidia is maybe a little more aware of that than Intel at the time, so That's all for this. one, hopefully, got some value out of it, even if it was just entertainment.

Seeing myself bewildered by mass media conglomerates that can't even get the name of what they're reporting on just when they are some of the largest and most established. companies in the world with a long EST history of reporting news, but that's okay, we will continue to make YouTube videos, thanks for watching, subscribe for more information, go to the store. gam access.net to help us directly if you want to help fund our discussion about the absurdity of it all and see you next time.