NVIDIA's Nemotron 3 AI Model Achieves Breakthrough Speed and Efficiency

Remember that most AI systems are proprietary, we have to pay a subscription for them, and no one knows how they work or what data they were trained on? Well, now hold on to your papers Fellow Scholars and check out this incredible work, and when I first saw it, my jaw hit the floor. They absolutely knocked it out of the park. They spilled all the secrets. This is an AI assistant that is free for all of us forever, but not just the model itself. They also gave us a 51-page research paper which might be the holy bible of creating such a system for now. Why is that?

Well, they show us every step of the way of how it was done, and the dataset it was trained on as well. That is extraordinary. Usually something is always missing. Not here. They call it Nemotron 3 Super and we are going to find out whether it is indeed super or not. Okay, so in goes 25 trillion tokens as training data, and out comes a 120 billion parameter AI assistant that is how smart exactly? It roughly matches the best closed frontier models from about a year and a half ago. Note that those models cost billions of dollars to train and every detail about them was kept in secret. And now, we just get this kind of

stuff for free. That is mind blowing. This is amazing for us, consumers and Fellow Scholars. So as you see, it is really smart. Up with some of the best open models out there in most tests, but note that it's still a bit behind some areas. Here's something that surprised me: in this result, they showcase two versions of the new model, BF16 and NVFP4. They perform roughly the same in terms of accuracy, so why the big fuss about this? Well, look at this. Holy mother of papers. Wow. Well, the NVFP4 version is about 3.5 times faster than their other model, and it is up to 7 times faster than similarly smart open models. So the story is not just the similarly smart part,

the story is that it is 7 times faster while it is similarly smart. Goodness. Okay, so how on Earth did they do that? So here are 4 secrets they gave us from the paper, in very simple words. Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Okay, NVFP4. What is that? This is a way for speeding up the AI to run a great deal faster by essentially compressing the mathematics it uses. Seeing a long number and rounding off a few digits. You get a smaller format. Less work! What's wrong with that? Well, everything. Normally, if you do that, you lose too much accuracy and the system will output nonsense. However, here, scientists did it the smart way:

they left the most sensitive calculations alone, and did this rounding for the rest, where it does not cause trouble. The result is that it runs up to 7 times faster than many other techniques. And we saw that it gives us no meaningful loss in accuracy. Magic. But there is more magic. When other AI techniques write their answer, they write it token by token. Let's simplify by saying word by word. Writing one word at a time. But not this one. This one calculates several future words at once. A whole sentence! Almost. Specifically, 7 tokens. And then the system verifies the 7 tokens in one go. Another massive speed up. They call it multi-token prediction. But why stop there? Let's add even more magic!

They showcased these weird things they call the mamba layers. What do these do? Well, traditional AI systems have a bit of a memory problem. They work like a student who constantly re-reads the textbook over and over again when they are given a question. Scientists at NVIDIA say, that's not the way to go. Memory is precious. So instead, read the book only once, and take highly compressed notes. So this kind of memory remembers important details about the conversation. However, it is smart enough to throw away the filler words. Thus, this system can process massive amounts of data efficiently. All this sounds glorious, but this still does not give us a working system. Why is that?

Well, this is why. You see that there is a lot of addition here? That is the problem. The AI generates your answer step by step, and because we rounded off the numbers, there is a little error. That's not a problem. Here's the problem. There are many steps, and the error is magnified through each step. Imagine trying to walk to your car, which is a 100 steps away, but you feel a bit sluggish today and every single one of your steps is a bit smaller than it was before. What's the result? Well, of course, after a 100 steps, you are still really far away from your car!

So what is the solution? Well, scientists solved this by adding back some random noise in the system. But wait, this noise is carefully crafted in a way that it averages to zero. So your new steps are sometimes smaller, and sometimes bigger than they used to be, but if you average them out, over a 100 steps, you will be exactly at your car. So good! They call this stochastic rounding and it is a genius idea. Now, not even this technique is perfect. For instance, when I give it my favorite question about assembling robotic cows, with lots of math, I like this guy a lot, but it thinks for almost an hour to get me an answer for that one. That's a lot.

So if I have workloads like that, I like to run it on a much faster Lambda instance. But still I think the AI game has suddenly changed. Closed systems used to dominate. Now, not anymore. It seems to me that Jensen at NVIDIA is not playing games here. It's in the news that they are going to invest tens of billions of dollars into fully open systems like this. I am not a money person, I don't know how that works exactly, but if we get to own more amazing free AI systems. Well, sign me up for this one! What a time to be alive! And there is just so much more in the paper, I would definitely love to come back for at least another video on it. Let me know in the comments if you would like that,

and if you enjoyed this, subscribe, and hit the bell.