A Deep Dive into Nanite Virtualized Geometry

Their YouTube channel has the excellent deep dive on the unreal engine's nanite virtual geometry system: ▶ this video The one on idTech8 global illumination from last year is pretty interesting too.

Nanite has a large pre-render step, but it also changes the rendering quite significantly. (Which in some cases includes a software rasterizer running on the GPU) This is a really good deep dive if you’re interested: ▶ this video

It attracts truly extraordinary talent. One of many examples: ▶ this video . There is more technical ambition in a single slide of that project than an entire department at places I've worked. However, it also tends to demand more and pay less, so there is churn and burnout and when it comes time to triage, the triage can cut very deep. If I were to dive into the JIRA for any one of these "missing compression" incidents, I wouldn't expect to find an easy win, I would expect to find "missing compression" beneath a pile of much worse bugs like "level 2 crashes on AMD cards." So it goes.

If this UE5 demo is using Nanite, then it really may well just be modeled geometry. Nanite really blurs the line between geometry and texture - in a sense it’s a shader that uses triangle mesh data as if it were a texture source. This siggraph session will expand your mind: ▶ this video

For real. The talent in the game industry is insane. Recently, I got my mind blown by the Unreal Nanite team: ▶ this video The core of it is a respectably sophisticated LOD building algorithm, as you might expect, but the sheer amount of engineering horsepower they put into driving it into production just boggles the mind.

Thanks for posting this! I'll be digging into it. It seems like a good basis for learning how the tech works. For those interested in more on continuous LOD, here's a SIGGRAPH talk where they go into some of the why and how of Unreal's Nanite (similar continuous LOD system) here: ▶ this video

The answer to that is in this hour-long SIGGRAPH video.[1] Some of the operations needed are not done well, or at all, by the GPU. [1] ▶ this video

There's a lot of techniques applied. It's not really "one new revolutionary thing" but a bunch of things and research over the past 20-30 years put together to make nanite possible. This video might shed some light ▶ this video

Yeah, but they had to go through hell to make it happen. It took a lot more effort than you might think. ▶ this video Some highlights: Seam staggering between levels for smooth swaps Current hardware rasterizers chug on small polys so they made a GPU software rasterizer (!) Actually it's a hybrid rasterizer, and gnarly features like derivatives are supported on both sides The GPU scheduler chugged so they abused Z-test wave repacking Streaming compression

No, but it is a full time job. It's getting much more complicated. Here's an hour long talk on how Unreal Engine 5's Nanite works.[1] This is a huge breakthrough in level of detail theory. Things that used to be O(N) are now O(1). With enough pre-computation, rendering cost does not go up with scene size. See this demo.[2] You can download the demo for Xbox X/S and PlayStation 5, and soon, in source form for PCs. Explore 16 square kilometers of photorealistic city in which you can see a close-up view of anything. The format of assets in memory for Nanite becomes far more complicated. GPUs need a redesign again; UE5 has to do more of the rendering on the CPUs than they'd like. What they need to do is simple but not well suited to current GPU designs. It's such a huge win this approach will take over, despite that. New game engines will need to have something comparable. Maybe using the same format. Epic doesn't seem to be making patent claims, and the code for all this is in the Unreal Engine 5 download. (Basic concept of how this works: A mesh becomes a hierarchy of meshes, allowing zoom in and out of on level of detail. The approach to subdivision is seamless. You divide the mesh into cells of about 20 triangles. Then, each cell is cut by one or two new long edges across the cell. These become the boundaries of a simpler cell with about half as many triangles, but no new vertices. The lower-resolution cell has no more edges than the higher-resolution cell. There's a neat gra

If you want to see this sort of brilliant hacking on a modern system, I recommend taking a closer look at Unreal Nanite. It isn't just auto-LOD. Oh no. That's the core, but they had to do this same sort of "work around the tools they are given" to make it actually happen. Tiny triangles chug in modern GPU hardware raster pipelines -- so they wrote a GPGPU software rasterizer. They needed more bandwidth, so they wrote a compression engine. Their shaders needed better scheduling, so they abused the Z-test to do it. It's nuts! ▶ this video

So I would say skill at GPU assembly is in-demand for the elite tier of GPU performance work. Not necessarily writing much of it (though see [1] for an example, this is the kernel of multisplit as used in Nvidia's Onesweep implementation), but definitely in being able to read it so you can understand what the compiled code is actually doing. I'll also cite as evidence of that the incredible work of the engineers on Nanite. They describe writing the core of the microtriangle software renderer in HLSL but analyzing the assembler output to optimize down to the cycle level, as described in their "deep dive into Nanite virtualized geometry" talk (timestamp points to the reference to instruction-level micro-optimization). [1]: link [2]: ▶ this video