Microsoft's research division has been hard at work trying to make ray tracing a more accessible feature for games of the future. A new patent from one of the team's senior architects describes how the memory and cache load for bounding volume hierarchies (BVH)— structures used to speed up ray tracing—can be reduced by using a similar approach to texture Level of Detail (LOD) application. 

While Microsoft's Xbox consoles would be the main beneficiary of this technique, AMD's traditionally ray tracing weak GPUs, and graphics cards that aren't blessed with a ton of VRAM are also a potential winners.

As with all such patents, the details within it are complex but also broad in terms of where they could be used. Its author, Mark Grossman, is a senior architect in Microsoft's research division, specialising in chip development, and he has a long and distinguished career in GPUs (starting with Silicon Graphics, before moving to ATI Technologies, then AMD, and finally Microsoft).

Last year, he had a patent approved for reducing the memory footprint of meshlets, small groups of polygons as used in mesh shaders. His latest patent (via Tom's Hardware) follows a near-identical format. To speed up the process of ray tracing, games use BVH structures, which are a little like a flowchart of the volumes that objects in a 3D world take up. BVHs are split into a large overall map, called a top-level acceleration structure (TLAS), and multiple smaller bottom-level ones (BLAS).

In Cyberpunk 2077, the average size of the TLAS is around 15 to 20MB, but the total BLAS is many times larger than this, running into hundreds or thousands of megabytes. If your graphics card has 16GB or more of VRAM, it's not really a problem but if you have 8GB or less, then it's not hard to see why it can become an issue. And this is especially true of consoles with shared memory pools.

Microsoft's technique avoids using the entire BVH in one go. Instead, small chunks of it are used at a time, and the details of this usage and the part of the BVH in question (called a node) are stored in two lists in the GPU's memory. With each successive frame, the ray tracing algorithm updates these lists, ensuring that only the nodes required to be used in the scene (the rest just not being visible or worth using) are stored in the VRAM.

Note that this system not only reduces the load on the graphics card's memory and the GPU's cache, but it also lowers the amount of ray tracing checks that need to be carried out. Rather than having to constantly traverse across the entire BVH structure to see what rays are interacting with which objects, only those in the lists (called a residency and recording map) need to be checked.

This is important for AMD's graphics processors because unlike Nvidia's GPUs, they don't have any dedicated hardware for accelerating the traversal process. Instead, it's all handled via compute shaders. Both vendors do have specialised units for testing ray-box and ray-triangle intersections, so they both check these things off very quickly, but RDNA 2 and newer GPUs spend quite a bit of time working through the BVH.

So while all of this work is to the benefit of the Xbox, it would also be a boon to any gaming PC sporting a Radeon graphics card, powered by a RDNA 2 or RDNA 3 GPU.

Yeah, this is all very complicated, and I'm almost certainly over-simplifying it, but the one thing that the best gaming PCs and consoles have in abundance is compute power. So it makes sense to leverage something that, while involving lots of extra but basic calculations, reduces memory and ray tracing loads.

Although Microsoft's patent is primarily targeted towards improving ray tracing performance on its Xbox Series X consoles, the technique could be implemented on any platform, as it's not specific toward any particular hardware. So if you have a mainstream graphics card, like a GeForce RTX 4060 or Radeon RX 7600, then you could well have the chance to enjoy ray tracing in a game in the not-too-distant future.