https://alextardif.com/BindlessProgramming.html

Alex Tardif
Graphics & Game Programmer

Portfolio Posts About Me Resume Contact

Bindless Oriented Graphics Programming

[bindlessdb]


Overview

In this post I begin by covering GPU-driven rendering at a high level
before diving into bindless as an accessible gateway to powerful
GPU-driven rendering techniques, and how to achieve this
pragmatically with low code complexity and mental overhead while
reaping the lion's share of the gains. Nothing I present is novel,
but my hope is that my approach to bindless may connect with you if
you have not yet made the leap.




GPU Driven Rendering

Stop me if you've seen or written this sort of thing before. Maybe
not exactly like this, but similar:


for(Draw& drawData : allDrawsForThisRenderPass)
{
    context.BindVertexAndIndexBuffers(drawData.geometry);
    context.WriteConstants(drawData.constants);
    context.BindConstantBuffer(drawData.constantBuffer);
    context.BindTextures(drawData.textures);
    context.Draw();
}


If you've played with graphics for a bit, you may have discovered
that there are scales that cannot be achieved with this loop at high
framerates. Although there are endless "it depends" that asterisk
this, generally I'd place previous-generation renderers with this
loop as being capable of individual draw calls in the thousands or
low tens of thousands, and needing to resort to complex and even
costly techniques to keep those counts low in scenes with many more
entities.

Sure, you can build a bunch of command contexts to split this into
jobs, tools that (properly) arrived with modern graphics APIs. This
can be cumbersome in practice though, as there are challenges in
working with graphics APIs that complicate how exactly you go about
doing it, and that's on top of adding multithreading to the equation.

I'm not about to lecture at length the benefits of GPU-driven
rendering, as it has been the subject of many of the most prolific
advancements in game graphics of the last decade or more, with
presentations and posts by the best in the industry. Here are some of
my favorites:

"GPU-Driven Rendering Pipelines" 2015 by Ulrich Haar, Sebastian
Aaltonen
https://advances.realtimerendering.com/s2015/
aaltonenhaar_siggraph2015_combined_final_footer_220dpi.pdf

"A Deferred Material Rendering System" 2015 by Tomasz Stachowiak
https://h3.gd/a-deferred-material-rendering-system/

"Optimizing the Graphics Pipeline With Compute" 2016 by Graham
Wihlidal
https://www.gdcvault.com/play/1023109/
Optimizing-the-Graphics-Pipeline-With

"Bindless Texturing for Deferred Rendering and Decals" 2016 by Matt
Pettineo
https://therealmjp.github.io/posts/
bindless-texturing-for-deferred-rendering-and-decals/

"Experiments in GPU-based Occlusion Culling" 2017 by Kostas
Anagnostou
https://interplayoflight.wordpress.com/2017/11/15/
experiments-in-gpu-based-occlusion-culling

"Nanite, A Deep Dive" 2021 by Brian Karis, Rune Stubbe, Graham
Wihlidal
https://advances.realtimerendering.com/s2021/
Karis_Nanite_SIGGRAPH_Advances_2021_final.pdf

"Visibility Buffer Rendering with Material Graphs" 2021 by John Hable
http://filmicworlds.com/blog/
visibility-buffer-rendering-with-material-graphs/

In its foundational form: rather than recording your draws on the
CPU, you instead build a buffer of draw call commands on the GPU,
formatted to the specification of the API. To see an example of that
format, take a look at D3D12: https://learn.microsoft.com/en-us/
windows/win32/direct3d12/indirect-drawing

You may notice that far more than just issuing a draw call can be
achieved through this interface, I'll come back to that.

This GPU generation of draws alleviates not only a significant
bottleneck from the CPU (recording the above loop) but likely also
avoiding the need to introduce more complexity to reach performance
targets, making host-side rendering code simpler while also providing
all the bones required to introduce GPU culling. Faster *and* easier
to work with is rare! There's a catch, though.

Because really, it wasn't exactly straightforward to get working. The
previous-generation requirements of GPU-driven rendering actually
came with quite a bit of its own complexity, at least if you didn't
want to issue a separate indirect instanced draw call for each
combination of mesh/material, limiting the benefits. Let's look back
at the loop and see what having an instance ID and/or draw ID lets us
eliminate from the loop:


BindGlobalBufferOfConstants();

for(Draw& drawData : allDrawsForThisRenderPass)
{
    context.BindVertexAndIndexBuffers(drawData.geometry);
    [S:context.WriteConstants(drawData.constants);:S]
    [S:context.BindConstantBuffer(drawData.constantBuffer);:S]
    context.BindTextures(drawData.textures);
    context.Draw();
}


If we know the ID of all possible instances, then we can put their
constants in a global buffer that we read from by Draw ID (or some
constant ID). It's probably a StructuredBuffer or a
ByteAddressBuffer, and we can index into it like this (in this case,
ByteAddressBuffer style):


MyConstants constantsForThisDraw = myBuffer.Load<MyConstants>(drawID * sizeof(MyConstants));


Alright, we took care of constants, and we're left with textures and
geometry. If we want to be able to index textures dynamically (though
uniform at the per-draw level), we could potentially use a big
texture array, but then that means we have format/size/array limit
constraints to deal with. Doable, but far from ideal. What others
have done instead is implement Virtual Texturing, where pages of
texture data are streamed in and out of memory into large texture
resources that are bound once for all draws.

Virtual texturing offers some additional advantages that I won't go
into here, but of course it also adds a meaningful amount of code and
data complexity in exchange to facilitate its existence. It is
hopefully not a surprising thing to say that there are people who
would rather avoid having to implement and maintain such a system for
general purpose rendering, but for now let's return to the loop:


BindGlobalBufferOfConstants();
BindGlobalVirtualTextures();

for(Draw& drawData : allDrawsForThisRenderPass)
{
    context.BindVertexAndIndexBuffers(drawData.geometry);
    [S:context.BindTextures(drawData.textures);:S]
    context.Draw();
}


Lastly, we have the geometry, which is generally solved by having a
giant megabuffer for vertices and another for indices that contain
all necessary geometry. While on the surface this might seem trivial,
considerations are required for how to handle, for example, 16-bit vs
32-bit indices, and potentially a wide variety of vertex formats. It
may also make decisions around asynchronous uploads more painful
depending on the platform (like when it's safe to schedule and sync),
and also around handling of procedurally generated geometry.

If we were to do this though, we would complete the necessary
requirements to fully "unlock" GPU driven rendering, as now the only
changes between draws is the pipeline state object, which become the
"buckets" with which draws are filtered. We can refactor the loop.


BindGlobalBufferOfConstants();
BindGlobalVirtualTextures();
BindGlobalVertexAndIndexBuffers();

for(Pipeline& pipeline : allPipelinesToDraw)
{
    context.DrawIndirect(pipeline.pso, pipeline.drawBuffer, pipeline.drawBufferOffset);
}


The once very hot CPU loop is now bounded only by the number of
pipelines, and so requires very little work to be done at all. This
is a good reminder that keeping PSO count low can mean a big
difference if your goal is high performance.

From here, compute shaders can take over by running culling passes
with the strengths of GPU parallelism and binning draws into indirect
draw buffers. Achieving this GPU driven rendering unlocks new degrees
of freedom in how much can be thrown at the GPU while also providing
significant performance improvements- orders of magnitude more. The
work to reach it that I hand-waved above can be quite a mountain to
climb though, performance coming at the cost of non-trivial amounts
of complexity.

So then, is there a middle-ground where I can still have high
performance at massive scales, while feeling like it's easily
maintainable? Yes!




Bindless to the Rescue

Modern "bindless" rendering - the act of arbitrarily accessing GPU
resources in shaders from large tables of descriptors (or even from
the descriptor heap itself) - is the most important advancement to
graphics APIs delivered in the last decade. Many people have beaten
this drum but I think it's still an underestimated and transformative
way to work when fully embraced, when platform support is available
of course. For example, DirectX 12 "Ultimate" compliance guarantees
it is supported.

If bindless is new to you, check out some links to get an idea first:

"Using Bindless Resources with DirectX Raytracing" by Matt Pettineo
from Raytracing Gems 2
https://therealmjp.github.io/posts/rtg2-bindless/
https://www.realtimerendering.com/raytracinggems/rtg2/index.html

"Bindless Rendering" series by Darius Bouma
https://blog.traverseresearch.nl/
bindless-rendering-setup-afeb678d77fc

"Bindless Descriptors" by Janos Turanszki
https://wickedengine.net/2021/04/bindless-descriptors/

"Binding Bindlessly" by me, though written before Shader Model 6.6
style bindless existed
https://alextardif.com/Bindless.html

Specification for SM 6.6 Dynamic Resources (bindless)
https://microsoft.github.io/DirectX-Specs/d3d/
HLSL_SM_6_6_DynamicResources.html

GPU driven rendering can be achieved with far less effort and code
with bindless. Instead of virtual texturing and mega-geometry being a
borderline necessity, we can simply store individual resources in an
array/table of resource views, and access them by indexing into those
tables (or directly to the descriptor heap) with constants, which can
also be accessed bindlessly. These tables can be implemented as
simply as a free-list of descriptors, and likely with frame-buffered
heaps to be able to update them easily frame-to-frame as resources
are streamed. As shown in the links above, it's altogether not much
work to get up and running compared to the alternative, and we'll see
soon how this allows for simply-written systems.

The component of GPU-driven rendering that bindless doesn't
implicitly solve is binding index buffers which are essential for
performance, but it is however an excellent tool for facilitating
that.

One option is to simply use mesh shaders, where the traditional bound
index buffer is not a part of the equation at all. For this, bindless
can be used to access geometry resources indirectly as long as you
know the descriptor indices of the buffers you want to access (via
constants for example), and there's no need to keep all of those
resources in a megabuffer if you don't want to.
https://microsoft.github.io/DirectX-Specs/d3d/MeshShader.html

If you're reading this though, I'll assume you're coming from a
traditional pipeline and/or aren't yet interested in tackling the
switch to mesh shaders. In this case DX12's ExecuteIndirect, Vulkan's
Device-Generated Commands, or something like these, is needed. With a
simplified bindless workflow, ExecuteIndirect's rebinding feature for
index buffers can be utilized to swap those resources between draws
as needed.

For DX12, this involves making use of
D3D12_INDIRECT_ARGUMENT_TYPE_INDEX_BUFFER_VIEW in the command
signature, and writing the index buffer view from a compute shader
along with the indirect draw. If the documentation I linked earlier
is confusing (here is the link again: https://learn.microsoft.com/
en-us/windows/win32/direct3d12/indirect-drawing) I'll try my own
explanation here:

In the section titled "Indirect Argument Buffer Structures,"
structures that are not used by the API are provided regardless, for
the purpose of you or I to be able to mimic the layout required by
D3D from a compute shader. In this case, the relevant arguments are
D3D12_DRAW_INDEXED_ARGUMENTS, D3D12_INDEX_BUFFER_VIEW, and plain old
root constants which are shown by example in the section towards the
bottom titled "Root Constants and Vertex Buffers". These are filled
out in a command signature via D3D12_INDIRECT_ARGUMENT_DESC, which is
a union of the different options.
https://learn.microsoft.com/en-us/windows/win32/api/d3d12/
ns-d3d12-d3d12_indirect_argument_desc

If we match root constants (no additional arguments required), the
index buffer view (https://learn.microsoft.com/en-us/windows/win32/
api/d3d12/ns-d3d12-d3d12_index_buffer_view), and the indexed draw
arguments (https://learn.microsoft.com/en-us/windows/win32/api/d3d12/
ns-d3d12-d3d12_draw_indexed_arguments), we would have an HLSL struct
that looks something like this:


struct MyCustomDrawArguments
{
    //Matches root constants. These let us indirectly access all the data we need for the draw via bindless
    uint instanceDataID;
    uint materialDataID;

    //Matches D3D12_INDEX_BUFFER_VIEW 
    uint64_t indexBufferLocation; //D3D12_GPU_VIRTUAL_ADDRESS, uint64_t available in SM 6.0+
    uint     indexBufferSizeInBytes;
    uint     indexBufferFormat;

    //Matches D3D12_DRAW_INDEXED_ARGUMENTS
    uint indexCountPerInstance;
    uint instanceCount;
    uint startIndexLocation;
    int  baseVertexLocation;
    uint startInstanceLocation;
};


We would keep a buffer of these sized to the worst case draw count
budget. On the CPU side, we would fill out a matching argument
description and command signature like this:


D3D12_INDIRECT_ARGUMENT_DESC indirectArguments[3] = {};
indirectArguments[0].Type = D3D12_INDIRECT_ARGUMENT_TYPE_CONSTANT;
indirectArguments[0].Constant.RootParameterIndex = 0;  // Match this with the root signature
indirectArguments[0].Constant.DestOffsetIn32BitValues = 0;
indirectArguments[0].Constant.Num32BitValuesToSet = 2; // My example uses 2 root constants
indirectArguments[1].Type = D3D12_INDIRECT_ARGUMENT_TYPE_INDEX_BUFFER_VIEW;
indirectArguments[2].Type = D3D12_INDIRECT_ARGUMENT_TYPE_DRAW_INDEXED;

D3D12_COMMAND_SIGNATURE_DESC commandSignatureDesc = {};
commandSignatureDesc.ByteStride = sizeof(MyCustomDrawArguments);
commandSignatureDesc.NumArgumentDescs = 3;
commandSignatureDesc.pArgumentDescs = indirectArguments;


We would then have a compute shader that consumes a buffer of draw
metadata to decide what to draw, and we don't want to fill that
buffer in an O(N) loop each frame or we'd be defeating the point.
Instead we can suballocate our potential draws out of that large
buffer using an allocator (and flag them as valid). The compute
shader can then skip considering invalid draws. This means a compute
dispatch might skip over a number of invalid draw IDs, and while this
can be improved as a later optimization, this is a very fast
operation and you can have pre-cull draw counts in the hundreds of
thousands being scanned like this before you would notice it become
meaningful in a GPU profiler. It's almost like a compute particle
shader, we can lean into the GPU here.


//HLSL
StructuredBuffer<Draw> DrawDataBuffer;
StructuredBuffer<Instance> InstanceDataBuffer;
StructuredBuffer<Material> MaterialDataBuffer;

//Compute shader can consume this to generate draw arguments
struct Draw
{
    //passed as push constants with the indirect draw call
    uint instanceID; //lookup in InstanceDataBuffer
    uint materialID; //lookup in MaterialDataBuffer

    //lookup further args like index buffer view
    uint drawArgsID;

    //flags to consider for drawing/culling/etc
    uint flags;
};

//This can be used for culling, and later during shading for grabbing vertices, etc
struct Instance
{
    float4x4 transform;
    uint4 vertexBufferIndices; //lookup in the descriptor heap
    AABB bounds;
};

//Used during shading
struct Material
{
    uint baseTextureIndex; //lookup in the descriptor heap
    uint normalsTextureIndex;
    //etc, various material properties
};


In those same compute shaders we can use indirection and bindless to
look up any data we need to perform culling, compact the valid draws
together, and then write them out to a buffer resource of
MyCustomDrawArguments, and the final draw count(s) to a count buffer.
I've skipped pseudo-coding this part as this is covered multiple
times over in the links I've provided - Kostas's article is an
example. It might sound intimidating at first but it's genuinely
altogether not much code. If you get stuck, I recommend writing debug
data to a buffer that you read back on the CPU to see what's
happening, such as draw ID, whether it passed culling, and if it was
actually written to the draw buffer.

By the end of these steps, you will have a functional GPU-driven
pipeline capable of supporting draw counts that rival many big games
that ship today, with minimal investment required to get there
compared to the alternatives. If you later want to go full Nanite or
something to advance it further you can, but my goal here is
bang-for-buck with your time investment.

Some vendors would not recommend using these features of
ExecuteIndirect due to performance issues caused by changing bindings
during command processing, but I've already worked with it first-hand
at high framerates on 6+ year old desktop hardware and
current-generation consoles, so it is absolutely possible to run well
with it today. It is also maybe not well known that Halo Infinite
shipped 60fps+ while (ab)using ExecuteIndirect's rebinding feature. I
deeply appreciate the people who paved the way to make this viable to
begin with; the things that we can take for granted today were
enabled cross-platform by extraordinary developers. There is
interesting history here:
https://www.supergoodcode.com/device-generated-commands/ Anyone
looking for Vulkan here, this is a great place to start.
https://docs.vulkan.org/refpages/latest/refpages/source/
VK_EXT_device_generated_commands.html
https://www.basnieuwenhuizen.nl/a-driver-on-the-gpu/
https://rg3.name/202409270942.html

In a post-workgraph world, I don't know how much attention and
performance improvement ExecuteIndirect/DGC will see going forward,
but it's in my top 5 because the workflow is excellent.




Rethinking Rendering

Bindless design isn't just about how we access resources in shaders
though. It can also enable simpler ways of working when designing
entire renderers. When all resources can be accessed by integer IDs,
new doors open.

How might those bindless draws be constructed, for example? What is
it that we actually need in a bindless, GPU driven pipeline? As we
saw above, the answer is draw-call arguments, the ones we usually
gather in a visitor-style O(N) loop like the one at the beginning,
probably writing constants to some intermediate buffer as we go.

Given that we know the resources we need on a drawable object, and
they can all now be referenced by ID constants on the GPU, then as
long as we can provide an ID for each unique resource up-front
(trivial), we could allocate/free draw calls out of a global "draw
buffer" as an entity is created and destroyed, in an event-driven
way:


void CreateDrawable(uint ID, GeometryID geometry, MaterialID material, etc)
{
    Drawable& drawable = DrawMap.insert(ID); //some way of mapping entity ID to draws for later
    drawable.drawID = DrawDataBuffer.AllocateDraw(); //keep ID for pushing updates

    //queues an event in a buffer to set Draw constants in VRAM via compute shader
    //...or just write via ReBAR if available, whatever you want
    TransactionHandler.CreateDraw(drawable.drawID, geometry, material, etc);
}

void UpdateMaterial(DrawID drawID, MaterialID material)
{
    //queues an event in a buffer to be processed by a compute shader to modify the draw's material
    //...or just write to that data via ReBAR if available, whatever you want
    TransactionHandler.UpdateMaterial(drawable.drawID, material);
}


I'm glossing over general resource loading, but this is pretty much
the holy grail of draw management to me - doing the minimum work
required on the CPU to set up some draws, and no per-frame scanning
or hot loop involved. The compute shaders that manage the draw data,
run culling, and ultimately compact draws into buffers for indirect
can handle the rest in a highly parallel fashion.

To me the most natural analogy is database programming. All the
important data sits in big (budgeted!) buffers and descriptor heaps
on the GPU as if they were tables in a SQL database linked by foreign
keys (IDs), allowing indirect access to virtually any data from any
shader as long as the connections exist. Events on the CPU push
transactions to those tables as needed (update properties, streaming,
etc), and the procedures (compute shaders) execute those
transactions.

Boring ending, right? For as long as I've worked in graphics, I've
sought ways to simplify the bits that sit between the engine and a
shader, especially as it necessarily expanded with each generation
and new technology. With modern bindless rendering, I feel empowered
to simplify to a degree I can be satisfied, and for me that's a bunch
of interconnected arrays, shaders that operate on them, and a
renderer that acts as a simple translation layer. This minimizes
surface area of the renderer (and API abstractions), removes some
mental overhead, and reduces opportunities for bugs. Further, every
rendering goal to be solved from then on can begin with the
assumption that any GPU accessible data can be trivially accessed,
leaving more time for doing the work rather than worrying about the
boilerplate apparatus.




Bonus Section: Bindless Tips

If you haven't used bindless before, here are some suggestions and
guard rails for implementation as you get started:

Only use bindless for read-only resources like geometry and textures.
You can create resources in their read-only state without having to
worry about tracking resource state for bindless access afterwards.
The validation layer is great at catching mistakes with barriers and
read-write access with explicitly bound resources, it's helpful to
lean into that until you have a good reason not to.

Whether you use bindless descriptor tables or index into the
descriptor heap, set clear bounds for where your bindless resources
exist to help you validate access yourself, both on the CPU and in
shaders. I recommend having separate sections for where buffers and
textures live to further validate.

Fill any unused descriptor in your bindless heap with error resources
(error texture, error buffer). This way you won't eat a GPU crash/
hang when you make a mistake, and it will be easy to see errors in
your renderer and in graphics debuggers from recognizable error
resources.

If you dynamically stream resources, frame-buffer your bindless
descriptors so that you can write changes to descriptors at specific
indices while the GPU is using those from the previous frame. It's a
little more work to also buffer the changes to update each frame's
descriptors, but it's worth it.

Consider how much easier texture replacement is with bindless,
through streaming or otherwise. All you have to do is replace the
descriptor at the associated index, and everything will just work
automatically.


Contact

  * alexdtardif@gmail.com
  * LinkedIn