Avoiding CPU synchronization#
Mesh topology and projected triangle coverage can change the amount of storage
needed by a call. By default, drtk.pixel_grid_mesh() reads the largest
vertex and face counts back to the CPU before allocating its outputs. The
transparent renderers and source-transmittance functions similarly read the
number of tile/triangle intersections before allocating their sorting buffers.
These reads make the CPU wait for the GPU.
Supply output_capacity to the mesh builder and tile_capacity to the
renderer or transmittance function to avoid those reads. Calls still allocate
ordinary PyTorch tensors; capacities specify their sizes. Actual counts remain
on the GPU. Geometry, topology, and appearance may change between calls while
the allocated shapes stay fixed.
Choosing capacities#
pixel_grid_mesh(..., output_capacity=(vertices, faces)) sets the maximum
vertex and face counts per mesh, rather than across the batch. Each mesh
occupies a compact prefix of its outputs. Spare entries use the documented
padding values and contribute no gradients. The renderer accepts the padded
face array directly.
For an H-by-W cell grid, conservative bounds valid for all topology decisions are:
Mesh |
Vertex capacity |
Face capacity |
|---|---|---|
Without feathering |
|
|
With feathering |
|
|
tile_capacity bounds the total number of tile/triangle intersections
across the batch. Tiles contain 4-by-4 image pixels. Each triangle’s
projected bounding box contributes an entry for every tile it overlaps.
Source-transmittance queries also include a one-pixel halo around that box.
Rendering and transmittance therefore may need different capacities for the
same mesh and image size.
A conservative bound for N meshes with F face slots and an H-by-W output is
N * F * ((H + 3) // 4) * ((W + 3) // 4). This assumes every triangle
covers the entire image and can be much too large for a finely tessellated mesh.
Use a tighter bound when the geometry’s projected extent is constrained.
Capacities must cover all inputs that will be replayed; counts observed in a
single frame are not a bound on later motion or topology changes.
Larger capacities consume more memory and increase padding and sorting work. Removing a count readback alone does not guarantee a faster call.
Exceeding a capacity raises an asynchronous CUDA device assertion. As with other CUDA device assertions, the process must be restarted afterward. The operation does not silently drop geometry to fit the requested size.
CUDA graph capture#
Fixed-capacity calls also support torch.cuda.CUDAGraph. Capturing and
replaying a sequence of GPU operations reduces CPU launch overhead, which can
be useful when many small operations run together.
This example captures mesh construction, rendering, and gradients. Input storage remains fixed; update its contents before replaying the graph.
import torch
from drtk import pixel_grid_mesh, render_transparent
n, h, w = 2, 8, 8
depth = torch.ones(n, h, w, device="cuda", requires_grad=True)
displacement = torch.zeros(n, h, w, 3, device="cuda", requires_grad=True)
intrinsics = torch.eye(3, device="cuda")[None]
alive = torch.ones(n, h, w, device="cuda", dtype=torch.bool)
right = torch.ones(n, h, w - 1, device="cuda", dtype=torch.bool)
down = torch.ones(n, h - 1, w, device="cuda", dtype=torch.bool)
texture = torch.rand(1, 3, h, w, device="cuda")
mesh_capacity = (4 * h * w, 2 * h * w)
tile_capacity = n * mesh_capacity[1] * ((h + 3) // 4) * ((w + 3) // 4)
def step():
v, uv, _, faces, _, _, _ = pixel_grid_mesh(
depth, displacement, intrinsics, alive, right, down,
output_capacity=mesh_capacity,
)
# The identity intrinsics make projected XY equal to X/Z, Y/Z.
screen = torch.cat((v[..., :2] / v[..., 2:], v[..., 2:]), dim=-1)
rgb, alpha, _ = render_transparent(
screen, faces, uv, texture, h, w, tile_capacity=tile_capacity,
)
gradients = torch.autograd.grad(
rgb.square().mean() + alpha.mean(), (depth, displacement),
)
return rgb, gradients
stream = torch.cuda.Stream()
stream.wait_stream(torch.cuda.current_stream())
with torch.cuda.stream(stream):
for _ in range(3):
step()
torch.cuda.current_stream().wait_stream(stream)
graph = torch.cuda.CUDAGraph()
with torch.cuda.graph(graph, stream=stream):
image, gradients = step()
with torch.no_grad():
displacement[..., 0].add_(0.05)
alive[1, 3, 3] = False
graph.replay()
# image and gradients now contain the result for the updated inputs.
The texture and MLP renderer variants and all source-transmittance variants
accept the same tile_capacity option. Feature sampling with
drtk.sample_with_transmittance() already has fixed output shapes and
needs no capacity argument.
CUDA graph replay uses the same tensor addresses and operation structure on
every iteration. Copy new data into the captured inputs rather than replacing
them with new tensors. Changing batch dimensions, image dimensions, or
capacities requires a new capture. Read results on the CPU only when needed;
calling .item() or .cpu() each iteration would introduce another wait.