GPU Access Tutorial

Introduction

The Zivid SDK keeps all captured data (point clouds, color images, depth maps, and SNRs) in GPU device memory throughout the processing pipeline. The GPU access APIs let you work with this data directly on the GPU without transferring it to CPU memory first, enabling zero-copy integration with frameworks like CUDA, OpenCL, PyTorch, CuPy, or OpenCV.

The central type is DeviceArray. It is a smart pointer to a GPU buffer with a reference count, which automatically frees the GPU memory when it is no longer in use. You get a DeviceArray from a PointCloud or Frame2D after capture, then pass its underlying device pointer to your GPU framework.

For a deeper reference of the memory layout per format, see the DeviceArray Reference reference article, and its DeviceArray — Stream, Queue and Context companion for the stream / queue ordering and context-sharing model.

Prerequisites

• Install Zivid Software and choose the GPU compute backend (CUDA or OpenCL) for your GPU.

• For Python: install zivid-python

Initialize, Connect and Capture

These are the standard Zivid steps and are not specific to GPU access: initialize the application (keeping it alive while the program runs), connect to the camera, and capture. See the capture tutorial for the details. The returned PointCloud and Frame2D objects hold their data in GPU device memory throughout.

C++

source

Zivid::Application zivid;
auto camera = zivid.connectCamera();
const auto frame = camera.capture2D3D(settings);

C#

source

var zivid = new Zivid.NET.Application();
var camera = zivid.ConnectCamera();
using (var frame = camera.Capture2D3D(settings))

Python

source

app = zivid.Application()
camera = app.connect_camera()

frame = camera.capture_2d_3d(settings)

The SDK selects a GPU based on the compute device configured in Config.yml. If no device is specified, it picks the GPU with the most compute cores. See Selecting Compute Device for how to choose a specific GPU.

Advanced: specify GPU context

For most CUDA use cases no explicit context setup is needed. The SDK integrates with the CUDA runtime context automatically. If you need to share a specific CUDA or OpenCL context (for example, to use the same context as PyTorch or your own OpenCL code), pass it to the Application constructor:

C++

// Share an existing CUDA context
CUcontext cuContext = ...;
Zivid::Application zivid{ Zivid::CUDAContextPtr{ cuContext } };

// Or share an existing OpenCL context
cl_context clContext = ...;
Zivid::Application zivid{ Zivid::OpenCLContextPtr{ clContext } };

Python

import zivid

# Share an existing CUDA context
app = zivid.Application(cuda_context=zivid.CUDAContextPtr(cu_context))

You can also retrieve the SDK’s own compute context and stream handle via ComputeDevice::nativeContext() and ComputeDevice::nativeStreamHandle() to share them with your own code without passing a context upfront.

For when a shared context is required and the difference between CUDA and OpenCL here, see GPU Context Sharing.

Get GPU Data from a 3D Point Cloud

After a 3D capture, retrieve DeviceArray objects for the point cloud buffers directly from GPU memory. All buffers stay on the GPU. No data is transferred to the CPU until you explicitly request it. Any function with the word copy in its name does exactly that: it copies data from the GPU to the CPU.

All functions that return a DeviceArray take a CUDA stream or OpenCL queue and synchronize the SDK’s compute into it before returning, so the returned buffer is already ordered against that stream/queue. Pass your own CUDA stream or OpenCL queue, or ComputeDevice::sdkStreamOrQueue() to use the SDK’s own stream when you do not have one of your own:

C++

const auto streamOrQueue = zivid.computeDevice().sdkStreamOrQueue();
const auto xyzBuffer = pointCloud.devicePointsXYZ(streamOrQueue);

C#

var streamOrQueue = zivid.ComputeDevice.SdkStreamOrQueue;
using (var xyzBuffer = pointCloud.DevicePointsXYZ(streamOrQueue))
{

Python

stream_or_queue = app.compute_device().sdk_stream_or_queue()
xyz_buffer = point_cloud.device_points_xyz(stream_or_queue)

The same pattern works for the other PointCloud buffers, such as XYZW (4×float, GPU-aligned), Z (float), SNR (float), normals (3×float), and color in any supported format:

C++

const auto xyzwBuffer = pointCloud.devicePointsXYZW(streamOrQueue);
const auto zBuffer = pointCloud.devicePointsZ(streamOrQueue);
const auto snrBuffer = pointCloud.deviceSNRs(streamOrQueue);
const auto normalBuffer = pointCloud.deviceNormalsXYZ(streamOrQueue);
const auto rgbaBuffer = pointCloud.imageDeviceArray<Zivid::ColorRGBA_SRGB>(streamOrQueue);

C#

using (var xyzwBuffer = pointCloud.DevicePointsXYZW(streamOrQueue))
using (var zBuffer = pointCloud.DevicePointsZ(streamOrQueue))
using (var snrBuffer = pointCloud.DeviceSNRs(streamOrQueue))
using (var normalBuffer = pointCloud.DeviceNormalsXYZ(streamOrQueue))
using (var rgbaBuffer = pointCloud.DeviceImageRGBA_SRGB(streamOrQueue))
{
    Console.WriteLine("Other buffers (elements): XYZW=" + xyzwBuffer.Size + ", Z=" + zBuffer.Size + ", SNR=" + snrBuffer.Size + ", Normals=" + normalBuffer.Size + ", RGBA=" + rgbaBuffer.Size);
}

Python

xyzw_buffer = point_cloud.device_points_xyzw(stream_or_queue)
z_buffer = point_cloud.device_points_z(stream_or_queue)
snr_buffer = point_cloud.device_snrs(stream_or_queue)
normals_buffer = point_cloud.device_normals_xyz(stream_or_queue)
rgba_buffer = point_cloud.device_image(stream_or_queue, zivid.PixelFormat.RGBA_SRGB)

Get GPU Data from a 2D Frame

After a 2D capture, retrieve a DeviceArray for the image:

C++

const auto imageBuffer = frame2D.imageDeviceArray<Zivid::ColorRGBA_SRGB>(streamOrQueue);

C#

using (var imageBuffer = frame2D.ImageDeviceArrayRGBA_SRGB(streamOrQueue))

Python

image_buffer = frame_2d.image_device_array(stream_or_queue, zivid.PixelFormat.RGBA_SRGB)

Available formats:

• ColorRGBA

• ColorRGBA_SRGB

• ColorBGRA

• ColorBGRA_SRGB

• ColorRGBAf

Inspect a DeviceArray

DeviceArray exposes metadata about the buffer:

C++

const auto shape = xyzBuffer.shape();
std::cout << "Backend: " << Zivid::toString(xyzBuffer.backend()) << '\n';
std::cout << "Shape: [" << shape[0] << ", " << shape[1] << ", " << shape[2] << "]\n";
std::cout << "Strides (elements): [" << xyzBuffer.strides()[0] << ", " << xyzBuffer.strides()[1] << "]\n";
std::cout << "SizeInBytes: " << xyzBuffer.sizeInBytes() << '\n';

C#

Console.WriteLine("Backend: " + xyzBuffer.Backend);
Console.WriteLine("Shape: [" + string.Join(", ", xyzBuffer.Shape()) + "]");
Console.WriteLine("Strides (elements): [" + string.Join(", ", xyzBuffer.Strides()) + "]");
Console.WriteLine("SizeInBytes: " + xyzBuffer.SizeInBytes);

Python

print(f"Backend: {xyz_buffer.backend}")
print(f"Shape: {xyz_buffer.shape}")
print(f"Strides (elements): {xyz_buffer.strides}")
print(f"SizeInBytes: {xyz_buffer.size_bytes}")

For what shape(), strides(), and stridesInBytes() return for each format, and how the elements are laid out in memory, see Data Layout.

Use the Device Pointer

Once you have a DeviceArray, pass its device pointer to your GPU framework for zero-copy processing. Because you supplied your stream or queue when you acquired the buffer, the SDK has already ordered its compute against it. The DeviceArray::devicePointer() call is therefore a plain accessor: it performs no synchronization and returns immediately, and you can use the pointer on that same stream/queue right away.

For exactly how the SDK orders its work against your stream or queue, see Stream and Queue Ordering.

CUDA

Provide your CUDA stream when you acquire the buffer. The SDK records an event on its internal stream and makes your stream wait on it, so the buffer is ordered against your stream and no CPU-side synchronization is needed:

C++

if(zivid.computeDevice().backend() == Zivid::ComputeBackend::cuda)
{
    const Zivid::CUDAStreamPtr cudaStream{};
    const auto cudaXyzBuffer = pointCloud.devicePointsXYZ(cudaStream);
    void *const devicePtr = cudaXyzBuffer.devicePointer();
    std::cout << "CUDA device pointer: " << devicePtr << '\n';
}

C#

if (xyzBuffer.Backend == Zivid.NET.ComputeBackend.Cuda)
{
    var cudaStream = new Zivid.NET.CUDAStreamPtr();
    using (var cudaXyzBuffer = pointCloud.DevicePointsXYZ(cudaStream))
    {
        var devicePtr = cudaXyzBuffer.DevicePointer();
        Console.WriteLine("CUDA device pointer: " + devicePtr);
    }
}

Python

if xyz_buffer.backend == zivid.ComputeBackend.cuda:
    cuda_stream = zivid.CUDAStreamPtr()
    cuda_xyz_buffer = point_cloud.device_points_xyz(cuda_stream)
    device_ptr = cuda_xyz_buffer.device_pointer()
    print(f"CUDA device pointer: {hex(device_ptr)}")

Note

Pass the stream from your GPU framework (e.g. stream.cudaPtr() in OpenCV, pytorch_stream.cuda_stream in PyTorch, or cupy_stream.ptr in CuPy) when you acquire the buffer to keep all operations on a single stream and avoid implicit synchronization.

OpenCL

Provide your OpenCL command queue when you acquire the buffer. The SDK inserts a barrier into your queue to wait for all prior SDK operations. devicePointer() then returns the raw cl_mem directly, no further sync needed:

C++

cl_command_queue userQueue = ...;  // must share context with Application
auto buffer = frame2D.imageDeviceArray<Zivid::ColorRGBA_SRGB>(Zivid::OpenCLCommandQueuePtr{ userQueue });
void *devicePtr = buffer.devicePointer();

C#

IntPtr userQueue = ...;  // cl_command_queue; must share context with Application
using (var buffer = frame2D.ImageDeviceArrayRGBA_SRGB(new Zivid.NET.OpenCLCommandQueuePtr { CommandQueue = userQueue }))
{
    var devicePtr = buffer.DevicePointer();
}

Python

buffer = frame_2d.image_device_array(zivid.OpenCLCommandQueuePtr(user_queue), zivid.PixelFormat.RGBA_SRGB)
device_ptr = buffer.device_pointer()

Framework interoperability

The device pointer is a generic GPU allocation, not tied to any particular framework. For the CUDA backend, it is a pointer allocated with cudaMallocAsync, so any framework that consumes CUDA device pointers can use it directly.

In Python, the simplest way to hand a DeviceArray to PyTorch is the CUDA Array Interface. zivid-python implements __cuda_array_interface__ on DeviceArray (CUDA backend only), so torch.as_tensor(device_array, device="cuda") imports the buffer into PyTorch zero-copy, with no manual pointer, shape, or stride handling:

Python

source

torch_image = torch.as_tensor(device_array, device="cuda")  # zero-copy via __cuda_array_interface__

Note

DLPack is a supported interop format. DeviceArray is not itself a DLPack tensor, but it exposes everything needed to build one: the device pointer (devicePointer()), the element data type (fixed by the format), the shape (shape()), the strides (strides() / stridesInBytes()), and the backend/device (backend()). Wrap these into a DLPack tensor for zero-copy hand-off to PyTorch, CuPy, or any other DLPack-consuming framework. This is primarily relevant to the CUDA backend, since that is what mainstream DLPack consumers support.

When you must produce a DLPack capsule yourself – for example in C++, or for a consumer that expects the __dlpack__ protocol directly – build a DLManagedTensor from the DeviceArray. The following wires its device pointer, shape, strides (in elements), element data type, and CUDA device into the tensor, deriving the DLPack data type generically from the format’s value type so the same code path handles both float point data and 8-bit color. The DeviceArray is kept alive by the tensor’s manager context, so the GPU memory stays valid for as long as a consumer holds the tensor:

C++

source

const auto shape = deviceArray.shape();
const auto strides = deviceArray.strides();

auto *context = new DLPackManagerContext<Format>{
    deviceArray,
    std::vector<int64_t>{ shape.begin(), shape.end() },
    std::vector<int64_t>{ strides.begin(), strides.end() },
};

auto *managedTensor = new DLManagedTensor{};
managedTensor->dl_tensor.data = context->deviceArray.devicePointer();
managedTensor->dl_tensor.device =
    DLDevice{ kDLCUDA, static_cast<int32_t>(cudaDeviceIdForPointer(context->deviceArray.devicePointer())) };
managedTensor->dl_tensor.ndim = static_cast<int32_t>(context->shape.size());
managedTensor->dl_tensor.dtype = dlDataType<Format>();
managedTensor->dl_tensor.shape = context->shape.data();
managedTensor->dl_tensor.strides = context->strides.data();
managedTensor->dl_tensor.byte_offset = 0;
managedTensor->manager_ctx = context;
managedTensor->deleter = &deleteDLManagedTensor<Format>;

Python

source

shape = tuple(int(dimension) for dimension in device_array.shape)
strides = tuple(int(stride) for stride in device_array.strides)

shape_array = (ctypes.c_int64 * len(shape))(*shape)
strides_array = (ctypes.c_int64 * len(strides))(*strides)

managed = _DLManagedTensor()
managed.dl_tensor.data = ctypes.c_void_p(device_array.device_pointer())
managed.dl_tensor.device = _DLDevice(_KDLCUDA, device_id)  # pylint: disable=no-value-for-parameter
managed.dl_tensor.ndim = len(shape)
managed.dl_tensor.dtype = _dl_data_type(device_array)
managed.dl_tensor.shape = shape_array
managed.dl_tensor.strides = strides_array
managed.dl_tensor.byte_offset = 0
managed.deleter = _release_managed_tensor  # pylint: disable=attribute-defined-outside-init

_MANAGER_CONTEXTS[ctypes.addressof(managed)] = (managed, shape_array, strides_array, device_array)

The resulting tensor can then be handed to any DLPack consumer. In Python, torch.from_dlpack imports it into PyTorch zero-copy:

Python

source

dlpack_source = ZividDeviceArrayDLPack(device_array, torch.cuda.current_device())
torch_image_via_capsule = torch.from_dlpack(dlpack_source)

Copy to CPU

When you need the data on the CPU, copy it explicitly. The DeviceArray::toArray2D() method schedules the device-to-host copy and returns immediately, before the copy has necessarily finished. Call synchronizeStream() on the same stream or queue you passed in before reading the host data:

C++

const auto hostPoints = xyzBuffer.toArray2D(streamOrQueue);
Zivid::synchronizeStream(streamOrQueue);

C#

var hostPoints = xyzBuffer.ToArray2D(streamOrQueue);
Zivid.NET.Compute.SynchronizeStream(streamOrQueue);

Python

host_points = xyz_buffer.copy_to_host_organized_array(stream_or_queue)
zivid.synchronize_stream(stream_or_queue)

Integration Examples

The following samples show complete end-to-end use of the GPU access APIs with popular frameworks:

• CaptureAndConvertImageWithOpenCVOnCuda (C++, OpenCV CUDA): Convert a 2D RGBA image to grayscale on the GPU using OpenCV CUDA without a CPU roundtrip.

• capture_and_process_image_with_cupy_on_cuda (Python, CuPy): Wrap a Zivid GPU buffer in a CuPy array for zero-copy GPU processing.

• capture_and_segment_image_with_pytorch_on_cuda (Python, PyTorch): Run semantic segmentation inference directly on GPU image data using a shared CUDA stream.

• capture_and_convert_to_dlpack_tensor_on_cuda (Python, PyTorch): Hand a DeviceArray to PyTorch two ways – the recommended CUDA Array Interface path (torch.as_tensor(device_array, device="cuda")) and an explicit, optional DLPack-capsule path built directly from the buffer.

• CaptureAndConvertToDlpackTensorOnCuda (C++, DLPack): Capture a 2D+3D frame and construct a DLManagedTensor for both the point cloud (PointXYZ) and color (ColorRGBA_SRGB), deriving the DLPack data type generically from the format and keeping each buffer alive via the tensor’s manager context for a DLPack consumer.

• capture_and_render_point_cloud_with_opengl_on_cuda (Python, OpenGL): Render a colored point cloud interactively by copying GPU buffers directly into OpenGL vertex buffers.

Conclusion

This tutorial shows how to use DeviceArray to access Zivid data on the GPU without copying it through CPU memory. Key points:

• Obtain a DeviceArray from PointCloud or Frame2D after capture, passing your CUDA stream or OpenCL queue (or computeDevice().sdkStreamOrQueue()).

• Call devicePointer() to get a raw device pointer for your GPU framework. No further synchronization is needed, since the buffer was already ordered against your stream/queue when you acquired it.

• Use toArray2D() followed by synchronizeStream() when you need the data on the CPU.

• Pass your own CUDA or OpenCL context to Application when you need full stream sharing with another framework.