GPU Access Tutorial

Introduction

The Zivid SDK keeps all captured data (point clouds, color images, and depth maps) in GPU device memory throughout the processing pipeline. The GPU access APIs let you work with this data directly on the GPU without transferring it to CPU memory first, enabling zero-copy integration with frameworks like CUDA, OpenCL, PyTorch, CuPy, or OpenCV.

The central type is DeviceArray, a reference-counted handle to a GPU buffer. You obtain it from a PointCloud or Frame2D after capture and pass the underlying device pointer to your GPU framework.

Prerequisites

• Install Zivid Software and choose the GPU compute backend (CUDA or OpenCL) for your GPU.

• For Python: install zivid-python

Initialize, Connect and Capture

These are the standard Zivid steps and are not specific to GPU access: initialize the application (keeping it alive while the program runs), connect to the camera, and capture. See the capture tutorial for the details. The returned PointCloud and Frame2D objects hold their data in GPU device memory throughout.

C++

source

Zivid::Application zivid;
auto camera = zivid.connectCamera();
const auto frame = camera.capture2D3D(settings);

C#

source

var zivid = new Zivid.NET.Application();
var camera = zivid.ConnectCamera();
using (var frame = camera.Capture2D3D(settings))

Python

source

app = zivid.Application()
camera = app.connect_camera()

frame = camera.capture_2d_3d(settings)

The SDK selects a GPU based on the compute device configured in Config.yml. If no device is specified, it picks the GPU with the most compute cores. See Selecting Compute Device for how to choose a specific GPU.

Advanced: specify GPU context

For most CUDA use cases no explicit context setup is needed. The SDK integrates with the CUDA runtime context automatically. If you need to share a specific CUDA or OpenCL context (for example, to use the same context as PyTorch or your own OpenCL code), pass it to the Application constructor:

C++

// Share an existing CUDA context
CUcontext cuContext = ...;
Zivid::Application zivid{ Zivid::CUDAContextPtr{ cuContext } };

// Or share an existing OpenCL context
cl_context clContext = ...;
Zivid::Application zivid{ Zivid::OpenCLContextPtr{ clContext } };

Python

import zivid

# Share an existing CUDA context
app = zivid.Application(cuda_context=zivid.CUDAContextPtr(cu_context))

You can also retrieve the SDK’s own compute context and stream handle via ComputeDevice::nativeContext() and ComputeDevice::nativeStreamHandle() to interop without passing a context upfront.

Get GPU Data from a 3D Point Cloud

After a 3D capture, retrieve DeviceArray objects for the point cloud buffers directly from GPU memory. All buffers stay on the GPU. No data is transferred to the CPU until you explicitly request it.

All functions that return a DeviceArray take a CUDA stream or OpenCL queue and synchronize the SDK’s compute into it before returning, so the returned buffer is already ordered against that stream/queue. Pass your own CUDA stream or OpenCL queue, or ComputeDevice::sdkStreamOrQueue() to use the SDK’s own stream when you do not have one of your own:

C++

const auto streamOrQueue = zivid.computeDevice().sdkStreamOrQueue();
const auto xyzBuffer = pointCloud.devicePointsXYZ(streamOrQueue);

C#

var streamOrQueue = zivid.ComputeDevice.SdkStreamOrQueue;
using (var xyzBuffer = pointCloud.DevicePointsXYZ(streamOrQueue))
{

Python

stream_or_queue = app.compute_device().sdk_stream_or_queue()
xyz_buffer = point_cloud.device_points_xyz(stream_or_queue)

The same pattern works for the other PointCloud buffers, such as XYZW (4×float, GPU-aligned), Z (float), SNR (float), normals (3×float), and color in any supported format:

C++

const auto xyzwBuffer = pointCloud.devicePointsXYZW(streamOrQueue);
const auto zBuffer = pointCloud.devicePointsZ(streamOrQueue);
const auto snrBuffer = pointCloud.deviceSNRs(streamOrQueue);
const auto normalBuffer = pointCloud.deviceNormalsXYZ(streamOrQueue);
const auto rgbaBuffer = pointCloud.imageDeviceArray<Zivid::ColorRGBA_SRGB>(streamOrQueue);

C#

using (var xyzwBuffer = pointCloud.DevicePointsXYZW(streamOrQueue))
using (var zBuffer = pointCloud.DevicePointsZ(streamOrQueue))
using (var snrBuffer = pointCloud.DeviceSNRs(streamOrQueue))
using (var normalBuffer = pointCloud.DeviceNormalsXYZ(streamOrQueue))
using (var rgbaBuffer = pointCloud.DeviceImageRGBA_SRGB(streamOrQueue))
{
    Console.WriteLine("Other buffers (elements): XYZW=" + xyzwBuffer.Size + ", Z=" + zBuffer.Size + ", SNR=" + snrBuffer.Size + ", Normals=" + normalBuffer.Size + ", RGBA=" + rgbaBuffer.Size);
}

Python

xyzw_buffer = point_cloud.device_points_xyzw(stream_or_queue)
z_buffer = point_cloud.device_points_z(stream_or_queue)
snr_buffer = point_cloud.device_snrs(stream_or_queue)
normals_buffer = point_cloud.device_normals_xyz(stream_or_queue)
rgba_buffer = point_cloud.device_image(stream_or_queue, zivid.PixelFormat.RGBA_SRGB)

Get GPU Data from a 2D Frame

After a 2D capture, retrieve a DeviceArray for the image:

C++

const auto imageBuffer = frame2D.imageDeviceArray<Zivid::ColorRGBA_SRGB>(streamOrQueue);

C#

using (var imageBuffer = frame2D.ImageDeviceArrayRGBA_SRGB(streamOrQueue))

Python

image_buffer = frame_2d.image_device_array(stream_or_queue, zivid.PixelFormat.RGBA_SRGB)

Available formats: ColorRGBA, ColorRGBA_SRGB, ColorBGRA, ColorBGRA_SRGB, ColorRGBAf.

Inspect a DeviceArray

DeviceArray exposes metadata about the buffer:

C++

const auto shape = xyzBuffer.shape();
std::cout << "Backend: " << Zivid::toString(xyzBuffer.backend()) << '\n';
std::cout << "Shape: [" << shape[0] << ", " << shape[1] << ", " << shape[2] << "]\n";
std::cout << "Strides (elements): [" << xyzBuffer.strides()[0] << ", " << xyzBuffer.strides()[1] << "]\n";
std::cout << "SizeInBytes: " << xyzBuffer.sizeInBytes() << '\n';

C#

Console.WriteLine("Backend: " + xyzBuffer.Backend);
Console.WriteLine("Shape: [" + string.Join(", ", xyzBuffer.Shape()) + "]");
Console.WriteLine("Strides (elements): [" + string.Join(", ", xyzBuffer.Strides()) + "]");
Console.WriteLine("SizeInBytes: " + xyzBuffer.SizeInBytes);

Python

print(f"Backend: {xyz_buffer.backend}")
print(f"Shape: {xyz_buffer.shape}")
print(f"Strides (elements): {xyz_buffer.strides}")
print(f"SizeInBytes: {xyz_buffer.size_bytes}")

Use the Device Pointer

Once you have a DeviceArray, pass its device pointer to your GPU framework for zero-copy processing. Because you supplied your stream or queue when you acquired the buffer, the SDK has already ordered its compute against it. The DeviceArray::devicePointer() call is therefore a plain accessor: it performs no synchronization and returns immediately, and you can use the pointer on that same stream/queue right away.

CUDA

Provide your CUDA stream at acquisition. The SDK records an event on its internal stream and makes your stream wait on it, so the buffer is ordered against your stream and no CPU-side synchronization is needed:

C++

if(zivid.computeDevice().backend() == Zivid::ComputeBackend::cuda)
{
    const Zivid::CUDAStreamPtr cudaStream{};
    const auto cudaXyzBuffer = pointCloud.devicePointsXYZ(cudaStream);
    void *const devicePtr = cudaXyzBuffer.devicePointer();
    std::cout << "CUDA device pointer: " << devicePtr << '\n';
}

C#

if (xyzBuffer.Backend == Zivid.NET.ComputeBackend.Cuda)
{
    var cudaStream = new Zivid.NET.CUDAStreamPtr();
    using (var cudaXyzBuffer = pointCloud.DevicePointsXYZ(cudaStream))
    {
        var devicePtr = cudaXyzBuffer.DevicePointer();
        Console.WriteLine("CUDA device pointer: " + devicePtr);
    }
}

Python

if xyz_buffer.backend == zivid.ComputeBackend.cuda:
    cuda_stream = zivid.CUDAStreamPtr()
    cuda_xyz_buffer = point_cloud.device_points_xyz(cuda_stream)
    device_ptr = cuda_xyz_buffer.device_pointer()
    print(f"CUDA device pointer: {hex(device_ptr)}")

Note

Pass the stream from your GPU framework (e.g. stream.cudaPtr() in OpenCV, pytorch_stream.cuda_stream in PyTorch, or cupy_stream.ptr in CuPy) at acquisition to keep all operations on a single stream and avoid implicit synchronization.

OpenCL

Provide your OpenCL command queue at acquisition. The SDK enqueues a barrier on your queue that waits for all prior SDK operations, so devicePointer() then returns the raw cl_mem without further synchronization:

C++

cl_command_queue userQueue = ...;  // must share context with Application
auto buffer = frame2D.imageDeviceArray<Zivid::ColorRGBA_SRGB>(Zivid::OpenCLCommandQueuePtr{ userQueue });
void *devicePtr = buffer.devicePointer();

C#

IntPtr userQueue = ...;  // cl_command_queue; must share context with Application
using (var buffer = frame2D.ImageDeviceArrayRGBA_SRGB(new Zivid.NET.OpenCLCommandQueuePtr { CommandQueue = userQueue }))
{
    var devicePtr = buffer.DevicePointer();
}

Python

buffer = frame_2d.image_device_array(zivid.OpenCLCommandQueuePtr(user_queue), zivid.PixelFormat.RGBA_SRGB)
device_ptr = buffer.device_pointer()

Copy to CPU

When you need the data on the CPU, copy it explicitly. The DeviceArray::toArray2D() method enqueues the device-to-host copy on the stream or queue you pass and returns immediately; call synchronizeStream() on that same stream/queue before reading the host data:

C++

const auto hostPoints = xyzBuffer.toArray2D(streamOrQueue);
Zivid::synchronizeStream(streamOrQueue);

C#

var hostPoints = xyzBuffer.ToArray2D(streamOrQueue);
Zivid.NET.Compute.SynchronizeStream(streamOrQueue);

Python

host_points = xyz_buffer.copy_to_host_organized_array(stream_or_queue)
zivid.synchronize_stream(stream_or_queue)

Integration Examples

The following samples show complete end-to-end use of the GPU access APIs with popular frameworks:

• CaptureAndConvertImageWithOpenCVOnCuda (C++, OpenCV CUDA): Convert a 2D RGBA image to grayscale on the GPU using OpenCV CUDA without a CPU roundtrip.

• capture_and_process_image_with_cupy_on_cuda (Python, CuPy): Wrap a Zivid GPU buffer in a CuPy array for zero-copy GPU processing.

• capture_and_segment_image_with_pytorch_on_cuda (Python, PyTorch): Run semantic segmentation inference directly on GPU image data using a shared CUDA stream.

• capture_and_render_point_cloud_with_opengl_on_cuda (Python, OpenGL): Render a colored point cloud interactively by copying GPU buffers directly into OpenGL vertex buffers.

Conclusion

This tutorial shows how to use DeviceArray to access Zivid data on the GPU without copying it through CPU memory. Key points:

• Obtain a DeviceArray from PointCloud or Frame2D after capture, passing your CUDA stream or OpenCL queue (or computeDevice().sdkStreamOrQueue()).

• Call devicePointer() to get a raw device pointer for your GPU framework. No further synchronization is needed, since the buffer was already ordered against your stream/queue at acquisition.

• Use toArray2D() followed by synchronizeStream() when you need the data on the CPU.

• Pass your own CUDA or OpenCL context to Application when you need full stream sharing with another framework.