DeviceArray — Stream, Queue and Context

This is the advanced companion to DeviceArray Reference. You need it only if you run your own GPU code against the buffer, for example a CUDA or OpenCL kernel, or a framework such as PyTorch, CuPy, or OpenCV. It covers how the SDK orders its GPU work against your stream or queue, and when you must share a CUDA or OpenCL context.

Stream and Queue Ordering 

A stream (CUDA) or command queue (OpenCL) is an ordered list of GPU work: whatever you place on it runs in order, one item after the next. The SDK runs its capture and processing on a stream (or queue) of its own, separate from any you create.

When you call a method that returns a DeviceArray, such as imageDeviceArray() or devicePointsXYZ(), you hand it your own stream or queue, wrapped in StreamOrQueue. The SDK then makes your stream wait until its own work has finished. Anything you place on your stream after the call is therefore guaranteed to see finished data, and the CPU never has to wait.

Two streams / queues in play 

        flowchart LR
  subgraph sdk [SDK-owned]
    SDKStream["SDK stream / queue"]
    captureOp(["capture / processing"])
  end
  subgraph user [User-owned]
    UserStream["User stream / queue"]
    userKernelOp(["your kernel or copy"])
  end

  captureOp --> SDKStream
  userKernelOp --> UserStream

  acqOp(["imageDeviceArray(userStream)"])
  SDKStream --> acqOp
  acqOp -.-> UserStream

  classDef zividClass fill:#4A8FA4,stroke:#34323D,color:#FFFFFF
  classDef api fill:#91D2C8,stroke:#4A8FA4,color:#000000
  class SDKStream,UserStream zividClass
  class captureOp,userKernelOp,acqOp api

The dashed arrow represents the ordering established by that call. The user stream is made to wait on an event recorded on the SDK stream. No CPU-side sync happens; both streams keep running asynchronously.

Timeline of a typical call sequence 

        sequenceDiagram
  participant Host
  participant SDK as SDK stream / queue
  participant User as User stream / queue

  Host->>SDK: capture / processing work
  Note right of SDK: producer runs asynchronously
  Host->>SDK: imageDeviceArray(userStream)
  SDK-->>User: record event, User waits on it
  Host-->>Host: DeviceArray returned (no host sync)
  Host->>User: launch consumer kernel(devicePointer)
  Note right of User: kernel is ordered after SDK work
  Host->>User: copyToHost(dst, userStream)
  Note right of User: D2H copy enqueued, call returns
  Host->>User: synchronizeStream(userStream)
  User-->>Host: blocks until copy completes
  Host->>Host: read dst on CPU

Key properties 

The call is non-blocking on the host and returns as soon as the ordering is set up.
devicePointer() is a plain accessor; it performs no synchronization.
The buffer is already valid to use on the same stream / queue you passed in.
Host-side accessors that copy to CPU (copyToHost(), toArray2D(), toArray1D(), and the free toImage(buffer, streamOrQueue) overloads) also return immediately.
Call synchronizeStream on the same stream / queue before reading the host destination.

Passing the SDK’s own stream / queue 

If you do not have a stream or queue of your own, pass computeDevice().sdkStreamOrQueue(). The SDK’s work and your work then share the same stream, so the wait step has nothing to wait for and costs nothing. This is the recommended pattern when you only need the raw device pointer for a framework that does not expose its own stream handle.

Choosing between StreamOrQueue types 

StreamOrQueue is a small tagged wrapper. Only the member matching the active backend is read; the other must be null.

On a CUDA build, pass CUDAStreamPtr{ stream }. A null stream is allowed and refers to CUDA’s default / legacy stream.
On an OpenCL build, pass OpenCLCommandQueuePtr{ queue }. The queue handle must be non-null.

Both wrappers convert implicitly to StreamOrQueue, so these methods take a single argument at the call site.

GPU Context Sharing 

Note

You only need this section if you create your own CUDA or OpenCL context, or want the SDK to share a context with another framework. With the default setup, where the SDK creates its own context and you use the CUDA runtime in your own code, everything already shares one context and there is nothing to do.

A context is the GPU runtime’s container for allocations, streams, and queues. A buffer allocated in one context cannot be read by a stream or queue that belongs to a different context. The buffer behind a DeviceArray lives in the context the SDK uses, so any stream or queue you use to consume it must belong to that same context. CUDA and OpenCL differ in how much of this they set up for you.

CUDA 

The CUDA runtime creates and manages one shared context per device for you, behind the scenes (CUDA calls this the primary context). As long as everything in your process talks to the same GPU through the CUDA runtime, all streams and device pointers share that one context automatically. This is what the SDK uses by default. The simplest setup is therefore to let the SDK create its own CUDA context and to use the CUDA runtime API in your own code as well. No extra setup is needed, and any CUstream you pass when requesting a device array is compatible.

If you instead have your own CUDA driver-API context (a CUcontext created with cuCtxCreate, or a context owned by another framework), you must give it to the SDK when constructing Application. Otherwise the SDK creates a separate context, and the DeviceArray it hands back is not usable from your streams.

CUcontext cuContext = ...;
Zivid::Application zivid{ Zivid::CUDAContextPtr{ cuContext } };

OpenCL 

OpenCL has no equivalent of a runtime-managed primary context. Every cl_mem and cl_command_queue is tied to a specific cl_context, and a queue can only enqueue work on buffers that live in the same context. If you pass an OpenCLCommandQueuePtr whose queue lives in a different context than the SDK’s, the ordering call fails and the device pointer cannot be safely consumed by your kernels.

You therefore have to make one context common to both the SDK and your code. There are two ways to do it.

Option 1: give the SDK your context up front. Construct Application with OpenCLContextPtr{ clContext }. The SDK creates its internal queue in that context, and any queue you already own in it is compatible.

cl_context clContext = ...;
Zivid::Application zivid{ Zivid::OpenCLContextPtr{ clContext } };

Option 2: adopt the SDK’s context. If you can create your OpenCL objects after Application is constructed, retrieve the SDK’s context and create your queues in it.

auto *sdkContext = static_cast<cl_context>(camera.computeDevice().nativeContext());
cl_command_queue userQueue = clCreateCommandQueueWithProperties(sdkContext, device, nullptr, &err);

In both cases you can pass OpenCLCommandQueuePtr{ userQueue } to any method that returns a DeviceArray, and the returned array is safe to use on that queue.

Note

createDeviceArrayView (OpenCL overload) has the same requirement. The cl_mem you wrap must be created in the SDK’s context, otherwise the SDK cannot enqueue work against it.