Runtime¶

The runtime is the component of Wave that is responsible for launching kernels. Wave compiles the kernels to a vmfb file that is then passed on to the iree-runtime [1].

iree-runtime accepts tensors using the widely adopted DLPack [2] format. Pytorch tensors are converted to DLPack tensors and passed to the runtime using the torch.to_dlpack function as shown below.

capsule = torch.to_dlpack(arg_tensor)

Wave Runtime¶

In scenarios where the latencies associated with torch.to_dlpack are unacceptable, the Wave runtime can be used to launch kernels directly. The Wave runtime is a C++ extension that uses nanobind [3] to interface with inputs from Wave. In order to enable the wave runtime, first install the wave-runtime pip package.

uv pip install -r requirements-wave-runtime.txt

Then modify the options as shown below.

options = WaveCompileOptions(
    ...
    wave_runtime=True,
    ...
)

# Compile the kernel.
kernel = wave_compile(options, kernel)

# Launch the kernel.
... = kernel(...)