In part 3, we finished wrapping a minimal subset of functions required to get a VADisplay up and running, however we won’t be able to verify our work without implementing some core functionality. It would also be nice if we had some sort of simple executable that we could use for testing the functions we have implemented so far, and in the suite of VAAPI tools, the simplest one I can think of is vainfo.

$ vainfo
Trying display: wayland
vainfo: VA-API version: 1.22 (libva 2.22.0)
vainfo: Driver version: Mesa Gallium driver 25.2.6-arch1.1 for AMD Radeon 760M Graphics (radeonsi, phoenix, LLVM 21.1.4, DRM 3.64, 6.17.6-arch1-1)
vainfo: Supported profile and entrypoints
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSlice
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointEncSlice
      VAProfileHEVCMain10             : VAEntrypointVLD
      VAProfileHEVCMain10             : VAEntrypointEncSlice
      VAProfileJPEGBaseline           : VAEntrypointVLD
      VAProfileVP9Profile0            : VAEntrypointVLD
      VAProfileVP9Profile2            : VAEntrypointVLD
      VAProfileAV1Profile0            : VAEntrypointVLD
      VAProfileAV1Profile0            : VAEntrypointEncSlice
      VAProfileNone                   : VAEntrypointVideoProc

It’s a basic utility for listing what version of VAAPI is installed on your system, and what profiles (codecs) and entrypoints (operations) your combination of hardware and driver supports. Such feature enumeration is going to be vital for all applications, so it would be a good first target to implement.

How to VAInfo?

If we look back to the example we’ve read through in part 1, we can see that there are query functions for listing entrypoints for a profile. And if we skim the function list in the API documentation, we can see a similar one for querying supported profiles too. We’ll also need to implement vaInitialize and vaTerminate, as well as a few ancillary queries to show library version and driver information. So let’s get started!

VADisplay and a VAAPI instance

To do anything with VAAPI, we’re going to need to initialise it on the VADisplay returned by the platform-specific module. It would be nice if the library’s end users didn’t have to worry about this detail, so the Display type Vaudeville will expose should take care of the whole display handle -> initialised VAAPI instance flow.

This display handle will also be required all over the user’s application, so I’ll create a separate DisplayOwner as well as the user-facing Display that will contain a reference counted handle to the DisplayOwner.

Something like the following:

struct DisplayOwner {
    display_handle: VADisplay,
    core: Core,
}

#[derive(Clone)]
struct Display {
    pub(crate) display: Arc<DisplayOwner>
}

Also note that I’ve opted to store a handle to the loaded dynamic library functions inside the DisplayOwner. This is hopefully going to make fetching the handles a little more efficient.

For getting a suitable display handle, let’s utilise the raw_window_handle interoperability crate. Then we can have a platform-agnostic Display::new() that calls out to the correct backend.

pub fn new(display_handle: RawDisplayHandle) -> Result<Self, DisplayCreationError> {
    let display = platforms::display_from_raw_handle(display_handle)?;
    Ok(Self::new_from_va_display(display)?)
}

platforms::display_from_raw_handle() matches on the RawDisplayHandle type:

pub(crate) fn display_from_raw_handle(
    handle: RawDisplayHandle,
) -> Result<VADisplay, DisplayConversionError> {
    match handle {
        RawDisplayHandle::Xlib(handle) => x11::display_from_xlib_handle(handle),
        RawDisplayHandle::Wayland(handle) => wayland::display_from_wayland_handle(handle),
        other => Err(DisplayConversionError::UnsupportedDisplayType(other)),
    }
}

And the platform-specific modules do the actual work:

pub(crate) fn display_from_xlib_handle(
    display_handle: XlibDisplayHandle,
) -> Result<VADisplay, DisplayConversionError> {
    use crate::display::platforms::DisplayConversionError::*;
    let lib = X11::get().map_err(|e| DylibLoad("X11", e))?;
    let display = match display_handle.display {
        None => {
            todo!("Request default display when using EGL?")
        }
        Some(ptr) => ptr.as_ptr(),
    };
    unsafe {
        let display = lib.vaGetDisplay(display);
        display.ok_or(VAGetDisplay)
    }
}

It would be nice to enable users of the library to insert an uninitialised VADisplay they acquired somewhere else into Vaudeville. To allow this without much duplication, the initialisation is put into the separate function Display::new_from_va_display(), which then calls the DisplayOwner to initialise and take ownership of the provided handle.

After initialisation, it’ll also hook the VAAPI logging callbacks into the corresponding Rust logging functions.

impl Display {
    /// Takes an **uninitialised** [`VADisplay`] handle and turns it into an initialised [`Display`]
    /// for use with Vaudeville.
    fn new_from_va_display(display_handle: VADisplay) -> Result<Self, DisplayInitError> {
        Ok(Self {
            display: DisplayOwner::new(display_handle)?.into(),
        })
    }
}

impl DisplayOwner {
    fn new(display_handle: VADisplay) -> Result<Self, DisplayInitError> {
        let core = Core::get()?;
        let mut minor = 0;
        let mut major = 0;

        unsafe { core.vaInitialize(display_handle, &mut major, &mut minor) }
            .to_result()?;

        unsafe {
            core.vaSetErrorCallback(display_handle, Self::error_handler, std::ptr::null_mut());
            core.vaSetInfoCallback(display_handle, Self::info_handler, std::ptr::null_mut());
        }

        Ok(Self {
            display_handle,
            core,
        })
    }

    extern "C" fn error_handler(_: *mut std::ffi::c_void, message: *const std::ffi::c_char) {
        let message = unsafe { CStr::from_ptr(message) };
        error!("{}", message.to_string_lossy());
    }

    extern "C" fn info_handler(_: *mut std::ffi::c_void, message: *const std::ffi::c_char) {
        let message = unsafe { CStr::from_ptr(message) };
        info!("{}", message.to_string_lossy());
    }
}

And to complete the Display’s lifecycle, let’s not forget about cleaning up after ourselves. To do this, let’s implement a custom destructor for DisplayOwner. To do this in Rust, we just need to implement Drop on the type.

impl Drop for DisplayOwner {
    fn drop(&mut self) {
        let status = unsafe { self.core.vaTerminate(self.display_handle) };
        if let Err(e) = status.to_result() {
            error!("Failed to terminate VAAPI Display: {}", e);
        }
    }
}

Starting vainfo

If we start a new binary crate within the workspace called vaudeville-vainfo, we can start writing some actual end user code.

First off, we want to use Winit to get a raw_window_handle::DisplayHandle we can feed into Display::new:

use winit::raw_window_handle::HasDisplayHandle;

fn main() {
    let event_loop = winit::event_loop::EventLoop::new().unwrap();
    let display_handle = event_loop.display_handle().unwrap();
    let display = vaudeville::display::Display::new(display_handle.as_raw()).unwrap();
}

Of course the real vainfo automatically tries multiple display options in order until it finds one it can work with, but let’s keep things simple at first.

With a display handle acquired, it’s time to query the data we need to log:

• VAAPI version and library implementation
• Driver name and version
• Supported profiles and entrypoints

VAAPI version is easy, we already receive it from the vaInitialize invocation. Let’s store that with the DisplayOwner so it can be requested by the user.

#[derive(Debug)]
pub(crate) struct DisplayOwner {
    pub display_handle: VADisplay,
    pub core: &'static Core,
    pub version: VAVersion,
}

[...]

fn new(display_handle: VADisplay) -> Result<Self, DisplayInitError> {
        [...]
        let mut version = VAVersion {
            major: 0,
            minor: 0,
        };

        unsafe { core.vaInitialize(display_handle, &mut version.major, &mut version.minor) }
        [...]
}

This lets us print the VA-API version, however the API doesn’t seem to expose the library version. I wonder where vainfo gets it from…

vainfo: VA-API version: 1.22 (libva 2.22.0)

The line within vainfo’s source code that prints this string can be found here. And if we run a search for what defines LIBVA_VERSION_S we find that it’s injected into the source code directly at build time. I guess we could put the Vaudeville version there instead.

To get the version of a crate to print in Rust, we can make use of the environment variables Cargo sets before calling rustc, CARGO_PKG_VERSION. Might as well expose it at the library level.

pub fn library_string() -> &'static str {
    concat!("Vaudeville ", env!("CARGO_PKG_VERSION"))
}

And now we can print the first line of our vainfo output:

println!("vainfo: VA-API version: {} ({})",
    display.version(),
    vaudeville::library_string()
);

The next line is going to be much simpler, it just gets the vendor string using vaQueryVendorString and prints the result. Just have to implement querying the vendor string for Display:

pub fn vendor_string(&self) -> Option<&CStr> {
    //SAFETY: VADisplay is known to be initialised, and the CStr's lifetime is bound to the
    //        Display so the returned string will stay valid.
    unsafe {
        let string = self
            .display
            .core
            .vaQueryVendorString(self.display.display_handle);
        if string != std::ptr::null() {
            Some(CStr::from_ptr(string))
        } else {
            None
        }
    }
}

…then use that implementation to print:

println!("vainfo: Driver version: {}", 
        display.vendor_string()
            .unwrap_or_default()
            .to_string_lossy()
);

let value: &std::ffi::CStr = Default::default();

#[stable(feature = "cstr_default", since = "1.10.0")]
impl Default for &CStr {
    #[inline]
    fn default() -> Self {
        const SLICE: &[c_char] = &[0];
        // SAFETY: `SLICE` is indeed pointing to a valid nul-terminated string.
        unsafe { CStr::from_ptr(SLICE.as_ptr()) }
    }
}

Querying profiles and entrypoints

The final step of the basic vainfo output is the list of profiles and entrypoints.

For this we just need to call vaQueryConfigProfiles to get the supported profiles. Then once we have the profiles, we can call vaQueryConfigEntrypoints with each profile.

As a safety mechanism, I’d like to wrap the profiles and entrypoints into structs so we can use the Rust type system to enforce that we only operate using profile and entrypoint combinations that are valid.

So let’s create a Profile struct that stores the Display and the associated VAProfile:

pub struct Profile {
    pub(crate) profile: VAProfile,
    pub(crate) display: Display,
}

Then we can add a method to Display that enumerates the supported profiles and wraps them:

pub fn profiles(&self) -> VAResult<impl Iterator<Item = Profile>> {
    // Allocate the required memory and initialise it to a valid placeholder
    let mut max_profile_count = self.max_num_profiles();
    let mut profile_vec = vec![VAProfile::None; max_profile_count as usize];

    // Request supported profiles
    unsafe {
        self.display.core.vaQueryConfigProfiles(
            self.handle(),
            profile_vec.as_mut_ptr(),
            &mut max_profile_count,
        ).to_result()?
    }

    // Make sure the returned profile count is valid.
    let max_profile_count = max_profile_count.try_into()
        .map_err(|_| {
            error!("Invalid max_profile_count received from VAAPI: {max_profile_count}");
            VAError::Unknown
        })?;

    // Wrap each VAProfile into a Profile as they are collected from the iterator
    Ok(profile_vec.into_iter()
        .take(max_profile_count)
        .map(|profile| {
            Profile {
                profile,
                display: self.clone(),
            })
        }))
}

Since we’re once again receiving enum values from C, we run into the same potential for UB that was mentioned in the previous part. And just like in the previous part, we are going to ignore it for the moment.

self.max_num_profiles() just calls vaMaxNumProfiles to get us the size of array to allocate for the profile list, then we just take the actual length of the array returned by the vaQueryConfigProfiles invocation and iterate over the response, validating each value as we encounter them and wrapping them into a Profile.

Getting entrypoints for a profile is practically the same code, just defined as a member function within Profile instead of Display, and also wrapping the associated VAProfile within the returned Entrypoint values, since not all profiles implement the same set of entrypoints.

Then we can just iterate over the options within our vainfo implementation to list everything:

println!("vainfo: Supported profile and entrypoints");
for profile in display.profiles().unwrap() {
    for entrypoint in profile.entrypoints().unwrap() {
        println!("{:?}: {:?}", profile.handle(), entrypoint.handle());
    }
}

And finally we can cargo run our very own vainfo!

/vaudeville-vainfo> cargo run vainfo
vainfo: VA-API version: 1.22 (Vaudeville 0.0.1)
vainfo: Driver version: Mesa Gallium driver 25.2.6-arch1.1 for AMD Radeon 760M Graphics (radeonsi, phoenix, LLVM 21.1.4, DRM 3.64, 6.17.6-arch1-1)
vainfo: Supported profile and entrypoints
H264ConstrainedBaseline: VLD
H264ConstrainedBaseline: EncSlice
H264Main: VLD
H264Main: EncSlice
H264High: VLD
H264High: EncSlice
HEVCMain: VLD
HEVCMain: EncSlice
HEVCMain10: VLD
HEVCMain10: EncSlice
JPEGBaseline: VLD
VP9Profile0: VLD
VP9Profile2: VLD
AV1Profile0: VLD
AV1Profile0: EncSlice
None: VideoProc

Woohoo! We finally have something to show for all our effort.

In the next part we will finally tackle some of the enum UB.