Part 2: Baby's first FFI

8 minutes read •


We’ve successfully skimmed the high-level overview of libva in the previous part, so with our newly acquired thorough understanding (ha!), let’s think about how we’re even going to bind to this very obviously C API from Rust.

Every language is different. C is extremely low level, C# has a garbage collector, Rust has its memory ownership model with the borrow checker, and these features all result in the languages’ respective compilers emitting completely different binaries that would violently explode if interleaved. But programmers can’t standardise on a single language, so an immense amount of effort has gone into developing standards that define ways for one language to call functions written in another language.

You could say that REST is a way to call a function that is written in another language. Same for gRPC. Web browsers call into code that isn’t just written in a different language, but is running on a completely separate computer too. But the highest performance language-independent interop standard is…

…that most languages can pretend to be C.

Pretending to be C

For Rust, this feature is called Foreign Function Interface (FFI). You can define a boundary of functions that travel from the cosy embrace of the borrow checker to the wild west of external code.

You can either have a statically linked bundle of external functions that get assimilated into our executable, or link to functions stored in shared libraries at runtime.

We’ll need to link against the drivers provided by the users’ machine, which will be stored in a shared library. Dynamic linking can be achieved in two ways: Before our program executes using ld, or manually linking to the shared object through our own code.

Both have their positives and negatives:

In this case, since we’re writing a library that others might depend on as an optional optimisation for their processing, it’s better to take matters into our own hands. This way we can throw an error to the library’s user if VA-API isn’t available on the system, and they can fall back to whatever other implementation they have in store.

Rust has a library for this purpose that makes the process trivial even across platforms: libloading. It’s what Ferrovanadium uses as well, so we’re staying on the beaten path.

Speaking of Ferrovanadium, I’m immediately going to lift an idea from that codebase and write a macro to simplify loading the VA-API functions.

Thinking in macro

Let’s first look at FeV’s implementation

macro_rules! dylib {
    (
        pub struct $strukt:ident;

        $(
            fn $func:ident( $( $name:ident : $t:ty ),* $(,)? ) $( -> $ret:ty )?;
        )+
    ) => {
        $(
            pub(crate) type $func = unsafe extern "C" fn( $( $name : $t ),* ) $( -> $ret )?;
        )+

        pub struct $strukt {
            $(
                $func: $func,
            )+
        }

        #[allow(unused)]
        impl $strukt {
            fn load() -> Result<Self, libloading::Error> {
                unsafe {
                    let libname = concat!(stringify!($strukt), ".so").replace('_', "-");
                    let lib = libloading::Library::new(&libname)?;

                    let this = Self {
                        $(
                            $func: *lib.get(concat!(stringify!($func), "\0").as_bytes())?,
                        )+
                    };

                    // Ensure the library is never unloaded.
                    std::mem::forget(lib);

                    Ok(this)
                }
            }

            pub fn get() -> Result<&'static Self, &'static libloading::Error> {
                static CELL: OnceLock<Result<$strukt, libloading::Error>> = OnceLock::new();
                match CELL.get_or_init(Self::load) {
                    Ok(this) => Ok(this),
                    Err(e) => Err(e),
                }
            }

            $(
                pub(crate) unsafe fn $func( &self, $( $name : $t ),* ) $( -> $ret )? {
                    (self.$func)($($name),*)
                }
            )+
        }
    };
}

That’s a lot of macro code, but there’s no reason to worry, let’s break it down slowly and carefully.

Basic declarative macros in Rust In case you (like me for the longest time) were terrified of Rust’s macro syntax, I can assure you that there’s nothing much to be afraid of, especially if you keep things simple.

We’re definitely going to write and interpret a few macros during the course of developing this as-of-yet-unnamed library, so the following is a very simple crash course mostly paraphrased from the excellent Little Book of Rust Macros:

Rust supports two types of macros: declarative and procedural ones. We’ll likely only need declarative ones, so procedural macros are left as an exercise for the reader.

To define a new declarative macro, we start by calling macro_rules!:

macro_rules! new_macro_name {

}

This creates a completely blank macro that can be invoked as new_macro_name!. A macro like this is going to replace itself and its list of parameters with some code that is defined within it, kind of like how C does it, except much smarter than a simple find-replace.

Rust doesn’t support function overloading in normal code, but declarative macros live and die by overloads. A macro can have multiple sets of parameters that it can accept, with each set being able to generate completely independent code.

Each of these overloads has to be listed between the braces following the new macro’s name, with a syntax not too dissimilar to match’s: (parameter list) => {substituted code};

Parameters have a type, which limits what sort of value they can match. Instead of having integers and strings and the like, we have Rust syntactic elements, like blocks, identifiers, expressions, lifetimes…

macro_rules! new_macro_name {
  ($param:ident) => {println!("{:?}", param)}
}

But we aren’t limited to just specifying explicit parameters, we can make our parameter list read small parts of a longer predefined sequence, like regex capture groups.

macro_rules! new_macro_name {
  (this is a $param:expr) => {println!("{:?}", param)}
}

new_macro_name!(this is a "string");

We can also capture a variable multiple times and then iterate over each capture

macro_rules! new_macro_name {
  ($($multiple:expr)*) => {
    $(println!("{:?}", multiple);)*
  };
}

There are a lot more supported parameter types, but this should be enough information to understand what we’re about to dig through. Read more

We define a macro called dylib that has a single rule. This rule matches one line that defines a struct (pub struct $strukt:ident;), then a list of function declarations in the shape fn $func:ident( $( $name:ident : $t:ty ),* $(,)? ) $( -> $ret:ty )?;.

(
    pub struct $strukt:ident;

    $(
        fn $func:ident( $( $name:ident : $t:ty ),* $(,)? ) $( -> $ret:ty )?;
    )+
)

The structure definition is pretty straightforward. It captures an identifier and stores it as $strukt. The next three lines are a bit more involved:

  $(
      fn $func:ident( $( $name:ident : $t:tx ),* $(,)? ) $( -> $ret:ty )?;
      // ^            ^   ^                   ^  ^       ^ Capture the optional return value type as $ret
      // |            |   |                   |  - There might be a single trailing comma
      // |            |   |                   - The list is separated by a comma
      // |            |   - Capture the name of the function parameter as $name, and the parameter type as $t
      // |            - This can happen multiple times, or not at all
  )+  // - Capture the function name as $func
//^ Capture at least one instance of the contained rule

The rule is then expanded into the following code:

// Create a type for each declared function that is an unsafe extern "C" function pointer
$(
    pub(crate) type $func = unsafe extern "C" fn( $( $name : $t ),* ) $( -> $ret )?;
)+

// Build a structure called $strukt that stores a function pointer for each function we've declared 
pub struct $strukt {
    $(
        $func: $func,
    )+
}

// Implement a few members for the structure above, with #[allow(unused)] to suppress some potentially errant lints
#[allow(unused)]
impl $strukt {

    // Define a function that loads the dynlib using libloading and returns a handle to the structure, or the error if the load fails.
    fn load() -> Result<Self, libloading::Error> {
        unsafe {
            // Divine the filename for the .so file from the name of the structure
            let libname = concat!(stringify!($strukt), ".so").replace('_', "-");
            
            // Load the library using libloading
            let lib = libloading::Library::new(&libname)?;
            
            // Individually fetch the location of the declared functions from the loaded library file.
            let this = Self {
                $(
                    $func: *lib.get(concat!(stringify!($func), "\0").as_bytes())?,
                )+
            };

            // Ensure the library is never unloaded.
            std::mem::forget(lib);
            
            // We're done
            Ok(this)
        }
    }
    
    // Define a function that gets the library handle for any public users.
    pub fn get() -> Result<&'static Self, &'static libloading::Error> {
        // Once we load a shared object, we don't have to do it again, so we store the handle in a static OnceLock
        static CELL: OnceLock<Result<$strukt, libloading::Error>> = OnceLock::new();
        
        // If the OnceLock has data, return that. Otherwise call the load function that we've defined earlier.
        match CELL.get_or_init(Self::load) {
            Ok(this) => Ok(this),
            Err(e) => Err(e),
        }
    }

    // Define convenience functions on the structure that call the individual functions using the pointers stored in the structure
    $(
        pub(crate) unsafe fn $func( &self, $( $name : $t ),* ) $( -> $ret )? {
            (self.$func)($($name),*)
        }
    )+
}

Simple enough, though I see a few points where this API could be improved:

[!WARNING] I’ve also made a mistake here that will come and bite me in a coming post.

Turns out that since if you drop the libloading::Library, the dynamic library gets unloaded and all function pointers are then pointing into invalid memory. Fun little SEGFAULT opportunity.

The solution is to store the Library inside the $strukt.

Decouple the structure name from the library name

Only requires a very simple edit. I think it would be the most ergonomic if the “fake” struct definition that the macro expands into the real deal had an additional field:

//...
(
  const LIB_NAME: &str = $libname:literal;
  pub struct $strukt:ident;

  $(
    fn $func:ident( $( $name:ident : $t:ty ),* $(,)? ) $( -> $ret:ty )?;
  )+
) => {
  //...

Of course, we don’t need to write any of that const LIB_NAME stuff before the parameter, but I like that it keeps the macro invocation looking like plain Rust code. And forcing the end user to write LIB_NAME means that it’s very hard for them to misunderstand what the parameter is for.

We’re about to completely rewrite how the library name is selected, so we’ll get back to where this needs to be edited in once that’s done.

VA-API isn’t exclusive to Linux

Currently, the filename is hardcoded to end in .so in load, which limits us to only those platforms that name their dynamic libraries that way. Thankfully, we don’t necessarily have to be familiar with how platforms differ in their naming schemes, because there’s libloading::library_filename.

Adding this to our code is as simple as replacing the line defining libname in load with

let libname = libloading::library_filename($libname)?;

Much nicer.

Better load failure handling

This one could be fixed by using OnceLock::get_or_try_init. It’s a nightly-only experimental API, and I’d prefer to stay on stable Rust, so let’s just make a note to fix this once get_or_try_init is stabilised.

What to load?

VA-API’s dynamic libraries are split into multiple blocks. There’s the core file, libva, that contains system-agnostic function calls operating on a display, then there are smaller, platform-specific libraries that implement features like converting from a window handle to a VADisplay or presenting a VASurface to a window.

We also have to think about which subset of a dynamic library we’d like to bind with, since we’ll have to manually specify the signature of each function individually. For now let’s not worry about compatibility with earlier versions of the library and fallbacks, though later on, it might be interesting to implement polyfills of sorts that implement a common subset of capabilities which achieve their functions differently depending on what calls are available.

So in the next part we are going to map out what we require for an absolutely minimal demo and get started on the actual FFI layer.