Chapter 102: Kernel Subsystems: Virtual File System (VFS) and Device Drivers

Chapter Objectives

• Understand the architectural role of the Virtual File System (VFS) as an abstraction layer in the Linux kernel.
• Analyze the four primary objects of the VFS model: Superblock, Inode, Dentry, and File objects.
• Implement a functional character device driver on the Raspberry Pi 5 that interfaces with the VFS.
• Configure the Linux kernel build system to support custom modules and cross-compilation for the ARMv8 architecture.
• Debug kernel-space interactions using procfs, sysfs, and kernel log buffers.

Introduction

In the realm of embedded Linux development, few concepts are as pervasive and powerful as the Unix philosophy that “everything is a file.” Whether an engineer is reading sensor data from an I2C bus, configuring network interfaces, or simply accessing a text document on an SD card, the mechanism for interaction remains consistently identical: open, read, write, and close. This unification of diverse hardware and software resources into a single, coherent namespace is not a happy accident, but the result of a sophisticated kernel subsystem known as the Virtual File System (VFS).

For the embedded developer working with platforms like the Raspberry Pi 5, the VFS serves as the critical bridge between user-space applications and the underlying hardware complexity. Without the VFS, a developer would need to write specific code to talk to a FAT32 filesystem, different code for EXT4, and entirely unique routines to communicate with a UART controller. The VFS abstracts these implementation details, allowing the system to treat a physical LED, a partition on an NVMe drive, and a process information list as uniform entities.

This chapter explores the theoretical underpinnings of the VFS, dissecting how the kernel routes a user’s request to the appropriate hardware driver or filesystem handler. We will move beyond the abstract by implementing a custom character device driver for the Raspberry Pi 5. By doing so, you will see exactly how the kernel creates the illusion of a file to represent physical hardware, providing a robust foundation for building complex embedded systems that interact seamlessly with the physical world.

Technical Background

The Universal Interface: Philosophy of the VFS

The Virtual File System acts as the kernel’s software switchboard. It is an abstraction layer that sits between the application layer (user space) and the concrete implementations of filesystems and device drivers. When a programmer calls a standard library function like fopen() in C or open() in Python, the C library invokes a system call (syscall) that traps into the kernel. The kernel does not immediately look at the disk; instead, it hands the request to the VFS. The VFS’s responsibility is to interpret this request and dispatch it to the specific code capable of handling it.

Imagine a highly organized international logistics center. Customers (user applications) submit orders (system calls) using a standard form, regardless of what they are shipping. The logistics managers (VFS) read the destination address. If the destination is a local warehouse (an EXT4 partition), they route the order to the local warehouse team. If the destination is a ship at sea (a network filesystem like NFS), they route it to the maritime team. If the destination is a specialized factory (a device driver), they route it there. The customer never needs to know the logistics of ships, warehouses, or factories; they only need to know how to fill out the standard form. This decoupling allows the Linux kernel to support dozens of different filesystems and thousands of hardware devices transparently.

graph TD
    %% Node Definitions
    Start((<b>App Layer</b><br>User Process)):::primary
    Syscall["<b>Syscall Trigger</b><br>open<i>/dev/vfs_led</i>"]:::primary
    KernelEntry{<b>Kernel Boundary</b><br>Validate Memory?}:::decision
    
    VFSSwitch[<b>VFS Switchboard</b><br>Lookup Path in Dcache]:::process
    InodeCheck{<b>Inode Found?</b>}:::decision
    
    DriverMatch[<b>VFS Dispatch</b><br>Match Major Number]:::process
    CdevCall["<b>Char Driver</b><br>Execute <i>driver_open()</i>"]:::system
    
    Error[<b>Error Return</b><br>EACCES / ENOENT]:::check
    Success((<b>Success</b><br>Return File Descriptor)):::endNode

    %% Workflow
    Start --> Syscall
    Syscall --> KernelEntry
    KernelEntry -- No --> Error
    KernelEntry -- Yes --> VFSSwitch
    
    VFSSwitch --> InodeCheck
    InodeCheck -- No --> Error
    InodeCheck -- Yes --> DriverMatch
    
    DriverMatch --> CdevCall
    CdevCall --> Success

    %% Class Definitions
    classDef primary fill:#1e3a8a,stroke:#1e3a8a,stroke-width:2px,color:#ffffff
    classDef endNode fill:#10b981,stroke:#10b981,stroke-width:2px,color:#ffffff
    classDef decision fill:#f59e0b,stroke:#f59e0b,stroke-width:1px,color:#ffffff
    classDef process fill:#0d9488,stroke:#0d9488,stroke-width:1px,color:#ffffff
    classDef check fill:#ef4444,stroke:#ef4444,stroke-width:1px,color:#ffffff
    classDef system fill:#8b5cf6,stroke:#8b5cf6,stroke-width:1px,color:#ffffff

The VFS Common Object Model

To manage this complexity, the VFS enforces a common object model. Regardless of how a specific filesystem stores data on a disk—or if the “disk” is actually a block of RAM or a network server—the VFS represents components using four primary objects: the Superblock, the Inode, the Dentry, and the File object. Understanding the interplay between these structures is essential for understanding how Linux organizes data.

The Superblock object is the anchor of a mounted filesystem. It contains the metadata describing the filesystem as a whole, such as the block size, the total number of blocks, and the filesystem type (e.g., whether it is ext4, vfat, or btrfs). When you mount a partition on the Raspberry Pi, the kernel reads the proprietary metadata from the disk and populates a generic VFS superblock structure in memory. This allows the kernel to query the status of the filesystem without needing to know the specific binary format of the disk headers.

The Inode (Index Node) is perhaps the most fundamental concept in Unix-like filesystems. An inode represents a specific object within the filesystem—a file or a directory. Crucially, the inode contains all the metadata about the file: permissions, owner, group, file size, timestamps, and pointers to the data blocks on the disk. However, the inode does not contain the file’s name. This distinction is vital. The inode is the kernel’s internal identifier for the file, essentially a unique number within that partition. This separation of identity (inode) from data allows features like hard links, where multiple filenames point to the exact same inode.

The Dentry (Directory Entry) is the object that links a filename to an inode. When the kernel traverses a path like /home/pi/logs/sensor.txt, it is the dentry cache (dcache) that facilitates this lookup. The VFS creates a dentry object for “home,” one for “pi,” one for “logs,” and one for “sensor.txt.” These objects are organized in a hierarchy that mirrors the directory structure. To improve performance, the dcache keeps these objects in RAM. If you access a file repeatedly, the VFS skips the expensive step of reading the disk to find the file and instead retrieves the inode directly from the dentry cache.

Finally, the File object represents an open file associated with a specific process. While the inode represents the static file on disk, the file object represents the dynamic interaction. It stores the current position of the cursor (file offset), the mode in which the file was opened (read-only, read-write), and a pointer to the underlying dentry. If two different processes open the same file, there will be two unique File objects and two file descriptors, but they will both point to the same Dentry and Inode.

Device Drivers and the Character Device

While filesystems manage data storage, device drivers manage hardware control. In the Linux model, device drivers are integrated into the VFS. This is achieved through special file types known as device nodes, typically found in the /dev directory. When a user opens a file like /dev/gpiomem or /dev/ttyUSB0, the VFS creates an inode just as it would for a text file. However, this inode contains a special flag indicating it is a character or block device, along with a “Major” and “Minor” number.

The Major Number serves as an index into the kernel’s device driver table. It tells the VFS which driver handles this file. For example, if a file has Major number 4, the VFS knows to route operations to the TTY (terminal) driver. The Minor Number is passed to the driver itself to distinguish between individual devices controlled by that driver (e.g., the first serial port vs. the second serial port).

Character devices (cdev) are the most common type of driver for embedded sensors and actuators. They process data as a stream of bytes, much like a pipe. When a user writes to a character device file, the VFS invokes the write function defined in the driver’s structure. This function effectively bridges the gap between software and the electrical signals on the Raspberry Pi 5’s GPIO pins.

System Calls and the Kernel Boundary

The transition from user space to kernel space is a strictly controlled operation. When an application calls read(), the CPU switches execution modes from unprivileged user mode (EL0 on ARMv8) to privileged kernel mode (EL1). The arguments passed by the user are validated—creating a security boundary. The kernel must ensure that the buffer provided by the user is a valid memory address and that the user has permission to read the requested file.

Once inside the kernel, the VFS looks up the file_operations structure associated with the open file. This structure is a collection of function pointers defined by the device driver. If the file is a standard disk file, these pointers point to filesystem-specific functions. If the file is a character device, they point to the driver’s functions. This polymorphism is the heart of the VFS; the kernel logic calling the function does not know or care whether it is talking to a hard drive or a temperature sensor.

sequenceDiagram
    autonumber
    participant U as User Space (EL0)
    participant V as VFS (EL1)
    participant D as LED Driver (EL1)
    participant H as GPIO Hardware (RPi 5)

    Note over U: App executes echo "1" > /dev/vfs_led
    U->>V: write(fd, "1", 1)
    Note right of V: VFS looks up f_ops->write
    V->>D: driver_write(buffer, count)
    
    rect rgb(240, 240, 255)
        Note over D: copy_from_user()
        D->>D: Parse '1'
        D->>H: gpiod_set_value(high)
        Note right of H: LED Physically Turns On
    end
    
    D-->>V: return 1 (bytes written)
    V-->>U: write() success

Practical Examples

In this section, we will develop a complete Linux kernel module (LKM) for the Raspberry Pi 5. This module will register a character device driver that allows users to control an LED by writing text to a file in /dev. We will cover the cross-compilation workflow, the driver code, and the testing procedure.

Build and Configuration Steps

Developing kernel modules requires the Linux kernel source code headers that match the kernel running on your target Raspberry Pi. Since compiling the entire kernel on the Pi itself can be slow, we will assume a cross-compilation environment on a Linux host (e.g., x86_64 Ubuntu) targeting the Raspberry Pi 5 (ARM64).

First, ensure your host environment has the necessary toolchain. You will need git, make, bison, flex, and the cross-compiler.

sudo apt update
sudo apt install git bc bison flex libssl-dev make libc6-dev libncurses5-dev
sudo apt install crossbuild-essential-arm64

Next, acquire the Raspberry Pi kernel source. We only need the headers and configuration to build a module, but downloading the full source is often the most reliable method.

mkdir -p ~/rpi-kernel
cd ~/rpi-kernel
git clone --depth=1 [https://github.com/raspberrypi/linux](https://github.com/raspberrypi/linux)
cd linux

You must configure the kernel source to match the Raspberry Pi 5. The RPi 5 uses the Broadcom BCM2712 chip, and the default configuration is bcm2712_defconfig.

# Set environment variables for cross-compilation
export ARCH=arm64
export CROSS_COMPILE=aarch64-linux-gnu-

# Apply the default configuration for RPi 5
make bcm2712_defconfig

# Prepare the modules (this ensures "Module.symvers" exists)
make modules_prepare

Tip: Ensure the kernel version of the source matches the running kernel on your Pi (uname -r). If they differ significantly, your module may refuse to load (insmod: ERROR: could not insert module: Invalid module format).

Code Snippets: The “Echo-LED” Driver

We will create a directory for our project and a file named vfs_led.c. This driver uses the kernel’s GPIO consumer interface to control a pin safely.

/* vfs_led.c - A simple VFS-based LED driver for Raspberry Pi 5 */

#include <linux/module.h>
#include <linux/init.h>
#include <linux/fs.h>
#include <linux/cdev.h>
#include <linux/uaccess.h>
#include <linux/gpio/consumer.h>
#include <linux/platform_device.h>

/* Meta Information */
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Embedded Linux Course");
MODULE_DESCRIPTION("A VFS interface for LED control");

/* Buffer for data storage */
#define BUFFER_SIZE 255
static char kernel_buffer[BUFFER_SIZE];

/* Variables for Device and Class */
static dev_t my_device_nr;
static struct class *my_class;
static struct cdev my_device;

/* GPIO Descriptor */
static struct gpio_desc *led_gpio;

/**
 * @brief Driver Open Function
 * Called when the user opens the device file
 */
static int driver_open(struct inode *device_file, struct file *instance) {
    printk("vfs_led: Device file opened\n");
    return 0;
}

/**
 * @brief Driver Close Function
 */
static int driver_close(struct inode *device_file, struct file *instance) {
    printk("vfs_led: Device file closed\n");
    return 0;
}

/**
 * @brief Driver Read Function
 * Allows the user to read the last written status
 */
static ssize_t driver_read(struct file *file, char *user_buffer, size_t count, loff_t *offs) {
    int to_copy, not_copied, delta;

    /* Calculate amount of data to copy */
    to_copy = min(count, (size_t)(BUFFER_SIZE - *offs));

    /* End of File check */
    if (to_copy <= 0) return 0;

    /* Copy data from kernel space to user space */
    not_copied = copy_to_user(user_buffer, kernel_buffer + *offs, to_copy);
    delta = to_copy - not_copied;

    *offs += delta;
    return delta;
}

/**
 * @brief Driver Write Function
 * Controls the LED based on user input
 */
static ssize_t driver_write(struct file *file, const char *user_buffer, size_t count, loff_t *offs) {
    int to_copy, not_copied, delta;
    char value;

    /* Get amount of data to copy */
    to_copy = min(count, (size_t)BUFFER_SIZE);

    /* Copy data from user space to kernel space */
    not_copied = copy_from_user(kernel_buffer, user_buffer, to_copy);
    delta = to_copy - not_copied;

    /* Parse the first character to control LED */
    if(delta > 0) {
        value = kernel_buffer[0];
        if(value == '1') {
            gpiod_set_value(led_gpio, 1);
            printk("vfs_led: LED ON\n");
        } else if(value == '0') {
            gpiod_set_value(led_gpio, 0);
            printk("vfs_led: LED OFF\n");
        }
    }

    return delta;
}

/* Map file operations to driver functions */
static struct file_operations fops = {
    .owner = THIS_MODULE,
    .open = driver_open,
    .release = driver_close,
    .read = driver_read,
    .write = driver_write
};

/**
 * @brief Module Init Function
 */
static int __init ModuleInit(void) {
    printk("vfs_led: Initializing module\n");

    /* 1. Allocate a device number dynamically */
    if(alloc_chrdev_region(&my_device_nr, 0, 1, "vfs_led") < 0) {
        printk("vfs_led: Could not allocate device number\n");
        return -1;
    }

    /* 2. Create device class (makes it appear in /sys/class) */
    if((my_class = class_create(THIS_MODULE, "vfs_led_class")) == NULL) {
        printk("vfs_led: Device class can not be created\n");
        goto ClassError;
    }

    /* 3. Create device file (makes /dev/vfs_led appear) */
    if(device_create(my_class, NULL, my_device_nr, NULL, "vfs_led") == NULL) {
        printk("vfs_led: Can not create device file\n");
        goto FileError;
    }

    /* 4. Initialize device file */
    cdev_init(&my_device, &fops);

    /* 5. Register device to kernel */
    if(cdev_add(&my_device, my_device_nr, 1) == -1) {
        printk("vfs_led: Registering of device to kernel failed\n");
        goto AddError;
    }

    /* 6. GPIO Setup - Hardcoding GPIO 17 for demonstration (Physical Pin 11) */
    /* Note: In production, platform drivers and Device Tree are preferred */
    led_gpio = gpio_to_desc(17); 
    if(!led_gpio) {
        printk("vfs_led: Error getting GPIO descriptor\n");
        goto GpioError;
    }
    
    if(gpiod_direction_output(led_gpio, 0) < 0) {
        printk("vfs_led: Error setting GPIO direction\n");
        goto GpioError;
    }

    return 0;

GpioError:
    cdev_del(&my_device);
AddError:
    device_destroy(my_class, my_device_nr);
FileError:
    class_destroy(my_class);
ClassError:
    unregister_chrdev_region(my_device_nr, 1);
    return -1;
}

/**
 * @brief Module Exit Function
 */
static void __exit ModuleExit(void) {
    gpiod_set_value(led_gpio, 0); /* Turn off LED */
    cdev_del(&my_device);
    device_destroy(my_class, my_device_nr);
    class_destroy(my_class);
    unregister_chrdev_region(my_device_nr, 1);
    printk("vfs_led: Module unloaded\n");
}

module_init(ModuleInit);
module_exit(ModuleExit);

File Structure Examples

To build this module, we need a Makefile. This file instructs the make utility to enter the kernel source directory, perform the build using our source file, and output the compiled module (.ko file).

Create a file named Makefile in the same directory as vfs_led.c:

# Makefile for Cross-Compiling the VFS LED Driver

# Point to the directory where we prepared the kernel source
KDIR := ~/rpi-kernel/linux

# The object file to build
obj-m += vfs_led.o

# Build targets
all:
	make -C $(KDIR) M=$(PWD) ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- modules

clean:
	make -C $(KDIR) M=$(PWD) clean

The directory structure on your host machine should look like this:

/home/user/project_drivers/
├── Makefile
└── vfs_led.c

Hardware Integration

For this exercise, we are controlling a simple LED. On the Raspberry Pi 5, the GPIO headers follow the standard 40-pin layout.

1. Connect the LED Anode (Long Leg) to Physical Pin 11 (GPIO 17).
2. Connect the LED Cathode (Short Leg) to a 330Ω resistor.
3. Connect the other end of the resistor to Physical Pin 6 (Ground).

Warning: Always verify your wiring before powering on the Raspberry Pi 5. The GPIO pins are 3.3V logic. Connecting them directly to 5V or shorting them to ground without a load can permanently damage the RP1 I/O controller.

Build, Flash, and Boot Procedures

With the source code and hardware ready, we proceed to compilation and deployment. Run the make command in your project directory on the host machine. If successful, you will see output indicating that vfs_led.ko has been generated.

Transfer this file to your Raspberry Pi 5. You can use scp (Secure Copy Protocol) over the network:

scp vfs_led.ko pi@<ip-address-of-pi>:~/

On the Raspberry Pi 5 terminal, load the module into the running kernel:

sudo insmod vfs_led.ko

Verify the module loaded successfully by checking the kernel ring buffer:

sudo dmesg | tail

You should see the message: vfs_led: Initializing module. Now, check for the device node. Because our code used class_create and device_create, udev should have automatically created the file entry for us.

ls -l /dev/vfs_led

Now, the moment of truth. We will use the VFS abstraction to control the hardware. By treating the LED as a file, we can use standard shell redirection:

# Turn the LED ON
echo "1" | sudo tee /dev/vfs_led

# Turn the LED OFF
echo "0" | sudo tee /dev/vfs_led

If the LED toggles, you have successfully bridged user space and hardware space using the VFS.

Common Mistakes & Troubleshooting

Developing for the kernel is unforgiving. A bug in user space crashes the application; a bug in kernel space can crash the entire operating system (Kernel Panic). Here are common pitfalls developers encounter when working with VFS and drivers.

Exercises

To solidify your understanding of the VFS and device drivers, perform the following exercises on your Raspberry Pi 5.

Exercise 1: The “Blinking” Script

Using the vfs_led driver you just created, write a shell script and a Python script that blinks the LED. The objective is to demonstrate that once the driver is registered with the VFS, the implementation language of the user application becomes irrelevant. Both scripts should simply open the file /dev/vfs_led and write characters to it.

Exercise 2: Adding Read Functionality

Modify the driver_read function in vfs_led.c. Currently, it returns the contents of the buffer. Change the logic so that it reads the actual state of the GPIO pin using gpiod_get_value(led_gpio). When a user runs cat /dev/vfs_led, it should report “1” if the LED is physically on and “0” if it is off, regardless of what is in the buffer. This exercises the round-trip communication from hardware to user space.

Exercise 3: Permission Management

By default, the device node /dev/vfs_led created by device_create is usually owned by root. Investigate how udev rules work. Create a file in /etc/udev/rules.d/99-vfs-led.rules that automatically sets the group of /dev/vfs_led to gpio and gives it write permissions (mode=”0660″). Add your user to the gpio group and verify you can control the LED without sudo. This highlights the relationship between the VFS and user-space permission managers.

Exercise 4: Exploring /sys

The Linux kernel exposes its object model via sysfs. Navigate to /sys/class/vfs_led_class on your Raspberry Pi. Explore the files in this directory. These are not files on a disk, but attributes generated dynamically by the kernel. Identify the relationship between the entries here and the class_create call in your C code.

Summary

• The Virtual File System (VFS) acts as an abstraction layer, allowing user-space applications to interact with filesystems and hardware devices using a unified set of system calls (open, read, write).
• The VFS architecture relies on four main objects: the Superblock (filesystem metadata), Inode (file metadata), Dentry (path and name resolution), and File (active open instance).
• Device Drivers register themselves with the VFS using Major and Minor numbers, allowing hardware peripherals to appear as nodes in the /dev directory.
• Developing Kernel Modules requires strict adherence to memory safety protocols, specifically using copy_from_user to bridge the gap between user space and kernel space.
• The Raspberry Pi 5 integration demonstrates how high-level software abstractions eventually translate to physical GPIO manipulations, demystifying the “everything is a file” philosophy.

Further Reading

1. Corbet, J., Rubini, A., & Kroah-Hartman, G. (2005). Linux Device Drivers, 3rd Edition. O’Reilly Media. (The definitive guide to Linux drivers).
2. Love, R. (2010). Linux Kernel Development, 3rd Edition. Addison-Wesley Professional. (Excellent coverage of VFS internals).
3. The Linux Kernel Organization. (2024). The Linux Kernel Documentation: VFS. kernel.org. (Official documentation on VFS structures).
4. Bootlin. (2024). Embedded Linux System Development Training Slides. (Up-to-date resources on kernel interfaces and cross-compilation).
5. Raspberry Pi Foundation. (2024). Raspberry Pi 5 Documentation: Kernel and Hardware. raspberrypi.com. (Specifics on the BCM2712 and RP1 architecture).
6. Simmonds, C. (2015). Mastering Embedded Linux Programming. Packt Publishing. (Practical approaches to building embedded systems).