Chapter 47: Special Filesystems: /proc/sys, and devtmpfs

Chapter Objectives

Upon completing this chapter, you will be able to:

  • Understand the concept of virtual filesystems and their critical role as an interface to the Linux kernel.
  • Explain the purpose and navigate the structure of the /proc filesystem to access process and system information.
  • Describe the /sys filesystem and its direct relationship to the kernel’s device model for hardware interaction.
  • Implement userspace scripts and tools to monitor system behavior and configure hardware by interacting with /proc and /sys.
  • Configure and utilize devtmpfs and udev for robust, dynamic device node management on an embedded system.
  • Debug common permission, driver, and configuration issues related to virtual filesystems in an embedded Linux environment.

Introduction

We are most familiar with those that manage data on persistent storage devices like SD cards, hard drives, or solid-state drives in the world of filesystems. Filesystems such as ext4 or FAT32 meticulously organize our documents, photos, and application binaries into files and directories on a physical medium. However, the Linux kernel employs a different, more abstract category of filesystems that do not store user data at all. These are virtual filesystems, and they serve not as storage containers, but as a dynamic and powerful Application Programming Interface (API) directly into the heart of the running kernel. This chapter explores the three most important of these: /proc/sys, and devtmpfs. Common Linux filesystems are:

Filesystem Storage Type Key Features Common Use Case
ext4 Block Devices (SD Card, eMMC, SSD) Journaling for reliability, widely used, large file support, good performance. Not power-cut safe without special care. Root filesystem on SD cards/eMMC for development systems or devices with reliable power.
SquashFS Block Devices / Memory Highly compressed, read-only filesystem. Excellent for reducing storage footprint. Firmware partitions that are updated as a whole unit. Ensures system integrity as it cannot be modified at runtime.
JFFS2 Raw NAND/NOR Flash (MTD) Designed for raw flash. Provides wear-leveling, compression, and is power-cut safe. Older or simpler devices with direct raw flash access. Often used for bootloaders or recovery partitions.
UBIFS Raw NAND Flash (MTD via UBI) Modern successor to JFFS2. Better performance on large NAND flash, better scalability, and faster mount times. The standard choice for read-write filesystems on modern embedded devices using raw NAND flash.
tmpfs RAM (Volatile) Extremely fast as it resides in memory. Contents are lost on reboot. Used for temporary data. Directories like /tmp, /var/run, and /var/log to avoid writing temporary data to flash storage.

Understanding these special filesystems is not merely an academic exercise; it is fundamental to modern Linux system administration and, most critically, to embedded systems development. They provide the primary mechanism through which userspace applications monitor system health, interact with hardware peripherals, and respond to changes in the system’s state. When you run common commands like topps, or lsusb, you are not invoking arcane magic but are simply using well-designed tools that read and format information presented by the kernel through /proc and /sys. For an embedded engineer working with a device like the Raspberry Pi 5, these filesystems are the bridge to controlling GPIO pins, reading from I2C sensors, and managing power states, often without writing a single line of kernel code. This chapter will demystify these file-based interfaces, transforming them from cryptic directories into a comprehensible and controllable view of the kernel’s inner world.

Technical Background

To truly appreciate the elegance of /proc/sys, and devtmpfs, we must first understand the abstraction layer that makes them possible: the Linux Virtual Filesystem (VFS). The VFS acts as a universal adapter for all filesystem interactions. When a userspace program calls open()read(), or write(), it uses a standardized system call. The VFS receives this call and translates it into the specific commands required by the underlying filesystem, whether it’s ext4 on an SD card or a network filesystem on a remote server. This design brilliantly decouples applications from the specifics of data storage.

Virtual filesystems take this abstraction a step further. They are registered with the VFS, so they look and feel like any other filesystem to userspace programs. However, the data they present is not read from a block device. Instead, when a program reads a file in a virtual filesystem, the VFS triggers a corresponding function within the kernel that generates the file’s content on the fly. This content is a direct representation of the kernel’s internal data structures—a live snapshot of system memory, process tables, or device states. Writing to certain files in these filesystems can trigger kernel functions that change system parameters or control hardware. This “everything is a file” philosophy is a cornerstone of Unix-like systems, and these special filesystems are its most powerful expression.

/proc: The Process and System Information Filesystem

The oldest of the virtual filesystems we will discuss is the procfs, universally mounted at /proc. As its name suggests, it was originally conceived in the early days of Linux as a mechanism for peering into the data structures of running processes. While its role has since expanded dramatically, this remains its primary function.

When you list the contents of /proc, you will see a collection of numbered directories alongside several other files. Each numbered directory corresponds to the Process ID (PID) of a currently running process on the system. These directories are ephemeral; they exist only for the lifetime of the process they represent. Exploring inside one of these directories, such as /proc/1234, is like performing a diagnostic scan on process 1234.

Within each process directory, a wealth of information is available. For instance, the file /proc/[pid]/cmdline contains the complete command, with arguments, that was used to launch the process. The /proc/[pid]/status file provides a more comprehensive, human-readable summary, including the process’s state (e.g., sleeping, running), its memory usage (VmSize, VmRSS), and the user and group IDs under which it is running. The /proc/[pid]/fd/ subdirectory is particularly useful for debugging, as it contains symbolic links to every file the process currently has open. This can be invaluable for diagnosing issues like “file in use” errors or resource leaks.

graph TB
    subgraph /proc Filesystem

        A["/proc"]
        style A fill:#1e3a8a,stroke:#1e3a8a,stroke-width:2px,color:#ffffff

        subgraph "Process-Specific Information"
            P1["PID 1 (init)"]
            P1234["PID 1234 (sshd)"]
            style P1 fill:#0d9488,stroke:#0d9488,stroke-width:1px,color:#ffffff
            style P1234 fill:#0d9488,stroke:#0d9488,stroke-width:1px,color:#ffffff

            subgraph "Directory: /proc/1234"
                F_cmdline["<b>cmdline</b><br><i>'sshd -D'</i>"]
                F_status["<b>status</b><br><i>State: Sleeping<br>VmRSS: 5124 kB</i>"]
                F_maps["<b>maps</b><br><i>Memory mappings</i>"]
                F_fd["<b>fd/</b> (directory)<br><i>Symlinks to open files</i>"]
            end
        end

        subgraph "System-Wide Information"
            S_cpuinfo["<b>cpuinfo</b><br><i>Processor type, features</i>"]
            S_meminfo["<b>meminfo</b><br><i>Memory usage stats</i>"]
            S_version["<b>version</b><br><i>Linux kernel version</i>"]
            S_interrupts["<b>interrupts</b><br><i>IRQ usage counts</i>"]
            S_cmdline["<b>cmdline</b><br><i>Kernel boot parameters</i>"]
            style S_cpuinfo fill:#8b5cf6,stroke:#8b5cf6,stroke-width:1px,color:#ffffff
            style S_meminfo fill:#8b5cf6,stroke:#8b5cf6,stroke-width:1px,color:#ffffff
            style S_version fill:#8b5cf6,stroke:#8b5cf6,stroke-width:1px,color:#ffffff
            style S_interrupts fill:#8b5cf6,stroke:#8b5cf6,stroke-width:1px,color:#ffffff
            style S_cmdline fill:#8b5cf6,stroke:#8b5cf6,stroke-width:1px,color:#ffffff
        end

        subgraph "Kernel Tunables"
            SYS["<b>sys/</b> (directory)"]
            style SYS fill:#f59e0b,stroke:#f59e0b,stroke-width:1px,color:#ffffff
            subgraph /proc/sys/net/ipv4
                IPF["<b>ip_forward</b><br><i>Read: 0<br>Write: 1 (to enable)</i>"]
            end
        end
    end

    A --> P1
    A --> P1234
    A --> S_cpuinfo
    A --> S_meminfo
    A --> S_version
    A --> S_interrupts
    A --> S_cmdline
    A --> SYS

    P1234 --> F_cmdline
    P1234 --> F_status
    P1234 --> F_maps
    P1234 --> F_fd

    SYS --> IPF

    classDef default fill:#f9f9f9,stroke:#333,stroke-width:2px;

Over time, /proc became a convenient location for the kernel to expose all sorts of system-wide information not related to a specific process. Files in the root of /proc, such as /proc/cpuinfo/proc/meminfo, and /proc/interrupts, provide detailed snapshots of the system’s hardware configuration and real-time operational statistics. Reading /proc/version reveals the exact kernel version that is running, while /proc/cmdline shows the parameters passed to the kernel at boot time.

Perhaps the most powerful aspect of /proc is its role as a runtime configuration interface, primarily through the /proc/sys directory. The files within this hierarchy do not just report information; many of them can be written to, allowing a system administrator with root privileges to tune kernel parameters without a reboot. A classic example is enabling IP packet forwarding for a device to act as a router. This is achieved simply by writing the character ‘1’ to the file /proc/sys/net/ipv4/ip_forward. This direct, immediate control is a powerful feature, but it also demands caution, as misconfiguration can lead to system instability. The unstructured growth of /proc eventually led to it being seen as a miscellaneous “dumping ground,” which prompted the development of a cleaner, more organized alternative for device and driver information: sysfs.

/sys: The System Filesystem and the Device Model

As the Linux kernel evolved, its ability to manage a vast array of complex hardware devices grew. The internal logic for this became known as the device model. This model organizes all system components—buses (like USB, I2C, SPI), devices (a USB mouse, an I2C temperature sensor), and the drivers that manage them—into a logical hierarchy. The kernel uses a fundamental data structure called a kobject (kernel object) to represent each of these items internally. These kobjects are linked together to form a tree that precisely mirrors the physical and logical arrangement of hardware in the system.

The /sys filesystem, or sysfs, was created to provide a clean, direct, and structured view of this internal device model to userspace. Unlike /proc, which grew organically and somewhat chaotically, /sys was designed from the ground up with a strict “one kobject, one directory” rule. The directory structure you see when you navigate /sys is a direct, one-to-one mapping of the kernel’s kobject hierarchy. This makes it an incredibly predictable and well-organized interface.

graph LR
    subgraph "Kernel Space (Internal Device Model)"
        direction TB
        K_Bus(Platform Bus<br>kobject)
        style K_Bus fill:#8b5cf6,stroke:#8b5cf6,stroke-width:2px,color:#ffffff
        K_Device(my_device<br>kobject)
        style K_Device fill:#0d9488,stroke:#0d9488,stroke-width:1px,color:#ffffff
        K_Driver(my_driver<br>kobject)
        style K_Driver fill:#0d9488,stroke:#0d9488,stroke-width:1px,color:#ffffff

        K_Bus -- "Contains" --> K_Device
        K_Device -- "Bound to" --> K_Driver
    end

    subgraph "Userspace View (/sys Filesystem)"
        direction TB
        S_Bus["/sys/bus/platform"]
        style S_Bus fill:#f8fafc,stroke:#64748b,stroke-width:1px,color:#1f2937
        S_Devices["/devices/my-device"]
        style S_Devices fill:#f8fafc,stroke:#64748b,stroke-width:1px,color:#1f2937
        S_Drivers["/drivers/my_driver"]
        style S_Drivers fill:#f8fafc,stroke:#64748b,stroke-width:1px,color:#1f2937
        S_Link["driver (symlink)"]
        style S_Link fill:#eab308,stroke:#eab308,stroke-width:1px,color:#1f2937

        S_Bus --> S_Devices
        S_Bus --> S_Drivers
        S_Devices --> S_Link
        S_Link -.-> S_Drivers
    end

    K_Bus -- "Mirrored As" --> S_Bus
    K_Device -- "Mirrored As" --> S_Devices
    K_Driver -- "Mirrored As" --> S_Drivers

    classDef default fill:#ffffff,stroke:#333,stroke-width:2px;

The /sys filesystem is organized into several key top-level directories, the most important of which are busclass, and devices.

The /sys/devices directory represents the true physical hierarchy of the system. It is the master map, showing how every device is connected, starting from a top-level “platform” device and branching down through various buses and bridges. While it is the most accurate representation, its structure can be complex and difficult to navigate for simple tasks.

The /sys/bus directory provides a different view, grouping devices by the type of bus they are connected to (e.g., i2cspiusb). Within /sys/bus/usb/devices, for example, you will find every USB device currently connected to the system. This view is useful for understanding how drivers are bound to devices on a specific bus.

For most day-to-day interaction, however, the /sys/class directory is the most useful. It provides an abstract, functional grouping of devices. For example, all network interfaces, whether they are onboard Ethernet, a USB Wi-Fi adapter, or a virtual bridge, will appear as subdirectories in /sys/class/net (e.g., eth0wlan0). All LEDs controllable by the system appear in /sys/class/leds, and all GPIO pins appear in /sys/class/gpio. This functional grouping allows applications to interact with devices based on what they do, without needing to know how they are physically connected.

The files within these directories are called attributes. They expose properties of the device or driver. Reading from an attribute file retrieves a value, like the MAC address of a network card from /sys/class/net/eth0/address. Writing to an attribute file can change a device’s state, such as controlling an LED’s brightness by writing a value to /sys/class/leds/led0/brightness. This mechanism is the modern, standard way for userspace to perform low-level hardware control in Linux.

devtmpfs: The Dynamic Device Filesystem

The final piece of our virtual filesystem puzzle is devtmpfs, which solves the long-standing problem of managing the /dev directory. The /dev directory is special; it contains device nodes or device files. These are not regular files; they are portals that connect userspace programs to device drivers in the kernel. When you read from or write to a device node like /dev/ttyS0 (a serial port), the VFS bypasses the storage-oriented filesystems and routes your I/O operations directly to the driver responsible for that serial port.

Historically, the /dev directory was static. A standard Linux distribution would ship with a /dev directory pre-populated with nodes for every conceivable device the system might ever have. This was inefficient, inflexible, and cluttered. If a new type of hardware was invented, a system administrator might need to manually create a device node for it using the mknod command.

The modern solution is a two-part system: devtmpfs and the udev daemon. devtmpfs is a lightweight, in-memory filesystem that the kernel mounts at /dev very early in the boot process. Its job is simple: whenever a kernel driver successfully initializes and registers a device, the kernel automatically creates a basic device node for it in devtmpfs. This ensures that device nodes only exist for hardware that is actually present and active.

However, devtmpfs only creates the essential nodes (e.g., sda for a disk). This is where the udev daemon comes in. Running in userspace, udev listens for events from the kernel that signal a change in the system’s device population (e.g., a device being added or removed). When devtmpfs creates a new node, udev receives an event. It then consults a set of rules, typically located in /etc/udev/rules.d/. Based on these rules, udev can perform a variety of essential “housekeeping” tasks: it can set specific ownership and permissions on the node, create stable, user-friendly symbolic links (e.g., /dev/disk/by-uuid/... which always points to the same disk regardless of its connection order), or even trigger scripts to load necessary firmware.

sequenceDiagram
    actor User
    participant Kernel
    participant devtmpfs
    participant udevd as udev Daemon

    rect rgb(230, 240, 255)
        Note over Kernel, udevd: Boot Process Starts
        Kernel->>Kernel: Initializes, mounts rootfs
        Kernel->>devtmpfs: Mounts empty devtmpfs at /dev
    end

    User->>Kernel: Plugs in USB Drive
    
    rect rgb(255, 250, 230)
        Note over Kernel, udevd: Device Detection & Node Creation
        Kernel->>Kernel: USB driver initializes
        Kernel->>devtmpfs: Creates node /dev/sda1
        devtmpfs-->>Kernel: Node created
        Kernel->>udevd: Sends "add" event for sda1
    end

    rect rgb(230, 255, 230)
        Note over udevd: Rule Processing & Device Setup
        udevd->>udevd: Receives event, checks rules in /etc/udev/rules.d/
        Note over udevd: Rule matches Vendor/Product ID
        udevd->>devtmpfs: Creates symlink /dev/my_backup_drive -> sda1
        udevd->>devtmpfs: Sets permissions on /dev/sda1
    end

For an embedded system like the Raspberry Pi 5, this dynamic duo is indispensable. It provides a clean, minimal /dev directory tailored to the detected hardware, supports seamless hot-plugging of devices like USB drives or sensors, and offers a powerful, customizable framework for managing device-specific policies from userspace.

Practical Examples

This section provides hands-on examples using a Raspberry Pi 5 running the standard Raspberry Pi OS. You will need the Raspberry Pi 5, a breadboard, one LED, a 330Ω resistor, and some jumper wires. All commands should be run in a terminal on the Raspberry Pi, either directly or via an SSH connection.

Exploring /proc for Process and System Information

Let’s begin by using /proc to inspect the system.

1. Inspecting a Running Process

First, we need a process to inspect. The SSH daemon, sshd, is a perfect candidate as it’s almost always running on a networked Raspberry Pi.

Bash
# Find the PID of the main sshd process
pid=$(pgrep -o sshd)
echo "The PID for sshd is: $pid"

# Now, let's explore its /proc entry
echo "--- Command Line ---"
# Use `cat` and `tr` to make the null-separated output readable
cat /proc/$pid/cmdline | tr '\0' ' '
echo "" # for a newline

echo "--- Process Status (excerpt) ---"
grep -E '^(Name|State|Pid|Uid|VmRSS)' /proc/$pid/status

echo "--- Open File Descriptors ---"
# List the files this process has open
ls -l /proc/$pid/fd

The output reveals the exact command used to start the daemon, its current state, and the files it’s using, including network sockets. This demonstrates how /proc provides a live, low-level view that tools like ps and lsof are built upon.

2. Scripting with /proc/meminfo

The files in /proc are plain text, making them easy to parse with scripts. Here is a simple Python script to monitor memory usage.

Create a file named mem_monitor.py:

Python
#!/usr/bin/env python3
import time

def get_memory_info():
    """Parses /proc/meminfo to get total and free memory."""
    mem_info = {}
    try:
        with open('/proc/meminfo', 'r') as f:
            for line in f:
                parts = line.split(':')
                if len(parts) == 2:
                    key = parts[0].strip()
                    value = parts[1].strip().split()[0] # Get the numeric value
                    mem_info[key] = int(value)
    except FileNotFoundError:
        print("Error: /proc/meminfo not found.")
        return None, None

    return mem_info.get('MemTotal'), mem_info.get('MemAvailable')

if __name__ == "__main__":
    print("--- Memory Monitor ---")
    print("Press Ctrl+C to exit.")
    try:
        while True:
            total_kb, avail_kb = get_memory_info()
            if total_kb is not None and avail_kb is not None:
                total_mb = total_kb / 1024
                avail_mb = avail_kb / 1024
                used_mb = total_mb - avail_mb
                used_percent = (used_mb / total_mb) * 100
                
                print(f"Total: {total_mb:.2f} MB | "
                      f"Available: {avail_mb:.2f} MB | "
                      f"Used: {used_percent:.2f}%")
            
            time.sleep(2)
    except KeyboardInterrupt:
        print("\nExiting.")

Make the script executable and run it:

Bash
chmod +x mem_monitor.py
./mem_monitor.py

T

his script demonstrates a common embedded use case: periodically polling a /proc file to monitor system health.

3. Tuning Kernel Parameters via /proc/sys

Warning: Modifying /proc/sys parameters can impact system stability. The following example is safe, but always research a parameter before changing it on a production system.

Let’s temporarily enable IP forwarding.

Bash
# First, check the current value. It should be 0 (disabled).
cat /proc/sys/net/ipv4/ip_forward

# Now, try to enable it. This will fail due to permissions.
# The shell performs the redirection before sudo runs the echo command.
sudo echo 1 > /proc/sys/net/ipv4/ip_forward
# bash: /proc/sys/net/ipv4/ip_forward: Permission denied

# The correct way is to use a shell with sudo's privileges
# or use the `tee` command.
sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"

# Verify the change
cat /proc/sys/net/ipv4/ip_forward
# The output should now be 1

# Remember to set it back to 0
sudo sh -c "echo 0 > /proc/sys/net/ipv4/ip_forward"

Controlling Hardware with /sys

The sysfs interface is the standard way to control simple hardware like GPIOs from userspace. Let’s blink an LED.

1. Hardware Setup

Connect the LED to your Raspberry Pi 5 as follows:

  • Connect the anode (longer leg) of the LED to GPIO pin 17.
  • Connect the cathode (shorter leg) of the LED to one end of the 330Ω resistor.
  • Connect the other end of the resistor to a Ground (GND) pin.

2. GPIO Control via Shell Commands

We will use the /sys/class/gpio interface.

Bash
# The GPIO pin we are using
GPIO_PIN=17

# Export the pin to make it available to userspace.
# This creates the /sys/class/gpio/gpio17 directory.
echo $GPIO_PIN | sudo tee /sys/class/gpio/export

# Set the pin's direction to output.
echo "out" | sudo tee /sys/class/gpio/gpio$GPIO_PIN/direction

# Turn the LED on
echo 1 | sudo tee /sys/class/gpio/gpio$GPIO_PIN/value
echo "LED should be ON. Waiting 3 seconds..."
sleep 3

# Turn the LED off
echo 0 | sudo tee /sys/class/gpio/gpio$GPIO_PIN/value
echo "LED should be OFF."

# Unexport the pin to clean up.
# This removes the /sys/class/gpio/gpio17 directory.
echo $GPIO_PIN | sudo tee /sys/class/gpio/unexport

Tip: We use echo <value> | sudo tee <file> here. The tee command reads from standard input and writes to both standard output and the specified file. Piping to sudo tee is a reliable way to write to root-owned files without the shell redirection issues seen earlier.

3. Automated Blinking with a Shell Script

Now, let’s create a script blink.sh to automate this.

Bash
#!/bin/bash

# A simple script to blink an LED connected to a GPIO pin.
# Usage: ./blink.sh <pin_number> <duration_seconds>

if [ "$#" -ne 2 ]; then
    echo "Usage: $0 <pin_number> <duration_seconds>"
    exit 1
fi

GPIO_PIN=$1
DURATION=$2
GPIO_PATH="/sys/class/gpio/gpio$GPIO_PIN"

# --- Cleanup function to run on exit ---
cleanup() {
    echo "Cleaning up..."
    echo 0 > "$GPIO_PATH/value" 2>/dev/null
    echo $GPIO_PIN > /sys/class/gpio/unexport 2>/dev/null
    exit 0
}

# Trap Ctrl+C (SIGINT) and script exit to run cleanup
trap cleanup SIGINT EXIT

# --- Main script ---
echo "Exporting GPIO pin $GPIO_PIN..."
echo $GPIO_PIN > /sys/class/gpio/export
# A small delay to allow the system to create the directory
sleep 0.1 

if [ ! -d "$GPIO_PATH" ]; then
    echo "Failed to export GPIO pin. Are you running as root?"
    exit 1
fi

echo "Setting direction to 'out'..."
echo "out" > "$GPIO_PATH/direction"

echo "Blinking LED on pin $GPIO_PIN for $DURATION seconds. Press Ctrl+C to stop."
end_time=$(( $(date +%s) + DURATION ))
while [ $(date +%s) -lt $end_time ]; do
    echo 1 > "$GPIO_PATH/value"
    sleep 0.5
    echo 0 > "$GPIO_PATH/value"
    sleep 0.5
done

Make it executable and run it as root:

Bash
chmod +x blink.sh
sudo ./blink.sh 17 10

This script demonstrates robust control, including cleanup, which is crucial in embedded applications.

Understanding devtmpfs and udev in Action

This example shows how devtmpfs and udev work together when a USB device is plugged in.

1. Observe the Initial State

Plug a USB flash drive into your Raspberry Pi 5. Now, use the dmesg command to see the kernel’s log.

Bash
dmesg | tail -n 10

You should see messages indicating a new USB device was detected, followed by the SCSI disk driver (sd) claiming the device and creating nodes like sda and sda1.

2. See the devtmpfs and udev Artifacts

The kernel, via devtmpfs, created the primary node. udev created the helpful links.

Bash
# See the raw device node created by devtmpfs
ls -l /dev/sda*

# See the stable symbolic links created by udev
ls -l /dev/disk/by-uuid/
ls -l /dev/disk/by-path/

The /dev/disk/by-uuid links are particularly valuable because they will always point to the correct partition, even if you plug in multiple drives in a different order.

3. Create a Custom udev Rule

Let’s create a rule that, when a specific USB drive is inserted, automatically creates a friendly symlink named /dev/my-backup-drive.

First, we need to find unique identifiers for our drive. The vendor and product IDs are perfect for this.

Bash
# Find your device in the list. Note its Bus and Device number.
lsusb
# Example output: Bus 001 Device 003: ID 0781:5581 SanDisk Corp. Ultra Fit

Here, the ID is 0781:5581. Now we create a rule file.

Bash
sudo nano /etc/udev/rules.d/99-usb-backup.rules

Add the following line to the file, replacing 0781 and 5581 with the IDs for your device.

Plaintext
# This rule creates a symlink for our specific backup drive
ACTION=="add", ATTRS{idVendor}=="0781", ATTRS{idProduct}=="5581", SYMLINK+="my-backup-drive"

Save the file (Ctrl+X, Y, Enter). Now, tell udev to reload its rules.

Bash
sudo udevadm control --reload-rules

Unplug your USB drive and plug it back in. Now check for the symlink:

Bash
ls -l /dev/my-backup-drive

You should see that the symlink has been successfully created, pointing to the correct sda device. This demonstrates the power of udev to enforce system policy on hardware events.

Common Mistakes & Troubleshooting

Navigating these virtual filesystems can sometimes be tricky. Here are some common pitfalls and their solutions.

Mistake / Issue Symptom(s) Troubleshooting / Solution
Permission Denied on Write Running sudo echo “1” > /sys/… fails with a “Permission denied” error message in the shell. The shell redirection happens before sudo runs.
Solution: Use a command that runs the write operation with root privileges, such as:
echo “1” | sudo tee /path/to/file
sudo sh -c ‘echo “1” > /path/to/file’
Device Node Not Found Hardware is connected (e.g., I2C sensor, USB device), but no corresponding node like /dev/i2c-1 or /dev/ttyUSB0 appears. This indicates a kernel driver issue.
Solution: Check kernel logs with dmesg for errors related to the device. The cause is often a missing kernel module (e.g., CONFIG_USB_SERIAL) or an incorrect Device Tree configuration.
Custom udev Rule Fails A new .rules file is created and udev is reloaded, but the expected symlink or script is not created/run when the device is plugged in. Usually a typo or incorrect attribute in the rule.
Solution: Use udevadm info -a -p /sys/path/to/device to inspect all available attributes. Compare this output carefully with your rule file to ensure keys like ATTRS{idVendor} are correct. Also check file permissions (should be 644).
Script Breaks After Update A script using a hardcoded path like /sys/devices/platform/… stops working after a kernel or system update. Physical device paths are not stable.
Solution: Modify the script to use the stable, functional paths provided under /sys/class/ (e.g., /sys/class/hwmon/hwmon0) or the stable symlinks created by udev in /dev/disk/by-uuid/.

Exercises

These exercises are designed to reinforce the concepts covered in this chapter, ranging from basic inspection to advanced configuration.

  1. Process Resource Monitor Script:
    • Objective: Write a shell script that takes a process name (e.g., kworkerpython3) as an argument and prints a formatted summary of its resource usage.
    • Requirements: The script should find the PID of the most resource-intensive process matching that name. It must then parse the /proc/[pid]/status and /proc/[pid]/io files to display the process’s PID, State, VmRSS (Resident Set Size memory), and read_bytes (total bytes read from storage).
    • Verification: Run the script against a known process, like bash, and compare its output with the raw data in the /proc files.
  2. System Uptime Formatter:
    • Objective: The /proc/uptime file provides two numbers: total system uptime in seconds and idle time in seconds. Write a Python script that reads this file and prints the system uptime in a human-readable format: “Uptime: X days, Y hours, Z minutes, W seconds.”
    • Requirements: The script must perform the necessary calculations to convert the total seconds into the day/hour/minute/second format.
    • Verification: Run your script and compare its output with the output of the standard uptime command. The “days, hours, minutes” part should match.
  3. Reading Onboard CPU Temperature:
    • Objective: The Raspberry Pi 5’s CPU temperature sensor is exposed via sysfs. Find the correct attribute file and write a script to monitor the CPU temperature.
    • Guidance: The temperature sensor is a “hardware monitoring” device, so start your search in /sys/class/hwmon. Find the directory (e.g., hwmon0), and within it, find the file that contains the temperature (its name is often tempX_input). The value is in millidegrees Celsius.
    • Requirements: Write a shell script that reads this file every 5 seconds, divides the value by 1000 to get degrees Celsius, and prints it to the console.
    • Verification: The script should output a reasonable temperature (e.g., CPU Temperature: 45.12 C).
  4. Custom udev Rule for USB-to-Serial Adapters:
    • Objective: Create a udev rule that assigns a persistent, friendly name to a USB-to-Serial adapter. These adapters typically appear as /dev/ttyUSB0, but this can change to /dev/ttyUSB1 if another is plugged in first.
    • Requirements: Use lsusb and udevadm info to find the vendor ID, product ID, and serial number of your adapter. Write a rule in /etc/udev/rules.d/ that uses these attributes to create a permanent symbolic link, for example, /dev/tty-my-adapter.
    • Verification: Reload the udev rules, re-plug the adapter, and verify that the /dev/tty-my-adapter symlink is created and points to the correct /dev/ttyUSBx device. This ensures that scripts and applications can always find the device using a predictable name.

Summary

This chapter provided a deep dive into the virtual filesystems that form the primary interface between userspace and the Linux kernel. By understanding and utilizing them, you gain direct control over the system’s processes and hardware.

  • Virtual Filesystems are an abstraction that presents kernel data structures as a file hierarchy, accessible with standard file I/O operations.
  • /proc (procfs) is the legacy virtual filesystem focused on exposing detailed information about running processes (/proc/[pid]) and a wide range of system-wide statistics and tunable parameters (/proc/cpuinfo/proc/sys).
  • /sys (sysfs) is the modern, structured filesystem that provides a direct representation of the kernel’s device model. It is organized by bus, class, and physical topology, and is the standard way to interact with hardware from userspace.
  • devtmpfs is a lightweight filesystem for /dev that works with the kernel to dynamically create device nodes only for hardware that is actually present and initialized.
  • udev is a userspace daemon that acts on kernel events to manage device nodes, set permissions, and create stable symbolic links, applying system policy to hardware.
  • Practical skills gained include scripting interactions with /proc and /sys, controlling GPIOs to blink LEDs, and writing custom udev rules to create persistent device names.

The knowledge from this chapter is a prerequisite for more advanced topics. As we move forward, we will see how the hardware that is exposed through /sys and devtmpfs is first described to the kernel using the Device Tree, the subject of our next chapter.

Further Reading

  1. The Linux Kernel Documentation for sysfs: The authoritative source. Available at https://docs.kernel.org/filesystems/sysfs.html
  2. The Linux Kernel Documentation for procfs: The official documentation for /proc. Available at https://docs.kernel.org/filesystems/proc.html
  3. man udev and man udev.rules: The system manual pages for the udev daemon and its rule syntax. They are the definitive reference for writing rules.
  4. Corbet, J., Rubini, A., & Kroah-Hartman, G. (2005). Linux Device Drivers, 3rd Edition. O’Reilly Media. While slightly dated, its chapters on kobjectsksets, and sysfs provide one of the clearest conceptual explanations of the device model available.
  5. Greg Kroah-Hartman’s Blog: As a key kernel developer and the original author of sysfs and udev, his blog posts often contain invaluable insights into the design and proper use of these subsystems.
  6. Raspberry Pi Documentation – GPIO: The official documentation from the Raspberry Pi Foundation on using GPIO pins, including the sysfs interface.
  7. LWN.net: A premier source for in-depth articles about Linux kernel development. Searching its archives for “sysfs” or “udev” will yield a history of their development and design decisions.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top