Chapter 78: IPC: (shm_open, ftruncate, mmap, munmap, shm_unlink)

Chapter Objectives

Upon completing this chapter, you will be able to:

• Understand the principles of POSIX shared memory as a high-performance Inter-Process Communication (IPC) mechanism.
• Implement a complete shared memory solution using the core POSIX API calls: shm_open, ftruncate, mmap, munmap, and shm_unlink.
• Configure and manage shared memory objects within the Linux virtual filesystem (/dev/shm).
• Design and debug multi-process applications on a Raspberry Pi 5 that efficiently share data using this technique.
• Analyze the performance trade-offs between POSIX shared memory and other IPC methods like pipes or sockets.
• Apply proper synchronization mechanisms (like semaphores, covered separately) to prevent race conditions in shared memory applications.

Introduction

In the world of embedded Linux, performance is paramount. Systems are often required to process vast amounts of data from sensors, network interfaces, or user inputs in real-time. This frequently involves multiple processes working in concert, each handling a specialized task. A camera process might capture raw video frames, a processing daemon could run computer vision algorithms, and a networking service might stream the results. The critical question then becomes: how do these processes exchange data with minimal overhead? While pipes, FIFOs, and sockets are robust IPC mechanisms, they all involve the overhead of kernel mediation—data must be copied from the user space of the sending process into a kernel buffer, and then copied again from the kernel buffer into the user space of the receiving process.

This is where POSIX shared memory offers a revolutionary advantage. It provides a mechanism for two or more processes to map a common memory region directly into their own virtual address spaces. Once this mapping is established, the kernel steps out of the way. A write operation performed by one process is instantaneously visible to all other participating processes, as if they were simply writing to a local variable. This “zero-copy” communication eliminates the system call overhead and buffer copying inherent in other IPC methods, making it the fastest and most efficient way to share large datasets. This chapter delves into the modern, file-backed approach to POSIX shared memory. You will learn how to create, manage, and utilize these powerful constructs to build high-performance, multi-process applications on your Raspberry Pi 5, a skill essential for any serious embedded Linux developer.

Technical Background

At its heart, the modern POSIX shared memory model is an elegant abstraction built upon the Linux kernel’s virtual filesystem (VFS) and memory management subsystems. Instead of introducing a completely separate namespace for shared memory objects, as older System V IPC did, the POSIX standard leverages the familiar concept of a file descriptor. The key insight is to treat a shared memory segment as a “memory-backed file” that doesn’t reside on a physical disk but exists purely as a kernel-managed object in RAM. This approach provides a clean, consistent, and permission-based API that integrates seamlessly with other file-oriented system calls.

The tmpfs Filesystem: The Home of Shared Memory

The foundation of this mechanism is a special-purpose filesystem called tmpfs. As its name implies, tmpfs is a temporary filesystem that stores all its files and directories in virtual memory. Anything written to tmpfs consumes RAM and will be lost upon reboot. On most modern Linux systems, including the Raspberry Pi OS, a tmpfs instance is mounted at /dev/shm. This directory serves as the public namespace for POSIX shared memory objects. When you create a shared memory object, you are effectively creating a file in this tmpfs mount. This has several important implications. First, standard filesystem tools like ls -l /dev/shm can be used to see active shared memory objects. Second, the standard Linux file permission model (owner, group, and other) applies directly to these objects, providing a familiar and robust security framework.

Creating and Opening a Shared Memory Object: shm_open

The journey into shared memory begins with the shm_open() system call. Its function is analogous to the standard open() call, but it is specifically designed for shared memory objects.

int shm_open(const char *name, int oflag, mode_t mode);

The name parameter is crucial. It must start with a forward slash (/) and must not contain any other slashes. For example, /my_shared_data is a valid name, but my_shared_data or /data/my_shared_data are not. This name corresponds to the filename that will be created under /dev/shm. So, shm_open("/my_shm_object", ...) will create a file at /dev/shm/my_shm_object.

The oflag argument is a bitmask that specifies how the object should be opened, mirroring the flags used in open(). The most common flags are:

• O_CREAT: Create the shared memory object if it does not exist.
• O_RDWR: Open the object for read-write access.
• O_RDONLY: Open for read-only access.
• O_EXCL: When used with O_CREAT, this ensures that the call fails if the object already exists. This is a vital atomic operation for guaranteeing that only one process—the “producer” or “server”—initializes the shared memory segment.

The mode argument specifies the file permissions (e.g., 0666 for read/write access for all users) to be applied to the new object if O_CREAT is used.

Upon success, shm_open() returns a standard, non-negative integer file descriptor. This descriptor is the handle that subsequent system calls will use to manipulate the object. If it fails, it returns -1 and sets errno.

Sizing the Object: ftruncate

A freshly created shared memory object has a size of zero bytes. It is merely a name in the tmpfs filesystem. Before it can be used to store data, it must be given a specific size. This is the job of the ftruncate() system call.

int ftruncate(int fd, off_t length);

This function takes the file descriptor returned by shm_open() and a length in bytes. It then extends (or truncates) the underlying object to match this length. This step is absolutely critical and is typically performed only once by the process that creates the object. All other processes that subsequently open the same object will see this pre-determined size. Attempting to access memory beyond this defined length will result in a SIGSEGV (Segmentation Fault) signal, just as with any other out-of-bounds memory access.

For example, to create a shared memory region to hold a structure SensorData, the creating process would call ftruncate(fd, sizeof(SensorData)).

Mapping Memory into the Address Space: mmap

At this point, we have a file descriptor pointing to a named, sized object in RAM. However, the process cannot yet access this memory directly. The final step is to map this object into the process’s own virtual address space using the powerful mmap() system call.

void *mmap(void *addr, size_t length, int prot, int flags, int fd, off_t offset);

mmap() is a versatile function used for many purposes, but in this context, its role is to connect the file descriptor from shm_open() to a usable memory pointer. Let’s break down the arguments:

addr: A suggested starting address for the mapping in the process’s virtual memory. It is almost always best practice to pass NULL here and let the kernel choose a suitable, available address. This makes the code more portable.

length: The number of bytes to map. This should match the size set by ftruncate().

prot: The desired memory protection for the mapping. This is a bitmask of PROT_READ (pages may be read), PROT_WRITE(pages may be written), or both. This cannot grant more permissions than were requested in shm_open(). For example, you cannot request PROT_WRITE if the object was opened with O_RDONLY.

flags: This controls the nature of the mapping. For shared memory, MAP_SHARED is essential. It specifies that any modifications made to the mapped memory will be written back to the underlying object and will be visible to all other processes that have mapped the same object with MAP_SHARED. The alternative, MAP_PRIVATE, would create a private, copy-on-write mapping, defeating the purpose of IPC.

fd: The file descriptor returned by shm_open().

offset: The offset within the shared memory object where the mapping should begin. For most use cases, this is 0, to map the entire object.

If mmap() is successful, it returns a pointer to the start of the mapped memory region. This pointer can be cast to the appropriate data type (e.g., a pointer to a struct) and used just like memory allocated with malloc(). If it fails, it returns MAP_FAILED (which is (void *) -1) and sets errno.

Cleanup: munmap and shm_unlink

Proper resource management is critical in any application. POSIX shared memory requires a two-step cleanup process. First, each process must unmap the memory region from its address space when it is finished. This is done with munmap():

int munmap(void *addr, size_t length);

The addr must be the exact pointer returned by mmap(), and length must be the same length that was mapped. This call removes the mapping from the process’s virtual address space. After unmapping, the pointer is no longer valid, and attempting to access it will result in a segmentation fault. It is also good practice to close() the file descriptor after unmapping.

Second, the shared memory object itself must be explicitly destroyed. Unmapping the memory does not remove the object from /dev/shm. It will persist in the tmpfs filesystem until the system reboots or until it is explicitly removed. This is a feature, not a bug; it allows processes to come and go, attaching to and detaching from a persistent shared data segment.

The responsibility for removing the object falls to one designated process (usually the creator). This is accomplished with shm_unlink():

int shm_unlink(const char *name);

The name must be the same name (e.g., "/my_shm_object") that was passed to shm_open(). shm_unlink() behaves similarly to the standard unlink() system call. It removes the name from the filesystem. The actual memory is not freed until the last process that has it open calls close() on its file descriptor. A common and robust pattern is for the creating process to call shm_unlink() immediately after all collaborating processes have successfully opened the object. This way, even if the creator process crashes, the object is guaranteed to be cleaned up once all other processes exit and close their descriptors.

flowchart LR
    subgraph Consumer Process
        K[Start] --> L{"shm_open<br><i>(No O_CREAT)</i>"};
        L --> M{Object Found?};
        M -- No --> N[Wait / Retry];
        M -- Yes --> O[mmap: Map Object to Memory];
        O --> P((Use Shared Memory<br><i>Read Data</i>));
        P --> Q[munmap: Unmap Memory];
        Q --> R[End];
    end
    subgraph Producer Process
        A[Start] --> B{"shm_open with<br>O_CREAT | O_EXCL"};
        B --> C{Object Exists?};
        C -- Yes --> D[Handle Error / Cleanup Stale];
        C -- No --> E[ftruncate: Set Size];
        E --> F[mmap: Map Object to Memory];
        F --> G((Use Shared Memory<br><i>Write Data</i>));
        G --> H[munmap: Unmap Memory];
        H --> I[shm_unlink: Remove Object Name];
        I --> J[End];
    end



    %% Styling
    style A fill:#1e3a8a,stroke:#1e3a8a,stroke-width:2px,color:#ffffff
    style J fill:#10b981,stroke:#10b981,stroke-width:2px,color:#ffffff
    style K fill:#1e3a8a,stroke:#1e3a8a,stroke-width:2px,color:#ffffff
    style R fill:#10b981,stroke:#10b981,stroke-width:2px,color:#ffffff
    
    style B fill:#0d9488,stroke:#0d9488,stroke-width:1px,color:#ffffff
    style E fill:#0d9488,stroke:#0d9488,stroke-width:1px,color:#ffffff
    style F fill:#0d9488,stroke:#0d9488,stroke-width:1px,color:#ffffff
    style G fill:#8b5cf6,stroke:#8b5cf6,stroke-width:1px,color:#ffffff
    style H fill:#0d9488,stroke:#0d9488,stroke-width:1px,color:#ffffff
    style I fill:#0d9488,stroke:#0d9488,stroke-width:1px,color:#ffffff
    
    style L fill:#0d9488,stroke:#0d9488,stroke-width:1px,color:#ffffff
    style O fill:#0d9488,stroke:#0d9488,stroke-width:1px,color:#ffffff
    style P fill:#8b5cf6,stroke:#8b5cf6,stroke-width:1px,color:#ffffff
    style Q fill:#0d9488,stroke:#0d9488,stroke-width:1px,color:#ffffff

    style C fill:#f59e0b,stroke:#f59e0b,stroke-width:1px,color:#ffffff
    style M fill:#f59e0b,stroke:#f59e0b,stroke-width:1px,color:#ffffff

    style D fill:#ef4444,stroke:#ef4444,stroke-width:1px,color:#ffffff
    style N fill:#eab308,stroke:#eab308,stroke-width:1px,color:#1f2937

Practical Examples

Let’s apply this theory to a practical scenario on the Raspberry Pi 5. We will create two programs: a “producer” that reads sensor data (we’ll simulate this for simplicity) and writes it into a shared memory segment, and a “consumer” that reads this data from shared memory and prints it to the console. This demonstrates a common embedded architecture where one high-priority process handles hardware interaction while other, lower-priority processes handle analysis, logging, or display.

Shared Data Structure

First, we need a common header file that defines the structure of our shared data. This ensures both the producer and consumer agree on the memory layout.

shm_common.h

#ifndef SHM_COMMON_H
#define SHM_COMMON_H

#include <fcntl.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <unistd.h>
#include <semaphore.h> // For synchronization, though we won't implement it here

#define SHM_NAME "/rpi_sensor_data"

// A simple structure to hold simulated sensor data
typedef struct {
    int sequence_num;
    float temperature;
    float humidity;
    char status_flag; // 'N' = New, 'R' = Read
} SensorData;

#endif // SHM_COMMON_H

The Producer Program

The producer is responsible for creating, sizing, and writing to the shared memory object.

producer.c

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include "shm_common.h"

void cleanup_and_exit(int status) {
    // It's good practice to unlink the shared memory object here,
    // especially in a real application with signal handlers.
    shm_unlink(SHM_NAME);
    exit(status);
}

int main() {
    printf("Producer starting...\n");

    // 1. Create the shared memory object
    // O_CREAT: Create it if it doesn't exist.
    // O_EXCL: Fail if it already exists (ensures we are the first).
    // O_RDWR: Open for reading and writing.
    int fd = shm_open(SHM_NAME, O_CREAT | O_EXCL | O_RDWR, 0666);
    if (fd == -1) {
        perror("shm_open");
        // If it already exists, maybe a previous run crashed.
        // Let's try to unlink it and try again.
        shm_unlink(SHM_NAME);
        fd = shm_open(SHM_NAME, O_CREAT | O_EXCL | O_RDWR, 0666);
        if (fd == -1) {
            perror("shm_open after unlink");
            exit(EXIT_FAILURE);
        }
    }
    printf("Shared memory object created with fd: %d\n", fd);

    // 2. Set the size of the shared memory object
    if (ftruncate(fd, sizeof(SensorData)) == -1) {
        perror("ftruncate");
        close(fd);
        cleanup_and_exit(EXIT_FAILURE);
    }
    printf("Shared memory object sized to %ld bytes\n", sizeof(SensorData));

    // 3. Map the shared memory object into our address space
    SensorData *shared_data = mmap(NULL, sizeof(SensorData), PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
    if (shared_data == MAP_FAILED) {
        perror("mmap");
        close(fd);
        cleanup_and_exit(EXIT_FAILURE);
    }
    // We no longer need the file descriptor after mapping
    close(fd);
    printf("Shared memory mapped at address: %p\n", (void*)shared_data);

    // Initialize the data
    shared_data->sequence_num = 0;
    shared_data->status_flag = 'R'; // Initial state: Read (producer can write)

    // 4. Write data periodically
    for (int i = 0; i < 20; ++i) {
        // A real application would use a semaphore or mutex here!
        // We are using a simple flag for demonstration.
        if (shared_data->status_flag == 'R') {
            printf("Writing data... (Seq: %d)\n", shared_data->sequence_num + 1);
            shared_data->sequence_num++;
            shared_data->temperature = 20.0f + (rand() % 100) / 10.0f;
            shared_data->humidity = 45.0f + (rand() % 200) / 10.0f;
            shared_data->status_flag = 'N'; // 'N' for New data
        } else {
            // Consumer hasn't read the last value yet, so we wait.
            printf("Waiting for consumer to read...\n");
            i--; // Retry this iteration
        }
        sleep(1); // Wait for 1 second
    }

    // 5. Cleanup
    printf("Producer finished. Unmapping memory and unlinking object.\n");
    if (munmap(shared_data, sizeof(SensorData)) == -1) {
        perror("munmap");
    }
    
    // This removes the object from /dev/shm
    if (shm_unlink(SHM_NAME) == -1) {
        perror("shm_unlink");
    }

    printf("Producer exiting.\n");
    return 0;
}

The Consumer Program

The consumer opens the existing object and reads data from it.

consumer.c

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include "shm_common.h"

int main() {
    printf("Consumer starting...\n");
    int fd;

    // Retry loop to wait for the producer to create the object
    for (int i = 0; i < 10; i++) {
        // 1. Open the existing shared memory object (read-only)
        fd = shm_open(SHM_NAME, O_RDWR, 0666); // Open RDWR to modify the flag
        if (fd != -1) {
            break;
        }
        printf("Waiting for producer to create shared memory...\n");
        sleep(1);
    }

    if (fd == -1) {
        perror("shm_open");
        exit(EXIT_FAILURE);
    }
    printf("Shared memory object opened with fd: %d\n", fd);

    // 2. Map the shared memory object
    SensorData *shared_data = mmap(NULL, sizeof(SensorData), PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
    if (shared_data == MAP_FAILED) {
        perror("mmap");
        close(fd);
        exit(EXIT_FAILURE);
    }
    close(fd);
    printf("Shared memory mapped at address: %p\n", (void*)shared_data);

    // 3. Read data periodically
    printf("Reading data from shared memory:\n");
    for (int i = 0; i < 20; ++i) {
        // A real application would use a semaphore or mutex here!
        if (shared_data->status_flag == 'N') {
            printf("Seq: %-3d | Temp: %5.2f C | Humidity: %5.2f %% | Flag: %c -> R\n",
                   shared_data->sequence_num,
                   shared_data->temperature,
                   shared_data->humidity,
                   shared_data->status_flag);
            shared_data->status_flag = 'R'; // Mark as Read
        } else {
            // No new data, wait.
            i--; // Retry
        }
        sleep(1);
    }

    // 4. Cleanup
    printf("Consumer finished. Unmapping memory.\n");
    if (munmap(shared_data, sizeof(SensorData)) == -1) {
        perror("munmap");
    }

    printf("Consumer exiting.\n");
    return 0;
}

Build and Run on Raspberry Pi 5

1. File Structure:Place all three files (shm_common.h, producer.c, consumer.c) in the same directory on your Raspberry Pi 5.

2. Compilation: Open a terminal on your Pi. Compile both programs. Since we are linking against the POSIX real-time extensions library, we must add the -lrt flag.

# Compile the producer
gcc producer.c -o producer -lrt

# Compile the consumer
gcc consumer.c -o consumer -lrt

2. Execution: You will need two separate terminal windows.

• In Terminal 1, run the producer:

./producer

• In Terminal 2, quickly run the consumer:

./consumer

4. Expected Output: Terminal 1 (Producer):

Producer starting...
Shared memory object created with fd: 3
Shared memory object sized to 16 bytes
Shared memory mapped at address: 0x7f9c8b5000
Writing data... (Seq: 1)
Waiting for consumer to read...
Writing data... (Seq: 2)
Waiting for consumer to read...
Writing data... (Seq: 3)
...
Producer finished. Unmapping memory and unlinking object.
Producer exiting.


Terminal 2 (Consumer):

Consumer starting...
Shared memory object opened with fd: 3
Shared memory mapped at address: 0x7fa2b4c000
Reading data from shared memory:
Seq: 1   | Temp:  23.45 C | Humidity:  52.80 % | Flag: N -> R
Seq: 2   | Temp:  28.12 C | Humidity:  49.10 % | Flag: N -> R
Seq: 3   | Temp:  21.90 C | Humidity:  61.50 % | Flag: N -> R
...
Consumer finished. Unmapping memory.
Consumer exiting.


Tip: While the programs are running, open a third terminal and inspect the /dev/shm directory. You will see the shared memory object listed as a file: ls -l /dev/shm/rpi_sensor_data. Its size will match sizeof(SensorData). After the producer exits and calls shm_unlink, this file will disappear.

Common Mistakes & Troubleshooting

Developing with shared memory can be tricky, as errors often manifest as subtle race conditions or abrupt segmentation faults. Here are some common pitfalls and how to avoid them.

sequenceDiagram
    actor Producer
    actor Consumer
    participant Kernel

    Producer->>Kernel: shm_open(), ftruncate(), mmap()
    Consumer->>Kernel: shm_open(), mmap()
    
    loop Unsynchronized Access
        Producer->>Producer: Prepare data (e.g., temp = 25.1)
        Producer->>Kernel: Write `temperature = 25.1` to SHM
        Producer->>Producer: Prepare data (e.g., humidity = 45.5)
        
        Note right of Consumer: Context Switch! <br> Consumer runs before Producer finishes.
        
        Consumer->>Kernel: Read SHM
        Consumer->>Consumer: Process Data: temp=25.1, humidity=<i>stale_value</i>
        
        Note left of Producer: Context Switch! <br> Producer resumes.

        Producer->>Kernel: Write `humidity = 45.5` to SHM
    end

    Note over Producer,Consumer: Data is now inconsistent! <br> Consumer has a mix of old and new data.

Exercises

1. Basic Data Swap: Create two programs, progA and progB. progA should create a shared memory segment large enough for two integers. It should write its process ID (PID) into the first integer and then wait. progB should open the segment, write its PID into the second integer, and then print both PIDs. Finally, progA should wake up, print both PIDs, and clean up the shared memory object. This reinforces the basic create-open-write-read-unlink cycle.
2. Shared Counter with Synchronization: Modify the producer/consumer example to use a POSIX named semaphore to protect the SensorData structure. The producer must sem_wait() before writing and sem_post() after. The consumer must do the same. Remove the status_flag and observe how the semaphore provides more robust synchronization. You will need to create and initialize the semaphore and place its name or a pointer to it in the shared memory.
3. Dynamic Array in Shared Memory: Design a system where a producer process can add variable-length strings to a shared memory segment. The shared segment should contain a header structure with a count of the number of strings and an array of offsets. The actual string data will follow this header. A consumer process should be able to attach at any time and print all the strings currently in the segment. This exercise challenges you to manage memory layout more dynamically.
4. Performance Analysis: Write a program that transfers 100 MB of data from a parent process to a child process, first using a pipe and then using POSIX shared memory. Use the clock_gettime() function with CLOCK_MONOTONIC to measure the time taken for each transfer. Run the test multiple times and calculate the average transfer time for both methods. Write a brief report on your findings, explaining the performance difference. This provides empirical evidence of shared memory’s efficiency.

Summary

• High-Performance IPC: POSIX shared memory is a zero-copy IPC mechanism, making it significantly faster than kernel-mediated methods like pipes or sockets for large data transfers between local processes.
• File-Backed Abstraction: Modern POSIX shared memory is implemented as memory-backed files in a tmpfs filesystem, typically mounted at /dev/shm. This allows the use of standard file permissions and tools for management.
• Core API: The lifecycle of a shared memory object is managed by a few key functions:
  • shm_open(): Creates or opens a named shared memory object, returning a file descriptor.
  • ftruncate(): Sets the size of a newly created object. This step is mandatory.
  • mmap(): Maps the object (represented by the file descriptor) into the process’s virtual address space, returning a usable pointer.
  • munmap(): Unmaps the object from the process’s address space.
  • shm_unlink(): Removes the named object from the system.
• Synchronization is Crucial: Shared memory itself provides no mechanism to prevent race conditions. Concurrent access must be managed using synchronization primitives like semaphores or mutexes to ensure data integrity.
• Cleanup is Manual: The system does not automatically remove shared memory objects on process exit. The application logic must explicitly call shm_unlink to prevent resource leaks.

Further Reading

1. POSIX Standard for shm_open (IEEE Std 1003.1): The official specification from The Open Group. The definitive source for function signatures, error codes, and expected behavior.
   • https://pubs.opengroup.org/onlinepubs/9699919799/functions/shm_open.html
2. Linux Programmer’s Manual (man pages): The built-in Linux documentation is an invaluable and authoritative resource.
   • man shm_open
   • man mmap
   • man shm_overview
3. “The Linux Programming Interface” by Michael Kerrisk: Chapter 54, “POSIX Shared Memory,” provides an exhaustive and exceptionally clear explanation of this topic, with robust code examples.
4. LWN.net Articles on Memory Management: This publication often features deep-dive articles into the workings of the Linux kernel’s memory management and filesystems, which provides excellent background context for how tmpfs and mmap operate.
   • https://lwn.net/Kernel/Index/#Memory_Management
5. Raspberry Pi Documentation – The Linux kernel: While not specific to IPC, understanding the underlying kernel on the target hardware is always beneficial.
   • https://www.raspberrypi.com/documentation/computers/linux_kernel.html