Chapter 30: C Programming Refresher: Basic File I/O in C

Chapter Objectives

By the end of this chapter, you will be able to:

• Understand the concept of file streams in C and their role in abstracting low-level I/O operations.
• Implement robust file handling by correctly using fopen() to open files in various modes and fclose() to close them.
• Perform formatted I/O operations to read and write text-based data using fprintf() and fscanf().
• Execute block-based I/O operations to efficiently read and write binary data using fread() and fwrite().
• Debug common file I/O errors, including null pointer returns, resource leaks, and incorrect mode usage.
• Apply file I/O concepts to create practical embedded applications like data loggers and configuration parsers on a Raspberry Pi 5.

Introduction

The ability to interact with the filesystem is not merely a convenience—it is a cornerstone of creating robust, intelligent, and maintainable devices In the world of embedded Linux. While previous chapters introduced you to the Linux environment and basic shell commands, this chapter delves into a fundamental programming skill: controlling the filesystem directly from a C application. Nearly every embedded system needs to persist data, whether it’s logging sensor readings, storing user configurations, or updating firmware. The C standard library provides a powerful, portable, and efficient set of tools for these very tasks.

This chapter serves as a vital refresher on standard file input/output (I/O) in C. We will move beyond simply printing to the console and learn how to create, read, and write to files stored on your Raspberry Pi 5’s filesystem. We will explore the six core functions that form the foundation of these operations: fopen(), fclose(), fprintf(), fscanf(), fwrite(), and fread(). Understanding these functions is essential for building applications that can save their state, generate logs for debugging, or interact with other processes through file-based communication. As we proceed, you will see how these simple building blocks enable complex behaviors, transforming your programs from transient processes into persistent, data-driven applications.

Technical Background

At the heart of C’s file I/O capabilities is the concept of a stream. A stream is a high-level abstraction that represents a flow of data from a source to a destination. This source or destination can be a physical device like a hard drive, a terminal, or even a network socket. The beauty of the stream concept is its ability to hide the complex, device-specific details of I/O operations. Whether you are writing to a file on an SD card or printing to the user’s console, your C program interacts with it through a consistent interface. This abstraction is managed by the FILE structure, a data type defined in the <stdio.h> header file that holds all the necessary information to manage the stream, including the file’s position indicator, error and end-of-file flags, and buffer details. You, as the programmer, never need to manipulate the members of the FILE structure directly; instead, you operate on a pointer to it, often called a file handle.

Opening and Closing Files: fopen() and fclose()

Before any interaction with a file can occur, a stream must be established and associated with it. This crucial first step is handled by the fopen() function. Its purpose is to locate the file, perform the necessary system-level requests to open it, and initialize a FILE structure to manage subsequent operations. The function takes two arguments: a string containing the path to the file and a mode string that specifies the intended type of access.

The mode string is critical as it dictates both what you can do with the file and how the system treats it. Common modes include "r" for reading, "w" for writing (which creates a new file or truncates an existing one to zero length), and "a" for appending (which creates a new file or opens an existing one for writing at the end).

graph TD
    subgraph "fopen(data.log, w)"
        A[Start: Call fopen] --> B{"File data.log Exists?"};
        B -- Yes --> C[Truncate File to Zero Length];
        B -- No --> D[Create New Empty File];
        C --> E[Open File for Writing];
        D --> E;

        E --> F[Return FILE* Pointer];
        A --> G{Permission Error?<br>Or Other Failure};
        G -- Yes --> H[Return NULL];
        G -- No --> B;
    end

    %% Styling
    classDef primary fill:#1e3a8a,stroke:#1e3a8a,stroke-width:2px,color:#ffffff;
    classDef success fill:#10b981,stroke:#10b981,stroke-width:2px,color:#ffffff;
    classDef decision fill:#f59e0b,stroke:#f59e0b,stroke-width:1px,color:#ffffff;
    classDef process fill:#0d9488,stroke:#0d9488,stroke-width:1px,color:#ffffff;
    classDef error fill:#ef4444,stroke:#ef4444,stroke-width:1px,color:#ffffff;

    class A primary;
    class B,G decision;
    class C,D,E process;
    class F success;
    class H error;

A pivotal aspect of using fopen() is error handling. If the function fails—for instance, if you try to open a non-existent file in read mode or lack the necessary permissions—it returns a NULL pointer. A robust program must always check the return value of fopen() before attempting to use the file handle. Failure to do so will result in the program attempting to dereference a NULL pointer, leading to a segmentation fault and a crash.

Once all operations on a file are complete, the associated stream must be closed using the fclose() function. This function takes a single argument: the FILE pointer returned by fopen(). Calling fclose() performs several essential cleanup tasks. First, it flushes any buffered data. For efficiency, write operations do not always immediately write to the disk; data is often held in a temporary buffer in memory. fclose() ensures that all pending data in this buffer is written to the physical file. Second, it releases the system-level resources associated with the file, allowing the operating system to make them available to other processes. Forgetting to close a file can lead to resource leaks, which can degrade system performance over time, and in the worst case, can result in data loss or corruption, as the output buffer may never be flushed.

Formatted I/O: fprintf() and fscanf()

For many embedded applications, data is best stored in a human-readable text format. Configuration files, event logs, and simple data records are common examples. The fprintf() and fscanf() functions are designed for these scenarios, handling the conversion of data between its in-memory binary representation and its on-disk text representation.

fprintf() is analogous to the familiar printf(), but instead of writing to the standard output, it writes to a specified file stream. Its first argument is the FILE pointer, followed by a format string and a variable number of arguments to be formatted and written. This is incredibly useful for creating structured log files. For example, you could write a timestamp, a sensor name, and a floating-point value to a file in a clean, delimited format.

Conversely, fscanf() reads from a file stream, parses the input according to a format string, and stores the converted values into the memory locations provided by its pointer arguments. It reads from the file, skipping whitespace, and attempts to match the patterns in the format string. While powerful for parsing simple, well-structured files, fscanf() has pitfalls. If the input data does not perfectly match the format string, the function may fail to parse, leave the file pointer in an unexpected position, or leave variables uninitialized. It returns the number of items successfully converted and assigned, which can be checked to verify the success of the read operation. For more complex parsing tasks, it is often more robust to read entire lines from a file using a function like fgets() and then parse the resulting string in memory.

Binary I/O: fread() and fwrite()

While text files are great for human readability, they are not always efficient for storing large amounts of data or complex data structures. Storing a 32-bit integer like 1234567890 requires ten bytes as a text string (“1234567890”), but only four bytes in its native binary form. When dealing with raw sensor data, image data, or serialized C structs, binary I/O is the superior choice. The fread() and fwrite() functions are designed for these block-based, non-formatted data transfers.

fwrite() writes a block of raw bytes from memory directly to a file stream. It takes four arguments: a pointer to the source of the data in memory (const void *ptr), the size of each element to be written (size_t size), the number of elements to write (size_t nmemb), and the destination FILE pointer. The total number of bytes written is size * nmemb. This is highly efficient for writing arrays or structures to a file in a single operation.

fread() is the counterpart to fwrite(). It reads a block of raw bytes from a file stream directly into a memory location. Its arguments mirror fwrite(): a pointer to the destination buffer in memory (void *ptr), the size of each element, the number of elements to read, and the source FILE pointer. It is crucial that the destination buffer is large enough to hold the requested data to avoid buffer overflows, a serious security vulnerability. fread() returns the number of elements successfully read. This value may be less than the number requested if the end of the file is reached or an error occurs. Programmers must check this return value to confirm how much data was actually read.

The choice between formatted and binary I/O is a fundamental design decision. Formatted I/O offers portability and readability, making debugging and manual editing easier. Binary I/O offers compactness and speed, as it avoids the overhead of data conversion, but the resulting files are not human-readable and can have portability issues between systems with different byte ordering (endianness). For many embedded systems, a hybrid approach is common: a text-based configuration file read with fscanf() and a high-throughput binary data log written with fwrite().

Practical Examples

Theory is best understood through practice. In this section, we will apply the file I/O functions to solve practical problems on your Raspberry Pi 5. We will write, compile, and run several C programs that demonstrate logging, configuration, and binary data handling.

For all examples, you will use the gcc compiler. You can compile a source file named example.c and create an executable named example with the following command:

gcc -o example example.c -Wall

The -Wall flag is highly recommended as it enables all compiler warnings, which can help you catch potential bugs early.

Example 1: Creating a Simple Event Logger with fprintf()

In this example, we’ll create a program that appends timestamped messages to a log file. This is a common requirement for tracking events or errors in an embedded application.

File Structure:

/home/pi/
└── file_io_examples/
    ├── event_logger.c
    └── events.log  (This file will be created by the program)

Code (event_logger.c):

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

int main() {
    // Open the log file in "append" mode.
    // This creates the file if it doesn't exist, or opens it for writing at the end.
    FILE *log_file = fopen("events.log", "a");

    // CRITICAL: Always check if fopen() succeeded.
    if (log_file == NULL) {
        perror("Error opening log file");
        return 1; // Return a non-zero value to indicate an error
    }

    // Get the current time.
    time_t now = time(NULL);
    struct tm *t = localtime(&now);

    // Format the timestamp string (e.g., "YYYY-MM-DD HH:MM:SS").
    char time_str[20];
    strftime(time_str, sizeof(time_str), "%Y-%m-%d %H:%M:%S", t);

    // Write a formatted log message to the file.
    fprintf(log_file, "[%s] System event: Sensor reading processed.\n", time_str);
    printf("Log message appended to events.log\n");

    // Close the file to flush the buffer and release resources.
    fclose(log_file);

    return 0;
}

Build and Run Steps:

1. Create a directory for your work: mkdir -p /home/pi/file_io_examples
2. Navigate into the directory: cd /home/pi/file_io_examples
3. Create the C source file using nano event_logger.c and paste the code above.
4. Compile the program: gcc -o event_logger event_logger.c -Wall
5. Run the program several times:./event_logger ./event_logger ./event_logger

Expected Output:

Each time you run the program, you will see:

Log message appended to events.log

After running it three times, you can inspect the log file:

cat events.log

The content of events.log will look similar to this:

[2025-07-08 22:30:15] System event: Sensor reading processed.
[2025-07-08 22:30:18] System event: Sensor reading processed.
[2025-07-08 22:30:21] System event: Sensor reading processed.

This example demonstrates the power of the "a" (append) mode for creating cumulative logs without overwriting previous entries.

Example 2: Reading a Configuration File with fscanf()

Many devices need to load settings on startup. This example shows how to read simple key-value pairs from a configuration file.

File Structure:

/home/pi/
└── file_io_examples/
    ├── config_parser.c
    └── device.conf

Configuration File (device.conf):

First, create the configuration file.

nano device.conf

Add the following content:

# Device Configuration
device_id   RPi5-001
baud_rate   115200
active      1

Code (config_parser.c):

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define MAX_ID_LEN 50

int main() {
    char device_id[MAX_ID_LEN];
    int baud_rate;
    int is_active;

    FILE *config_file = fopen("device.conf", "r");

    if (config_file == NULL) {
        perror("Error opening config file");
        return 1;
    }

    // Use fscanf to parse the file.
    // The format string looks for a string ("device_id"), then a value string,
    // then a string ("baud_rate"), then an integer, etc.
    // The initial strings are consumed but not assigned.
    int items_read = fscanf(config_file, "%*s %49s %*s %d %*s %d", 
                            device_id, &baud_rate, &is_active);

    // Check if all expected items were read.
    if (items_read == 3) {
        printf("Configuration loaded successfully:\n");
        printf("  Device ID: %s\n", device_id);
        printf("  Baud Rate: %d\n", baud_rate);
        printf("  Device Active: %s\n", is_active ? "Yes" : "No");
    } else {
        fprintf(stderr, "Error parsing configuration file. Read %d items.\n", items_read);
    }

    fclose(config_file);

    return 0;
}

Tip: The %*s in the fscanf format string is a powerful feature. The asterisk (*) tells the function to read a string but to discard it instead of assigning it to a variable. This is perfect for skipping over known labels in a file. The %49s is a security measure to prevent buffer overflows when reading the device_id.

Build and Run Steps:

1. Ensure you are in the /home/pi/file_io_examples directory.
2. Create config_parser.c with the code above.
3. Compile: 
   gcc -o config_parser config_parser.c -Wall
4. Run: 
   ./config_parser

Expected Output:

Configuration loaded successfully:
  Device ID: RPi5-001
  Baud Rate: 115200
  Device Active: Yes

Example 3: Copying a Binary File with fread() and fwrite()

This example demonstrates how to handle binary data by creating a simple file copy utility. This technique is the basis for tasks like reading sensor data from a device file or processing image data.

File Structure:

/home/pi/
└── file_io_examples/
    ├── binary_copier.c
    ├── source_file.bin
    └── destination_file.bin (Created by the program)

Build and Run Steps:

1. First, create a dummy binary file to act as our source. We can use the dd command for this. This command creates a 1MB file filled with random data.dd if=/dev/urandom of=source_file.bin bs=1M count=1
2. Create binary_copier.c with the code below.

Code (binary_copier.c):

#include <stdio.h>
#include <stdlib.h>

#define BUFFER_SIZE 4096 // Use a 4KB buffer

int main() {
    // Open the source file in binary read mode ("rb").
    FILE *source = fopen("source_file.bin", "rb");
    if (source == NULL) {
        perror("Error opening source file");
        return 1;
    }

    // Open the destination file in binary write mode ("wb").
    FILE *destination = fopen("destination_file.bin", "wb");
    if (destination == NULL) {
        perror("Error opening destination file");
        fclose(source); // Clean up the already opened file
        return 1;
    }

    unsigned char buffer[BUFFER_SIZE];
    size_t bytes_read;

    // Loop until the end of the source file.
    while ((bytes_read = fread(buffer, 1, BUFFER_SIZE, source)) > 0) {
        // Write the chunk of data we just read to the destination.
        size_t bytes_written = fwrite(buffer, 1, bytes_read, destination);
        if (bytes_written < bytes_read) {
            perror("Error writing to destination file");
            fclose(source);
            fclose(destination);
            return 1;
        }
    }

    printf("File copied successfully.\n");

    // Close both files.
    fclose(source);
    fclose(destination);

    return 0;
}

3. Compile: 
   gcc -o binary_copier binary_copier.c -Wall
4. Run: 
   ./binary_copier
5. Verify the copy was successful. The md5sum command calculates a checksum for a file. If the two files are identical, their checksums will match.
   md5sum source_file.bin destination_file.bin

Expected Output:

File copied successfully.

And the md5sum command will output two identical hashes:

d8e8fca2dc0f896fd7cb4cb0031ba249  source_file.bin
d8e8fca2dc0f896fd7cb4cb0031ba249  destination_file.bin

This confirms that the binary data was copied byte-for-byte, demonstrating the correct use of fread and fwrite.

Common Mistakes & Troubleshooting

File I/O is a common source of bugs, especially for those new to C programming. Being aware of these common pitfalls can save you hours of debugging.

Exercises

These exercises will help you solidify your understanding of C file I/O.

1. Enhanced Event Logger
   • Objective: Modify the event_logger.c example to accept a log message as a command-line argument.
   • Guidance:
     1. Modify the main function signature to int main(int argc, char *argv[]).
     2. Check if argc is equal to 2. If not, print a usage message (e.g., Usage: ./event_logger "Your message") and exit.
     3. Instead of the hardcoded message, use argv[1] in your fprintf() call.
   • Verification: Run ./event_logger "Test message 1" and check that events.log contains the new, custom message.
2. Configuration Writer
   • Objective: Write a program that creates the device.conf file used in Example 2.
   • Guidance:
     1. Create a new C file, config_writer.c.
     2. Open a file named device.conf in write mode ("w").
     3. Use three separate fprintf() calls to write the device_id, baud_rate, and active key-value pairs to the file, matching the format from Example 2 exactly.
     4. Remember to include newlines (\n) at the end of each line.
   • Verification: Run your config_writer program and then run the config_parser from Example 2. The parser should be able to read the file you generated without any errors.
3. Structured Binary Data Logger
   • Objective: Create a program that simulates reading sensor data and logs it to a binary file using a struct.
   • Guidance:
     1. Define a struct to hold a reading:struct SensorReading { int sensor_id; float temperature; float humidity; };
     2. Open a file sensor_data.bin in binary append mode ("ab").
     3. Create an instance of struct SensorReading and populate it with some sample data (e.g., ID: 101, Temp: 25.5, Humidity: 45.2).
     4. Use fwrite() to write the entire struct to the file in one call. The ptr will be the address of your struct variable, size will be sizeof(struct SensorReading), and nmemb will be 1.
   • Verification: Run the program multiple times with different data. Use the ls -l sensor_data.bin command to check that the file size increases by sizeof(struct SensorReading) each time.
4. Binary Data Reader
   • Objective: Write a program to read and display the data from the sensor_data.bin file created in Exercise 3.
   • Guidance:
     1. Open sensor_data.bin in binary read mode ("rb").
     2. Create an instance of struct SensorReading to act as a buffer.
     3. Use a while loop with fread() to read one struct at a time from the file until fread() returns 0 (end of file).
     4. Inside the loop, use printf() to display the sensor_id, temperature, and humidity from the struct you just read.
   • Verification: The output of your program should be a human-readable list of all the sensor readings you logged in Exercise 3.

Summary

• File I/O is managed through streams, an abstraction represented by the FILE structure in <stdio.h>.
• fopen() is used to open a file and associate it with a stream, returning a FILE* handle. Its mode string ("r", "w", "a", "rb", etc.) is critical.
• Always check the return value of fopen() for NULL to prevent segmentation faults.
• fclose() is essential for flushing data buffers and releasing system resources. For every fopen(), there must be a fclose().
• fprintf() and fscanf() are used for formatted text I/O, converting data between in-memory types and their string representations. They are ideal for logs and configuration files.
• fwrite() and fread() are used for unformatted binary I/O, performing direct memory-to-file transfers. They are efficient for large data blocks, arrays, and structs.
• Choosing between text and binary I/O involves a trade-off between human readability (text) and storage/speed efficiency (binary).

Further Reading

1. The C Programming Language, 2nd Edition by Brian W. Kernighan and Dennis M. Ritchie – Chapter 7, “Input and Output,” is the canonical reference for the C standard I/O library.
2. GNU C Library (glibc) Manual – The “I/O on Streams” section provides exhaustive documentation on every function in <stdio.h>. (https://www.gnu.org/software/libc/manual/html_node/I_002fO-on-Streams.html)
3. Linux man pages – The manual pages for fopen(3), fprintf(3), and fread(3) provide concise, authoritative technical details. You can access them on your Raspberry Pi with man 3 fopen.
4. SEI CERT C Coding Standard – The FIO (File I/O) section offers best practices for secure and robust file handling. (https://wiki.sei.cmu.edu/confluence/pages/viewpage.action?pageId=87151932)
5. “Beej’s Guide to C Programming” – A well-regarded, accessible guide that covers file I/O in a practical, easy-to-understand manner. (https://beej.us/guide/bgc/)