Chapter 286: Creating Reusable Libraries

Chapter Objectives

By the end of this chapter, you will be able to:

• Understand the importance of modularity and reusability in embedded projects.
• Structure your code into a custom, reusable ESP-IDF component.
• Create the necessary CMakeLists.txt file to register a component with the build system.
• Use a custom local library (component) within your main application.
• Understand how to package a component for distribution using Git.
• Learn the basics of preparing a component for the ESP-IDF Component Registry.

Introduction

As your projects grow in complexity, the “write everything in main.c” approach quickly becomes unmanageable. Code becomes difficult to read, debug, and, most importantly, reuse in future projects. Professional embedded development relies on modularity: breaking down a large system into smaller, self-contained, and reusable pieces of code. We call these pieces libraries.

In the ESP-IDF ecosystem, the primary way to create a reusable library is by structuring it as a component. The ESP-IDF build system is specifically designed to discover, compile, and link these components into your final application automatically. This chapter will teach you how to move from monolithic code to a clean, modular design by creating your own reusable components. This is a cornerstone of efficient and scalable firmware development.

Theory

What is a Library?

In programming, a library is a collection of pre-compiled code, functions, data structures, and other resources that can be used by other programs. Instead of reinventing the wheel every time you need to perform a common task (like controlling a motor or parsing a JSON string), you can simply include a library that already provides that functionality.

In the context of C and ESP-IDF, we primarily deal with static libraries. When you build your project, the compiler and linker find the library code you’ve used and copy it directly into your final executable firmware image. This is different from dynamic libraries, which are loaded at runtime, a concept less common in smaller embedded systems.

The ESP-IDF Component Model

ESP-IDF’s architecture is built around the concept of components. A component is the fundamental building block of an ESP-IDF project. Even the code you write in the main directory is, in fact, a component—the main component of your application. Other functionalities provided by Espressif, such as the Wi-Fi stack, FreeRTOS, and various drivers, are also organized as pre-built components.

A component is essentially a self-contained directory that includes:

• Source code files (.c or .cpp).
• Header files (.h or .hpp), typically in a dedicated include subdirectory.
• A CMakeLists.txt file, which tells the build system how to compile the component.

By creating your own components, you are creating reusable libraries that cleanly integrate with the ESP-IDF build system.

Component Structure

To be recognized by the build system, a component must have a specific directory structure. Let’s imagine we’re creating a library to manage a custom sensor. We’ll call the component custom_sensor.

my_project/
├── components/
│   └── custom_sensor/
│       ├── include/
│       │   └── custom_sensor.h
│       ├── custom_sensor.c
│       └── CMakeLists.txt
├── main/
│   ├── main.c
│   └── CMakeLists.txt
├── CMakeLists.txt
└── sdkconfig

• components/: This is a special directory where you place your custom components. The build system will automatically search here.
• custom_sensor/: The root directory of our library component.
• include/: This subdirectory is crucial. It contains the public header files for your library. Any header file in this directory will be accessible to other components in the project (like main).
• custom_sensor.c: The source file containing the actual logic of our library.
• CMakeLists.txt: The component’s build script. This is the heart of the component registration process.

The Component CMakeLists.txt File

This file instructs the build system on how to handle the component’s files. For a basic library, it’s quite simple.

# CMakeLists.txt for the 'custom_sensor' component

idf_component_register(SRCS "custom_sensor.c"
                       INCLUDE_DIRS "include"
                       PRIV_REQUIRES esp_timer)

Let’s break down the idf_component_register function:

• SRCS: Lists the source files to be compiled as part of this component.
• INCLUDE_DIRS: Specifies which directories contain public header files. By convention, this is almost always "include". The build system adds this path to the global “include search paths” for the entire project.
• REQUIRES / PRIV_REQUIRES: Lists other components that this component depends on. REQUIRES is for public dependencies (if a component includes our header, it also needs to know about our dependencies’ headers), while PRIV_REQUIRES is for private dependencies used only in our .c files. Here, we might need a timer, so we specify a private dependency on the esp_timer component.

Distributing Components

Once you have a useful component, you’ll want to share it across multiple projects or with other developers.

1. Git Repositories: The most common method is to place your component in its own Git repository. Other projects can then pull it in as a Git submodule or, more modernly, by specifying the repository URL in a manifest file (idf_component.yml).
2. ESP-IDF Component Registry: For public, open-source components, you can publish them to the official ESP-IDF Component Registry. This makes your library discoverable and easily manageable through the idf.py component command, similar to package managers like pip or npm.

graph TD
    subgraph "Your Project: my_app"
        A["<b>main</b> Component<br>(main.c)"]
        B["Custom <b>blinker</b> Component<br>(Local: /components)"]
    end

    subgraph "External Sources"
        C["<b>wifi</b> Component<br>(ESP-IDF Framework)"]
        D["<b>cJSON</b> Component<br>(ESP-IDF Component Registry)"]
    end
    
    subgraph "Build System (idf.py build)"
        E{CMake Build Tool}
        F["Compiler (xtensa-esp32-elf-gcc)"]
        G[Linker]
    end

    H["Application Firmware<br>(my_app.bin)"]

    A --> E
    B --> E
    C --> E
    D --> E
    E --> F
    F --> G
    G --> H

    classDef main fill:#EDE9FE,stroke:#5B21B6,stroke-width:2px,color:#5B21B6;
    classDef custom fill:#DBEAFE,stroke:#2563EB,stroke-width:1px,color:#1E40AF;
    classDef registry fill:#DBEAFE,stroke:#2563EB,stroke-width:1px,color:#1E40AF;
    classDef idf fill:#DBEAFE,stroke:#2563EB,stroke-width:1px,color:#1E40AF;
    classDef build fill:#FEF3C7,stroke:#D97706,stroke-width:1px,color:#92400E;
    classDef final fill:#D1FAE5,stroke:#059669,stroke-width:2px,color:#065F46;

    class A main;
    class B,C,D custom;
    class E,F,G build;
    class H final;

Practical Examples

Let’s build a simple, practical library to encapsulate the logic for blinking an LED. This “Blinker” library will allow any application to start blinking a specific GPIO pin at a given frequency with a single function call.

Example 1: Creating a Local “Blinker” Component

1. Project Setup

Start with a standard hello_world project created from the ESP-IDF templates. If you don’t have one, create it now in VS Code. Your initial project structure should look like this:

my_blinker_project/
├── main/
│   ├── main.c
│   └── CMakeLists.txt
├── CMakeLists.txt
└── sdkconfig

2. Create the Component Directory Structure

Inside your project’s root directory (my_blinker_project), create the following directory structure for our new blinker component:

1. Create a directory named components.
2. Inside components, create a directory named blinker.
3. Inside blinker, create a directory named include.

Your structure should now be:

my_blinker_project/
├── components/
│   └── blinker/
│       └── include/
├── main/
...

3. Create the Header File

Inside components/blinker/include/, create a new file named blinker.h. This file will define the public interface of our library.

components/blinker/include/blinker.h

#pragma once

#include "driver/gpio.h"
#include "esp_err.h"

#ifdef __cplusplus
extern "C" {
#endif

/**
 * @brief Initializes the blinker functionality for a specific GPIO pin.
 * * This function sets up the specified GPIO pin as an output. It must be called
 * before any other blinker functions.
 *
 * @param gpio_num The GPIO pin to be configured for blinking.
 * @return 
 * - ESP_OK on success
 * - ESP_ERR_INVALID_ARG if the GPIO number is invalid
 */
esp_err_t blinker_init(gpio_num_t gpio_num);

/**
 * @brief Starts blinking the initialized LED.
 *
 * A background task is created to toggle the GPIO pin at the specified frequency.
 *
 * @param frequency_hz The frequency of the blink in Hertz (Hz). 
 * e.g., 1 means one full on/off cycle per second.
 * @return 
 * - ESP_OK on success
 * - ESP_FAIL if the blinker was not initialized first or task creation failed.
 */
esp_err_t blinker_start(uint32_t frequency_hz);

#ifdef __cplusplus
}
#endif

Tip: The #pragma once, ifdef __cplusplus, and extern "C" guards are best practices. They prevent multiple inclusion errors and ensure the C functions in your library can be correctly called from C++ code.

4. Create the Source File

Inside components/blinker/, create a new file named blinker.c. This is the implementation of our library.

components/blinker/blinker.c

#include "blinker.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
#include "esp_check.h"

// Define a tag for logging
static const char *TAG = "BLINKER";

// Module-level static variable to hold the GPIO pin number
static gpio_num_t s_led_pin = -1;

// The task that will handle the blinking
static void blinker_task(void *pvParameters) {
    uint32_t frequency_hz = (uint32_t)pvParameters;
    uint32_t delay_ms = 500 / frequency_hz; // 500ms for on, 500ms for off = 1 cycle
    bool is_on = false;

    ESP_LOGI(TAG, "Blinker task started on GPIO %d at %d Hz.", s_led_pin, frequency_hz);

    while (1) {
        is_on = !is_on;
        gpio_set_level(s_led_pin, is_on);
        vTaskDelay(pdMS_TO_TICKS(delay_ms));
    }
}

// Implementation of the public API function
esp_err_t blinker_init(gpio_num_t gpio_num) {
    // Use ESP_CHECK to validate arguments. It logs an error and returns on failure.
    ESP_RETURN_ON_FALSE(GPIO_IS_VALID_OUTPUT_GPIO(gpio_num), ESP_ERR_INVALID_ARG, TAG, "Invalid GPIO pin");
    
    s_led_pin = gpio_num;
    
    // Configure the GPIO
    gpio_config_t io_conf = {
        .pin_bit_mask = (1ULL << s_led_pin),
        .mode = GPIO_MODE_OUTPUT,
        .pull_up_en = 0,
        .pull_down_en = 0,
        .intr_type = GPIO_INTR_DISABLE
    };

    return gpio_config(&io_conf);
}

// Implementation of the public API function
esp_err_t blinker_start(uint32_t frequency_hz) {
    ESP_RETURN_ON_FALSE(s_led_pin != -1, ESP_FAIL, TAG, "Blinker not initialized. Call blinker_init() first.");
    ESP_RETURN_ON_FALSE(frequency_hz > 0, ESP_ERR_INVALID_ARG, TAG, "Frequency must be > 0");

    // Create the blinker task. Pass frequency as the task parameter.
    // The xTaskCreate function returns pdPASS (which is 1) on success.
    BaseType_t res = xTaskCreate(blinker_task, "blinker_task", 2048, (void*)frequency_hz, 5, NULL);

    return (res == pdPASS) ? ESP_OK : ESP_FAIL;
}

sequenceDiagram
    actor User as app_main
    participant BL as blinker_start()
    participant RTOS as FreeRTOS Kernel
    participant BT as blinker_task
    participant GPIO as GPIO Driver

    User->>BL: Calls blinker_start(2)
    activate BL
    BL-->>BL: Checks if initialized (s_led_pin != -1)
    BL->>RTOS: xTaskCreate(blinker_task, "blinker_task", ...)
    activate RTOS
    RTOS-->>BL: Returns pdPASS (success)
    deactivate RTOS
    BL-->>User: Returns ESP_OK
    deactivate BL
    
    Note over RTOS, BT: RTOS schedules and starts blinker_task
    
    activate BT
    BT->>BT: Enters infinite while(1) loop
    loop Blink Cycle
        BT->>BT: Toggles LED state (is_on = !is_on)
        BT->>GPIO: gpio_set_level(pin, state)
        activate GPIO
        GPIO-->>BT: 
        deactivate GPIO
        BT->>RTOS: vTaskDelay(250ms)
        activate RTOS
        Note right of RTOS: Task is blocked, other tasks can run
        RTOS-->>BT: Delay finished
        deactivate RTOS
    end
    deactivate BT

5. Create the Component CMakeLists.txt

Finally, create the CMakeLists.txt file inside components/blinker/.

components/blinker/CMakeLists.txt

idf_component_register(SRCS "blinker.c"
                       INCLUDE_DIRS "include"
                       REQUIRES freertos
                       PRIV_REQUIRES esp_log esp_check)

• SRCS: We list our source file blinker.c.
• INCLUDE_DIRS: We expose our public headers in the include directory.
• REQUIRES: We publicly depend on freertos because our header uses uint32_t which might be defined via FreeRTOS headers. While it’s also in esp_err.h, being explicit is good practice.
• PRIV_REQUIRES: Our blinker.c implementation uses logging (esp_log) and argument checking (esp_check), so we add these as private dependencies.

6. Use the Library in main.c

Now, modify your main/main.c to use your new library.

main/main.c

#include <stdio.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
#include "blinker.h" // <-- Include our new library header

// Most dev boards have a built-in LED on GPIO 2. Check your board's schematic.
#define BLINK_GPIO GPIO_NUM_2 

void app_main(void)
{
    ESP_LOGI("MAIN", "Initializing the blinker library...");

    // Initialize the blinker on our chosen GPIO pin
    esp_err_t ret = blinker_init(BLINK_GPIO);
    if (ret != ESP_OK) {
        ESP_LOGE("MAIN", "Blinker init failed: %s", esp_err_to_name(ret));
        return;
    }

    ESP_LOGI("MAIN", "Starting the blinker at 2 Hz...");

    // Start the blinker at 2 cycles per second
    ret = blinker_start(2);
     if (ret != ESP_OK) {
        ESP_LOGE("MAIN", "Blinker start failed: %s", esp_err_to_name(ret));
    }

    // The main task can now do other things or simply exit,
    // as the blinker runs in its own task.
    ESP_LOGI("MAIN", "app_main finished. Blinker is running in the background.");
}

Warning: Make sure the BLINK_GPIO definition matches an actual LED on your development board. For many ESP32 boards, this is GPIO_NUM_2. For ESP32-C3, it’s often GPIO_NUM_8.

7. Build, Flash, and Observe

1. Save all your files.
2. Open the terminal in VS Code.
3. Run the build command: idf.py build. You should see the build system automatically find and compile your blinker component before compiling main.
4. Flash the project to your board: idf.py flash.
5. Open the monitor: idf.py monitor.
6. You should see the log messages from both main and your blinker component, and the LED on your board should be blinking twice per second.

Congratulations! You have successfully created and used a reusable local library.

Variant Notes

The process of creating a component-based library is identical across all ESP32 variants (ESP32, ESP32-S2, ESP32-S3, ESP32-C3, ESP32-C6, ESP32-H2). This is a feature of the ESP-IDF software framework and its build system, not the hardware itself.

However, the content of your library might be variant-specific. For example:

• A library utilizing the USB-OTG peripheral would only work on ESP32-S2 and ESP32-S3.
• A library using a peripheral exclusive to the ESP32-C6 would not compile if you tried to use it in a project targeting the original ESP32.

When writing a library intended for cross-variant use, you should use C preprocessor directives to handle differences:

#if CONFIG_IDF_TARGET_ESP32S3
    // Code specific to ESP32-S3
#elif CONFIG_IDF_TARGET_ESP32C3
    // Code specific to ESP32-C3
#else
    // Generic code or error for unsupported targets
#endif

These CONFIG_IDF_TARGET_* macros are automatically defined by the build system based on the target you select in idf.py set-target [target].

Common Mistakes & Troubleshooting Tips

Exercises

1. Stop the Blinker: Add a new function to the blinker library called esp_err_t blinker_stop(void);. This function should find and delete the FreeRTOS task created by blinker_start, effectively stopping the LED from blinking. Update main.c to call this function after a 10-second delay.
2. Create a Simple UART Logger Library: Create a new component named uart_logger. This library should have an uart_logger_init() function that configures a UART port (e.g., UART1) and a function uart_logger_send(const char* message) that sends a string over that UART. This is useful for creating a dedicated diagnostic output that doesn’t interfere with the main monitor.
3. Package the Blinker Library:a. Move the components/blinker directory out of your project and into its own new directory.b. Initialize a Git repository in that directory and commit the blinker component files.c. In your original my_blinker_project, delete the local components/blinker directory.d. Create an idf_component.yml manifest file in the root of my_blinker_project and add your blinker library as a dependency, pointing to its local Git path. (Refer to the ESP-IDF Component Manager documentation for the exact syntax).e. Run idf.py reconfigure and verify that the build system now fetches your component from the Git location and the project still compiles and runs correctly.

Summary

• Reusable code in ESP-IDF is packaged into components.
• A component is a directory containing source files, header files (in an include subfolder), and a CMakeLists.txt file.
• The idf_component_register function in CMakeLists.txt is essential for registering the component with the build system.
• SRCS defines the source files, INCLUDE_DIRS defines the public header directories, and REQUIRES lists dependencies.
• Custom components are typically placed in the components directory of a project for local use.
• For distribution, components can be stored in Git repositories or published to the ESP-IDF Component Registry.
• The component creation process is identical across all ESP32 variants, but the library’s internal logic may need to be adapted for variant-specific hardware.

Further Reading

• ESP-IDF Build System Documentation: https://docs.espressif.com/projects/esp-idf/en/v5.2.1/api-reference/build-system.html
• ESP-IDF Component Manager (idf.py component): https://docs.espressif.com/projects/esp-idf/en/v5.2.1/esp32/api-guides/tools/idf-component-manager.html
• ESP-IDF Component Registry: https://components.espressif.com/