Chapter 95: Certificate Management and Verification

Chapter Objectives

After completing this chapter, you will be able to:

• Understand various methods for storing TLS certificates (Root CAs, client certificates) on an ESP32.
• Implement embedding certificates directly into the application firmware.
• Learn how to store and retrieve certificates from Non-Volatile Storage (NVS).
• Explore storing and retrieving certificates from a flash filesystem like SPIFFS or LittleFS.
• Effectively use the ESP-IDF built-in certificate bundle for common public CAs.
• Understand the process of parsing and utilizing PEM and DER certificate formats.
• Grasp the importance of secure certificate storage and provisioning.
• Implement robust server certificate verification, including Common Name (CN) and Subject Alternative Name (SAN) checks.
• Discuss strategies for certificate updates and lifecycle management in embedded systems.
• Troubleshoot common issues related to certificate loading, parsing, and verification.

Introduction

Chapter 94 laid the groundwork for secure communication using HTTPS and TLS/SSL, emphasizing the role of digital certificates in authenticating servers and establishing encrypted channels. However, simply knowing that certificates are needed is only half the battle. Effective certificate management—how these certificates are stored, accessed, verified, and updated—is paramount for maintaining the security and reliability of your ESP32-based IoT applications.

If a device cannot trust the server it’s talking to, or if its trust anchors (Root CA certificates) are compromised or outdated, the entire security model collapses. This chapter focuses on the practical aspects of handling X.509 certificates on the ESP32. We will explore different methods for storing trusted CA certificates, from embedding them directly in the firmware to loading them from flash storage. We’ll also delve deeper into the verification process performed by the client and discuss how to manage these critical pieces of cryptographic identity throughout the lifecycle of your device. Proper certificate management is not just a technical detail; it’s a fundamental requirement for building secure and maintainable IoT systems.

Theory

Recap: X.509 Certificates and Trust

As discussed in Chapter 94, X.509 certificates are digital documents that bind an identity (like a server’s domain name) to a public key. This binding is asserted by a Certificate Authority (CA) that signs the certificate. Trust in this system is established through a chain of trust, where a server’s certificate is signed by an intermediate CA, which might be signed by another intermediate CA, ultimately leading back to a Root CA certificate that the client (our ESP32) inherently trusts.

The ESP32, when acting as an HTTPS client, must:

1. Receive the server’s certificate (and any intermediates).
2. Verify the signature chain up to a trusted Root CA.
3. Check the validity period of each certificate in the chain.
4. Verify that the hostname it connected to matches the Common Name (CN) or a Subject Alternative Name (SAN) in the server’s certificate.

To perform step 2, the ESP32 needs access to a collection of trusted Root CA certificates. This chapter focuses on how to provide and manage these certificates.

Certificate Formats

Digital certificates are typically encountered in a few standard formats:

1. PEM (Privacy Enhanced Mail):
   • This is the most common format for certificates, private keys, and certificate requests.
   • It’s a Base64 encoded DER certificate, wrapped with ASCII headers and footers.
   • Example (Root CA certificate):
     -----BEGIN CERTIFICATE-----
MIIE0zCCA7ugAwIBAgIQGNrRniZ96LtKIVjNzGs7SjANBgkqhkiG9w0BAQUFADCB
...
(Base64 encoded data)
...
-----END CERTIFICATE-----
   • PEM files can contain multiple certificates concatenated together (e.g., a server certificate followed by its intermediate CA certificates).
   • The esp_http_client and esp_tls APIs in ESP-IDF primarily expect CA certificates in PEM format when provided as strings or embedded.
2. DER (Distinguished Encoding Rules):
   • A binary format for X.509 certificates.
   • PEM is essentially a Base64 encoded version of DER.
   • While less common for manual handling by developers (who usually prefer text-based PEM), systems often convert PEM to DER internally for parsing.
   • If you have a certificate in DER format and need to use it with APIs expecting PEM, you would typically convert it (e.g., using OpenSSL: openssl x509 -inform der -in certificate.der -outform pem -out certificate.pem).

Methods for Storing Certificates on ESP32

There are several ways to make trusted CA certificates available to your ESP32 application:

1. Embedding in Firmware:
   • How: The PEM-formatted certificate string is included directly in your C code as a const char* string, or linked as binary data from a .pem file during compilation.
   • Pros:
     • Simple to implement for a small, fixed set of CAs.
     • Certificates are available immediately at boot.
     • No reliance on a filesystem or NVS being initialized or populated.
   • Cons:
     • Updating certificates requires a full firmware update (OTA).
     • Less flexible if you need to support many CAs or change them frequently.
     • Increases firmware size, especially if many or large certificates are embedded.
     • Storing private keys this way (for client certificates) is generally discouraged unless the firmware itself is encrypted and secure boot is enabled.
2. ESP-IDF Certificate Bundle:
   • How: ESP-IDF provides a mechanism to link a pre-compiled bundle of common public Root CA certificates (sourced from Mozilla) into the firmware. This is enabled via menuconfig (Component config -> mbedTLS -> Certificate Bundle). The esp_http_client can be configured to use this bundle automatically via config.crt_bundle_attach = esp_crt_bundle_attach;.
   • Pros:
     • Very convenient for connecting to common public web servers.
     • No need to manually find and embed individual CA certificates for many popular services.
     • The bundle is maintained and updated by Espressif with ESP-IDF releases.
   • Cons:
     • Increases firmware size by the size of the bundle (can be tens to over a hundred KB).
     • The bundle might not contain very new, niche, or private CAs.
     • Updating the bundle still requires an ESP-IDF update and firmware rebuild/reflash.
3. Storing in NVS (Non-Volatile Storage):
   • How: Certificates (PEM strings) are written to and read from an NVS partition.
   • Pros:
     • Allows updating certificates without a full firmware OTA (e.g., via a separate provisioning process over BLE, Wi-Fi, or UART).
     • More flexible than embedding if certificates change or need to be customized per device.
   • Cons:
     • NVS has a limited number of key-value pairs and a maximum value size (around 4000 bytes for strings, though larger blobs are possible). Long certificate chains might exceed this if stored as a single string.
     • Slightly more complex to implement reading/writing logic.
     • Slower access compared to embedded certificates.
     • Requires NVS to be initialized.
4. Storing in a Flash Filesystem (SPIFFS, LittleFS, FatFS on SD Card):
   • How: Certificate files (e.g., .pem files) are stored in a filesystem partition on the ESP32’s flash or an SD card. The application reads these files at runtime.
   • Pros:
     • Most flexible for managing a large number of certificates or large certificate files.
     • Certificates can be easily updated by replacing files in the filesystem (e.g., via OTA, HTTP download, or physical access if using an SD card).
     • Well-suited for scenarios where the set of trusted CAs is dynamic.
   • Cons:
     • Adds the overhead of a filesystem (flash space, RAM for filesystem structures).
     • More complex to implement file I/O and management.
     • Slower access than embedded certificates.
     • Requires the filesystem to be mounted and initialized.
     • Flash wear can be a concern if certificates are updated very frequently on internal flash.

graph TD
    subgraph ESP32 Device
        direction LR
        ESP32["<br><b>ESP32 Microcontroller</b>"]

        subgraph "On-Chip/Firmware Storage"
            direction TB
            Embed["<br><b>1. Embedded in Firmware</b><br>(Directly in code or as<br>linked binary data - RO Data)<br><i>Example: const char* ca_pem;</i>"]
            Bundle["<br><b>2. ESP-IDF Cert Bundle</b><br>(Pre-compiled bundle of common CAs,<br>linked into firmware)"]
        end
        
        subgraph "On-Board Flash Storage (Partitions)"
            direction TB
            NVS["<br><b>3. NVS Partition</b><br>(Non-Volatile Storage<br>for key-value pairs)"]
            FS["<br><b>4. Flash Filesystem</b><br>(SPIFFS / LittleFS partition<br>for storing .pem files)"]
        end

        ESP32 --- Embed
        ESP32 --- Bundle
        ESP32 --- NVS
        ESP32 --- FS
    end

    subgraph "External Storage (Optional)"
        direction TB
        SDCard["<br><b>5. SD Card</b><br>(Via SPI interface,<br>using FatFS filesystem)"]
    end
    
    ESP32 -.-> SDCard


    %% Styling
    classDef default fill:#transparent,stroke:#4B5563,color:#1F2937;
    classDef esp32_chip fill:#EDE9FE,stroke:#5B21B6,stroke-width:2px,color:#5B21B6
    classDef firmware_storage fill:#DBEAFE,stroke:#2563EB,stroke-width:1px,color:#1E40AF
    classDef flash_storage fill:#FEF3C7,stroke:#D97706,stroke-width:1px,color:#92400E
    classDef external_storage fill:#D1FAE5,stroke:#059669,stroke-width:1px,color:#065F46

    class ESP32 esp32_chip;
    class Embed,Bundle firmware_storage;
    class NVS,FS flash_storage;
    class SDCard external_storage;

Server Certificate Verification Details

When esp_http_client (using esp_tls and mbedTLS) performs server certificate verification, it typically checks:

1. Trust Chain: Can the server’s certificate be linked back to a trusted Root CA provided by the application (via cert_pem, crt_bundle_attach, or other esp_tls configurations)?
2. Validity Period: Are all certificates in the chain (server, intermediates, root) currently valid based on their “Not Before” and “Not After” dates? This critically depends on the ESP32 having accurate system time. If the ESP32’s clock is significantly off, it may incorrectly reject valid certificates or accept expired ones.
3. Common Name (CN) / Subject Alternative Name (SAN) Verification:
   • The client checks if the hostname it attempted to connect to matches the Common Name field in the certificate’s Subject or, preferably, an entry in the Subject Alternative Name (SAN) extension.
   • SAN is more flexible and preferred over CN for hostname matching, as it can list multiple hostnames (DNS names) and IP addresses.
   • If there’s no match, it indicates a potential man-in-the-middle attack or misconfiguration, and the connection should be rejected.
   • The esp_http_client_config_t has a field skip_cert_common_name_check (or similar, name might vary slightly across IDF versions). This should always be false (default) in production. Setting it to true disables this crucial check and makes your connection vulnerable.

flowchart TD
    A["Start: ESP32 receives Server Certificate(s)"] --> B{Is Certificate Chain Provided?};
    B -- Yes --> C{Parse Server Certificate};
    B -- No --> FAIL_NO_CHAIN[Fail: No Certificate Provided];

    C --> D{Verify Signature of Server Cert <br> using Issuer's Public Key};
    D -- Valid --> E{Is Issuer an Intermediate CA?};
    D -- Invalid --> FAIL_SIG[Fail: Signature Invalid];

    E -- Yes --> F{"Locate & Parse Issuer's (Intermediate CA) Cert"};
    F --> G{Verify Signature of Intermediate CA Cert <br> using its Issuer's Public Key};
    G -- Valid --> H{Is this Issuer a known Root CA?};
    G -- Invalid --> FAIL_SIG_INTERMEDIATE[Fail: Intermediate CA Signature Invalid];
    
    E -- No (Issuer is expected Root CA) --> H;

    H -- Yes, Root CA Found & Trusted --> I{Check Validity Period of ALL Certs in Chain};
    H -- No, or Root CA Not Trusted --> FAIL_TRUST[Fail: Untrusted Root CA or Chain Incomplete];

    I -- All Valid --> J{Check Hostname against CN/SAN in Server Cert};
    I -- Any Expired/NotYetValid --> FAIL_VALIDITY["Fail: Certificate Expired or Not Yet Valid <br> (Check ESP32 System Time!)"];

    J -- Match --> SUCCESS[Success: Server Authenticated! Proceed with TLS Handshake.];
    J -- Mismatch --> FAIL_HOSTNAME["Fail: Hostname Mismatch (CN/SAN)"];

    %% Styling
    classDef default fill:#transparent,stroke:#4B5563,color:#1F2937;
    classDef start_node fill:#EDE9FE,stroke:#5B21B6,stroke-width:2px,color:#5B21B6
    classDef process_node fill:#DBEAFE,stroke:#2563EB,stroke-width:1px,color:#1E40AF
    classDef decision_node fill:#FEF3C7,stroke:#D97706,stroke-width:1px,color:#92400E
    classDef check_node fill:#FEE2E2,stroke:#DC2626,stroke-width:1px,color:#991B1B
    classDef success_node fill:#D1FAE5,stroke:#059669,stroke-width:2px,color:#065F46
    classDef fail_node fill:#FECACA,stroke:#B91C1C,stroke-width:2px,color:#7F1D1D 

    class A start_node;
    class C,F,G process_node;
    class B,D,E,H,I,J decision_node;
    class SUCCESS success_node;
    class FAIL_NO_CHAIN,FAIL_SIG,FAIL_SIG_INTERMEDIATE,FAIL_TRUST,FAIL_VALIDITY,FAIL_HOSTNAME fail_node;

Client Certificates (Mutual TLS – mTLS)

While this chapter primarily focuses on the ESP32 verifying the server, some scenarios require mutual TLS (mTLS), where the server also verifies the client’s identity. In this case, the ESP32 would need its own client certificate and corresponding private key.

• Client Certificate and Private Key: The ESP32 would be provisioned with a client certificate (issued by a CA trusted by the server) and its associated private key.
• Configuration: The esp_http_client_config_t has fields for providing the client certificate and private key:
  • client_cert_pem: Pointer to the client’s certificate in PEM format.
  • client_key_pem: Pointer to the client’s private key in PEM format.
  • (And corresponding client_cert_len, client_key_len if not null-terminated).

• Security of Private Key: Storing and handling the client’s private key on the ESP32 requires utmost care. It’s highly sensitive.
  • Ideally, use an ESP32 variant with a secure element or hardware security module (HSM) for private key storage and operations (e.g., ESP32-H2, or external ATECC608).
  • If stored in flash, the flash partition should be encrypted.
  • Embedding private keys directly in firmware source code is generally a bad practice unless the entire firmware image is robustly protected.

mTLS is common in device-to-cloud communication for platforms like AWS IoT, Azure IoT Hub, and Google Cloud IoT Core, where each device needs to strongly authenticate itself.

Certificate Lifecycle Management

Certificates have a limited lifespan. Root CAs, intermediate CAs, and server/client certificates all expire.

graph TD
    A[1- Provisioning/Enrollment] --> B(2- Secure Storage);
    B --> C{3- Active Use & Verification};
    C --> D(4- Monitoring & Auditing);
    D --> E{Is Certificate Nearing Expiry?};
    E -- Yes --> F[5- Renewal/Re-issuance];
    F --> A;
    E -- No --> C;

    C --> G{Compromise or Revocation Needed?};
    G -- Yes --> H["6- Revocation (If applicable/possible)"];
    H --> I[7- Decommissioning/Replacement];
    G -- No --> C;

    D -- "(Found Issue)" --> H; 

    subgraph "Key Stages"
        direction LR
        A; B; C; D; F; H; I;
    end
    
    %% Styling
    classDef default fill:#transparent,stroke:#4B5563,color:#1F2937;
    classDef stage_process fill:#DBEAFE,stroke:#2563EB,stroke-width:1px,color:#1E40AF
    classDef stage_decision fill:#FEF3C7,stroke:#D97706,stroke-width:1px,color:#92400E
    classDef stage_action fill:#EDE9FE,stroke:#5B21B6,stroke-width:2px,color:#5B21B6 
    classDef stage_critical fill:#FEE2E2,stroke:#DC2626,stroke-width:1px,color:#991B1B 

    class A,B,C,D stage_process;
    class E,G stage_decision;
    class F,H,I stage_action;

• Monitoring Expiry: Your system needs a strategy for monitoring certificate expiry.
• Updating Certificates:
  • Root CAs: The ESP-IDF certificate bundle is updated with new ESP-IDF releases. If you embed custom Root CAs or store them in NVS/filesystem, you need a secure mechanism to update them (e.g., via a firmware OTA that includes new certs, or a specific provisioning process).
  • Server Certificates: Servers renew their certificates. As long as your ESP32 trusts the server’s Root CA (and any intermediates are correctly provided by the server or also trusted), renewed server certificates should validate without client-side changes.
  • Client Certificates: If your ESP32 uses a client certificate for mTLS, you’ll need a process to renew and reprovision it before it expires. This is a significant operational consideration for large IoT deployments.
• Revocation: While CRL (Certificate Revocation List) and OCSP (Online Certificate Status Protocol) are standard mechanisms for checking if a certificate has been revoked before its expiry, client-side implementation on constrained devices like ESP32 can be challenging due to resource and network overhead. esp_http_client does not perform these checks by default.

Practical Examples

These examples assume you have Wi-Fi configured and basic HTTPS knowledge from Chapter 94.

Example 1: Using the ESP-IDF Certificate Bundle

This is often the simplest way to connect to common public HTTPS servers.

Project Setup:

(Similar to Chapter 94, Example 1)

1. Ensure CONFIG_MBEDTLS_CERTIFICATE_BUNDLE is enabled in menuconfig (Component config -> mbedTLS -> Certificate Bundle -> [*] Provide bundle of common CA certificates).

main/main.c (Relevant part for configuration):

// ... (Includes and Wi-Fi setup from Chapter 94, Example 1) ...
// ... (_http_event_handler from Chapter 94, Example 1) ...

static void https_using_bundle_task(void *pvParameters)
{
    // ... (Wait for Wi-Fi) ...
    ESP_LOGI(TAG, "Connected to Wi-Fi, starting HTTPS GET task using Cert Bundle.");

    char local_response_buffer[MAX_HTTP_OUTPUT_BUFFER] = {0};

    esp_http_client_config_t config = {
        .url = "https://www.google.com", // Or another common HTTPS site
        .event_handler = _http_event_handler, // Re-use handler from Ch 94
        .user_data = local_response_buffer,
        .transport_type = HTTP_TRANSPORT_OVER_SSL,
        .crt_bundle_attach = esp_crt_bundle_attach, // Key line for using the bundle
    };
    esp_http_client_handle_t client = esp_http_client_init(&config);

    esp_err_t err = esp_http_client_perform(client);
    if (err == ESP_OK) {
        ESP_LOGI(TAG, "HTTPS GET Status = %d, content_length = %lld",
                 esp_http_client_get_status_code(client),
                 esp_http_client_get_content_length(client));
    } else {
        ESP_LOGE(TAG, "HTTPS GET request failed: %s (0x%x)", esp_err_to_name(err), err);
        // Check logs for TLS errors if any
    }

    esp_http_client_cleanup(client);
    vTaskDelete(NULL);
}

void app_main(void)
{
    // ... (NVS init, Wi-Fi init) ...
    // wifi_init_sta(); // Assuming this function is defined as in Ch 94

    xTaskCreate(&https_using_bundle_task, "https_bundle_task", 8192 * 2, NULL, 5, NULL);
}

Build and Flash: Standard procedure.

Observe: The ESP32 should connect to https://www.google.com successfully, relying on the CA certificates within the ESP-IDF bundle for trust.

Example 2: Embedding a Specific Root CA Certificate

This example shows how to embed a PEM certificate string directly in your code.

Project Setup:

1. Obtain the Root CA certificate for your target server in PEM format (e.g., save it as my_ca.pem).
2. You can either paste the string directly into your C code or use target_add_binary_data in CMakeLists.txt to link it.

Method A: Pasting PEM string in C code:

main/main.c:

// ... (Includes and Wi-Fi setup) ...
// ... (_http_event_handler) ...

// Example: A fictional Root CA certificate PEM string
// Replace this with the actual PEM content of your target Root CA
static const char *MY_SERVER_ROOT_CA_PEM = \
"-----BEGIN CERTIFICATE-----\n"
"MIIDdzCCAl+gAwIBAgIEAgAAuTANBgkqhkiG9w0BAQUFADBaMQswCQYDVQQGEwJJ\n"
"RTESMBAGA1UEChMJQmFsdGltb3JlMRMwEQYDVQQLEwpDeWJlclRydXN0MSIwIAYD\n"
// ... (many more lines of Base64 data) ...
"VQQDExxCYWx0aW1vcmUgQ3liZXJUcnVzdCBSb290MB4XDTAwMDUxMjE4NDYwMFoX\n"
"DTI1MDUxMjIzNTkwMFowWjELMAkGA1UEBhMCSUUxEjAQBgNVBAoTCUJhbHRpbW9y\n"
"-----END CERTIFICATE-----\n";


static void https_embedded_cert_task(void *pvParameters)
{
    // ... (Wait for Wi-Fi) ...
    ESP_LOGI(TAG, "Connected to Wi-Fi, starting HTTPS GET task with embedded cert.");

    char local_response_buffer[MAX_HTTP_OUTPUT_BUFFER] = {0};

    esp_http_client_config_t config = {
        .url = "https://your-specific-server.com/api/data", // Replace with your server
        .event_handler = _http_event_handler,
        .user_data = local_response_buffer,
        .transport_type = HTTP_TRANSPORT_OVER_SSL,
        .cert_pem = MY_SERVER_ROOT_CA_PEM, // Use the embedded certificate
    };
    esp_http_client_handle_t client = esp_http_client_init(&config);
    // ... (perform, cleanup as before) ...
    esp_err_t err = esp_http_client_perform(client);
    if (err == ESP_OK) {
        ESP_LOGI(TAG, "HTTPS GET Status = %d, content_length = %lld",
                 esp_http_client_get_status_code(client),
                 esp_http_client_get_content_length(client));
    } else {
        ESP_LOGE(TAG, "HTTPS GET request failed: %s (0x%x)", esp_err_to_name(err), err);
    }
    esp_http_client_cleanup(client);
    vTaskDelete(NULL);
}

// ... (app_main to call this task) ...

Method B: Linking PEM file as binary data:

Save your CA certificate as my_server_ca.pem in your main directory.

In main/CMakeLists.txt, add:

target_add_binary_data(${COMPONENT_TARGET} "my_server_ca.pem" TEXT)

In main/main.c:

// ... (Includes and Wi-Fi setup) ...
// ... (_http_event_handler) ...

// Declare symbols for the embedded binary data
extern const char my_server_ca_pem_start[] asm("_binary_my_server_ca_pem_start");
extern const char my_server_ca_pem_end[]   asm("_binary_my_server_ca_pem_end");

static void https_linked_cert_task(void *pvParameters)
{
    // ... (Wait for Wi-Fi) ...
    ESP_LOGI(TAG, "Connected to Wi-Fi, starting HTTPS GET task with linked cert.");

    // Calculate length if needed, or ensure cert_pem is null-terminated by how it's linked
    // const unsigned int my_server_ca_pem_len = my_server_ca_pem_end - my_server_ca_pem_start;

    char local_response_buffer[MAX_HTTP_OUTPUT_BUFFER] = {0};

    esp_http_client_config_t config = {
        .url = "https://your-specific-server.com/api/data", // Replace
        .event_handler = _http_event_handler,
        .user_data = local_response_buffer,
        .transport_type = HTTP_TRANSPORT_OVER_SSL,
        .cert_pem = my_server_ca_pem_start,
        // .cert_len = my_server_ca_pem_len, // Use if cert_pem is not null-terminated
    };
    esp_http_client_handle_t client = esp_http_client_init(&config);
    // ... (perform, cleanup as before) ...
    esp_err_t err = esp_http_client_perform(client);
    if (err == ESP_OK) {
        ESP_LOGI(TAG, "HTTPS GET Status = %d, content_length = %lld",
                esp_http_client_get_status_code(client),
                esp_http_client_get_content_length(client));
    } else {
        ESP_LOGE(TAG, "HTTPS GET request failed: %s (0x%x)", esp_err_to_name(err), err);
    }
    esp_http_client_cleanup(client);
    vTaskDelete(NULL);
}
// ... (app_main to call this task) ...

Build and Flash: Standard procedure.

Observe: The connection should succeed if your-specific-server.com‘s certificate chain is validated by my_server_ca.pem.

Example 3: Loading Root CA from NVS (Conceptual)

This example outlines the steps. A full, robust implementation would involve provisioning the certificate into NVS first.

Provisioning (One-time or via a separate mechanism):

#include "nvs_flash.h"
#include "nvs.h"

const char* NVS_CERT_NAMESPACE = "certs";
const char* NVS_ROOT_CA_KEY = "root_ca";

// Fictional CA content for demonstration
const char* ca_to_store_in_nvs = \
"-----BEGIN CERTIFICATE-----\n"
"MIIF2DCCA8CgAwIBAgIQAcq7p34iG2AnS144X4o2BDANBgkqhkiG9w0BAQsFADBG\n"
// ... rest of actual PEM content ...
"-----END CERTIFICATE-----\n";


void provision_ca_to_nvs(void) {
    nvs_handle_t nvs_handle;
    esp_err_t err = nvs_open(NVS_CERT_NAMESPACE, NVS_READWRITE, &nvs_handle);
    if (err != ESP_OK) {
        ESP_LOGE(TAG, "Error (%s) opening NVS handle!", esp_err_to_name(err));
        return;
    }
    err = nvs_set_str(nvs_handle, NVS_ROOT_CA_KEY, ca_to_store_in_nvs);
    if (err != ESP_OK) {
        ESP_LOGE(TAG, "Error (%s) writing CA to NVS!", esp_err_to_name(err));
    } else {
        ESP_LOGI(TAG, "CA certificate written to NVS successfully.");
        err = nvs_commit(nvs_handle);
        if (err != ESP_OK) {
            ESP_LOGE(TAG, "Error (%s) committing NVS!", esp_err_to_name(err));
        }
    }
    nvs_close(nvs_handle);
}

Loading and Using in HTTP Client Task:

// ... (Includes, Wi-Fi setup, _http_event_handler) ...

static char* loaded_ca_from_nvs = NULL; // Global or dynamically managed buffer

esp_err_t load_ca_from_nvs(void) {
    nvs_handle_t nvs_handle;
    esp_err_t err = nvs_open(NVS_CERT_NAMESPACE, NVS_READONLY, &nvs_handle);
    if (err != ESP_OK) {
        ESP_LOGE(TAG, "Error (%s) opening NVS handle for reading!", esp_err_to_name(err));
        return err;
    }

    size_t required_size = 0;
    err = nvs_get_str(nvs_handle, NVS_ROOT_CA_KEY, NULL, &required_size);
    if (err != ESP_OK && err != ESP_ERR_NVS_NOT_FOUND) {
        ESP_LOGE(TAG, "Error (%s) getting CA size from NVS!", esp_err_to_name(err));
        nvs_close(nvs_handle);
        return err;
    }

    if (required_size == 0) {
        ESP_LOGE(TAG, "CA not found in NVS or size is 0.");
        nvs_close(nvs_handle);
        return ESP_ERR_NVS_NOT_FOUND;
    }

    if (loaded_ca_from_nvs) free(loaded_ca_from_nvs); // Free previous if any
    loaded_ca_from_nvs = malloc(required_size);
    if (loaded_ca_from_nvs == NULL) {
        ESP_LOGE(TAG, "Failed to allocate memory for CA from NVS (%d bytes)", required_size);
        nvs_close(nvs_handle);
        return ESP_ERR_NO_MEM;
    }

    err = nvs_get_str(nvs_handle, NVS_ROOT_CA_KEY, loaded_ca_from_nvs, &required_size);
    if (err != ESP_OK) {
        ESP_LOGE(TAG, "Error (%s) reading CA from NVS!", esp_err_to_name(err));
        free(loaded_ca_from_nvs);
        loaded_ca_from_nvs = NULL;
    } else {
        ESP_LOGI(TAG, "CA certificate loaded from NVS successfully.");
    }
    nvs_close(nvs_handle);
    return err;
}

static void https_nvs_cert_task(void *pvParameters) {
    // ... (Wait for Wi-Fi) ...
    if (load_ca_from_nvs() != ESP_OK || loaded_ca_from_nvs == NULL) {
        ESP_LOGE(TAG, "Failed to load CA from NVS. Aborting HTTPS task.");
        vTaskDelete(NULL);
        return;
    }

    char local_response_buffer[MAX_HTTP_OUTPUT_BUFFER] = {0};
    esp_http_client_config_t config = {
        .url = "https://your-specific-server.com/api/data", // Replace
        .event_handler = _http_event_handler,
        .user_data = local_response_buffer,
        .transport_type = HTTP_TRANSPORT_OVER_SSL,
        .cert_pem = loaded_ca_from_nvs, // Use CA loaded from NVS
    };
    esp_http_client_handle_t client = esp_http_client_init(&config);
    // ... (perform, cleanup) ...
    // IMPORTANT: Free loaded_ca_from_nvs after esp_http_client_cleanup(client) if it's no longer needed
    // or manage its lifecycle carefully if the client is reused.
    // For this example, assuming it's freed when task ends or before next load.
    esp_err_t err = esp_http_client_perform(client);
    if (err == ESP_OK) { /* ... log success ... */ }
    else { /* ... log error ... */ }
    
    esp_http_client_cleanup(client);
    if (loaded_ca_from_nvs) {
        free(loaded_ca_from_nvs);
        loaded_ca_from_nvs = NULL;
    }
    vTaskDelete(NULL);
}

void app_main(void) {
    // ... (NVS init, Wi-Fi init) ...
    // Call provision_ca_to_nvs() once, perhaps based on a flag or first boot.
    // provision_ca_to_nvs(); 
    xTaskCreate(&https_nvs_cert_task, "https_nvs_task", 8192 * 2, NULL, 5, NULL);
}

Note: For SPIFFS/LittleFS, the concept is similar: provision the .pem file to the filesystem, then read its content into a buffer at runtime and pass that buffer to config.cert_pem. Remember to handle file I/O errors.

Variant Notes

• Flash Encryption and Secure Boot: If certificates (especially client private keys) are stored in flash (NVS or filesystem), enabling ESP-IDF’s flash encryption feature is highly recommended to protect them from physical extraction. Secure Boot helps ensure that only authorized firmware (which respects this protection) can run. These features are available on most ESP32 variants.
• Memory for Certificate Parsing: Parsing X.509 certificates, especially chains, can be memory-intensive. Variants with more RAM (e.g., ESP32-S3, some ESP32 original modules with PSRAM) will handle complex certificate scenarios more easily. The mbedTLS configuration can be tweaked in menuconfig to optimize memory usage if needed, sometimes at the cost of features or performance.
• Secure Element / HSM Integration: For applications requiring the highest level of security for client private keys (used in mTLS), consider variants like ESP32-H2 which may have enhanced secure storage capabilities, or integrate an external Secure Element (like ATECC608A) via I2C. esp-cryptoauthlib can be used for this, and esp_tls can be configured to use private keys stored in such secure elements. This is an advanced topic beyond simple cert_pem usage.

The choice of certificate storage method often depends on the application’s security requirements, update strategy, and the number/size of certificates involved.

Common Mistakes & Troubleshooting Tips

Exercises

1. Implement Certificate Loading from SPIFFS:
   • Create a project that initializes and mounts a SPIFFS filesystem.
   • Before building and flashing, prepare a ca.pem file with a Root CA certificate and include it in a SPIFFS image that gets flashed to the device.
   • Write C code to read ca.pem from SPIFFS into a dynamically allocated buffer.
   • Use this loaded certificate with esp_http_client to connect to an HTTPS server that is trusted by this CA.
   • Ensure proper error handling for file operations and memory management.
2. Selective Certificate Usage:
   • Embed two different Root CA PEM strings in your firmware (e.g., ca1_pem and ca2_pem).
   • Write an application that connects to two different HTTPS servers, where server1.com is trusted by ca1_pem and server2.com is trusted by ca2_pem.
   • Dynamically select the correct cert_pem in the esp_http_client_config_t based on which server you are connecting to.
3. Client Certificate (mTLS) Setup with a Test Server:
   • Advanced: This requires setting up a server that supports mTLS or using a public mTLS test endpoint if available.
   • Generate a client certificate and private key pair (e.g., using OpenSSL).
   • Configure your test server to require client certificates and trust the CA that signed your client certificate.
   • Embed the client certificate (client_cert_pem) and client private key (client_key_pem) in your ESP32 firmware (use target_add_binary_data for security rather than pasting keys in code).
   • Also provide the server’s Root CA certificate (cert_pem or bundle).
   • Attempt an HTTPS request to your mTLS-enabled server.
   • Focus: Correctly configuring esp_http_client with client cert/key and server CA.
   • Warning: Handle private keys with extreme care. For production, avoid embedding private keys directly if possible; use secure elements.

Summary

• Effective certificate management is vital for secure HTTPS communication on ESP32.
• Certificates can be embedded in firmware, loaded from NVS or a flash filesystem (SPIFFS/LittleFS), or accessed via the ESP-IDF Certificate Bundle. Each method has trade-offs in terms of flexibility, updateability, and resource usage.
• PEM is the common text-based format for certificates used with ESP-IDF APIs.
• Robust server certificate verification involves checking the trust chain, validity period (requiring accurate system time), and hostname (CN/SAN matching).
• For mutual TLS (mTLS), the ESP32 also needs its own client certificate and private key, which must be stored securely.
• Certificate lifecycle (expiry, updates, revocation) needs consideration in product design.
• Common issues often relate to incorrect certificate formatting, providing the wrong type of certificate (e.g., server cert instead of Root CA), or errors in loading from storage.

Further Reading

• ESP-IDF Programming Guide – NVS:
  • NVS API Documentation
• ESP-IDF Programming Guide – SPIFFS / LittleFS:
  • SPIFFS Filesystem
  • LittleFS Filesystem
• OpenSSL Command Line Tool Documentation: (Useful for inspecting and converting certificates)
  • OpenSSL Website
• mbedTLS esp_crt_bundle_attach source:
  • Inspect the source code for esp_crt_bundle.c in your ESP-IDF components directory (components/mbedtls/esp_crt_bundle/esp_crt_bundle.c) to see how it’s implemented.
• Best Practices for IoT Security (General):
  • Look for whitepapers and guides from organizations like NIST, ENISA, or IoT Security Foundation.