Chapter 247: Partition Encryption Techniques

Chapter Objectives

By the end of this chapter, you will be able to:

• Understand the relationship between Flash Encryption and partition-level encryption.
• Enable and configure Flash Encryption for an ESP-IDF project.
• Mark specific partitions for encryption within a custom partition table.
• Understand the “first boot” process for enabling encryption and generating keys.
• Read from and write to encrypted partitions transparently.
• Recognize the security implications and limitations of Flash Encryption.
• Identify differences in encryption features across ESP32 variants.

Introduction

In the previous chapter, we learned how to organize flash memory by creating custom partition tables. While this gives us structural control, it does nothing to protect the data itself. If a malicious actor gains physical access to your device, they could read the raw contents of the flash chip, exposing sensitive information like Wi-Fi credentials, private keys, or proprietary firmware.

Partition encryption, built upon the ESP32’s powerful Flash Encryption feature, solves this problem. By enabling it, you can render the data in specified partitions unreadable without the device’s unique, hardware-fused key. This chapter will guide you through the theory, configuration, and practical application of encrypting individual partitions, adding a critical layer of physical security to your embedded projects.

Theory

Partition encryption is not a standalone feature; it is an integral part of the Flash Encryption security mechanism provided by all ESP32 variants. It’s essential to understand the core concepts of Flash Encryption before applying it to partitions.

1. What is Flash Encryption?

Flash Encryption is a hardware-accelerated feature that encrypts the contents of the SPI flash memory. When enabled, all data written to the flash is encrypted on-the-fly by a dedicated hardware block, and all data read is decrypted on-the-fly. The encryption key is stored in the device’s eFuses, which are a special type of one-time-programmable memory. Once a key is written to the eFuses and the corresponding lock bits are set, the key cannot be read out or modified via software, making it a secure, device-specific secret.

%%{init: {'theme': 'base', 'themeVariables': { 'fontFamily': '"Open Sans", sans-serif'}}}%%
graph TD
    subgraph "CPU Domain (Plaintext)"
        A("Application Code<br>e.g., esp_partition_read()")
        B("Application Code<br>e.g., esp_partition_write()")
    end

    subgraph "Flash Controller (Hardware AES Engine)"
        C{Transparent<br>Decryption}
        D{Transparent<br>Encryption}
    end

    subgraph "External SPI Flash (Encrypted)"
        E["Encrypted Data<br>on Flash Chip"]
    end
    
    A -- "Requests Data" --> C
    C -- "Reads Encrypted Data" --> E
    E -- "Provides Encrypted Data" --> C
    C -- "Returns Plaintext Data" --> A

    B -- "Writes Plaintext Data" --> D
    D -- "Sends Encrypted Data" --> E

    %% Styling
    classDef cpu fill:#DBEAFE,stroke:#2563EB,stroke-width:1px,color:#1E40AF
    classDef hardware fill:#EDE9FE,stroke:#5B21B6,stroke-width:2px,color:#5B21B6
    classDef flash fill:#FEE2E2,stroke:#DC2626,stroke-width:1px,color:#991B1B
    
    class A,B cpu
    class C,D hardware
    class E flash

The entire process is transparent to the application. Your code, using standard functions like esp_partition_read() and esp_partition_write(), interacts with decrypted data. The hardware handles the cryptographic heavy lifting.

2. The Encryption Key

The encryption key is a 256-bit AES key. It can be generated in two ways:

1. Device-Generated: On the very first boot after enabling Flash Encryption, if no key is pre-provisioned, the ESP32’s hardware random number generator (RNG) creates a key and burns it into the eFuses. This is the most common method for development and many production scenarios, as each device will have a unique key.
2. Host-Generated: For production environments where all devices in a batch must share the same key, you can generate a key on a host computer and flash it to the device during the manufacturing process. This is an advanced use case.

Warning: Once Flash Encryption is enabled and the key is burned into the eFuses, the process is irreversible. A device with Flash Encryption enabled can only boot encrypted firmware. You cannot disable it. Furthermore, if the eFuse key is lost or damaged, the encrypted data on the flash is irrecoverable.

3. How Partitions Get Encrypted

When you enable Flash Encryption, you can choose which partitions to encrypt. This is controlled by the flags column in your partition table .csv file.

By adding the encrypted flag to a partition’s entry, you are telling the bootloader and the flashing tools that this partition’s contents must be encrypted on the flash chip.

• App Partitions (app): Application partitions are always encrypted if Flash Encryption is enabled, regardless of the flag. This is a fundamental security measure to protect your intellectual property.
• Data Partitions (data): Data partitions are only encrypted if you explicitly add the encrypted flag in the partition table. This gives you the flexibility to leave non-sensitive data partitions unencrypted to save on minor performance overhead. Partitions like nvs and phy_init are typically left unencrypted.

4. The First Boot: The Point of No Return

The process of enabling encryption is critical:

%%{init: {'theme': 'base', 'themeVariables': { 'fontFamily': '"Open Sans", sans-serif'}}}%%
sequenceDiagram
    participant BTL as 1st Stage Bootloader
    participant eFuse as eFuse Controller
    participant Flash as SPI Flash Memory
    
    BTL->>Flash: Read App Header
    Flash-->>BTL: App built with encryption enabled
    
    BTL->>eFuse: Check encryption status
    eFuse-->>BTL: eFuse bits are not set (encryption is off)
    
    BTL->>BTL: <b>Point of No Return Begins</b>
    
    Note right of BTL: This happens ONLY ONCE!
    
    BTL->>eFuse: 1. Generate & Burn AES Key
    activate eFuse
    eFuse-->>BTL: Key is now fused
    deactivate eFuse

    BTL->>Flash: 2. Read plaintext partitions (app, flagged data, etc.)
    BTL->>BTL: Encrypt partitions in RAM
    BTL->>Flash: Write back encrypted partitions
    
    BTL->>eFuse: 3. Burn eFuse bits to<br>permanently enable Flash Encryption
    activate eFuse
    eFuse-->>BTL: Mode is now permanently ON
    deactivate eFuse
    
    BTL->>BTL: 4. Trigger system reset to<br>re-boot in encrypted mode

1. Build: You build your project with Flash Encryption enabled in menuconfig. The build tools create an encrypted “plaintext” binary.
2. Flash: You flash the device as usual. At this stage, the binaries on the flash are not yet encrypted. The eFuses are also not yet programmed.
3. First Boot: The device boots. The 1st stage bootloader detects that Flash Encryption is enabled in the app binary’s header. It then performs the one-time encryption process:
   • It generates a new AES key (or uses a pre-flashed one).
   • It burns this key into the eFuses.
   • It encrypts the app partitions, the partition table itself, and any data partitions marked with the encrypted flag in-place.
   • Finally, it permanently sets the eFuse bits that enable Flash Encryption mode.
4. Subsequent Boots: On every boot after this, the bootloader expects to find encrypted firmware and data. All read/write operations to the flash are now transparently decrypted/encrypted by the hardware.

Practical Examples

Let’s modify the project from the previous chapter to encrypt our custom storage partition.

Example: Encrypting the ‘storage’ Partition

Our goal is to secure the data within the storage partition we created previously.

1. Update the Custom Partition .csv File

Modify your partitions_custom.csv file to add the encrypted flag to the storage partition.

# ESP-IDF Partition Table
# Name,   Type, SubType, Offset,   Size,   Flags
nvs,      data, nvs,     ,         28K,
otadata,  data, ota,     ,         8K,
ota_0,    app,  ota_0,   ,         1856K,
ota_1,    app,  ota_1,   ,         1856K,
storage,  data, 0x99,    ,         256K,   encrypted

This is the only change needed in the partition table. The build system will now know that this partition requires encryption.

2. Enable Flash Encryption in menuconfig

This is the most critical step.

1. Open the Project Configuration menu (>ESP-IDF: SDK Configuration editor (menuconfig)).
2. Navigate to Security features —>.
3. Check the box for [*] Enable flash encryption on boot.
4. Leave the Flash Encryption Mode as Development (NOT SECURE). The Release mode imposes stricter checks and is intended for production. For this example, development mode is sufficient.
5. Save the configuration and exit.

Tip: In development mode, the bootloader allows re-flashing of plaintext application binaries over an encrypted device. It re-encrypts the new binary on the next boot using the existing key. In Release mode, this is forbidden to prevent tampering.

3. The Code (No Changes Needed!)

Here is the beauty of transparent Flash Encryption. The C code from the previous chapter for writing to and reading from the storage partition does not need to be changed at all.

Your main/main.c file remains the same:

#include <stdio.h>
#include <string.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
#include "esp_partition.h"

static const char *TAG = "ENCRYPT_PARTITION_EXAMPLE";

void app_main(void)
{
    ESP_LOGI(TAG, "Starting Partition Encryption Example...");

    const esp_partition_t *storage_partition = esp_partition_find_first(ESP_PARTITION_TYPE_DATA, 0x99, "storage");
    
    if (!storage_partition) {
        ESP_LOGE(TAG, "Failed to find 'storage' partition.");
        while(1) { vTaskDelay(pdMS_TO_TICKS(1000)); }
    } else {
        ESP_LOGI(TAG, "Found 'storage' partition: size=0x%x, offset=0x%x", storage_partition->size, storage_partition->address);
    }
    
    // The esp_partition functions work identically on encrypted partitions.
    // The hardware handles all encryption/decryption transparently.

    const char *message = "This is a secret message on an encrypted partition!";
    char read_buffer[128] = {0};

    ESP_LOGI(TAG, "Writing secret data to encrypted partition...");
    esp_err_t err = esp_partition_write(storage_partition, 0, message, strlen(message) + 1);
    if (err != ESP_OK) {
        ESP_LOGE(TAG, "Failed to write data: %s", esp_err_to_name(err));
    } else {
        ESP_LOGI(TAG, "Successfully wrote data.");
    }

    ESP_LOGI(TAG, "Reading secret data back...");
    err = esp_partition_read(storage_partition, 0, read_buffer, sizeof(read_buffer));
    if (err != ESP_OK) {
        ESP_LOGE(TAG, "Failed to read data: %s", esp_err_to_name(err));
    } else {
        ESP_LOGI(TAG, "Successfully read data: '%s'", read_buffer);
    }

    if (strcmp(message, read_buffer) == 0) {
        ESP_LOGI(TAG, "Data verification successful! The secret is safe.");
    } else {
        ESP_LOGE(TAG, "Data verification FAILED!");
    }

    ESP_LOGI(TAG, "Example finished.");
}

4. Build, Flash, and Monitor the First Boot

This process is different from a normal flash. Pay close attention to the monitor output.

1. Build: Build the project as usual.
2. Flash: Flash the project to a clean (never-before-encrypted) ESP32.
3. Monitor: Open the serial monitor immediately. You will see the special first-boot log messages.

The monitor output will look something like this:

...
I (32) boot: ESP-IDF v5.x-dirty 2nd stage bootloader
...
I (72) boot: Loading partition table
I (82) boot: Checking flash encryption...
I (89) boot: Detected flash encryption in menuconfig, but not enabled in eFuse.
I (89) boot: Therefore, enabling flash encryption...
I (95) boot: Generating new flash encryption key...
I (134) boot: Burned flash encryption key
I (134) boot: Writing EFUSE to enable flash encryption...
I (139) boot: Reading partition table from flash...
I (145) boot: Encrypting partition 2 at offset 0x10000...
I (314) boot: Encrypting partition 3 at offset 0x1a0000...
I (502) boot: Encrypting partition 4 at offset 0x3b0000...
I (528) boot: Flash encryption completed
I (533) boot: Resetting...
ets Jul 29 2019 12:21:46
...
rst:0x3 (SW_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
...
I (72) boot: Checking flash encryption...
I (73) boot: Flash encryption is enabled (Development mode)
...
I (288) ENCRYPT_PARTITION_EXAMPLE: Starting Partition Encryption Example...
I (298) ENCRYPT_PARTITION_EXAMPLE: Found 'storage' partition: size=0x40000, offset=0x3b0000
I (298) ENCRYPT_PARTITION_EXAMPLE: Writing secret data to encrypted partition...
I (318) ENCRYPT_PARTITION_EXAMPLE: Successfully wrote data.
I (318) ENCRYPT_PARTITION_EXAMPLE: Reading secret data back...
I (328) ENCRYPT_PARTITION_EXAMPLE: Successfully read data: 'This is a secret message on an encrypted partition!'
I (338) ENCRYPT_PARTITION_EXAMPLE: Data verification successful! The secret is safe.
I (348) ENCRYPT_PARTITION_EXAMPLE: Example finished.

The application runs perfectly, completely unaware that the underlying data on the flash chip is now scrambled by AES encryption.

Variant Notes

Flash Encryption is a core security feature available on all ESP32 variants, but there are minor implementation differences.

• ESP32: Uses one AES key for both Flash Encryption and Secure Boot.
• ESP32-S2, S3, C3, C6, H2: These newer chips have dedicated, separate eFuse key blocks for Flash Encryption and Secure Boot, which is a more robust design. They also feature more advanced hardware support for cryptography. The ESP32-S2, for example, has an “XTS” mode which provides enhanced protection against data tampering.
• Configuration: The menuconfig options and the encrypted flag in the partition table work identically across all variants. The ESP-IDF abstracts the underlying hardware differences.

The practical steps shown in this chapter apply universally. Always refer to the specific datasheet and Technical Reference Manual for your variant for the deepest architectural details.

Common Mistakes & Troubleshooting Tips

Exercises

1. Encrypting the NVS Partition: In many applications, the NVS partition holds sensitive data like Wi-Fi credentials. Modify your partitions_custom.csv to add the encrypted flag to the nvs partition. After flashing and the first boot, use the NVS example code to write a key-value pair and verify that the NVS library continues to function correctly, proving that the encryption is transparent.
2. Verify Encryption with esptool: Take the project from this chapter. After flashing and encrypting, use the command line tool esptool.py to read the entire flash memory to a file (esptool.py --port COMx read_flash 0 4MB flash_dump.bin). Open flash_dump.bin in a hex editor. Find the offset of your storage partition (from the monitor logs) and observe the random-looking, encrypted data. Compare this to the plaintext string in your code to visually confirm that the data is not readable.

Summary

• Partition encryption is a function of the master Flash Encryption feature.
• Enabling Flash Encryption is a one-time, irreversible process that burns a unique AES key into the device’s eFuses.
• Partitions are marked for encryption by adding the encrypted flag in the partition table CSV file.
• Application partitions are always encrypted when the feature is on; data partitions are encrypted only if flagged.
• The encryption/decryption process is hardware-accelerated and transparent to the application code; esp_partition and other APIs work without modification.
• The first boot after flashing with encryption enabled is a critical step where the device self-encrypts its storage.
• Always use Development Mode for development and be extremely careful with keys in Release Mode.

Further Reading

• ESP-IDF Flash Encryption Documentation: https://docs.espressif.com/projects/esp-idf/en/v5.2.1/security/flash-encryption.html
• ESP32 Technical Reference Manual (TRM): Contains detailed information on the AES hardware accelerator and eFuse block for your specific variant. (Links available on the Espressif website).
• Secure Boot Documentation: A related security feature that ensures only signed, authentic firmware can run on the device. https://docs.espressif.com/projects/esp-idf/en/v5.2.1/security/secure-boot-v2.html