Chapter 8: Understanding ESP32 Boot Process

Chapter Objectives

Upon completing this chapter, you will be able to:

• Describe the sequence of events from power-on/reset to the execution of your application code (app_main).
• Identify the roles of the ROM Bootloader (1st stage) and the SPI Bootloader (2nd stage).
• Understand the purpose and structure of the Partition Table in Flash memory.
• Explain how strapping pins influence the boot mode (Run vs. Download).
• Use ESP-IDF tools (idf.py, esptool.py, parttool.py) to examine partitions and interact with the bootloader.
• Recognize key differences in the boot process across various ESP32 variants.
• Troubleshoot common boot-related problems.
• Appreciate the basics of Secure Boot and Flash Encryption features.

Introduction

When you apply power or reset an ESP32 chip, a complex sequence of operations occurs before your carefully crafted app_main function ever gets called. This sequence, known as the boot process, is fundamental to the chip’s operation. Understanding it is crucial not only for basic development but also for debugging startup issues, implementing firmware updates (Over-The-Air or OTA), managing flash memory layout, and utilizing security features.

This chapter demystifies the ESP32 boot process. We will trace the journey from the initial hardware reset, through the execution of internal ROM code, the loading of a secondary bootloader from Flash, the interpretation of the partition table, and finally, the loading and execution of your main application firmware.

Theory

The ESP32 boot process involves multiple stages, transitioning from internal, unchangeable code in ROM to configurable software loaded from external SPI Flash memory.

%%{ init: { 'theme': 'base', 'themeVariables': { 'fontFamily': 'Open Sans'} } }%%
graph TD
    A["Reset Signal<br>(POR, External, Watchdog, etc.)"] --> B{"ROM Bootloader (1st Stage)"};
    B --> C{Check Strapping Pins};
    C -- Download Mode (e.g., GPIO0 LOW) --> D[UART/USB Interaction<br>esptool.py for flashing];
    C -- Run Mode (e.g., GPIO0 HIGH) --> E["Load SPI Bootloader (2nd Stage)<br>from Flash @ 0x1000 or 0x0"];
    E --> F[Read Partition Table<br>from Flash @ 0x8000];
    F --> G{"Select App Partition<br>(Factory, OTA_0, OTA_1)"};
    G --> H["Verify & Load Application<br>(Integrity Check/Secure Boot)"];
    H --> I["Jump to App Entry Point<br>(e.g., call_start_cpu0)"];
    I --> J["Application Initialization<br>(Peripherals, Heap, FreeRTOS)"];
    J --> K["Run app_main()"];

    %% Styling
    classDef startNode fill:#EDE9FE,stroke:#5B21B6,stroke-width:2px,color:#5B21B6;
    classDef processNode fill:#DBEAFE,stroke:#2563EB,stroke-width:1px,color:#1E40AF;
    classDef decisionNode fill:#FEF3C7,stroke:#D97706,stroke-width:1px,color:#92400E;
    classDef checkNode fill:#FEE2E2,stroke:#DC2626,stroke-width:1px,color:#991B1B;
    classDef endNode fill:#D1FAE5,stroke:#059669,stroke-width:2px,color:#065F46;

    class A startNode;
    class B,E,F,H,I,J processNode;
    class C,G decisionNode;
    class D processNode; 
    class K endNode;

1. Reset and Hardware Initialization

The process begins when the chip comes out of reset. This can be triggered by:

• Power-On Reset (POR): Applying power to the chip.
• External Reset: Asserting the CHIP_PU (Enable) pin low and then high.
• Watchdog Timer Reset: If a hardware or software watchdog timer expires.
• Software Reset: Triggered intentionally by application code.
• Deep Sleep Wake-up: Waking from a low-power deep sleep state (which involves a reset).

Upon reset, basic hardware initialization occurs, and the CPU(s) begin executing code from a fixed address known as the Reset Vector.

2. ROM Bootloader (1st Stage Bootloader)

The Reset Vector points into the chip’s internal Read-Only Memory (ROM). This ROM contains the 1st Stage Bootloader, a small, immutable piece of code burned into the silicon during manufacturing. Its primary responsibilities are:

What are Strapping Pins?

Strapping pins are specific GPIOs whose voltage levels (High/Low/Floating) are sampled by the ROM Bootloader immediately after reset. These levels determine fundamental chip configurations, most importantly the boot mode. Common strapping pins include GPIO0, GPIO2, MTDI (GPIO12), GPIO4, GPIO5, depending on the specific ESP32 variant. Consult the datasheet for your variant!

• Download Mode: If the strapping pins indicate Download Mode (typically by holding GPIO0 LOW during reset on many ESP32 variants), the ROM Bootloader enters a state where it listens for commands over UART0 (or USB Serial/JTAG on newer variants like S2, S3, C3, C6). Tools like esptool.py use this mode to write firmware (bootloader, partition table, application) to the Flash memory.
• Run Mode (SPI Boot): If the strapping pins indicate normal execution mode (typically GPIO0 HIGH or Floating), the ROM Bootloader attempts to load the 2nd Stage Bootloader from the external SPI Flash memory, usually located at offset 0x1000.

3. SPI Bootloader (2nd Stage Bootloader)

If in Run Mode, the ROM Bootloader loads the 2nd Stage Bootloader (often called bootloader.bin) from the SPI Flash into internal SRAM (IRAM/DRAM). This bootloader is part of the ESP-IDF project and is flashed along with your application. It’s significantly more complex than the ROM bootloader and performs tasks like:

• Initializing more peripherals (including SPI Flash interface with correct settings like speed and mode read from the image header).
• Configuring memory mapping (MMU) and enabling Flash cache (XIP).
• Reading the Partition Table from Flash (usually at offset 0x8000).
• Checking for Over-The-Air (OTA) update instructions (if configured).
• Selecting which application partition to boot (e.g., factory, ota_0, ota_1).
• Performing integrity checks (checksum/hash) or signature verification (if Secure Boot is enabled) on the selected application image.
• Loading the application image segments (code and data) from Flash into the appropriate RAM regions (IRAM/DRAM).
• Finally, jumping to the application’s entry point.

Tip: The source code for the 2nd Stage Bootloader is available within the ESP-IDF components (/components/bootloader). You generally don’t need to modify it, but understanding its flow (bootloader_start.c) can be insightful.

4. Partition Table

The external SPI Flash memory is not treated as one monolithic block. Instead, it’s organized using a Partition Table. This table, also flashed onto the device, defines named regions (partitions) within the Flash, specifying their type, subtype, offset, and size.

Common partition types include:

• app: Stores application firmware. There’s usually a factory app and potentially ota_0, ota_1, etc., for OTA updates.
• data: Stores data, such as:
  • nvs: Non-Volatile Storage for key-value pairs.
  • phy: PHY calibration data.
  • spiffs, littlefs, fat: Filesystem data.
  • coredump: Core dump storage for debugging crashes.
• bootloader: Although the 2nd stage bootloader resides at 0x1000, this partition type isn’t typically listed in the main table itself as its location is fixed.

The 2nd Stage Bootloader reads this table (located at 0x8000 by default) to find the location and size of the application partition it needs to load. Partition tables are defined in simple CSV format within your ESP-IDF project (e.g., partitions.csv) and compiled into a binary format (partition_table.bin) during the build process.

5. Application Loading and Execution

Once the 2nd Stage Bootloader identifies a valid application partition (e.g., factory), it performs the following:

6. Application Startup

The application code now takes over. The initial application startup code (part of ESP-IDF, before app_main) performs tasks like:

From this point, your custom application logic begins executing.

7. Secure Boot and Flash Encryption

ESP32 variants offer security features:

• Flash Encryption: Encrypts the contents of the SPI Flash (bootloader, partition table, applications, data). The keys are stored in eFuses and are hardware-protected. When enabled, the hardware transparently decrypts code/data as it’s read from Flash into the cache or RAM. This prevents unauthorized reading of the firmware from the Flash chip.
• Secure Boot: Ensures that only authentically signed firmware can be executed. The ROM Bootloader and 2nd Stage Bootloader verify the digital signature of the next stage bootloader and application, respectively, using keys stored in eFuses. This prevents running unauthorized or tampered firmware.

Warning: Enabling Flash Encryption and Secure Boot involves burning eFuses, which are one-time programmable (OTP) bits. Once burned, they cannot be easily reversed. Incorrect configuration can permanently brick the device or lock you out. Approach these features with extreme caution and thorough understanding, starting with development kits before production hardware.

%%{ init: { 'theme': 'base', 'themeVariables': { 'fontFamily': 'Open Sans' } } }%%
graph TD
    A["Start: Considering Security Features<br>(Secure Boot / Flash Encryption)"] --> B{"Understand eFuses are<br>One-Time Programmable (OTP)"};
    B --> C["Action: Thoroughly Read ESP-IDF<br>Security Documentation!"];
    C --> D{"Have you backed up<br>all generated keys securely?"};
    D -- No --> E["STOP! Backup keys first.<br>Losing keys can be catastrophic."];
    E --> D;
    D -- Yes --> F{"Tested entire process on a<br>non-production development kit?"};
    F -- No --> G["STOP! Test on dev kit first.<br>Avoid bricking production hardware."];
    G --> F;
    F -- Yes --> H["Proceed with Extreme Caution<br>on Production Hardware"];
    H --> I["Potential Outcome 1:<br>Security Features Enabled Successfully"];
    H --> J["Potential Outcome 2:<br>Misconfiguration / Key Loss<br>Device Potentially Bricked or Locked"];

    %% Styling
    classDef startNode fill:#EDE9FE,stroke:#5B21B6,stroke-width:2px,color:#5B21B6;
    classDef processNode fill:#DBEAFE,stroke:#2563EB,stroke-width:1px,color:#1E40AF;
    classDef decisionNode fill:#FEF3C7,stroke:#D97706,stroke-width:1px,color:#92400E;
    classDef warningNode fill:#FEE2E2,stroke:#DC2626,stroke-width:1px,color:#991B1B;
    classDef successNode fill:#D1FAE5,stroke:#059669,stroke-width:2px,color:#065F46;

    class A startNode;
    class B,D,F decisionNode;
    class C,H processNode;
    class E,G,J warningNode;
    class I successNode;

Practical Examples

Example 1: Examining the Partition Table

You can easily view the partition table flashed onto your device using ESP-IDF tools.

1. Prerequisites: Have an ESP-IDF project built and flashed to a device. Connect the device.
2. Method 1: Using idf.py
   • Open the ESP-IDF Terminal in VS Code.
   • Navigate to your project directory.
   • Run the command:idf.py -p <YOUR_PORT> partition_table
(Replace <YOUR_PORT> with your device’s serial port).
   • Observe: The tool will connect to the device, read the partition table from Flash offset 0x8000, and print it in a readable format, showing partition names, types, subtypes, offsets, and sizes.
3. Method 2: Using parttool.py
   • parttool.py offers more specific partition operations.
   • Run the command to list partitions:parttool.py -p <YOUR_PORT> read_partitions --offset 0x8000 # Default offset
   • To get info about a specific partition (e.g., “nvs”):parttool.py -p <YOUR_PORT> get_partition_info --partition-name nvs
   • Observe: Similar output to idf.py partition_table, but parttool.py can also be used to read/write specific partitions (use with caution!).

Example 2: Entering Download Mode (Strapping Pins)

This example describes the physical procedure to force the ESP32 into Download Mode, allowing tools like esptool.py or idf.py flash to write to the Flash.

1. Identify Strapping Pins: Consult the datasheet for your specific ESP32 variant and your development board’s schematic to identify the primary boot mode pin (usually GPIO0) and the enable/reset pin (EN or RST).
2. Procedure (Typical for many ESP32 boards):
   • Connect the ESP32 board to your computer via USB.
   • Press and hold down the “BOOT” or “FLASH” button on your board (this usually pulls GPIO0 LOW).
   • While holding the BOOT button, press and release the “EN” or “RST” button (this resets the chip).
   • Release the “BOOT” button.
3. Verification: The ESP32 should now be in Download Mode. You can verify this by running a command like:esptool.py -p <YOUR_PORT> read_mac
oridf.py -p <YOUR_PORT> monitor
If successful, esptool.py will connect and read the MAC address, or the monitor might show a “waiting for download” message from the ROM bootloader. If it fails or starts running the application, the strapping pins were likely not sampled correctly during reset.
4. Exiting Download Mode: Simply press and release the “EN” or “RST” button again (without holding BOOT) to reset the chip into normal Run Mode.

Tip: Most development boards have buttons wired correctly to simplify this process. The ESP-IDF flashing tools (idf.py flash) often use DTR/RTS serial lines to automatically reset the board and manipulate the BOOT pin, putting it into download mode without manual button presses, but understanding the manual process is crucial for troubleshooting.

Variant Notes

While the overall boot process concept is similar, there are notable differences across ESP32 variants:

• Strapping Pins: The specific GPIOs used for strapping and their interpretation can differ significantly. Always check the datasheet for your specific SoC.
• Download Interface: Newer chips (S2, S3, C3, C6) often allow firmware download via the built-in USB Serial/JTAG controller, not just UART0.
• Secure Boot: Secure Boot v2 (available on newer chips) offers improved security and flexibility over v1.
• Flash Encryption: Algorithm (AES-256 vs AES-128) and implementation details vary.
• 2nd Loader Offset: On newer chips (S3, C3, C6, H2), the 2nd stage bootloader can be placed at offset 0x0 instead of 0x1000, simplifying the memory map slightly. This is configurable in menuconfig.
• ROM Code: The exact capabilities and behavior of the immutable ROM code differ between chip families.

Common Mistakes & Troubleshooting Tips

Exercises

1. Exploring with esptool.py:
   • Put your ESP32 board into Download Mode using the strapping pins/buttons.
   • Open a standard terminal or command prompt (not necessarily the ESP-IDF one, as esptool.py is often installed system-wide with ESP-IDF).
   • Run the following esptool.py commands (replace <YOUR_PORT>):
     • esptool.py -p <YOUR_PORT> read_mac (Reads the base MAC address)
     • esptool.py -p <YOUR_PORT> flash_id (Reads Flash manufacturer and device ID)
     • esptool.py -p <YOUR_PORT> chip_id (Reads the unique Chip ID)
     • esptool.py -p <YOUR_PORT> read_flash_status --bytes 2 (Reads Flash status register)
   • Observe the output for each command. What information can you gather about your specific chip and Flash memory directly from the ROM bootloader? Reset the board to exit Download Mode afterwards.
2. Custom Partition Table:
   • In your ESP-IDF project directory, find the partitions.csv file (or create one if using a default).
   • Make a backup copy of the original file.
   • Edit partitions.csv to add a new custom data partition. For example, add a line like:mydata, data, spiffs, , 1M,(Ensure the offset is left blank so the tool calculates it, and adjust the size 1M as needed, ensuring it doesn’t overlap with other partitions or exceed Flash capacity).
   • Save the file.
   • Run idf.py build. Observe the build output – it should mention processing the partition table.
   • Flash the entire project: idf.py -p <YOUR_PORT> flash (This will write the new partition_table.bin).
   • Verify the new table: idf.py -p <YOUR_PORT> partition_table. Check if your mydata partition is listed with the correct details. (You haven’t added code to use this partition yet, just defined it).

Summary

• The ESP32 boot process starts with the 1st Stage ROM Bootloader executing after reset.
• Strapping pins determine the boot mode (Run or Download). Download mode allows flashing via UART/USB using tools like esptool.py.
• In Run mode, the ROM loader loads the 2nd Stage SPI Bootloader (bootloader.bin) from Flash offset 0x1000 (or 0x0 on newer chips).
• The 2nd Stage Bootloader reads the Partition Table (from Flash offset 0x8000 by default) to understand the Flash layout.
• The Partition Table defines regions for applications (app), data (nvs, phy, filesystems), etc.
• The 2nd Stage Bootloader verifies and loads the selected Application image into RAM (IRAM/DRAM) and jumps to its entry point.
• Application startup code initializes peripherals, heap, FreeRTOS, and finally calls app_main().
• Secure Boot verifies firmware signatures, while Flash Encryption protects Flash contents from unauthorized access. Both rely on eFuses and require careful handling.
• Boot processes and specific features (strapping pins, security versions) vary across ESP32 variants. Always consult the datasheet and ESP-IDF documentation for your target.
• Common boot issues involve incorrect strapping, flashing errors, corrupted images/partitions, or early application crashes.

Further Reading

• ESP-IDF Programming Guide: Bootloader: https://docs.espressif.com/projects/esp-idf/en/v5.4/esp32/api-guides/bootloader.html (Select your target chip in the documentation)
• ESP-IDF Programming Guide: Partition Tables: https://docs.espressif.com/projects/esp-idf/en/v5.4/esp32/api-guides/partition-tables.html
• Esptool.py Documentation: https://github.com/espressif/esptool/
• ESP-IDF Programming Guide: Secure Boot V2: https://docs.espressif.com/projects/esp-idf/en/v5.4/esp32/security/secure-boot-v2.html
• ESP-IDF Programming Guide: Flash Encryption: https://docs.espressif.com/projects/esp-idf/en/v5.4/esp32/security/flash-encryption.html
• ESP32 Series Datasheets: (Available on the Espressif website) – Contain definitive information on strapping pins and hardware specifics.