Chapter 102: MQTT Client Implementation with ESP-MQTT

Chapter Objectives

After completing this chapter, you will be able to:

• Configure and initialize the esp-mqtt client component in an ESP-IDF project.
• Establish a connection between an ESP32 device and an MQTT broker.
• Implement event handling for various MQTT client states and received messages.
• Publish messages to specific MQTT topics from your ESP32.
• Subscribe to MQTT topics and process incoming messages on your ESP32.
• Understand and apply Quality of Service (QoS) levels during MQTT communication.
• Implement basic MQTT features like Last Will and Testament (LWT) and Retained Messages.
• Troubleshoot common issues encountered during MQTT client development on ESP32.

Introduction

In Chapter 101, we explored the fundamental concepts and architecture of the MQTT protocol, understanding its lightweight, publish-subscribe nature, and its suitability for IoT applications. Now, it’s time to translate that theory into practice. This chapter will guide you through the process of implementing an MQTT client on your ESP32 device using the official ESP-IDF framework.

The ESP-IDF provides a robust and flexible MQTT client component, esp-mqtt, which simplifies the complexities of the protocol, allowing you to focus on your application logic. By the end of this chapter, you will be capable of connecting your ESP32 to an MQTT broker, publishing sensor data, and subscribing to control commands, opening up a vast array of possibilities for your IoT projects. We will use VS Code with the Espressif IDF extension as our development environment, ensuring a smooth and efficient coding experience.

Theory

The esp-mqtt Component

The esp-mqtt component is the official MQTT client library provided by Espressif within the ESP-IDF framework. It’s built on top of the MbedTLS library for secure connections (TLS/SSL) and leverages ESP-IDF’s networking capabilities. It provides a high-level API to manage MQTT connections, subscriptions, and publications, abstracting away the underlying TCP/IP and MQTT protocol details.

Core Implementation Concepts

Implementing an MQTT client with esp-mqtt primarily involves these steps:

1. Wi-Fi (or Network) Connectivity: Before your ESP32 can connect to an MQTT broker, it needs network access. This typically means connecting to a Wi-Fi network.
2. MQTT Client Configuration: Defining the parameters for your MQTT connection, such as the broker’s URI, client ID, username, password, QoS levels, and other features like LWT and clean session.
3. Event Handling: Setting up a callback function to process various events generated by the MQTT client, such as successful connection, disconnection, incoming messages, and errors.
4. Client Initialization and Start: Creating an MQTT client instance and initiating the connection process.
5. Publishing and Subscribing: Using dedicated API calls to send messages to topics and register interest in receiving messages from topics.

Let’s delve into these concepts in more detail.

1. MQTT Client Configuration (esp_mqtt_client_config_t)

The esp_mqtt_client_config_t structure is used to configure your MQTT client. You populate this structure with various parameters before initializing the client.

typedef struct esp_mqtt_client_config {
    const char *uri;                /*!< Broker URI. For example: "mqtt://mqtt.eclipseprojects.io" */
    const char *host;               /*!< Broker host address */
    int port;                       /*!< Broker port */
    const char *client_id;          /*!< MQTT client ID. If NULL, a random ID will be generated. */
    const char *username;           /*!< MQTT username */
    const char *password;           /*!< MQTT password */
    int keepalive;                  /*!< MQTT keepalive interval in seconds. Default is 120 seconds. */
    bool disable_clean_session;     /*!< Set to true to enable persistent session. Default is false (clean session). */
    int qos;                        /*!< Default QoS level for subscriptions. */
    bool disable_auto_reconnect;    /*!< Set to true to disable auto-reconnection. Default is false (auto-reconnect enabled). */
    // ... (other fields for LWT, TLS, etc.)
} esp_mqtt_client_config_t;

Key Configuration Parameters:

2. Event Handling (esp_mqtt_event_handle_cb_t)

The esp-mqtt client operates asynchronously. Instead of blocking calls, it uses an event-driven model. You register a callback function that the MQTT client will invoke whenever a significant event occurs.

typedef esp_err_t (*esp_mqtt_event_handle_cb_t)(esp_mqtt_event_handle_t event);

The event parameter passed to your callback function is a pointer to an esp_mqtt_event_t structure, which contains information about the specific event.

typedef struct {
    esp_mqtt_event_id_t event_id;   /*!< Event ID */
    esp_mqtt_client_handle_t client; /*!< MQTT client handle */
    void *user_context;             /*!< User context, can be set in mqtt client configuration */
    const char *data;               /*!< Data associated with the event (e.g., message payload) */
    int data_len;                   /*!< Length of data */
    const char *topic;              /*!< Topic associated with the event */
    int topic_len;                  /*!< Length of topic */
    int msg_id;                     /*!< Message ID (for PUBLISH, SUBSCRIBE, UNSUBSCRIBE) */
    int qos;                        /*!< QoS level */
    bool retain;                    /*!< Retain flag */
    bool dup;                       /*!< Duplicate flag */
    // ... (other fields)
} esp_mqtt_event_t;

Common esp_mqtt_event_id_t values you’ll handle:

graph TD
    subgraph ESP32 Application
        A["ESP32 MQTT Client <br> (esp-mqtt library operations)"]
        B{"MQTT Event Occurs <br> (e.g., Network, Broker Message)"}
        C["esp-mqtt library detects event <br> & prepares esp_mqtt_event_t data"]
        D["Invokes Registered Callback: <br> <b>your_mqtt_event_handler(event)</b>"]
    end

    subgraph Your Custom Code

        E{"Inside Callback: <br> switch (event->event_id)"}
        F["Handle MQTT_EVENT_CONNECTED <br> (e.g., subscribe to topics)"]
        G["Handle MQTT_EVENT_DISCONNECTED <br> (e.g., log, attempt reconnect if auto_reconnect disabled)"]
        H["Handle MQTT_EVENT_DATA <br> (e.g., process incoming message: <br> event->topic, event->data)"]
        I["Handle MQTT_EVENT_PUBLISHED <br> (e.g., confirm message sent for QoS > 0)"]
        J["Handle MQTT_EVENT_SUBSCRIBED / UNSUBSCRIBED <br> (e.g., log confirmation)"]
        K["Handle MQTT_EVENT_ERROR <br> (e.g., log error details: event->error_handle)"]
        L["Handle Other Events..."]
        M["Event Processed <br> Callback Returns"]
    end

    A --> B;
    B --> C;
    C --> D;
    D --> E;
    E -- MQTT_EVENT_CONNECTED --> F;
    E -- MQTT_EVENT_DISCONNECTED --> G;
    E -- MQTT_EVENT_DATA --> H;
    E -- MQTT_EVENT_PUBLISHED --> I;
    E -- MQTT_EVENT_SUBSCRIBED --> J;
    E -- MQTT_EVENT_ERROR --> K;
    E -- Other --> L;
    F --> M;
    G --> M;
    H --> M;
    I --> M;
    J --> M;
    K --> M;
    L --> M;

    classDef primary fill:#EDE9FE,stroke:#5B21B6,stroke-width:2px,color:#5B21B6;
    classDef process fill:#DBEAFE,stroke:#2563EB,stroke-width:1px,color:#1E40AF;
    classDef decision fill:#FEF3C7,stroke:#D97706,stroke-width:1px,color:#92400E;
    classDef endo fill:#D1FAE5,stroke:#059669,stroke-width:2px,color:#065F46;
    classDef check fill:#FEE2E2,stroke:#DC2626,stroke-width:1px,color:#991B1B;

    class A primary;
    class B process;
    class C process;
    class D process;
    class E decision;
    class F process;
    class G process;
    class H process;
    class I process;
    class J process;
    class K check;
    class L process;
    class M endo;

3. Client Initialization and Start

After configuring the client and defining the event handler, you initialize the client and start the connection process.

• esp_mqtt_client_handle_t esp_mqtt_client_init(const esp_mqtt_client_config_t *config): Initializes the MQTT client with the given configuration. Returns a handle to the client instance.
• esp_err_t esp_mqtt_client_start(esp_mqtt_client_handle_t client): Starts the MQTT client, which attempts to connect to the broker. This function is non-blocking. The connection status will be reported via events.

4. Publishing and Subscribing

Once connected, you can publish messages and subscribe to topics.

• int esp_mqtt_client_publish(esp_mqtt_client_handle_t client, const char *topic, const char *data, int len, int qos, int retain): Publishes a message.
  • topic: The topic string.
  • data: The message payload.
  • len: Length of the payload. If 0, it’s an empty message (useful for clearing retained messages).
  • qos: Quality of Service (0, 1, or 2).
  • retain: 0 for false, 1 for true.
  • Returns the msg_id if successful, or -1 on error.
• int esp_mqtt_client_subscribe(esp_mqtt_client_handle_t client, const char *topic, int qos): Subscribes to a topic.
  • topic: The topic filter string (can include wildcards).
  • qos: Desired QoS for this subscription.
  • Returns the msg_id if successful, or -1 on error.
• int esp_mqtt_client_unsubscribe(esp_mqtt_client_handle_t client, const char *topic): Unsubscribes from a topic.

sequenceDiagram
    participant ESP32_Client as ESP32 Client
    participant MQTT_Broker as MQTT Broker
    participant Other_Client as Other MQTT Client (Optional)

    ESP32_Client->>+MQTT_Broker: CONNECT (ClientId, KeepAlive, CleanSession, LWT Opt.)
    MQTT_Broker-->>-ESP32_Client: CONNACK (Return Code, Session Present Opt.)

    alt Connection Successful
        ESP32_Client->>+MQTT_Broker: SUBSCRIBE (Topic: "control/led", QoS: 1)
        MQTT_Broker-->>-ESP32_Client: SUBACK (MsgID, Granted QoS: [1])

        ESP32_Client->>+MQTT_Broker: PUBLISH (Topic: "sensor/temp", Payload: "25C", QoS: 1, Retain: false)
        MQTT_Broker-->>-ESP32_Client: PUBACK (MsgID for QoS 1 publish)

        Note over ESP32_Client, MQTT_Broker: ESP32 can also publish with QoS 0 (no ack) or QoS 2 (more steps)

        Other_Client->>+MQTT_Broker: PUBLISH (Topic: "control/led", Payload: "ON", QoS: 0)
        MQTT_Broker-->>-ESP32_Client: PUBLISH (Topic: "control/led", Payload: "ON") (Forwarded to subscriber)

        ESP32_Client->>+MQTT_Broker: UNSUBSCRIBE (Topic: "control/led")
        MQTT_Broker-->>-ESP32_Client: UNSUBACK (MsgID)

        ESP32_Client->>+MQTT_Broker: PINGREQ (if no other traffic within keepalive)
        MQTT_Broker-->>-ESP32_Client: PINGRESP

        ESP32_Client->>+MQTT_Broker: DISCONNECT
    else Connection Failed (e.g., bad credentials, broker down)
        Note over ESP32_Client: Handle CONNACK error, retry connection based on policy.
    end

Secure Connections (MQTTS/TLS)

While this chapter focuses on basic MQTT, it’s crucial to understand that esp-mqtt fully supports MQTTS (MQTT over TLS/SSL) for secure communication. This involves providing CA certificates, client certificates, and private keys in your esp_mqtt_client_config_t. We will delve into MQTTS security in detail in a later chapter (e.g., Chapter 105). For now, we’ll use unencrypted connections for simplicity in our examples.

Practical Examples

This example will demonstrate how to set up an ESP32 as an MQTT client. It will connect to a Wi-Fi network, then connect to a public MQTT broker, publish a “Hello from ESP32!” message to a specific topic, and subscribe to another topic to receive messages.

Project Structure

mqtt_client_example/
├── CMakeLists.txt
├── main/
│   ├── CMakeLists.txt
│   └── main.c
└── sdkconfig.defaults

Code Snippets

1. sdkconfig.defaults

This file sets default configuration options for your project, including Wi-Fi credentials and MQTT logging levels. Replace YOUR_WIFI_SSID and YOUR_WIFI_PASSWORD with your actual Wi-Fi network details.

# Wi-Fi Configuration
CONFIG_ESP_WIFI_SSID="YOUR_WIFI_SSID"
CONFIG_ESP_WIFI_PASSWORD="YOUR_WIFI_PASSWORD"
CONFIG_ESP_WIFI_STA_CONNECT_TIMEOUT=10000

# Component logging levels
CONFIG_LOG_DEFAULT_LEVEL_INFO=y
CONFIG_LOG_COLORS=y
CONFIG_LOG_TIMESTAMP=y
CONFIG_MQTT_CLIENT_LOG_LEVEL_INFO=y
CONFIG_LWIP_DNS_SUPPORT_MDNS=y

2. CMakeLists.txt (Project Level)

This is the top-level CMake file for your project.

# The following lines of code are for the project's CMakeLists.txt
# (usually at the root of your project folder)

cmake_minimum_required(VERSION 3.16)

# Include the ESP-IDF build system
include($ENV{IDF_PATH}/tools/cmake/project.cmake)

# Define your project name
project(mqtt_client_example)

3. main/CMakeLists.txt

This CMake file defines the source files for your main component and links necessary ESP-IDF components.

# The following lines of code are for the main/CMakeLists.txt
# (inside your main component folder)

# Define the source files for this component
set(COMPONENT_SRCS "main.c")

# Link required ESP-IDF components
# `esp_event` for event loop, `nvs_flash` for Wi-Fi credentials storage,
# `esp_wifi` for Wi-Fi, `mqtt` for MQTT client, `lwip` for TCP/IP stack.
set(COMPONENT_REQUIRES esp_event nvs_flash esp_wifi mqtt lwip)

# Add the component to the build
register_component()

4. main/main.c

This is the core application logic. It includes Wi-Fi setup, the MQTT event handler, and the main application task.

#include <stdio.h>
#include <string.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/event_groups.h"
#include "esp_system.h"
#include "esp_wifi.h"
#include "esp_event.h"
#include "esp_log.h"
#include "nvs_flash.h"
#include "mqtt_client.h" // Include the MQTT client library

// Define Wi-Fi credentials from sdkconfig.defaults
#define WIFI_SSID CONFIG_ESP_WIFI_SSID
#define WIFI_PASSWORD CONFIG_ESP_WIFI_PASSWORD

// Define MQTT Broker URI (using a public test broker for demonstration)
// WARNING: Data sent to public brokers is unencrypted and visible to others.
//          Do not use for sensitive information.
#define MQTT_BROKER_URI "mqtt://broker.hivemq.com:1883" // Or "mqtt://test.mosquitto.org:1883"

// Define MQTT topics
#define MQTT_PUBLISH_TOPIC "esp32/sensor/temperature"
#define MQTT_SUBSCRIBE_TOPIC "esp32/control/light"

// Event group for Wi-Fi connection status
static EventGroupHandle_t s_wifi_event_group;
#define WIFI_CONNECTED_BIT BIT0

static const char *TAG = "MQTT_EXAMPLE"; // Tag for ESP_LOG messages

esp_mqtt_client_handle_t client_handle = NULL; // Global MQTT client handle

// --- Wi-Fi Event Handler ---
static void wifi_event_handler(void* arg, esp_event_base_t event_base,
                               int32_t event_id, void* event_data) {
    if (event_base == WIFI_EVENT && event_id == WIFI_EVENT_STA_START) {
        esp_wifi_connect();
    } else if (event_base == WIFI_EVENT && event_id == WIFI_EVENT_STA_DISCONNECTED) {
        ESP_LOGI(TAG, "Wi-Fi disconnected, trying to reconnect...");
        esp_wifi_connect();
        xEventGroupClearBits(s_wifi_event_group, WIFI_CONNECTED_BIT);
    } else if (event_base == IP_EVENT && event_id == IP_EVENT_STA_GOT_IP) {
        ip_event_got_ip_t* event = (ip_event_got_ip_t*) event_data;
        ESP_LOGI(TAG, "Got IP: " IPSTR, IP2STR(&event->ip_info.ip));
        xEventGroupSetBits(s_wifi_event_group, WIFI_CONNECTED_BIT);
    }
}

// --- Wi-Fi Initialization ---
static void wifi_init(void) {
    s_wifi_event_group = xEventGroupCreate();

    ESP_ERROR_CHECK(nvs_flash_init()); // Initialize NVS for Wi-Fi storage
    ESP_ERROR_CHECK(esp_netif_init()); // Initialize TCP/IP stack
    ESP_ERROR_CHECK(esp_event_loop_create_default()); // Create default event loop
    esp_netif_create_default_wifi_sta(); // Create Wi-Fi station interface

    wifi_init_config_t cfg = WIFI_INIT_CONFIG_DEFAULT();
    ESP_ERROR_CHECK(esp_wifi_init(&cfg)); // Initialize Wi-Fi driver

    // Register Wi-Fi event handlers
    ESP_ERROR_CHECK(esp_event_handler_instance_register(WIFI_EVENT,
                                                        ESP_EVENT_ANY_ID,
                                                        &wifi_event_handler,
                                                        NULL,
                                                        NULL));
    ESP_ERROR_CHECK(esp_event_handler_instance_register(IP_EVENT,
                                                        IP_EVENT_STA_GOT_IP,
                                                        &wifi_event_handler,
                                                        NULL,
                                                        NULL));

    wifi_config_t wifi_config = {
        .sta = {
            .ssid = WIFI_SSID,
            .password = WIFI_PASSWORD,
            .threshold.authmode = WIFI_AUTH_WPA2_PSK,
            .sae_pwe_h2e = WPA3_SAE_PWE_BOTH,
        },
    };
    ESP_ERROR_CHECK(esp_wifi_set_mode(WIFI_MODE_STA)); // Set Wi-Fi to station mode
    ESP_ERROR_CHECK(esp_wifi_set_config(WIFI_IF_STA, &wifi_config)); // Apply Wi-Fi configuration
    ESP_ERROR_CHECK(esp_wifi_start()); // Start Wi-Fi
    ESP_LOGI(TAG, "Wi-Fi initialization finished.");

    // Wait for Wi-Fi connection
    ESP_LOGI(TAG, "Waiting for Wi-Fi connection to %s...", WIFI_SSID);
    xEventGroupWaitBits(s_wifi_event_group, WIFI_CONNECTED_BIT, pdFALSE, pdFALSE, portMAX_DELAY);
    ESP_LOGI(TAG, "Wi-Fi connected successfully!");
}

// --- MQTT Event Handler ---
static esp_err_t mqtt_event_handler_cb(esp_mqtt_event_handle_t event) {
    esp_mqtt_client_handle_t client = event->client;
    int msg_id;

    // Use a switch statement to handle different MQTT event types
    switch (event->event_id) {
        case MQTT_EVENT_CONNECTED:
            ESP_LOGI(TAG, "MQTT_EVENT_CONNECTED");
            // On connection, subscribe to the control topic
            msg_id = esp_mqtt_client_subscribe(client, MQTT_SUBSCRIBE_TOPIC, 1); // QoS 1
            ESP_LOGI(TAG, "Sent subscribe successful, msg_id=%d", msg_id);

            // You can also publish a retained message here to indicate device status
            // For example, to announce the device is online:
            msg_id = esp_mqtt_client_publish(client, "esp32/status/online", "online", 0, 1, 1); // QoS 1, Retain = true
            ESP_LOGI(TAG, "Sent retained online status, msg_id=%d", msg_id);
            break;

        case MQTT_EVENT_DISCONNECTED:
            ESP_LOGI(TAG, "MQTT_EVENT_DISCONNECTED");
            break;

        case MQTT_EVENT_SUBSCRIBED:
            ESP_LOGI(TAG, "MQTT_EVENT_SUBSCRIBED, msg_id=%d", event->msg_id);
            break;

        case MQTT_EVENT_UNSUBSCRIBED:
            ESP_LOGI(TAG, "MQTT_EVENT_UNSUBSCRIBED, msg_id=%d", event->msg_id);
            break;

        case MQTT_EVENT_PUBLISHED:
            ESP_LOGI(TAG, "MQTT_EVENT_PUBLISHED, msg_id=%d", event->msg_id);
            break;

        case MQTT_EVENT_DATA:
            ESP_LOGI(TAG, "MQTT_EVENT_DATA");
            // Print the received topic and data
            printf("TOPIC=%.*s\r\n", event->topic_len, event->topic);
            printf("DATA=%.*s\r\n", event->data_len, event->data);

            // Example: If the light control topic receives "ON", turn on an imaginary light.
            if (strncmp(event->topic, MQTT_SUBSCRIBE_TOPIC, event->topic_len) == 0) {
                if (strncmp(event->data, "ON", event->data_len) == 0) {
                    ESP_LOGI(TAG, "Received 'ON' command for light!");
                    // Add your light control logic here (e.g., GPIO control)
                } else if (strncmp(event->data, "OFF", event->data_len) == 0) {
                    ESP_LOGI(TAG, "Received 'OFF' command for light!");
                    // Add your light control logic here
                }
            }
            break;

        case MQTT_EVENT_ERROR:
            ESP_LOGI(TAG, "MQTT_EVENT_ERROR");
            if (event->error_handle->error_type == MQTT_ERROR_TYPE_TCP_TRANSPORT) {
                ESP_LOGI(TAG, "Last error code reported from mqtt event: 0x%x", event->error_handle->esp_transport_return_code);
            }
            break;

        default:
            ESP_LOGI(TAG, "Other event id:%d", event->event_id);
            break;
    }
    return ESP_OK;
}

static void mqtt_app_start(void) {
    // Configure the MQTT client
    esp_mqtt_client_config_t mqtt_cfg = {
        .uri = MQTT_BROKER_URI, // Set the broker URI
        .client_id = "esp32_client_001", // A unique client ID
        .lwt_topic = "esp32/status/lastwill", // LWT topic
        .lwt_msg = "offline", // LWT message
        .lwt_qos = 1, // LWT QoS
        .lwt_retain = 1, // LWT retain flag (true)
        .keepalive = 60, // Keep-alive interval in seconds
        // No username/password for public broker. For private brokers, add:
        // .username = "your_username",
        // .password = "your_password",
        // .disable_clean_session = true, // Set to true for persistent session
    };

    // Initialize the MQTT client with the configuration and event handler
    client_handle = esp_mqtt_client_init(&mqtt_cfg);
    if (client_handle == NULL) {
        ESP_LOGE(TAG, "Failed to initialize MQTT client");
        return;
    }

    // Register the event handler for MQTT events
    esp_mqtt_client_register_event(client_handle, ESP_EVENT_ANY_ID, mqtt_event_handler_cb, NULL);

    // Start the MQTT client. This initiates connection to the broker.
    ESP_ERROR_CHECK(esp_mqtt_client_start(client_handle));
    ESP_LOGI(TAG, "MQTT client started.");
}

// Main application task
void app_main(void) {
    // 1. Initialize Wi-Fi
    wifi_init();

    // 2. Start MQTT application after Wi-Fi is connected
    mqtt_app_start();

    // Loop for publishing messages periodically
    int msg_count = 0;
    char payload[50];

    while (true) {
        // Publish a message every 10 seconds
        sprintf(payload, "Hello from ESP32! Message #%d", msg_count++);
        if (client_handle != NULL) { // Check if client handle is valid
            // Publish to the sensor topic with QoS 1, not retained
            int msg_id = esp_mqtt_client_publish(client_handle, MQTT_PUBLISH_TOPIC, payload, 0, 1, 0);
            ESP_LOGI(TAG, "Published message: '%s' to topic '%s', msg_id=%d", payload, MQTT_PUBLISH_TOPIC, msg_id);
        }
        vTaskDelay(pdMS_TO_TICKS(10000)); // Delay for 10 seconds
    }
}

Build Instructions

1. Set up ESP-IDF and VS Code: Ensure you have ESP-IDF v5.x installed and configured with the Espressif IDF extension in VS Code. If not, refer to the official Espressif documentation for setup.
2. Create Project: Open VS Code, go to File > Open Folder..., and select the mqtt_client_example directory you created.
3. Configure Wi-Fi: Open sdkconfig.defaults and replace YOUR_WIFI_SSID and YOUR_WIFI_PASSWORD with your actual Wi-Fi credentials. Alternatively, you can run idf.py menuconfig (or use the VS Code ESP-IDF extension’s “SDK Configuration Editor” button) and navigate to Example Connection Configuration to set them.
4. Build:
   • In VS Code, open the Command Palette (Ctrl+Shift+P or Cmd+Shift+P).
   • Type ESP-IDF: Build your project.
   • Select your target chip (e.g., esp32, esp32s3, esp32c3).
   • The build process will start in the terminal.

Run/Flash/Observe Steps

1. Connect ESP32: Connect your ESP32 development board to your computer via USB.
2. Select Port: In VS Code, check the status bar at the bottom. Ensure the correct COM port/serial port for your ESP32 is selected. If not, click on it and choose the right one.
3. Flash:
   • Open the Command Palette (Ctrl+Shift+P or Cmd+Shift+P).
   • Type ESP-IDF: Flash your project.
   • This will compile (if not already done) and then flash the firmware to your ESP32.
4. Monitor:
   • Open the Command Palette.
   • Type ESP-IDF: Monitor your device.
   • This will open a serial monitor, displaying the logs from your ESP32.

Expected Output (in serial monitor):

You should observe the following in the serial monitor:

• Wi-Fi connection attempts and successful IP address acquisition.
• MQTT client connection messages (MQTT_EVENT_CONNECTED).
• Subscription messages (MQTT_EVENT_SUBSCRIBED).
• Messages being published periodically (Published message: ...).
• If you publish a message to esp32/control/light (e.g., “ON” or “OFF”) using a separate MQTT client tool (like MQTT Explorer or mosquitto_pub from Chapter 101), you will see:I (XXXX) MQTT_EXAMPLE: MQTT_EVENT_DATA TOPIC=esp32/control/light DATA=ON I (XXXX) MQTT_EXAMPLE: Received 'ON' command for light!
• If you disconnect your ESP32 without graceful shutdown (e.g., power off), and then you were subscribed to esp32/status/lastwill, you would see “offline” published to esp32/status/lastwill by the broker.

sequenceDiagram
    autonumber
    participant ESP32
    participant MQTT_Broker as "MQTT Broker <br> (e.g., broker.hivemq.com)"
    participant Client_Tool as "MQTT Client Tool <br> (e.g., MQTT Explorer)"

    ESP32->>+MQTT_Broker: CONNECT (URI: broker.hivemq.com, <br>ClientID: "esp32_client_001", <br>LWT: "esp32/status/lastwill" = "offline")
    MQTT_Broker-->>-ESP32: CONNACK (Success)
    Note over ESP32: MQTT_EVENT_CONNECTED

    ESP32->>+MQTT_Broker: SUBSCRIBE (Topic: "esp32/control/light", QoS: 1)
    MQTT_Broker-->>-ESP32: SUBACK (MsgID, Granted QoS: [1])
    Note over ESP32: MQTT_EVENT_SUBSCRIBED

    ESP32->>+MQTT_Broker: PUBLISH (Topic: "esp32/status/online", <br>Payload: "online", QoS: 1, Retain: true)
    MQTT_Broker-->>-ESP32: PUBACK (MsgID)
    Note over ESP32: MQTT_EVENT_PUBLISHED

    loop Periodic Temperature Publishing (every 10s)
        ESP32->>+MQTT_Broker: PUBLISH (Topic: "esp32/sensor/temperature", <br>Payload: "Hello #X", QoS: 1, Retain: false)
        MQTT_Broker-->>-ESP32: PUBACK (MsgID)
    end

    Client_Tool->>+MQTT_Broker: CONNECT (URI: broker.hivemq.com)
    MQTT_Broker-->>-Client_Tool: CONNACK (Success)

    Client_Tool->>+MQTT_Broker: SUBSCRIBE (Topic: "esp32/sensor/temperature")
    MQTT_Broker-->>-Client_Tool: SUBACK
    Note over Client_Tool, MQTT_Broker: Client tool now receives <br>ESP32's temperature messages.

    Client_Tool->>+MQTT_Broker: SUBSCRIBE (Topic: "esp32/status/#")
    MQTT_Broker-->>-Client_Tool: SUBACK
    MQTT_Broker-->>Client_Tool: PUBLISH (Topic: "esp32/status/online",<br> Payload: "online", Retain: true) <br>(Retained message delivered)


    Client_Tool->>+MQTT_Broker: PUBLISH (Topic: "esp32/control/light", <br>Payload: "ON", QoS: 0)
    MQTT_Broker-->>ESP32: PUBLISH (Topic: "esp32/control/light", Payload: "ON") <br>(Forwarded message)
    Note over ESP32: MQTT_EVENT_DATA received. <br> Processes "ON" command.

    Client_Tool->>+MQTT_Broker: PUBLISH (Topic: "esp32/control/light", <br>Payload: "OFF", QoS: 0)
    MQTT_Broker-->>ESP32: PUBLISH (Topic: "esp32/control/light", <br>Payload: "OFF") (Forwarded message)
    Note over ESP32: MQTT_EVENT_DATA received. <br> Processes "OFF" command.

    alt ESP32 Disconnects Ungracefully <br>(e.g., power off)
        Note over ESP32, MQTT_Broker: Connection lost. <br>LWT is triggered by Broker.
        MQTT_Broker-->>Client_Tool: PUBLISH (Topic: "esp32/status/lastwill", <br>Payload: "offline", Retain: true)
    end

To interact with your ESP32:

1. Install an MQTT Client Tool:
   • GUI: MQTTX (recommended for ease of use).
   • CLI: mosquitto_pub and mosquitto_sub (if you have Mosquitto installed).
2. Connect to the Broker: Use broker.hivemq.com (or test.mosquitto.org) on port 1883.
3. Subscribe: In your MQTT client tool, subscribe to esp32/sensor/temperature to see the messages published by your ESP32.
4. Publish: Publish messages to esp32/control/light (e.g., ON or OFF) to send commands to your ESP32.

Variant Notes

The esp-mqtt component is designed to be highly portable across all ESP32 series microcontrollers supported by ESP-IDF v5.x, including ESP32, ESP32-S2, ESP32-S3, ESP32-C3, ESP32-C6, and ESP32-H2. The core MQTT API and functionality remain consistent regardless of the specific chip. However, there are some subtle differences in performance and underlying hardware capabilities:

1. Wi-Fi Module Differences:
   • All listed variants support Wi-Fi. The esp-mqtt client relies on the standard ESP-IDF Wi-Fi driver. You generally won’t see behavioral differences at the MQTT application level due to the Wi-Fi module itself, beyond raw throughput capabilities of the chip.
   • ESP32-C6/H2 (802.15.4): These chips also support IEEE 802.15.4 (for Thread/Zigbee). If your MQTT broker is on an IP network, you would typically need a border router or gateway to bridge the 802.15.4 network to the IP network. The MQTT client on the ESP32-C6/H2 itself would still use the standard IP stack and esp-mqtt component once it has IP connectivity, whether directly via Wi-Fi or indirectly via a border router.
2. Resource Constraints (RAM/Flash):
   • While MQTT is lightweight, the memory footprint of the esp-mqtt client itself, plus the TLS stack (if MQTTS is used), and your application code, can vary.
   • More resource-constrained variants (e.g., ESP32-C3 with smaller PSRAM/Flash configurations) might have limitations on the complexity of simultaneous subscriptions, the maximum payload size that can be buffered, or the number of concurrent connections if you were to implement multiple clients. However, for a single, typical MQTT client, all variants are generally capable.
3. TLS Hardware Acceleration:
   • For secure MQTT connections (MQTTS), cryptographic operations are performed to encrypt and decrypt data. These operations can be computationally intensive.
   • Newer ESP32 variants (ESP32-S2, ESP32-S3, ESP32-C2, ESP32-C3, ESP32-C6, ESP32-H2) feature dedicated hardware accelerators for common cryptographic algorithms (AES, SHA, RSA, ECC). This significantly offloads the CPU, speeds up TLS handshakes, reduces power consumption, and improves overall performance when using MQTTS.
   • The original ESP32 also has some hardware acceleration for certain crypto functions, but the newer chips generally offer more comprehensive and efficient support.
   • When using esp-mqtt with TLS, the library automatically leverages these hardware accelerators if available, so you typically don’t need to change your application code. The performance benefits are automatic.
4. CPU Performance:
   • Different ESP32 variants have varying CPU clock speeds and core counts (e.g., original ESP32, S2, S3 are dual-core; C3, C6, H2 are single-core). For typical MQTT message rates, this difference is usually not a bottleneck. However, if your application involves heavy data processing alongside MQTT communication, or extremely high message throughput, a faster or multi-core variant might offer better performance.

In summary, the code provided in the practical example is directly applicable to all modern ESP32 variants. The differences are largely under the hood, affecting performance and resource utilization, especially for secure and high-throughput applications.

Common Mistakes & Troubleshooting Tips

Here are some common pitfalls developers encounter when implementing MQTT clients on ESP32 and how to address them:

Exercises

These exercises will help solidify your understanding and practical skills in implementing MQTT clients on ESP32.

1. Dynamic Client ID and LWT Enhancement:
   • Modify the main.c example to generate a dynamic client_id using the ESP32’s MAC address, as suggested in the troubleshooting tips.
   • Enhance the LWT message: instead of just “offline”, make it a JSON string like {"device_id": "YOUR_DYNAMIC_CLIENT_ID", "status": "offline", "timestamp": "CURRENT_TIME"}. (Hint: You’ll need to generate a timestamp string or use a placeholder for simplicity).
   • Test this by flashing the ESP32, subscribing to its LWT topic (e.g., esp32/status/lastwill) with an MQTT client tool, and then unexpectedly power cycling the ESP32. Observe the LWT message.
2. Toggle an LED via MQTT:
   • Add a simple LED control to your ESP32 application. Choose an unused GPIO pin (e.g., GPIO2 for built-in LED on some boards, or connect an external LED with a resistor).
   • In the MQTT_EVENT_DATA handler, parse the incoming message on esp32/control/light. If the payload is “ON”, turn the LED on. If it’s “OFF”, turn the LED off.
   • Implement basic GPIO initialization for the LED.
   • Publish “ON” and “OFF” messages from your MQTT client tool to test.
3. Publish Sensor Data (Simulated):
   • Modify the periodic publishing loop. Instead of just sending “Hello from ESP32!”, simulate temperature and humidity readings.
   • Generate random or incrementally changing float values for temperature (e.g., 20.0-30.0 °C) and humidity (e.g., 40.0-70.0%).
   • Publish these as a JSON payload to a new topic, e.g., esp32/home/livingroom/data. The payload could look like: {"temperature": 25.3, "humidity": 55.7}.
   • Set the QoS to 1 and the Retain flag to true for these sensor readings.
   • Observe the published messages using your MQTT client tool. Disconnect and reconnect your client tool; do you immediately receive the last retained sensor data?
4. QoS Level Exploration:
   • In your main.c, modify the esp_mqtt_client_publish call for the “Hello from ESP32!” message to use QoS 0.
   • Flash and observe. What happens if you momentarily disconnect and reconnect your MQTT client tool? Does it miss any “Hello” messages compared to using QoS 1 or 2?
   • (Optional but recommended): Research and consider how you would implement client-side message deduplication if you were to use QoS 1 and wanted to ensure “exactly-once” processing at the application level.
5. Wildcard Subscription Practice:
   • Introduce a new simulated sensor: esp32/home/kitchen/sensor/light_level.
   • Modify the ESP32 to publish random light level values (e.g., 0-100) to this topic every 5 seconds.
   • In your MQTT client tool, try subscribing to the following topics and observe which messages you receive:
     • esp32/home/#
     • esp32/home/+/sensor/#
     • esp32/home/+/+/light_level
     • esp32/# (Use with caution on busy brokers as it will show all messages)

Summary

• The esp-mqtt component in ESP-IDF provides a robust, official client library for MQTT communication on ESP32 devices.
• MQTT client implementation involves: Wi-Fi connectivity, client configuration (esp_mqtt_client_config_t), asynchronous event handling, and using API calls for publishing and subscribing.
• The esp_mqtt_event_handler_cb callback function is crucial for reacting to various MQTT events such as connection status, received messages (MQTT_EVENT_DATA), and acknowledgments.
• esp_mqtt_client_publish() sends messages to a topic with specified QoS and retain flag. esp_mqtt_client_subscribe() registers interest in receiving messages from a topic.
• Key MQTT features like Last Will and Testament (LWT) for graceful disconnections, Keep Alive for connection supervision, and Retain Flag for storing the last good message are configurable via esp_mqtt_client_config_t.
• All ESP32 variants (ESP32, S2, S3, C3, C6, H2) support the esp-mqtt component with consistent API, though newer variants offer performance benefits, especially for TLS with hardware acceleration.
• Common troubleshooting steps include verifying Wi-Fi credentials, broker URI/port, ensuring unique client IDs, correctly handling MQTT event data, and checking CMake component linkages.

Further Reading

• ESP-IDF MQTT Client Documentation (esp-mqtt): The official Espressif documentation is the definitive source for esp-mqtt API details and advanced usage.
  • ESP-IDF MQTT Client API Reference
• MQTT Version 5.0 Specification (OASIS Standard): For in-depth protocol understanding.
  • MQTT v5.0 Specification
• HiveMQ MQTT Essentials: An excellent resource for foundational MQTT concepts, often complementing the technical specifications.
  • HiveMQ MQTT Essentials
• Mosquitto MQTT Broker Documentation: If you plan to set up your own local broker for testing.
  • Mosquitto Documentation
• Espressif Wi-Fi Station Example: For more detailed Wi-Fi setup variations.
  • ESP-IDF Wi-Fi Station Example (Github)