Chapter 49: Mesh Network Diagnostics and Healing

Chapter Objectives

By the end of this chapter, you will be able to:

• Understand common issues that can arise in an ESP-MESH network.
• Utilize ESP-IDF APIs to gather diagnostic information about mesh nodes and the network topology.
• Interpret mesh events to diagnose connectivity problems and network changes.
• Explain the self-healing mechanisms inherent in ESP-MESH, such as node re-parenting and root failover.
• Implement code to log diagnostic data and observe self-healing processes.
• Apply troubleshooting techniques to resolve common mesh network problems.
• Develop strategies for improving mesh network stability and performance based on diagnostic insights.

Introduction

Previous chapters have guided us through setting up ESP-MESH networks, configuring root nodes, and managing internal nodes. While ESP-MESH is designed to be self-organizing and self-healing, real-world deployments can encounter challenges due to RF interference, node failures, configuration errors, or suboptimal network layouts. When issues arise, or when you simply need to understand the current state and performance of your mesh, effective diagnostics become indispensable.

Furthermore, the “self-healing” aspect of ESP-MESH is one of its most powerful features. Understanding how nodes automatically recover from disconnections or how the network adapts to the loss of a critical node like the root is key to appreciating its robustness. This chapter focuses on the tools and techniques for diagnosing your ESP-MESH network’s health and observing its healing capabilities. We will explore ESP-IDF APIs that provide insights into the mesh topology, node status, and connectivity, and learn how to interpret events to troubleshoot problems and verify the network’s resilience.

Theory

Understanding Mesh Health: Common Issues

A healthy mesh network is characterized by stable connections, efficient data routing, and appropriate layer distribution. Common issues that can degrade mesh health include:

ESP-MESH Diagnostic APIs

ESP-IDF provides several functions to query the state of the mesh and its nodes. These are invaluable for diagnostics:

• esp_mesh_get_parent_bssid(mesh_addr_t *bssid):
  • Retrieves the MAC address (BSSID) of the current parent node for the calling node.
  • If the node is the root or not connected, it returns an error.
• esp_mesh_get_layer():
  • Returns the current layer of the calling node within the mesh hierarchy. Root is Layer 1.
• esp_mesh_is_root():
  • Returns true if the calling node is currently the root node, false otherwise.
• esp_mesh_get_routing_table(mesh_addr_t *table, uint16_t size, uint16_t *table_size):
  • Retrieves the mesh routing table. The routing table contains the MAC addresses of other nodes known to the current node (directly or indirectly).
  • table: A buffer to store the MAC addresses.
  • size: The size of the provided buffer in bytes.
  • table_size: Output parameter indicating the actual number of entries (nodes) in the routing table.
• esp_mesh_get_routing_table_size(uint16_t *table_size):
  • A simpler way to get just the number of nodes in the routing table.
• esp_mesh_get_ débit_enfant_info(mesh_assoc_t *assoc, uint8_t *num) (Conceptual – esp_mesh_get_assoc_child_count or similar might be available or require iterating routing table):
  • While there isn’t a direct single function like esp_mesh_get_children_info that returns a list of all children and their details, you can infer direct children by examining the routing table or by handling MESH_EVENT_CHILD_CONNECTED/DISCONNECTED. The root node knows its direct children.
  • esp_wifi_ap_get_sta_list() can be used by a parent node (including root) to get a list of directly connected stations (children in the mesh context). The wifi_sta_list_t structure contains MAC addresses and RSSI values.
• esp_wifi_sta_get_rssi(int *rssi) (on Root for Router) / esp_wifi_ap_get_sta_list() (for Children’s RSSI):
  • The root node, acting as a station to the router, can call esp_wifi_sta_get_rssi() to get the signal strength to the router.
  • Any parent node can use esp_wifi_ap_get_sta_list() to get information about its connected children, including their RSSI. This is crucial for understanding link quality.
• esp_mesh_get_id(mesh_addr_t *mesh_id):
  • Gets the 6-byte MUID of the mesh the node is part of.
• esp_mesh_get_voted_real_layer():
  • Gets the actual layer of the node in the mesh, which might differ from the initially assigned layer during topology adjustments.

Interpreting Mesh Events for Diagnostics

Mesh events (event base MESH_EVENT) provide real-time information about network dynamics:

Self-Healing Mechanisms in ESP-MESH

ESP-MESH is designed with inherent resilience:

1. Node Re-parenting:
   • If an internal node loses connection to its parent (due to parent failure, signal loss, etc.), it automatically triggers a scan for a new parent.
   • It will attempt to connect to another available parent node within the same mesh, potentially changing its layer or position in the tree.
   • This is the most common form of self-healing and ensures that isolated node failures (other than the root) don’t necessarily bring down large parts of the network.
2. Root Node Failover (Voting Mechanism):
   • If the root node fails or loses its connection to the external router for an extended period, the mesh can elect a new root node from among the existing internal nodes.
   • Eligibility: Nodes that are “root-eligible” (typically Layer 2 nodes with good stability, though configurable) can participate in voting.
   • Voting Trigger: Triggered by prolonged absence of root beacons or specific conditions.
   • Process: Eligible nodes may “vote” for a new root. esp_mesh_set_いえいえ_vote_percentage(1) means a node needs 100% of votes from its children to become root (this is a simplification; the actual mechanism is more complex and involves candidate selection). esp_mesh_set_is_preferred_root(true) can make a node more likely to become root.
   • Outcome: A new root is elected, which then attempts to connect to the external router. The mesh re-organizes around this new root.
   • This provides resilience against root node failure, maintaining internal mesh communication and potentially restoring external connectivity if the new root can connect to the router.
3. Dynamic Routing Updates:
   • As nodes join, leave, or re-parent, the routing tables within each node are updated.
   • This ensures that data packets can find valid paths to their destinations even as the network topology changes.

%%{
  init: {
    'theme': 'base',
    'themeVariables': {
      'fontFamily': 'Open Sans',
      'textColor': '#1F2937',
      'lineColor': '#A78BFA', 
      'actorBorder': '#5B21B6',
      'actorFill': '#EDE9FE',
      'actorTextColor': '#5B21B6',
      'sequenceNumberColor': '#1E40AF',
      'loopTextColor': '#2563EB',
      'noteBkgColor': '#FEF3C7',
      'noteTextColor': '#92400E',
      'activationBorderColor': '#2563EB',
      'activationBkgColor': '#DBEAFE',
      'criticalBkgColor': '#FEE2E2',
      'criticalBorderColor': '#DC2626'
    }
  }
}%%
sequenceDiagram
    actor OldRoot as "Original Root (R1)"
    participant MeshNodes as "Mesh Nodes (Layer 2+)"
    actor CandidateA as "Candidate Node A (L2)"
    actor CandidateB as "Candidate Node B (L2)"
    participant ExtRouter as "External Wi-Fi Router"

    OldRoot->>MeshNodes: Normal Beacons & Data Flow
    
    critical Root Failure
        OldRoot--xMeshNodes: R1 Fails / Loses Router Link (Beacons Stop)
    end

    MeshNodes->>MeshNodes: Detect Root Absence (e.g., beacon timeout)
    Note over MeshNodes: MESH_EVENT_VOTE_STARTED
    
    Note over MeshNodes, CandidateB: Nodes eligible for root (e.g., L2, stable) become candidates.
    
    MeshNodes->>CandidateA: "Vote" for Candidate A (based on criteria)
    MeshNodes->>CandidateB: "Vote" for Candidate B (based on criteria)
    
    alt Candidate A Wins Election
        CandidateA->>CandidateA: Accumulates Enough Votes
        CandidateA->>MeshNodes: Announce Self as New Root (R_New)
        Note over MeshNodes: MESH_EVENT_VOTE_STOPPED
        Note over MeshNodes: MESH_EVENT_ROOT_ADDRESS (R_New's MAC)
        CandidateA->>ExtRouter: Attempt Wi-Fi Connection (STA mode)
        activate ExtRouter
        ExtRouter-->>CandidateA: Wi-Fi Connection Successful
        CandidateA->>ExtRouter: DHCP Request
        ExtRouter-->>CandidateA: IP Address Assigned
        deactivate ExtRouter
        Note over CandidateA, MeshNodes: MESH_EVENT_ROOT_GOT_IP (as R_New)
        Note over CandidateA, MeshNodes: Mesh re-organizes under R_New.<br>External connectivity restored.
    else Candidate B Wins Election or Other Scenario
        CandidateB->>CandidateB: (Similar process if B wins)
        Note over MeshNodes: If no candidate succeeds quickly,<br>mesh might remain without an active IP root.
    end

Logging Strategies for Diagnostics

• Use ESP_LOGx Macros: Employ ESP_LOGI, ESP_LOGW, ESP_LOGE, ESP_LOGD, ESP_LOGV with meaningful tags.
• Increase Log Verbosity: For detailed mesh diagnostics, you might temporarily increase the log level for the esp_mesh component (and potentially wifi) via menuconfig (Component config -> Log output -> Default log verbosity and Maximum log verbosity, then per-component levels).
• Event-Specific Logging: Log detailed information within your mesh event handler, especially for disconnection events (including reason codes) and routing table changes.
• Periodic Status Logging: Implement a task that periodically queries and logs key metrics like current layer, parent BSSID, number of children (if a parent), and RSSI to parent/router.

Practical Examples

Example 1: Periodic Node Status Logger

This example shows how a node (can be root or internal) can periodically log its own mesh status.

1. Project Setup:

• Use an existing ESP-MESH project (either root or internal node from Chapters 46/47).
• Ensure ESP-MESH is initialized and started.

2. Code Implementation (add this task to your existing app_main):

// In your main.c file (for either root or internal node)

// ... (include existing headers: esp_log, esp_mesh, freertos/FreeRTOS, freertos/task etc.) ...

static const char *DIAG_TAG = "MESH_DIAG";

static void log_mesh_status_task(void *pvParameters)
{
    ESP_LOGI(DIAG_TAG, "Mesh Status Logger Task Started.");
    mesh_addr_t parent_bssid;
    wifi_sta_list_t sta_list; // For getting children info if this node is a parent
    int current_layer;

    while (1) {
        vTaskDelay(pdMS_TO_TICKS(30000)); // Log every 30 seconds

        if (!esp_mesh_is_peer_started()) {
            ESP_LOGI(DIAG_TAG, "Mesh is not started/active.");
            continue;
        }

        current_layer = esp_mesh_get_layer();
        ESP_LOGI(DIAG_TAG, "--- Mesh Status ---");
        ESP_LOGI(DIAG_TAG, "Current Layer: %d", current_layer);

        if (esp_mesh_is_root()) {
            ESP_LOGI(DIAG_TAG, "Role: Root Node");
            // Check connection to router
            wifi_ap_record_t ap_info;
            if (esp_wifi_sta_get_ap_info(&ap_info) == ESP_OK) {
                ESP_LOGI(DIAG_TAG, "Connected to Router: SSID='%s', RSSI=%d", ap_info.ssid, ap_info.rssi);
            } else {
                ESP_LOGW(DIAG_TAG, "Not connected to router or unable to get AP info.");
            }
            // Get number of children
            uint16_t children_num = 0;
            // A simple way: count from routing table, but this includes non-children too.
            // More accurately, if root, it acts as AP.
            if(esp_wifi_ap_get_sta_list(&sta_list) == ESP_OK) {
                ESP_LOGI(DIAG_TAG, "Direct Children Count: %d", sta_list.num);
                for (int i = 0; i < sta_list.num; i++) {
                    ESP_LOGI(DIAG_TAG, "  Child %d MAC: "MACSTR", RSSI: %d", i + 1,
                             MAC2STR(sta_list.sta[i].mac), sta_list.sta[i].rssi);
                }
            }


        } else { // Internal Node
            ESP_LOGI(DIAG_TAG, "Role: Internal Node");
            if (esp_mesh_get_parent_bssid(&parent_bssid) == ESP_OK) {
                ESP_LOGI(DIAG_TAG, "Parent BSSID: " MACSTR, MAC2STR(parent_bssid.addr));
                // Note: Getting RSSI to parent for a mesh child is not straightforward with a single API call
                // like esp_wifi_sta_get_rssi() which is for STA mode to an AP.
                // The parent node would know the RSSI of this child.
            } else {
                ESP_LOGW(DIAG_TAG, "Not connected to a parent.");
            }
             // Check if this internal node is also a parent
            if(esp_wifi_ap_get_sta_list(&sta_list) == ESP_OK && sta_list.num > 0) {
                ESP_LOGI(DIAG_TAG, "Acting as Parent. Direct Children Count: %d", sta_list.num);
                 for (int i = 0; i < sta_list.num; i++) {
                    ESP_LOGI(DIAG_TAG, "  Child %d MAC: "MACSTR", RSSI: %d", i + 1,
                             MAC2STR(sta_list.sta[i].mac), sta_list.sta[i].rssi);
                }
            }
        }

        uint16_t routing_table_size;
        esp_mesh_get_routing_table_size(&routing_table_size);
        ESP_LOGI(DIAG_TAG, "Routing Table Size: %d", routing_table_size);
        // Optionally, print the whole routing table if small enough
        // mesh_addr_t route_table[CONFIG_MESH_ROUTE_TABLE_SIZE_MAX]; // Max possible size
        // if (routing_table_size > 0 && routing_table_size <= CONFIG_MESH_ROUTE_TABLE_SIZE_MAX) {
        //     esp_mesh_get_routing_table(route_table, sizeof(route_table), &routing_table_size);
        //     for (int i = 0; i < routing_table_size; i++) {
        //         ESP_LOGD(DIAG_TAG, "  Route %d: "MACSTR, i, MAC2STR(route_table[i].addr));
        //     }
        // }
        ESP_LOGI(DIAG_TAG, "--- End Status ---");
    }
}

// In your app_main, after mesh is started:
// xTaskCreate(log_mesh_status_task, "mesh_status_logger", 3072, NULL, 5, NULL);

To use:

• Add the log_mesh_status_task function to your existing root or internal node code.
• In app_main, after esp_mesh_start() is called and the mesh is expected to be forming, create this task:xTaskCreate(log_mesh_status_task, “mesh_status_logger”, 4096, NULL, 5, NULL);
• Build, flash, and observe the periodic status logs on the serial monitor.

Example 2: Observing Node Re-parenting (Self-Healing)

This example requires three ESP32 boards:

• Board R: Configured as the Root Node.
• Board P: Configured as an Internal Node (will act as a Parent for Board C).
• Board C: Configured as an Internal Node (will connect to Board P initially).

Setup:

1. Flash Board R with root node code (e.g., from Chapter 47). Ensure it connects to your router.
2. Flash Board P with internal node code (e.g., from Chapter 48). It should connect to Board R.
3. Flash Board C with internal node code. It should connect to Board P.
   • You might need to slightly delay the startup of Board C or P to encourage this specific topology initially, or place them physically to favor these connections.
   • All nodes must share the same Mesh ID, Password, and Channel.

Observation Steps:

1. Power on all three boards.
2. Observe their serial monitors:
   • Board R: Should be root, layer 1. Should show P connected as a child.
   • Board P: Should connect to R (parent BSSID matches R’s MAC). Layer should be 2. Should show C connected as its child.
   • Board C: Should connect to P (parent BSSID matches P’s MAC). Layer should be 3.
3. Once the topology is stable (R <- P <- C), power off or reset Board P.
4. Observe Board C’s serial monitor:
   • It should log MESH_EVENT_PARENT_DISCONNECTED (reason might be beacon timeout or similar).
   • It should then start scanning for a new parent.
   • After a short while, it should log MESH_EVENT_PARENT_CONNECTED again. This time, its parent BSSID should match Board R’s MAC address.
   • Its MESH_EVENT_LAYER_CHANGE should show its layer changing, likely to 2.
5. Observe Board R’s serial monitor:
   • It should log MESH_EVENT_CHILD_DISCONNECTED for Board P.
   • It should then log MESH_EVENT_CHILD_CONNECTED for Board C.

This sequence demonstrates self-healing: Board C automatically re-parented to Board R when its intermediate parent (Board P) became unavailable.

Variant Notes

The diagnostic APIs and self-healing mechanisms described are core features of the ESP-MESH stack and are generally consistent across all WiFi-enabled ESP32 variants:

• ESP32, ESP32-S2, ESP32-S3, ESP32-C3, ESP32-C6.

The underlying principles of event generation, routing table management, and parent selection logic are the same.

• Performance Impact of Diagnostics: Frequent polling of diagnostic APIs (like esp_mesh_get_routing_table) on resource-constrained variants or in very large networks might have a minor performance overhead. Use diagnostic tasks with reasonable polling intervals.
• Log Output: The volume and detail of log output can be substantial, especially with higher verbosity. Ensure your serial monitor setup can handle it, or consider logging to other destinations (like flash or a remote server via the root node) for long-term diagnostics.
• ESP32-H2: Does not support WiFi or ESP-MESH, so this chapter is not applicable.

Common Mistakes & Troubleshooting Tips

Exercises

1. RSSI to Parent Logger:
   • Modify the internal node example. When MESH_EVENT_PARENT_CONNECTED occurs, the parent node (if it’s an ESP32 you also control) should log the RSSI of the newly connected child.
   • To achieve this, the parent node’s MESH_EVENT_CHILD_CONNECTED handler would need to iterate through its station list (using esp_wifi_ap_get_sta_list()) to find the new child’s MAC and log its RSSI.
   • This exercise helps understand link quality from the parent’s perspective.
2. Simulate and Log Root Failover:
   • Set up a mesh with one designated root (Root A) and at least one other node (Node B) that is also configured with router credentials (making it a potential root).
   • Ensure both nodes have esp_mesh_set_いえいえ_vote_percentage(1) (or similar to allow voting).
   • Once the mesh is stable with Root A as the active root, power off Root A.
   • Observe the serial logs on Node B. It should eventually detect the root loss, participate in voting (you might see MESH_EVENT_VOTE_STARTED), and then potentially become the new root (indicated by MESH_EVENT_PARENT_CONNECTED to the router and esp_mesh_is_root() returning true).
3. Network Healing Visualization (Conceptual):
   • On the root node, create a task that periodically gets the routing table.
   • Send this routing table information (e.g., list of MACs and their layers if determinable) via MQTT or HTTP to a simple web page or a host application.
   • The web page/application should try to visualize the connected nodes.
   • Induce changes in the mesh (power off an intermediate node). Observe how the visualized topology changes as the network heals. (This is more advanced, focus on logging the changes if visualization is too complex).

Summary

• Effective mesh network management relies on robust diagnostics and understanding self-healing processes.
• ESP-IDF provides APIs like esp_mesh_get_parent_bssid(), esp_mesh_get_layer(), and esp_mesh_get_routing_table() for querying node and network status.
• Mesh events (MESH_EVENT_PARENT_DISCONNECTED, MESH_EVENT_NO_PARENT_FOUND, etc.) offer real-time insights into network dynamics and issues.
• ESP-MESH’s self-healing includes automatic re-parenting of nodes when a parent link is lost and a voting mechanism for root node failover.
• Strategic logging and periodic status checks are crucial for troubleshooting and monitoring mesh health.
• Understanding the reasons behind disconnections and the behavior of nodes during healing helps in building more resilient mesh applications.

Further Reading

• ESP-IDF ESP-MESH Programming Guide: https://docs.espressif.com/projects/esp-idf/en/v5.4/esp32/api-guides/mesh.html (Especially sections on events and network management).
• ESP-IDF ESP-MESH API Reference (esp_mesh.h): https://docs.espressif.com/projects/esp-idf/en/v5.4/esp32/api-reference/network/esp_mesh.html
• ESP-IDF Wi-Fi Driver API Reference (esp_wifi.h): For APIs like esp_wifi_ap_get_sta_list().
• ESP-MESH Examples in ESP-IDF: $IDF_PATH/examples/mesh/ provide practical code for various scenarios.