Leave a Comment / By /

Chapter 118: CoAP Observe Pattern in ESP-IDF

Chapter Objectives

By the end of this chapter, you will be able to:

  • Understand the purpose and benefits of the CoAP Observe pattern (RFC 7641).
  • Explain how a client registers and de-registers its interest in observing a resource.
  • Describe how a CoAP server manages observers and sends notifications.
  • Differentiate between Confirmable (CON) and Non-confirmable (NON) notifications.
  • Implement an observable resource on an ESP32 CoAP server.
  • Develop an ESP32 CoAP client that can observe a resource and handle notifications.
  • Understand the role of sequence numbers and ETags in the Observe pattern.
  • Recognize considerations for using Observe with different ESP32 variants.

Introduction

In our previous CoAP chapters, we’ve seen how clients can retrieve resource states (GET) or update them (PUT/POST). This typically involves a client polling a server periodically if it needs to monitor changes. However, polling can be inefficient, especially for constrained devices and networks, as it generates unnecessary traffic and consumes energy even when the resource state hasn’t changed.

The CoAP Observe pattern provides a much more efficient solution. It allows a CoAP client to register its interest in a resource with a CoAP server. The server then notifies the client whenever the state of that resource changes. This publish/subscribe-like mechanism is crucial for applications requiring timely updates, such as monitoring sensor data, receiving alerts, or synchronizing states between devices, without the overhead of constant polling. This chapter explores the theory and practical implementation of the CoAP Observe pattern on ESP32 devices.

Theory

What is the Observe Pattern?

The CoAP Observe pattern, defined in RFC 7641, allows a CoAP client to “observe” a resource on a CoAP server. When a client observes a resource, it’s essentially subscribing to changes in that resource’s state. The server, in turn, agrees to send notifications to the client whenever the observed resource’s state changes.

This is analogous to subscribing to a newsletter. Instead of buying a newspaper every day to see if there’s news you care about (polling), you subscribe once, and the publisher sends you the newsletter whenever new content is available (notification).

The Observe Option

The core of the Observe pattern is the Observe Option (Option number 6). This option is included in CoAP messages to manage the observation relationship.

  • Client Request (Registering/De-registering):
    • To register for observation, a client sends a GET request to the resource and includes an Observe Option with a value of 0 (Register).
    • To de-register, a client sends a GET request with an Observe Option value of 1 (Deregister). Alternatively, the server can implicitly de-register a client (e.g., if a notification times out or the client explicitly rejects a notification with a Reset message).
  • Server Response (Establishing Observation):
    • If the server supports observation for the requested resource and accepts the registration, it includes the Observe Option in its 2.xx success response (e.g., 2.05 Content). The value of this option in the response is a sequence number, starting typically from 0 or a low number, and incrementing for each subsequent notification.
  • Server Notifications (Sending Updates):
    • When the state of an observed resource changes, the server sends a new CoAP response (typically 2.05 Content) to each registered observer. This response is unsolicited by an immediate GET from the client but is a direct result of the observation relationship.
    • This notification message must include the Observe Option with an incrementing sequence number. This sequence number helps the client detect lost or out-of-order notifications.
    • Notifications can be sent as Confirmable (CON) or Non-confirmable (NON) messages.
      • CON notifications: Require an ACK from the client. If the server doesn’t receive an ACK, it will retransmit the notification. This provides reliability but adds overhead. If a CON notification repeatedly fails, the server may assume the client is no longer reachable and de-register it.
      • NON notifications: Are “fire and forget.” They are less reliable but have lower overhead. They are suitable for data that changes frequently, where occasional loss is acceptable.
sequenceDiagram
    actor Client
    participant Server

    rect rgb(255, 255, 255)
        note over Client, Server: Initial State: Resource has some state
    end

    Client->>+Server: GET /sensor (Observe: 0 - Register)
    activate Server
    Server-->>-Client: 2.05 Content (Observe: N, Payload: initial_state)
    deactivate Server


    loop Resource State Changes
        note over Server: Resource state changes on server
        Server->>+Client: 2.05 Content (Observe: N+1, Payload: new_state)
        activate Client
        opt If Notification was CON
            Client-->>-Server: ACK
        end
        deactivate Client

        note over Server: Resource state changes again
        Server->>+Client: 2.05 Content (Observe: N+2, Payload: newer_state)
        activate Client
        opt If Notification was CON
            Client-->>-Server: ACK
        end
        deactivate Client

    end

Notification Freshness and Sequence Numbers

The Observe Option in notifications contains a sequence number (typically 2 or 3 bytes). This number is chosen by the server and should ideally be monotonically increasing for each notification sent for a specific observation relationship. This allows the client to:

  • Detect reordered notifications (e.g., if sequence N+2 arrives before N+1).
  • Detect lost notifications (e.g., if sequence N+2 arrives after N, implying N+1 was lost).
Scenario Example (Last Known Seq -> Current Seq) Client Action / Interpretation Notes
Initial Registration Response N/A -> 0 (or S0) Accept response. Store S0 as the current sequence number. Observation established. This is the first notification in the observation.
In-Order Notification Sn -> Sn+1 Accept notification. Update current sequence number to Sn+1. Normal operation.
Reordered Notification (Late) Sn+1 -> Sn (where Sn+1 was already processed) Discard Sn as it’s older than already processed Sn+1. Client has already seen a newer notification.
Reordered Notification (Early) Sn -> Sn+2 (Sn+1 not yet received) Accept Sn+2. Update current sequence. Recognize Sn+1 might be lost or delayed. If Sn+1 arrives later, it will be handled as “Reordered (Late)”. Indicates a potential gap.
Lost Notification(s) Sn -> Sn+k (where k > 1) Accept Sn+k. Recognize that k-1 notifications (Sn+1 to Sn+k-1) were lost. Client might need to decide if the gap is acceptable or if re-registration is needed.
Duplicate Notification Sn -> Sn Discard the duplicate Sn. Server might have retransmitted a CON notification if ACK was lost.
Sequence Number Wrap-Around Smax -> S0 (e.g., 16777215 -> 0 for 24-bit sequence) Accept S0. Update current sequence. Handle comparison carefully (RFC 7641 specifies rules). Infrequent but possible. Client must compare based on a window around the expected sequence.
Re-registration Response Sold_obs -> Snew_obs_0 If client initiated re-registration, treat Snew_obs_0 as the start of a new observation sequence. The new sequence might be unrelated to the old one.

The sequence number space is large enough that wrap-around is infrequent. However, clients should be prepared for it. RFC 7641 specifies rules for comparing sequence numbers to handle wrap-around and reordering.

ETags and Conditional Observation

The ETag (Entity Tag) option can be used in conjunction with Observe.

  • When a server sends an initial response to an Observe registration or a notification, it can include an ETag representing the current version of the resource’s state.
  • If a client temporarily loses connection or wants to re-establish an observation, it can send a GET request with the Observe option and include the last ETag it received (using the If-Match or If-None-Match options).
    • If the resource state hasn’t changed (ETag still matches), the server can respond with 2.03 Valid without sending the full payload, confirming the observation is still active with the known state.
    • If the state has changed, the server sends 2.05 Content with the new payload and new ETag.
sequenceDiagram
    actor Client
    participant Server


    %% Initial Observation
    Client->>+Server: GET /resource (Observe: 0)
    Server-->>-Client: 2.05 Content (Observe: S0, ETag: E0, Payload: Data0)
    Client->>Client: Store ETag E0, Sequence S0

    %% ... Time passes, client might have disconnected or wants to re-validate ...

    alt Re-establishing/Validating Observation
        Client->>+Server: GET /resource (Observe: 0, If-Match: E0)
        activate Server
        alt Resource Unchanged
            Server-->>-Client: 2.03 Valid (Observe: S_current, ETag: E0)

            Client->>Client: Observation confirmed with existing E0

        else Resource Changed
            Server-->>-Client: 2.05 Content (Observe: S_new, ETag: E1, Payload: Data1)

            Client->>Client: Store ETag E1, Sequence S_new
           
        end
    end

    %% Subsequent Notifications
    note over Server: Resource state changes
    Server->>+Client: 2.05 Content (Observe: S_new+1, ETag: E2, Payload: Data2)
    activate Client
    Client-->>-Server: ACK (if CON)
    deactivate Client
    Client->>Client: Store ETag E2

This helps reduce redundant data transmission, especially when re-establishing observations.

De-registration

A client can explicitly de-register by:

  1. Sending a GET request with the Observe Option set to 1 (Deregister).
  2. Sending a Reset (RST) message in reply to a CON notification.
Method Initiator Mechanism Description
Explicit GET Deregister Client Client sends a GET request for the observed resource with the Observe Option set to 1 (Deregister). This is the standard way for a client to explicitly end an observation. The server should acknowledge this, typically with a 2.xx response.
Reset (RST) on CON Notification Client Client sends a Reset (RST) message in reply to a Confirmable (CON) notification from the server. This implicitly tells the server the client no longer wishes to observe or cannot process the notification. The server should remove the client from the observer list.
CON Notification Timeout Server (Implicit) Server sends a CON notification, but it times out after several retransmissions without receiving an ACK from the client. The server assumes the client is no longer reachable or interested and removes it from the observer list for that resource.
Resource Deletion Server (Implicit) The observed resource is deleted from the server. The server should remove all observers for that resource. It may optionally send a final 4.04 Not Found notification to observers.
Server Shutdown / Resource Constraints Server (Implicit) The server is shutting down, or it runs out of resources (e.g., memory) to maintain the observation relationship. The server may de-register observers. Well-behaved servers might attempt to notify clients if possible (e.g., with a 5.03 Service Unavailable).
Client “Forgets” Observation Client (Implicit) Client simply stops processing notifications or crashes without explicit de-registration. If notifications are NON, the server might not know. If CON, it will eventually time out (see “CON Notification Timeout”).

A server can implicitly de-register a client if:

  1. A CON notification times out after several retransmissions.
  2. The resource is deleted.
  3. The server is shutting down or runs out of resources to maintain the observation.

libcoap Support for Observe

The libcoap library (and thus esp-coap) provides significant built-in support for the Observe pattern:

  • Server-side:
    • coap_resource_set_get_observable(resource, 1): Marks a resource as observable.
    • coap_resource_notify_observers(resource, query): Triggers notifications to all registered observers for that resource. libcoap handles sending the actual PDU with the current resource state and incrementing the Observe sequence number.
    • It manages the list of observers for each resource.
  • Client-side:
    • When sending a GET request, coap_pdu_add_observe_option(pdu, COAP_OBSERVE_ESTABLISH) can be used to register.
    • A response handler will receive notifications. The client needs to check for the Observe option in incoming responses to identify them as notifications.
    • libcoap can automatically re-register if an observation times out (e.g., due to MAX_AGE elapsing without fresh notifications), if configured.
Function / Macro Side Purpose Brief Description / Conceptual Usage
coap_resource_set_get_observable(resource, 1) Server Mark a resource as observable. Call this during resource initialization to enable the Observe feature for GET requests on this resource.
coap_resource_set_dirty(resource, query_filter) Server Mark an observable resource as “dirty,” indicating its state has changed and notifications should be sent. Call when the resource’s underlying data changes. coap_io_process() will then send notifications. query_filter can be NULL.
coap_touch_resource(resource) Server Alternative to coap_resource_set_dirty(). Marks a resource as changed. Similar effect to coap_resource_set_dirty(); triggers notifications on the next suitable occasion.
coap_resource_notify_observers(resource, query) Server (More direct way, though set_dirty is common) Triggers notifications to observers. libcoap often handles this internally after set_dirty. Direct use might be for specific scenarios. Note: This function is not always a public API or might be an internal mechanism triggered by set_dirty and coap_io_process. The primary method is set_dirty.
COAP_RESOURCE_FLAGS_NOTIFY_CON / COAP_RESOURCE_FLAGS_NOTIFY_NON Server Resource flags to specify notification type. Set during coap_resource_init() or similar to indicate if notifications should be Confirmable (CON) or Non-confirmable (NON). Default is often CON.
coap_pdu_add_observe_option(pdu, COAP_OBSERVE_ESTABLISH) Client Add Observe option to a PDU to register for observation. Used when creating a GET request to start observing. COAP_OBSERVE_ESTABLISH is typically 0.
coap_pdu_add_observe_option(pdu, COAP_OBSERVE_CANCEL) Client Add Observe option to a PDU to de-register. Used when creating a GET request to stop observing. COAP_OBSERVE_CANCEL is typically 1.
coap_check_option(received_pdu, COAP_OPTION_OBSERVE, &opt_iter) Client Check for the Observe option in a received PDU. Used in the response handler to identify notifications and extract the sequence number.
coap_decode_var_bytes(opt_value, opt_length) Client Decode the Observe option’s sequence number. Used to get the integer value of the sequence number from the option data.
Response Handler (user-defined) Client Process incoming messages, including notifications. Checks for the Observe option, handles sequence numbers, and processes the payload of notifications. Registered with coap_register_response_handler().

Practical Examples

Ensure your ESP-IDF project includes the esp-coap component and Wi-Fi is configured. The Wi-Fi initialization code (wifi_init_sta) is assumed to be similar to previous chapters and is included for completeness.

Example 1: ESP32 CoAP Server with an Observable Resource

This server will offer a resource /time that updates with the system uptime every few seconds. Clients can observe this resource.

1. Project Setup:

Create/use an ESP-IDF project, configure Wi-Fi and esp-coap.

2. main/coap_server_observe_main.c:

C
#include <string.h>
#include <sys/socket.h>
#include <netdb.h>
#include <freertos/FreeRTOS.h>
#include <freertos/task.h>
#include <freertos/timers.h> // For timer
#include <freertos/event_groups.h>
#include "esp_log.h"
#include "nvs_flash.h"
#include "esp_event.h"
#include "esp_wifi.h"
#include "esp_coap.h"
#include "esp_timer.h" // For esp_timer_get_time

static const char *TAG = "COAP_OBSERVE_SRV";

static coap_resource_t *resource_time = NULL;
static coap_context_t *coap_ctx = NULL; // Global context for timer callback

/* --- Resource Handler for /time (GET, Observable) --- */
static void time_get_handler(coap_resource_t *resource,
                             coap_session_t *session,
                             const coap_pdu_t *request,
                             const coap_string_t *query,
                             coap_pdu_t *response)
{
    char time_str[32];
    uint64_t now_us = esp_timer_get_time();
    snprintf(time_str, sizeof(time_str), "Uptime: %llu.%03llu s", now_us / 1000000, (now_us % 1000000) / 1000);

    coap_pdu_set_code(response, COAP_RESPONSE_CODE_CONTENT);
    coap_add_data_large_response(resource, session, request, response, query,
                                 COAP_MEDIATYPE_TEXT_PLAIN,
                                 strlen(time_str),
                                 (const uint8_t *)time_str,
                                 NULL, NULL);
    // ESP_LOGI(TAG, "/time GET request processed");
}

/* --- Timer callback to trigger notifications --- */
static void notify_time_observers_callback(TimerHandle_t xTimer)
{
    if (coap_ctx && resource_time) {
        // ESP_LOGI(TAG, "Timer expired, triggering notify for /time");
        // coap_resource_notify_observers will check if the resource has observers
        // and if the resource is "dirty" (needs notification).
        // To ensure it sends, we can mark it dirty.
        coap_resource_set_dirty(resource_time, NULL); // Second arg is query filter, NULL for all
        // Alternatively, coap_touch_resource(resource_time) also works.

        // Forcing a periodic notification regardless of actual data change (for demo):
        // If the resource data itself is not updated by another mechanism before calling notify,
        // libcoap might optimize and not send if it thinks the data hasn't changed.
        // In a real scenario, you'd update your resource's state representation *before* calling notify.
        // Here, the time_get_handler dynamically generates the time, so each call to notify
        // will result in the handler being invoked to get the *current* time for the notification payload.
    } else {
        ESP_LOGW(TAG, "Timer expired, but CoAP context or resource not ready.");
    }
    // Note: coap_io_process() must be running in the CoAP server task for notifications to be sent.
    // coap_resource_notify_observers itself does not send; it queues the need for notification.
}


/* --- Wi-Fi Setup (identical to Chapter 116/117) --- */
#define EXAMPLE_ESP_WIFI_SSID      CONFIG_ESP_WIFI_SSID
#define EXAMPLE_ESP_WIFI_PASS      CONFIG_ESP_WIFI_PASSWORD
static EventGroupHandle_t wifi_event_group;
const int WIFI_CONNECTED_BIT = BIT0;
// ... (wifi_event_handler and wifi_init_sta functions as in previous chapters) ...
static void wifi_event_handler(void* arg, esp_event_base_t event_base,
                               int32_t event_id, void* event_data) {
    if (event_base == WIFI_EVENT && event_id == WIFI_EVENT_STA_START) {
        esp_wifi_connect();
    } else if (event_base == WIFI_EVENT && event_id == WIFI_EVENT_STA_DISCONNECTED) {
        ESP_LOGI(TAG, "Disconnected. Connecting to the AP again...");
        esp_wifi_connect();
    } else if (event_base == IP_EVENT && event_id == IP_EVENT_STA_GOT_IP) {
        ip_event_got_ip_t* event = (ip_event_got_ip_t*) event_data;
        ESP_LOGI(TAG, "Got IP:" IPSTR, IP2STR(&event->ip_info.ip));
        xEventGroupSetBits(wifi_event_group, WIFI_CONNECTED_BIT);
    }
}

void wifi_init_sta(void) {
    wifi_event_group = xEventGroupCreate();
    ESP_ERROR_CHECK(esp_netif_init());
    ESP_ERROR_CHECK(esp_event_loop_create_default());
    esp_netif_create_default_wifi_sta();
    wifi_init_config_t cfg = WIFI_INIT_CONFIG_DEFAULT();
    ESP_ERROR_CHECK(esp_wifi_init(&cfg));
    esp_event_handler_instance_t instance_any_id;
    esp_event_handler_instance_t instance_got_ip;
    ESP_ERROR_CHECK(esp_event_handler_instance_register(WIFI_EVENT, ESP_EVENT_ANY_ID, &wifi_event_handler, NULL, &instance_any_id));
    ESP_ERROR_CHECK(esp_event_handler_instance_register(IP_EVENT, IP_EVENT_STA_GOT_IP, &wifi_event_handler, NULL, &instance_got_ip));
    wifi_config_t wifi_config = { .sta = { .ssid = EXAMPLE_ESP_WIFI_SSID, .password = EXAMPLE_ESP_WIFI_PASS, .threshold.authmode = WIFI_AUTH_WPA2_PSK }};
    ESP_ERROR_CHECK(esp_wifi_set_mode(WIFI_MODE_STA) );
    ESP_ERROR_CHECK(esp_wifi_set_config(WIFI_IF_STA, &wifi_config) );
    ESP_ERROR_CHECK(esp_wifi_start() );
    ESP_LOGI(TAG, "wifi_init_sta finished.");
    xEventGroupWaitBits(wifi_event_group, WIFI_CONNECTED_BIT, pdFALSE, pdFALSE, portMAX_DELAY);
    ESP_LOGI(TAG, "Connected to ap SSID:%s", EXAMPLE_ESP_WIFI_SSID);
}


/* --- CoAP Server Task --- */
static void coap_server_task(void *pvParameters)
{
    coap_address_t serv_addr;
    TimerHandle_t notify_timer = NULL;

    xEventGroupWaitBits(wifi_event_group, WIFI_CONNECTED_BIT, pdFALSE, pdTRUE, portMAX_DELAY);
    ESP_LOGI(TAG, "Wi-Fi Connected. Starting CoAP server...");

    coap_startup();
    coap_address_init(&serv_addr);
    serv_addr.addr.sin.sin_family      = AF_INET;
    serv_addr.addr.sin.sin_addr.s_addr = INADDR_ANY;
    serv_addr.addr.sin.sin_port        = htons(COAP_DEFAULT_PORT);

    coap_ctx = coap_new_context(&serv_addr); // Assign to global coap_ctx
    if (!coap_ctx) {
        ESP_LOGE(TAG, "Failed to create CoAP context");
        goto finish_server_obs;
    }
    coap_context_set_keepalive(coap_ctx, COAP_DEFAULT_KEEPALIVE);

    // Create /time resource
    resource_time = coap_resource_init(coap_make_str_const("time"), COAP_RESOURCE_FLAGS_NOTIFY_CON);
    if (!resource_time) {
        ESP_LOGE(TAG, "Failed to create /time resource");
        goto finish_server_obs;
    }
    coap_register_handler(resource_time, COAP_REQUEST_GET, time_get_handler);
    coap_resource_set_get_observable(resource_time, 1); // Make it observable
    coap_add_attr(resource_time, coap_make_str_const("rt"), coap_make_str_const("uptime_sensor"), 0);
    coap_add_attr(resource_time, coap_make_str_const("ct"), coap_make_str_const("0"), 0); // text/plain
    coap_add_resource(coap_ctx, resource_time);

    // Create a FreeRTOS timer to periodically trigger notifications
    notify_timer = xTimerCreate("NotifyTimer",          // Timer name
                                pdMS_TO_TICKS(5000),    // 5 seconds period
                                pdTRUE,                 // Auto-reload
                                (void *)0,              // Timer ID (not used here)
                                notify_time_observers_callback); // Callback
    if (notify_timer == NULL) {
        ESP_LOGE(TAG, "Failed to create notify timer");
    } else {
        if (xTimerStart(notify_timer, 0) != pdPASS) {
            ESP_LOGE(TAG, "Failed to start notify timer");
        } else {
            ESP_LOGI(TAG, "Notify timer started, period 5s");
        }
    }

    ESP_LOGI(TAG, "CoAP server started on port %d", COAP_DEFAULT_PORT);

    while (1) {
        // coap_io_process also handles sending queued notifications
        int result = coap_io_process(coap_ctx, COAP_IO_WAIT);
        if (result < 0) {
            ESP_LOGE(TAG, "coap_io_process error: %d", result);
            break;
        }
        // Check if any resource is dirty and needs explicit notification trigger
        // This is more for complex scenarios. For simple periodic, timer + set_dirty is enough.
        // coap_check_dirty_resources(coap_ctx); // This is not a public libcoap API function.
        // Instead, rely on coap_resource_set_dirty() and coap_io_process()
    }

finish_server_obs:
    if (notify_timer) xTimerDelete(notify_timer, portMAX_DELAY);
    if (resource_time) coap_delete_resource(coap_ctx, resource_time);
    if (coap_ctx) coap_free_context(coap_ctx);
    coap_cleanup();
    vTaskDelete(NULL);
}

void app_main(void)
{
    ESP_ERROR_CHECK(nvs_flash_init());
    wifi_init_sta();
    xTaskCreate(coap_server_task, "coap_server_task", 8192, NULL, 5, NULL);
}

3. Build Instructions:

Bash
idf.py set-target esp32 # or other Wi-Fi variant
idf.py menuconfig # Configure WiFi credentials
idf.py build

4. Run/Flash/Observe:

  1. Flash the firmware: idf.py -p /dev/ttyUSB0 flash monitor.
  2. Note the ESP32’s IP address.
  3. Use a CoAP client tool that supports Observe. libcoap-cellar/coap-client is excellent for this.To observe the /time resource:coap-client -m get -s 30 coap://<ESP32_IP_ADDRESS>/time -o - # Observe, output to stdout # -s 30: subscribe for 30 seconds # -o - : output notifications to stdout
    You should see an initial response with the uptime, followed by new uptime notifications every 5 seconds. The Observe sequence number in the CoAP header should increment with each notification.

Example 2: ESP32 CoAP Client to Observe a Resource

This client will observe the /time resource from the server in Example 1.

1. Project Setup:

Similar to the server.

2. main/coap_client_observe_main.c:

C
#include <string.h>
#include <sys/socket.h>
#include <netdb.h>
#include <freertos/FreeRTOS.h>
#include <freertos/task.h>
#include <freertos/event_groups.h>
#include "esp_log.h"
#include "nvs_flash.h"
#include "esp_event.h"
#include "esp_wifi.h"
#include "esp_coap.h"

static const char *TAG = "COAP_OBSERVE_CLI";

#define COAP_SERVER_IP_ADDR    "192.168.1.100" // CHANGE THIS to your server's IP
#define COAP_DEFAULT_PORT_STR  "5683"
#define COAP_TARGET_URI        "/time"

static bool observation_active = false;
static uint32_t last_observe_seq = 0; // To track observe sequence numbers

/* --- Wi-Fi Setup (identical to server example) --- */
#define EXAMPLE_ESP_WIFI_SSID      CONFIG_ESP_WIFI_SSID
#define EXAMPLE_ESP_WIFI_PASS      CONFIG_ESP_WIFI_PASSWORD
static EventGroupHandle_t wifi_event_group;
const int WIFI_CONNECTED_BIT = BIT0;
// ... (wifi_event_handler and wifi_init_sta functions as in Example 1) ...
static void wifi_event_handler(void* arg, esp_event_base_t event_base,
                               int32_t event_id, void* event_data) {
    if (event_base == WIFI_EVENT && event_id == WIFI_EVENT_STA_START) {
        esp_wifi_connect();
    } else if (event_base == WIFI_EVENT && event_id == WIFI_EVENT_STA_DISCONNECTED) {
        ESP_LOGI(TAG, "Disconnected. Connecting to the AP again...");
        esp_wifi_connect();
    } else if (event_base == IP_EVENT && event_id == IP_EVENT_STA_GOT_IP) {
        ip_event_got_ip_t* event = (ip_event_got_ip_t*) event_data;
        ESP_LOGI(TAG, "Got IP:" IPSTR, IP2STR(&event->ip_info.ip));
        xEventGroupSetBits(wifi_event_group, WIFI_CONNECTED_BIT);
    }
}

void wifi_init_sta(void) {
    wifi_event_group = xEventGroupCreate();
    ESP_ERROR_CHECK(esp_netif_init());
    ESP_ERROR_CHECK(esp_event_loop_create_default());
    esp_netif_create_default_wifi_sta();
    wifi_init_config_t cfg = WIFI_INIT_CONFIG_DEFAULT();
    ESP_ERROR_CHECK(esp_wifi_init(&cfg));
    esp_event_handler_instance_t instance_any_id;
    esp_event_handler_instance_t instance_got_ip;
    ESP_ERROR_CHECK(esp_event_handler_instance_register(WIFI_EVENT, ESP_EVENT_ANY_ID, &wifi_event_handler, NULL, &instance_any_id));
    ESP_ERROR_CHECK(esp_event_handler_instance_register(IP_EVENT, IP_EVENT_STA_GOT_IP, &wifi_event_handler, NULL, &instance_got_ip));
    wifi_config_t wifi_config = { .sta = { .ssid = EXAMPLE_ESP_WIFI_SSID, .password = EXAMPLE_ESP_WIFI_PASS, .threshold.authmode = WIFI_AUTH_WPA2_PSK }};
    ESP_ERROR_CHECK(esp_wifi_set_mode(WIFI_MODE_STA) );
    ESP_ERROR_CHECK(esp_wifi_set_config(WIFI_IF_STA, &wifi_config) );
    ESP_ERROR_CHECK(esp_wifi_start() );
    ESP_LOGI(TAG, "wifi_init_sta finished.");
    xEventGroupWaitBits(wifi_event_group, WIFI_CONNECTED_BIT, pdFALSE, pdFALSE, portMAX_DELAY);
    ESP_LOGI(TAG, "Connected to ap SSID:%s", EXAMPLE_ESP_WIFI_SSID);
}


/* --- CoAP Response/Notification Handler --- */
static coap_response_handler_t observe_response_handler(coap_session_t *session,
                                                        const coap_pdu_t *sent,
                                                        const coap_pdu_t *received,
                                                        const coap_mid_t mid)
{
    const uint8_t *data = NULL;
    size_t data_len = 0;
    coap_pdu_code_t rcv_code = coap_pdu_get_code(received);
    coap_opt_iterator_t opt_iter;
    coap_opt_t *observe_opt;
    uint32_t current_observe_seq = 0;
    bool is_notification = false;

    observe_opt = coap_check_option(received, COAP_OPTION_OBSERVE, &opt_iter);
    if (observe_opt) {
        is_notification = true;
        current_observe_seq = coap_decode_var_bytes(coap_opt_value(observe_opt), coap_opt_length(observe_opt));
    }

    if (COAP_RESPONSE_CLASS(rcv_code) == 2) { // Success (e.g., 2.05 Content)
        if (coap_get_data(received, &data_len, &data)) {
            if (is_notification) {
                ESP_LOGI(TAG, "Notification (Seq: %lu, Prev Seq: %lu): %.*s",
                         (unsigned long)current_observe_seq, (unsigned long)last_observe_seq, (int)data_len, (char *)data);
                // Basic check for sequence, more robust checking needed for wrap-around / reordering
                if (current_observe_seq > last_observe_seq || (last_observe_seq > (1 << 23) && current_observe_seq < (1 << 8))) { // Handle wrap-around simply
                    last_observe_seq = current_observe_seq;
                } else if (current_observe_seq < last_observe_seq && observation_active) {
                     ESP_LOGW(TAG, "Out of order notification or re-registration? Current: %lu, Last: %lu", (unsigned long)current_observe_seq, (unsigned long)last_observe_seq);
                     // Potentially reset last_observe_seq if it's a new observation after a lapse
                } else if (current_observe_seq == last_observe_seq && observation_active && sent == NULL) {
                     // sent == NULL implies this is an unsolicited response (notification)
                     ESP_LOGW(TAG, "Duplicate notification sequence: %lu", (unsigned long)current_observe_seq);
                }
                observation_active = true; // Mark active on first valid notification
            } else { // Initial response to GET Observe
                ESP_LOGI(TAG, "Initial Observe Response: %.*s", (int)data_len, (char *)data);
                if (observe_opt) { // Should be present if server accepted observation
                    last_observe_seq = current_observe_seq;
                    observation_active = true;
                    ESP_LOGI(TAG, "Observation established. Initial Seq: %lu", (unsigned long)last_observe_seq);
                }
            }
        }
    } else { // Error response
        ESP_LOGE(TAG, "CoAP request/observation failed (Code: %d.%02d)",
                 COAP_RESPONSE_GET_CLASS(rcv_code), COAP_RESPONSE_GET_CODE(rcv_code));
        observation_active = false; // Stop observation on error
    }
    return COAP_RESPONSE_OK;
}

/* --- CoAP Client Task --- */
static void coap_client_task(void *pvParameters)
{
    coap_context_t *ctx = NULL;
    coap_session_t *session = NULL;
    coap_address_t dst_addr;
    static coap_uri_t uri;
    coap_pdu_t *pdu = NULL;
    unsigned char opt_buf[4]; // Buffer for observe option value

    xEventGroupWaitBits(wifi_event_group, WIFI_CONNECTED_BIT, pdFALSE, pdTRUE, portMAX_DELAY);
    ESP_LOGI(TAG, "Wi-Fi Connected. Starting CoAP client for Observe...");

    coap_startup();

    struct addrinfo *res_addr, *ainfo;
    struct addrinfo hints;
    memset(&hints, 0, sizeof(hints));
    hints.ai_socktype = SOCK_DGRAM;
    hints.ai_family = AF_INET;
    if (getaddrinfo(COAP_SERVER_IP_ADDR, COAP_DEFAULT_PORT_STR, &hints, &res_addr) != 0) {
        ESP_LOGE(TAG, "DNS lookup failed for: %s", COAP_SERVER_IP_ADDR);
        goto finish_client_obs;
    }
    ainfo = res_addr;
    coap_address_init(&dst_addr);
    memcpy(&dst_addr.addr.sin, ainfo->ai_addr, sizeof(dst_addr.addr.sin));
    dst_addr.addr.sin.sin_port = htons(atoi(COAP_DEFAULT_PORT_STR));
    freeaddrinfo(res_addr);

    ctx = coap_new_context(NULL);
    if (!ctx) {
        ESP_LOGE(TAG, "Failed to create CoAP context");
        goto finish_client_obs;
    }
    coap_register_response_handler(ctx, observe_response_handler);
    // Enable libcoap's automatic re-registration for observations if they "expire"
    // coap_context_set_observe_max_non(ctx, 10); // Example: max 10 NON notifications before expecting a CON or re-reg
    // coap_context_set_observe_max_age(ctx, 120); // Example: max 120s before client re-registers

    session = coap_new_client_session(ctx, NULL, &dst_addr, COAP_PROTO_UDP);
    if (!session) {
        ESP_LOGE(TAG, "Failed to create CoAP client session");
        goto finish_client_obs;
    }

    // Prepare PDU for GET request with Observe option
    pdu = coap_new_pdu(COAP_MESSAGE_CON, COAP_REQUEST_CODE_GET, coap_new_message_id(session));
    if (!pdu) {
        ESP_LOGE(TAG, "Failed to create PDU");
        goto finish_client_obs;
    }

    // Set URI path
    if (coap_string_to_uri((const uint8_t *)COAP_TARGET_URI, strlen(COAP_TARGET_URI), &uri) != 0) {
         ESP_LOGE(TAG, "Cannot parse URI: %s", COAP_TARGET_URI);
         coap_delete_pdu(pdu);
         goto finish_client_obs;
    }
    if (uri.path.length) {
        coap_add_option(pdu, COAP_OPTION_URI_PATH, uri.path.length, uri.path.s);
    }

    // Add Observe option (value 0 to register)
    // coap_encode_var_safe will write 0 into opt_buf. Length will be 0 if value is 0.
    // For observe=0, the option length is 0.
    coap_add_option(pdu, COAP_OPTION_OBSERVE, coap_encode_var_safe(opt_buf, sizeof(opt_buf), COAP_OBSERVE_ESTABLISH), opt_buf);

    ESP_LOGI(TAG, "Sending CoAP GET Observe request to %s%s", COAP_SERVER_IP_ADDR, COAP_TARGET_URI);
    if (coap_send(session, pdu) == COAP_INVALID_MID) {
        ESP_LOGE(TAG, "Failed to send CoAP Observe request");
    }

    // Client will now passively receive notifications.
    // The coap_io_process loop handles receiving these.
    // We'll run this for a while, then demonstrate de-registration.
    TickType_t observation_start_time = xTaskGetTickCount();
    uint32_t observation_duration_ms = 30000; // Observe for 30 seconds

    while(xTaskGetTickCount() - observation_start_time < pdMS_TO_TICKS(observation_duration_ms)) {
        int result = coap_io_process(ctx, 1000); // Process I/O, 1s timeout
        if (result < 0) {
            ESP_LOGE(TAG, "coap_io_process error: %d", result);
            break;
        }
        // If observation_active becomes false due to an error, we might break early.
        if (!observation_active && (xTaskGetTickCount() - observation_start_time > pdMS_TO_TICKS(5000))) { // Give some time for initial setup
            ESP_LOGW(TAG, "Observation seems to have failed or stopped.");
            break;
        }
    }

    // De-register observation (optional demonstration)
    if (observation_active) {
        ESP_LOGI(TAG, "De-registering observation...");
        pdu = coap_new_pdu(COAP_MESSAGE_CON, COAP_REQUEST_CODE_GET, coap_new_message_id(session));
        if (pdu) {
            if (uri.path.length) {
                coap_add_option(pdu, COAP_OPTION_URI_PATH, uri.path.length, uri.path.s);
            }
            // Add Observe option (value 1 to de-register)
            coap_add_option(pdu, COAP_OPTION_OBSERVE, coap_encode_var_safe(opt_buf, sizeof(opt_buf), COAP_OBSERVE_CANCEL), opt_buf);
            if (coap_send(session, pdu) == COAP_INVALID_MID) {
                ESP_LOGE(TAG, "Failed to send CoAP de-register request");
            }
            // Wait a moment for de-registration to be processed
            coap_io_process(ctx, 2000);
        }
        observation_active = false;
    }


finish_client_obs:
    if (session) coap_free_session(session);
    if (ctx) coap_free_context(ctx);
    coap_cleanup();
    ESP_LOGI(TAG, "Client task finished.");
    vTaskDelete(NULL);
}

void app_main(void)
{
    ESP_ERROR_CHECK(nvs_flash_init());
    wifi_init_sta();
    ESP_LOGI(TAG, "Target CoAP Server IP: %s", COAP_SERVER_IP_ADDR);
    xTaskCreate(coap_client_task, "coap_client_task", 8192, NULL, 5, NULL);
}

3. Build/Flash/Observe:

  1. Ensure the server (Example 1) is running.
  2. Update COAP_SERVER_IP_ADDR in coap_client_observe_main.c.
  3. Build and flash the client firmware.
  4. Observe the client’s serial monitor. It will:
    • Send a GET request with Observe=0.
    • Receive an initial response.
    • Receive periodic notifications with updated uptime and incrementing Observe sequence numbers.
    • After about 30 seconds, it will attempt to de-register by sending GET with Observe=1.

Variant Notes

  • ESP32, ESP32-S2, ESP32-S3, ESP32-C3, ESP32-C6 (Wi-Fi variants):
    • All these variants are well-suited for implementing both CoAP Observe clients and servers.
    • The primary consideration is the number of concurrent observation relationships a server can maintain, which depends on available RAM for storing observer information (session details, last sequence number, etc.). libcoap manages this, but resource limits exist.
    • For clients, handling notifications is generally lightweight.
  • ESP32-H2 (802.15.4/Thread/Zigbee & Bluetooth LE):
    • The Observe pattern is highly relevant for CoAP over Thread/6LoWPAN, as it minimizes traffic on these low-power, low-bandwidth networks.
    • The libcoap mechanisms remain the same. Network configuration for Thread would be different (as covered in relevant volumes).
    • Managing sleep cycles on battery-powered observer clients in Thread networks requires careful coordination with notification schedules or using techniques like sleepy end device polling for CON notifications if applicable. NON notifications might be missed if the device is asleep.
  • Notification Types (CON vs. NON):
    • The server example uses COAP_RESOURCE_FLAGS_NOTIFY_CON to indicate that notifications should be Confirmable. This can be changed (e.g. to COAP_RESOURCE_FLAGS_NOTIFY_NON if it existed, or by modifying resource flags directly if libcoap supports it, though typically it’s a resource-wide setting).
    • CON notifications increase reliability but also traffic and server load (tracking ACKs). NON notifications are lighter but risk loss. The choice depends on the application’s requirements for data freshness and reliability. libcoap defaults to CON notifications for observed resources.

Common Mistakes & Troubleshooting Tips

Mistake / Issue Symptom(s) Troubleshooting / Solution
Server Not Marking Resource as Observable Client attempts to observe (sends GET with Observe:0), but server does not include Observe option in response, or client never receives notifications. Server might respond with a normal GET response without establishing observation. Fix: Ensure coap_resource_set_get_observable(resource, 1); is called during resource initialization on the server for the specific resource. Verify the resource flags are set correctly.
Server Not Triggering Notifications Resource state changes on the server, but clients do not receive notifications. Initial observation registration might succeed. Fix:
  • Ensure coap_resource_set_dirty(resource, NULL); (or coap_touch_resource(resource);) is called after the resource’s underlying data has changed.
  • Verify that the main CoAP processing loop (e.g., while(1) { coap_io_process(ctx, timeout); }) is running continuously and frequently enough in the server task.
  • Check if the resource handler (e.g., time_get_handler) is correctly providing the updated data when libcoap prepares the notification.
Client Not Handling Observe Sequence Numbers Correctly Client receives notifications but may misinterpret them, process old data, or fail to detect lost/duplicate updates. May appear as erratic data updates. Fix: Implement robust logic in the client’s response handler to:
  • Store the last received Observe sequence number for each observation.
  • Compare new sequence numbers with the stored one, accounting for wrap-around as per RFC 7641.
  • Correctly handle out-of-order, duplicate, or lost notifications. (See “Observe Sequence Number Handling” table).
Observation Silently Stops Client stops receiving notifications. If NON notifications are used, server is unaware. If CON, server might eventually de-register after timeouts. Client may not realize the observation is broken. Fix:
  • Server (CON): Relies on CON retransmission timeouts to clean up dead observers.
  • Client: Implement an application-level timeout. If no notification is received for an expected period (e.g., greater than resource’s Max-Age or a custom timeout), attempt to re-register (send GET with Observe:0). Log re-registration attempts. libcoap might have settings for automatic re-registration (e.g., coap_context_set_observe_max_age).
Incorrect De-registration Client attempts to de-register, but observation continues, or server logs errors. Client might keep receiving notifications. Fix:
  • Ensure client sends a GET request (not POST/PUT) for the target resource.
  • The GET request must include the Observe option with value 1 (COAP_OBSERVE_CANCEL).
  • Verify server correctly parses the Observe:1 option and removes the client from the observer list.
  • Alternatively, client can send RST in reply to a CON notification.
Firewall/NAT Issues Blocking Notifications Initial GET request from client to server might work, but subsequent unsolicited notifications from server to client are blocked. Fix:
  • Ensure any stateful firewalls or NAT devices correctly map/allow incoming UDP packets from the server to the client’s ephemeral port used for the CoAP session.
  • Consider using CON notifications, as the client’s ACK might help keep NAT bindings alive.
  • For long-lived observations across NATs, techniques like CoAP over TCP/TLS or application-layer keep-alives might be needed (though Observe is primarily UDP).
Mismatched CoAP Library Versions or Configurations Subtle issues in Observe behavior, especially around sequence numbers, retransmissions, or option handling. Fix:
  • Ensure client and server are using compatible versions of libcoap or other CoAP stacks.
  • Check default timeouts and retransmission parameters (e.g., ACK_TIMEOUT, MAX_RETRANSMIT) on both client and server.
  • Enable detailed logging in libcoap on both ends to trace PDU exchanges.

Exercises

  1. Observable Counter Resource:
    • Modify the server (Example 1). Create a new resource /counter.
    • This resource should have an integer value that increments every time a PUT request is made to a different resource, say /increment-trigger.
    • Make /counter observable. When /increment-trigger receives a PUT, the server should increment the counter and notify observers of /counter.
    • Test with the client (Example 2) observing /counter and another CoAP client sending PUTs to /increment-trigger.
  2. Client-Side MAX_AGE Handling:
    • Modify the client (Example 2). Assume the /time resource has a Max-Age of 15 seconds (the server doesn’t need to explicitly send this option for this exercise, just assume it).
    • If the client doesn’t receive a notification for /time within, say, 20 seconds (allowing some leeway), it should automatically attempt to re-register its observation by sending a new GET request with Observe: 0. Log these re-registration attempts.
  3. NON vs. CON Notifications:
    • Research how libcoap allows specifying whether notifications for an observed resource should be Confirmable (CON) or Non-confirmable (NON). (Hint: It’s often related to resource flags set during initialization, e.g., COAP_RESOURCE_FLAGS_NOTIFY_CON vs. potentially other flags or session settings if available).
    • Modify the server (Example 1) to send NON notifications for the /time resource if possible with esp-coap‘s current libcoap binding.
    • Observe the difference in network traffic (e.g., lack of ACKs from client to server for notifications) using a tool like Wireshark if your setup allows, or infer from client/server behavior. Discuss the trade-offs.

Summary

  • The CoAP Observe pattern (RFC 7641) enables servers to notify clients about resource state changes, avoiding inefficient polling.
  • Clients register interest using a GET request with the Observe: 0 option.
  • Servers acknowledge registration and send subsequent updates (notifications) as CoAP responses containing an incrementing Observe sequence number.
  • Notifications can be Confirmable (CON) for reliability or Non-confirmable (NON) for lower overhead.
  • Clients de-register using Observe: 1 in a GET request or by sending RST to a CON notification. Servers can also implicitly de-register clients.
  • Observe sequence numbers help detect lost or reordered notifications. ETags can optimize re-registration.
  • libcoap (via esp-coap) provides functions like coap_resource_set_get_observable() and coap_resource_set_dirty() (or coap_touch_resource()) to simplify server-side implementation, and manages much of the client-side observation lifecycle.
Aspect Confirmable (CON) Notifications Non-confirmable (NON) Notifications
Reliability Higher. Server expects an Acknowledgement (ACK) from the client. If no ACK is received, the server retransmits the notification. Lower. Server sends the notification (“fire and forget”) and does not expect an ACK. Notifications can be lost without the server knowing.
Overhead Higher. Requires ACKs from client to server, increasing network traffic and processing load on both ends. Server needs to track ACKs and manage retransmissions. Lower. No ACKs mean less traffic and less state to manage on the server for individual notifications.
Client Liveness Detection Better. If a server repeatedly fails to get an ACK for CON notifications, it can assume the client is offline and de-register the observation. Poorer. Server has no direct feedback if client is receiving NON notifications. Observation might silently break.
Energy Consumption Higher for client (must send ACKs) and server (retransmissions, state tracking). Lower for client (no ACKs to send) and server (fewer retransmissions). More suitable for very constrained or battery-powered clients if occasional data loss is acceptable.
Use Case Examples Critical alerts, state synchronization where every update matters, commands. (libcoap default for observe) Frequently changing sensor data where some loss is tolerable (e.g., temperature updates every second), streaming-like data.
libcoap Flag (Server Resource) Typically COAP_RESOURCE_FLAGS_NOTIFY_CON (often the default). May require specific flag like COAP_RESOURCE_FLAGS_NOTIFY_NON if supported/available, or by modifying resource attributes.

Further Reading

Related Posts

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top