Leave a Comment / By /

Chapter 117: CoAP Block-wise Transfers

Chapter Objectives

By the end of this chapter, you will be able to:

  • Understand the need for block-wise transfers in CoAP.
  • Explain the CoAP Block1 and Block2 options and their roles.
  • Describe the negotiation process for block size and transfer.
  • Implement a CoAP server on an ESP32 capable of serving large resources using Block2.
  • Implement a CoAP client on an ESP32 capable of retrieving large resources using Block2.
  • Implement a CoAP client on an ESP32 capable of sending large data using Block1.
  • Implement a CoAP server on an ESP32 capable of receiving large data using Block1.
  • Recognize the implications of block-wise transfers on ESP32 resource usage.

Introduction

In Chapter 115, we introduced the CoAP protocol, and in Chapter 116, we explored resource discovery. CoAP is designed for constrained devices and networks, which often implies limitations on the size of datagrams that can be handled efficiently (e.g., fitting within a single MTU to avoid IP fragmentation). However, many IoT applications require the transfer of data larger than what a single CoAP message can comfortably carry – think of firmware updates, large sensor data logs, or configuration files.

CoAP addresses this challenge through block-wise transfers. This mechanism allows a large resource representation (for a GET response) or a large request payload (for PUT/POST) to be transferred as a sequence of smaller blocks. Each block fits within a single CoAP message. This chapter will delve into the theory and practical implementation of CoAP block-wise transfers on ESP32 devices using ESP-IDF and the esp-coap (libcoap) library.

Theory

The Problem: Large Data in Constrained Environments

CoAP messages are typically transported over UDP. UDP datagrams have a theoretical maximum size, but practical limits are often imposed by the underlying network’s Maximum Transmission Unit (MTU). For IPv6, the minimum MTU is 1280 bytes, and for IPv4, it’s common to see 1500 bytes on Ethernet or lower values on other link layers like 802.15.4 (127 bytes physical layer PDU, leading to much smaller IP MTUs after overhead).

Feature Single CoAP Message CoAP Block-wise Transfer
Data Size Small (fits within MTU) Large (exceeds MTU, e.g., firmware updates, logs)
IP Fragmentation Avoided (ideal) or occurs if payload exceeds MTU Avoids IP fragmentation by CoAP-level segmentation
Resource Strain Lower for small data Manages strain by processing data in manageable chunks
Complexity Simpler, single request/response More complex, involves state management and multiple exchanges
Reliability Handled by CON messages for the single PDU Each block transfer can use CON messages for reliability
Use Cases Sensor readings, simple commands, small configurations Firmware updates, large sensor data logs, configuration files

Sending large CoAP payloads in a single datagram can lead to:

  1. IP Fragmentation: If the UDP datagram exceeds the path MTU, IP fragmentation occurs. This is generally undesirable in constrained networks because it adds overhead, complexity, and increases the chance of packet loss (losing one fragment means losing the entire datagram). Some constrained network stacks might not even fully support IP fragmentation.
  2. Resource Strain: Constrained devices might not have enough RAM to buffer and process very large single messages.

Block-wise transfers provide a CoAP-level solution to fragment and reassemble data, avoiding IP fragmentation and allowing devices to process large payloads in manageable chunks.

CoAP Block Options

CoAP defines two main options for block-wise transfers, specified in RFC 7959:

  1. Block2 Option (for responses): Used in GET, POST, PUT, or FETCH requests to indicate interest in receiving a large response payload in blocks. It’s also used in the corresponding 2.xx (Success) or 4.xx/5.xx (Error) responses to transfer the blocks.
  2. Block1 Option (for requests): Used in PUT or POST requests to transfer a large request payload in blocks. It’s also used in the corresponding 2.xx responses to acknowledge received blocks and potentially request the next ones.

Additionally, there are size options:

  • Size2 Option: Used in a request by a client to indicate the total expected size of the resource representation it is about to request using Block2. Used in a response by a server to indicate the total size of the resource being transferred.
  • Size1 Option: Used in a request by a client to indicate the total size of the payload it is about to send using Block1. Used in a response by a server to acknowledge the total size.
Option Name Purpose Usage RFC
Block2 Indicates interest in/transfers large response payloads in blocks. Used in GET/POST/PUT/FETCH requests (client) and 2.xx/4.xx/5.xx responses (server). RFC 7959
Block1 Transfers large request payloads in blocks. Used in PUT/POST requests (client) and 2.xx responses (server for acknowledgment). RFC 7959
Size2 Indicates the total expected/actual size of a resource representation (response). Used by client in request (expected), by server in response (actual). RFC 7959
Size1 Indicates the total expected/actual size of a request payload. Used by client in request (actual), by server in response (acknowledgment). RFC 7959

How Block-wise Transfers Work

Both Block1 and Block2 options share a similar structure and operational principles. The option value consists of three parts:

  • NUM (Block Number): A variable-length integer (0-1,048,575) indicating the sequence number of the block, starting from 0.
  • M (More Bit): A single bit indicating whether more blocks follow (M=1) or if this is the last block (M=0).
  • SZX (Block Size): A 3-bit integer indicating the size of the block. The actual size is 2^(SZX + 4) bytes (e.g., SZX=0 means 16 bytes, SZX=6 means 1024 bytes). The maximum SZX is 6 (1024 bytes). SZX=7 is reserved.

Negotiation:

The client and server can negotiate the block size. A client might request a certain block size, but the server can choose a smaller size if it prefers (e.g., due to its own buffer limitations). The chosen size must be consistent throughout a single block-wise transfer sequence.

Block2 Operation (Server Sending, Client Receiving Large Data)

  1. Client Request:
    • The client sends a GET request for a resource.
    • It can include a Block2 option with NUM=0 and a desired SZX. It might also include a Size2 option if it knows the expected size.
    • Example: GET /large-resource Block2: NUM=0, M=0, SZX=6 (requesting first block, 1024 bytes).
  2. Server Response (First Block):
    • The server receives the request. If the resource is large and it supports block-wise transfer, it responds with the first block.
    • The response (e.g., 2.05 Content) includes a Block2 option indicating the block number (NUM=0), whether more blocks follow (M=1 if not the last block), and the SZX it has chosen (which might be smaller than the client requested).
    • It may also include a Size2 option indicating the total size of the resource.
    • Example: 2.05 Content Block2: NUM=0, M=1, SZX=5 (512 bytes) Size2: 8000 Payload: [first 512 bytes]
  3. Client Requests Subsequent Blocks:
    • The client receives the first block. If the M bit is 1, it knows more blocks are coming.
    • It requests the next block by sending another GET request for the same resource URI, but with an updated Block2 option: NUM incremented, M=0 (client isn’t sending more blocks, just requesting), and the SZX confirmed by the server.
    • Example: GET /large-resource Block2: NUM=1, M=0, SZX=5
  4. Server Responds with Subsequent Blocks:
    • The server sends the requested block with updated NUM and M bit.
    • Example: 2.05 Content Block2: NUM=1, M=1, SZX=5 Payload: [next 512 bytes]
  5. Final Block:
    • This continues until the server sends the last block, which will have M=0.
    • Example: 2.05 Content Block2: NUM=15, M=0, SZX=5 Payload: [last 352 bytes of 8000] (assuming 8000 bytes total, 15*512 + 352)
graph TD
    %% Styling for nodes
    classDef startNode fill:#EDE9FE,stroke:#5B21B6,stroke-width:2px,color:#5B21B6;
    classDef endNode fill:#D1FAE5,stroke:#059669,stroke-width:2px,color:#065F46;
    classDef processNode fill:#DBEAFE,stroke:#2563EB,stroke-width:1px,color:#1E40AF;
    classDef decisionNode fill:#FEF3C7,stroke:#D97706,stroke-width:1px,color:#92400E;
    classDef checkNode fill:#FEE2E2,stroke:#DC2626,stroke-width:1px,color:#991B1B;

    subgraph Client
        C1[Client: <br>Initiate GET]:::startNode
        C2{Client: <br>Receive Block?}:::decisionNode
        C3[Client: <br>Assemble Data]:::processNode
        C4[Client: <br>Transfer Complete]:::endNode
    end

    subgraph Server
        S1[Server: <br>Resource Available]:::startNode
        S2{Server: <br>Receive GET<br>with Block2?}:::decisionNode
        S3[Server: <br>Send Block]:::processNode
        S4[Server: <br>All Blocks Sent]:::endNode
    end

    C1 -- GET /resource <br>Block2: NUM=0, SZX=? --> S2
    S2 -- Yes --> S3
    S3 -- 2.05 Content <br>Block2: NUM=0, M=1, SZX=? <br>+ Payload --> C2
    C2 -- Yes, M=1 --> C3
    C3 -- Request Next Block --> C1
    S3 -- If Last Block (M=0) --> S4
    C2 -- No, M=0 --> C4
    C2 -- Error --> C4
    S2 -- No (Error/Invalid) --> S4

Block1 Operation (Client Sending, Server Receiving Large Data)

  1. Client Request (First Block):
    • The client sends a PUT or POST request.
    • It includes the first chunk of data in the payload.
    • It includes a Block1 option with NUM=0, M=1 (if more blocks follow), and a chosen SZX.
    • It may also include a Size1 option indicating the total size of the request payload.
    • Example: PUT /large-data Block1: NUM=0, M=1, SZX=4 (256 bytes) Size1: 2000 Payload: [first 256 bytes]
  2. Server Response (Acknowledgement):
    • The server receives the first block.
    • It responds with 2.31 Continue if it’s ready for the next block. This response includes a Block1 option acknowledging the received block (NUM, M=1 if it wants more, SZX matching client or smaller if negotiated).
    • If the server has already received the full payload (e.g., if M was 0 on the client’s first block and the server accepts it), it might respond with 2.01 Created or 2.04 Changed.
    • Example: 2.31 Continue Block1: NUM=0, M=1, SZX=4
  3. Client Sends Subsequent Blocks:
    • The client receives the 2.31 Continue.
    • It sends the next block with an incremented NUM in the Block1 option.
    • Example: PUT /large-data Block1: NUM=1, M=1, SZX=4 Payload: [next 256 bytes]
  4. Server Acknowledges Subsequent Blocks:
    • The server continues to respond with 2.31 Continue and the acknowledged Block1 option.
  5. Final Block and Server Final Response:
    • The client sends the last block with M=0.
    • Example: PUT /large-data Block1: NUM=7, M=0, SZX=4 Payload: [last 208 bytes of 2000]
    • The server, upon receiving the final block, processes the complete payload and sends a final success/failure response (e.g., 2.01 Created, 2.04 Changed). This response should include a Block1 option echoing the client’s final block (NUM, M=0, SZX).
    • Example: 2.04 Changed Block1: NUM=7, M=0, SZX=4
graph TD
    %% Styling for nodes
    classDef startNode fill:#EDE9FE,stroke:#5B21B6,stroke-width:2px,color:#5B21B6;
    classDef endNode fill:#D1FAE5,stroke:#059669,stroke-width:2px,color:#065F46;
    classDef processNode fill:#DBEAFE,stroke:#2563EB,stroke-width:1px,color:#1E40AF;
    classDef decisionNode fill:#FEF3C7,stroke:#D97706,stroke-width:1px,color:#92400E;
    classDef checkNode fill:#FEE2E2,stroke:#DC2626,stroke-width:1px,color:#991B1B;

    subgraph Client
        C1[Client: <br>Initiate PUT/POST]:::startNode
        C2{Client: <br>More Blocks to Send?}:::decisionNode
        C3[Client: <br>Send Next Block]:::processNode
        C4[Client: <br>Receive Final Response]:::endNode
    end

    subgraph Server
        S1[Server: <br>Ready to Receive]:::startNode
        S2{Server: <br>Receive Block<br>with Block1?}:::decisionNode
        S3[Server: <br>Assemble Data]:::processNode
        S4[Server: <br>Send Acknowledgement]:::processNode
        S5[Server: <br>Process Complete Payload]:::endNode
    end

    C1 -- PUT/POST /data <br>Block1: NUM=0, M=1, SZX=? <br>+ Payload --> S2
    S2 -- Yes --> S3
    S3 -- Data Appended --> S4
    S4 -- 2.31 Continue <br>Block1: NUM=0, M=1, SZX=? --> C2
    C2 -- Yes --> C3
    C3 -- PUT/POST /data <br>Block1: NUM=N, M=?, SZX=? <br>+ Payload --> S2
    S2 -- If M=0 (Last Block) --> S5
    S5 -- 2.01 Created / 2.04 Changed <br>Block1: NUM=N, M=0, SZX=? --> C4
    C2 -- No (Last Block Sent) --> C4
    S2 -- No (Error/Invalid) --> S5

Key Considerations:

  • Reliability: CoAP’s CON (Confirmable) messages ensure reliability for each block transfer.
  • Statelessness vs. Statefulness: While CoAP aims for stateless interactions, block-wise transfers inherently introduce state (e.g., current block number, total size) that both client and server must manage for the duration of the transfer.
  • Timeouts: If a block or its acknowledgment is lost, CoAP retransmission mechanisms apply. Prolonged failures can lead to the transfer being aborted.
  • libcoap Handling: The libcoap library (used by esp-coap) handles much of the low-level mechanics of block-wise transfers, such as adding Block1/Block2 options and managing retransmissions. However, the application is still responsible for providing (for sending) or consuming (for receiving) the actual data, potentially in chunks if it’s too large to fit in RAM all at once.

Practical Examples

We will use the esp-coap component. Ensure your ESP-IDF project is configured to include this component and Wi-Fi is set up. The Wi-Fi initialization code (wifi_init_sta) will be similar to Chapter 116 and is included here for completeness.

Example 1: ESP32 CoAP Server – Serving a Large Resource (Block2)

This server will offer a resource /large-text that contains a long string, demonstrating Block2 transfers.

1. Project Setup:

Create/use an ESP-IDF project, configure Wi-Fi and esp-coap.

2. main/coap_server_blockwise_main.c:

C
#include <string.h>
#include <sys/socket.h>
#include <netdb.h>
#include <freertos/FreeRTOS.h>
#include <freertos/task.h>
#include <freertos/event_groups.h>
#include "esp_log.h"
#include "nvs_flash.h"
#include "esp_event.h"
#include "esp_wifi.h"
#include "esp_coap.h"

static const char *TAG = "COAP_SERVER_BLK";

// A large string resource (approx 1.5KB to demonstrate multiple blocks)
const char *large_resource_data =
    "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. " // 119
    "Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor " // 120
    "in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, " // 120
    "sunt in culpa qui officia deserunt mollit anim id est laborum. Donec quam felis, ultricies nec, pellentesque eu, pretium quis, sem. " // 120
    "Nulla consequat massa quis enim. Donec pede justo, fringilla vel, aliquet nec, vulputate eget, arcu. In enim justo, rhoncus ut, " // 119
    "imperdiet a, venenatis vitae, justo. Nullam dictum felis eu pede mollis pretium. Integer tincidunt. Cras dapibus. Vivamus elementum " // 120
    "semper nisi. Aenean vulputate eleifend tellus. Aenean leo ligula, porttitor eu, consequat vitae, eleifend ac, enim. Aliquam lorem ante, " // 120
    "dapibus in, viverra quis, feugiat a, tellus. Phasellus viverra nulla ut metus varius laoreet. Quisque rutrum. Aenean imperdiet. " // 119
    "Etiam ultricies nisi vel augue. Curabitur ullamcorper ultricies nisi. Nam eget dui. Etiam rhoncus. Maecenas tempus, tellus eget " // 119
    "condimentum rhoncus, sem quam semper libero, sit amet adipiscing sem neque sed ipsum. Nam quam nunc, blandit vel, luctus pulvinar, " // 120
    "hendrerit id, lorem. Maecenas nec odio et ante tincidunt tempus. Donec vitae sapien ut libero venenatis faucibus. Nullam quis ante. " // 120
    "Etiam sit amet orci eget eros faucibus tincidunt. Duis leo. Sed fringilla mauris sit amet nibh. Donec sodales sagittis magna. " // 119
    "Sed consequat, leo eget bibendum sodales, augue velit cursus nunc, quis gravida magna mi a libero. Fusce vulputate eleifend sapien. " // 120
    "End of large data."; // 20
    // Total ~1455 chars

static size_t large_resource_len = 0; // Will be set by strlen

/* --- Resource Handler for /large-text (GET with Block2) --- */
static void large_text_get_handler(coap_resource_t *resource,
                                   coap_session_t *session,
                                   const coap_pdu_t *request,
                                   const coap_string_t *query,
                                   coap_pdu_t *response)
{
    // libcoap handles Block2 negotiation and providing data in chunks via this handler.
    // We just need to provide the total data and its length.
    // coap_add_data_large_response() will take care of segmenting it.

    coap_pdu_set_code(response, COAP_RESPONSE_CODE_CONTENT);
    // No need to explicitly set Content-Format if it's text/plain (default for coap_add_data_large_response)
    // or you can set it via coap_insert_option(response, COAP_OPTION_CONTENT_FORMAT, ...)

    // The key function for block-wise response is coap_add_data_large_response.
    // It uses the request's Block2 option (if any) to determine which chunk to send.
    // It also correctly sets the Block2 option in the response.
    int res = coap_add_data_large_response(resource, session, request, response, query,
                                           COAP_MEDIATYPE_TEXT_PLAIN, // Content-Type
                                           large_resource_len,
                                           (const uint8_t *)large_resource_data,
                                           NULL, NULL); // No etag, no token

    if (res == 0) {
        ESP_LOGE(TAG, "Failed to add large data to response for /large-text");
        // coap_add_data_large_response might fail if, for example, the requested block number is out of range.
        // libcoap usually handles this by sending an error PDU itself, but good to be aware.
        // If this handler is called, it implies libcoap expects us to provide *some* data for a block.
    } else {
        // ESP_LOGI(TAG, "Sent a block for /large-text");
    }
}


/* --- Wi-Fi Setup (identical to Chapter 116) --- */
#define EXAMPLE_ESP_WIFI_SSID      CONFIG_ESP_WIFI_SSID
#define EXAMPLE_ESP_WIFI_PASS      CONFIG_ESP_WIFI_PASSWORD

static EventGroupHandle_t wifi_event_group;
const int WIFI_CONNECTED_BIT = BIT0;

static void wifi_event_handler(void* arg, esp_event_base_t event_base,
                               int32_t event_id, void* event_data) {
    if (event_base == WIFI_EVENT && event_id == WIFI_EVENT_STA_START) {
        esp_wifi_connect();
    } else if (event_base == WIFI_EVENT && event_id == WIFI_EVENT_STA_DISCONNECTED) {
        ESP_LOGI(TAG, "Disconnected. Connecting to the AP again...");
        esp_wifi_connect();
    } else if (event_base == IP_EVENT && event_id == IP_EVENT_STA_GOT_IP) {
        ip_event_got_ip_t* event = (ip_event_got_ip_t*) event_data;
        ESP_LOGI(TAG, "Got IP:" IPSTR, IP2STR(&event->ip_info.ip));
        xEventGroupSetBits(wifi_event_group, WIFI_CONNECTED_BIT);
    }
}

void wifi_init_sta(void) {
    wifi_event_group = xEventGroupCreate();
    ESP_ERROR_CHECK(esp_netif_init());
    ESP_ERROR_CHECK(esp_event_loop_create_default());
    esp_netif_create_default_wifi_sta();
    wifi_init_config_t cfg = WIFI_INIT_CONFIG_DEFAULT();
    ESP_ERROR_CHECK(esp_wifi_init(&cfg));
    esp_event_handler_instance_t instance_any_id;
    esp_event_handler_instance_t instance_got_ip;
    ESP_ERROR_CHECK(esp_event_handler_instance_register(WIFI_EVENT, ESP_EVENT_ANY_ID, &wifi_event_handler, NULL, &instance_any_id));
    ESP_ERROR_CHECK(esp_event_handler_instance_register(IP_EVENT, IP_EVENT_STA_GOT_IP, &wifi_event_handler, NULL, &instance_got_ip));
    wifi_config_t wifi_config = { .sta = { .ssid = EXAMPLE_ESP_WIFI_SSID, .password = EXAMPLE_ESP_WIFI_PASS, .threshold.authmode = WIFI_AUTH_WPA2_PSK }};
    ESP_ERROR_CHECK(esp_wifi_set_mode(WIFI_MODE_STA) );
    ESP_ERROR_CHECK(esp_wifi_set_config(WIFI_IF_STA, &wifi_config) );
    ESP_ERROR_CHECK(esp_wifi_start() );
    ESP_LOGI(TAG, "wifi_init_sta finished.");
    xEventGroupWaitBits(wifi_event_group, WIFI_CONNECTED_BIT, pdFALSE, pdFALSE, portMAX_DELAY);
    ESP_LOGI(TAG, "Connected to ap SSID:%s", EXAMPLE_ESP_WIFI_SSID);
}

/* --- CoAP Server Task --- */
static void coap_server_task(void *pvParameters)
{
    coap_context_t *ctx = NULL;
    coap_address_t serv_addr;
    coap_resource_t *res_large_text = NULL;

    large_resource_len = strlen(large_resource_data); // Calculate actual length
    ESP_LOGI(TAG, "Large resource size: %d bytes", large_resource_len);

    xEventGroupWaitBits(wifi_event_group, WIFI_CONNECTED_BIT, pdFALSE, pdTRUE, portMAX_DELAY);
    ESP_LOGI(TAG, "Wi-Fi Connected. Starting CoAP server...");

    coap_startup();
    coap_address_init(&serv_addr);
    serv_addr.addr.sin.sin_family      = AF_INET;
    serv_addr.addr.sin.sin_addr.s_addr = INADDR_ANY;
    serv_addr.addr.sin.sin_port        = htons(COAP_DEFAULT_PORT);

    ctx = coap_new_context(&serv_addr);
    if (!ctx) {
        ESP_LOGE(TAG, "Failed to create CoAP context");
        goto finish_server;
    }
    coap_context_set_keepalive(ctx, COAP_DEFAULT_KEEPALIVE);

    // Create /large-text resource
    res_large_text = coap_resource_init(coap_make_str_const("large-text"), COAP_RESOURCE_FLAGS_NOTIFY_CON); // Use CON for notifications if observable
    if (!res_large_text) {
        ESP_LOGE(TAG, "Failed to create /large-text resource");
        goto finish_server;
    }
    coap_register_handler(res_large_text, COAP_REQUEST_GET, large_text_get_handler);
    // Add attributes if needed, e.g., rt, if
    // coap_add_attr(res_large_text, coap_make_str_const("rt"), coap_make_str_const("large.text.payload"), 0);
    coap_resource_set_get_observable(res_large_text, 1); // Make it observable for potential future use
    coap_add_resource(ctx, res_large_text);

    ESP_LOGI(TAG, "CoAP server started on port %d", COAP_DEFAULT_PORT);

    while (1) {
        int result = coap_io_process(ctx, COAP_IO_WAIT);
        if (result < 0) {
            ESP_LOGE(TAG, "coap_io_process error: %d", result);
            break;
        }
    }

finish_server:
    if (res_large_text) coap_delete_resource(ctx, res_large_text); // Important to free resources
    if (ctx) coap_free_context(ctx);
    coap_cleanup();
    vTaskDelete(NULL);
}

void app_main(void)
{
    ESP_ERROR_CHECK(nvs_flash_init());
    wifi_init_sta();
    xTaskCreate(coap_server_task, "coap_server_task", 8192, NULL, 5, NULL);
}

3. Build Instructions:

Bash
idf.py set-target esp32 # or other Wi-Fi variant
idf.py menuconfig # Configure WiFi credentials
idf.py build

4. Run/Flash/Observe:

  1. Flash the firmware: idf.py -p /dev/ttyUSB0 flash monitor.
  2. Note the ESP32’s IP address.
  3. Use a CoAP client tool that supports block-wise transfers. libcoap-cellar/coap-client is good for this.coap-client -m get coap://<ESP32_IP_ADDRESS>/large-text -B 10 # Try to get it in 10 seconds, -s for block size # Example with specific block size request (e.g., 64 bytes, SZX=2) # coap-client -m get coap://<ESP32_IP_ADDRESS>/large-text -s 2
    The client should automatically negotiate and retrieve the resource in multiple blocks. You’ll see the full “Lorem ipsum…” text assembled.Observe the server logs; you might see multiple calls to large_text_get_handler or internal libcoap logs indicating block transfers if debug level is high.coap_add_data_large_response is designed to be called once per incoming request for a block. libcoap internally tracks which block is being requested based on the Block2 option in the PDU.

Tip: The coap_add_data_large_response() function in libcoap is smart. When a client requests a resource that you’ve registered with this handler, and the client uses Block2 options, libcoap will call your handler for each block request. Your handler’s job is to provide the entire data pointer and length. libcoap then picks out the correct segment based on the Block2 NUM and SZX from the client’s request and sends just that segment, adding the correct Block2 option to the response.

Example 2: ESP32 CoAP Client – Retrieving a Large Resource (Block2)

This client will connect to the server from Example 1 and fetch /large-text.

1. Project Setup:

Similar to the server.

2. main/coap_client_blockwise_main.c:

C
#include <string.h>
#include <sys/socket.h>
#include <netdb.h>
#include <freertos/FreeRTOS.h>
#include <freertos/task.h>
#include <freertos/event_groups.h>
#include "esp_log.h"
#include "nvs_flash.h"
#include "esp_event.h"
#include "esp_wifi.h"
#include "esp_coap.h"

static const char *TAG = "COAP_CLIENT_BLK";

#define COAP_SERVER_IP_ADDR    "192.168.1.100" // CHANGE THIS to your server's IP
#define COAP_DEFAULT_PORT_STR  "5683"
#define COAP_TARGET_URI        "/large-text"

// Buffer to assemble the large response
#define MAX_RESPONSE_SIZE 2048 // Adjust if your resource is larger
static uint8_t response_buffer[MAX_RESPONSE_SIZE];
static size_t response_buffer_len = 0;
static bool transfer_complete = false;
static bool transfer_error = false;

/* --- Wi-Fi Setup (identical to server example) --- */
#define EXAMPLE_ESP_WIFI_SSID      CONFIG_ESP_WIFI_SSID
#define EXAMPLE_ESP_WIFI_PASS      CONFIG_ESP_WIFI_PASSWORD
static EventGroupHandle_t wifi_event_group;
const int WIFI_CONNECTED_BIT = BIT0;
// ... (wifi_event_handler and wifi_init_sta functions as in Example 1) ...
static void wifi_event_handler(void* arg, esp_event_base_t event_base,
                               int32_t event_id, void* event_data) {
    if (event_base == WIFI_EVENT && event_id == WIFI_EVENT_STA_START) {
        esp_wifi_connect();
    } else if (event_base == WIFI_EVENT && event_id == WIFI_EVENT_STA_DISCONNECTED) {
        ESP_LOGI(TAG, "Disconnected. Connecting to the AP again...");
        esp_wifi_connect();
    } else if (event_base == IP_EVENT && event_id == IP_EVENT_STA_GOT_IP) {
        ip_event_got_ip_t* event = (ip_event_got_ip_t*) event_data;
        ESP_LOGI(TAG, "Got IP:" IPSTR, IP2STR(&event->ip_info.ip));
        xEventGroupSetBits(wifi_event_group, WIFI_CONNECTED_BIT);
    }
}

void wifi_init_sta(void) {
    wifi_event_group = xEventGroupCreate();
    ESP_ERROR_CHECK(esp_netif_init());
    ESP_ERROR_CHECK(esp_event_loop_create_default());
    esp_netif_create_default_wifi_sta();
    wifi_init_config_t cfg = WIFI_INIT_CONFIG_DEFAULT();
    ESP_ERROR_CHECK(esp_wifi_init(&cfg));
    esp_event_handler_instance_t instance_any_id;
    esp_event_handler_instance_t instance_got_ip;
    ESP_ERROR_CHECK(esp_event_handler_instance_register(WIFI_EVENT, ESP_EVENT_ANY_ID, &wifi_event_handler, NULL, &instance_any_id));
    ESP_ERROR_CHECK(esp_event_handler_instance_register(IP_EVENT, IP_EVENT_STA_GOT_IP, &wifi_event_handler, NULL, &instance_got_ip));
    wifi_config_t wifi_config = { .sta = { .ssid = EXAMPLE_ESP_WIFI_SSID, .password = EXAMPLE_ESP_WIFI_PASS, .threshold.authmode = WIFI_AUTH_WPA2_PSK }};
    ESP_ERROR_CHECK(esp_wifi_set_mode(WIFI_MODE_STA) );
    ESP_ERROR_CHECK(esp_wifi_set_config(WIFI_IF_STA, &wifi_config) );
    ESP_ERROR_CHECK(esp_wifi_start() );
    ESP_LOGI(TAG, "wifi_init_sta finished.");
    xEventGroupWaitBits(wifi_event_group, WIFI_CONNECTED_BIT, pdFALSE, pdFALSE, portMAX_DELAY);
    ESP_LOGI(TAG, "Connected to ap SSID:%s", EXAMPLE_ESP_WIFI_SSID);
}


/* --- CoAP Response Handler for Block2 --- */
// This handler will be called for each block received.
static coap_response_handler_t block_response_handler(coap_session_t *session,
                                                 const coap_pdu_t *sent,
                                                 const coap_pdu_t *received,
                                                 const coap_mid_t mid)
{
    const uint8_t *data = NULL;
    size_t data_len = 0;
    coap_pdu_code_t rcv_code = coap_pdu_get_code(received);
    coap_opt_iterator_t opt_iter;
    coap_opt_t *block_opt;

    if (COAP_RESPONSE_CLASS(rcv_code) == 2) { // Success (e.g., 2.05 Content)
        // Check for Block2 option
        block_opt = coap_check_option(received, COAP_OPTION_BLOCK2, &opt_iter);
        if (block_opt) {
            uint16_t blk_szx = coap_opt_block_szx(block_opt);
            uint32_t blk_num = coap_opt_block_num(block_opt);
            uint8_t blk_m = coap_opt_block_m(block_opt); // More bit

            ESP_LOGI(TAG, "Received block: NUM=%d, M=%d, SZX=%d (size %d)",
                     blk_num, blk_m, blk_szx, 1 << (blk_szx + 4));

            if (coap_get_data(received, &data_len, &data)) {
                if ((response_buffer_len + data_len) <= MAX_RESPONSE_SIZE) {
                    memcpy(response_buffer + response_buffer_len, data, data_len);
                    response_buffer_len += data_len;
                    ESP_LOGI(TAG, "Copied %d bytes to buffer, total %d bytes", data_len, response_buffer_len);
                } else {
                    ESP_LOGE(TAG, "Response buffer overflow!");
                    transfer_error = true;
                    transfer_complete = true; // Stop trying
                    return COAP_RESPONSE_OK; // Acknowledge this problematic block
                }
            }

            if (blk_m == 0) { // This was the last block
                ESP_LOGI(TAG, "All blocks received. Total size: %d", response_buffer_len);
                transfer_complete = true;
                // The complete data is in response_buffer
                printf("\n--- Assembled Response ---\n%.*s\n--------------------------\n",
                       (int)response_buffer_len, (char*)response_buffer);
            } else {
                // libcoap will automatically request the next block if M=1
                // by re-sending the original request with an updated Block2 option.
            }
        } else { // Not a block-wise transfer, or first block without Block2 (shouldn't happen if server is compliant)
            ESP_LOGI(TAG, "Received non-block response or first block.");
            if (coap_get_data(received, &data_len, &data)) {
                 if ((response_buffer_len + data_len) <= MAX_RESPONSE_SIZE) {
                    memcpy(response_buffer + response_buffer_len, data, data_len);
                    response_buffer_len += data_len;
                } else {
                    ESP_LOGE(TAG, "Response buffer overflow!");
                    transfer_error = true;
                }
            }
            transfer_complete = true; // Assume single PDU transfer is complete
             printf("\n--- Assembled Response (single) ---\n%.*s\n--------------------------\n",
                       (int)response_buffer_len, (char*)response_buffer);
        }
    } else { // Error response
        ESP_LOGE(TAG, "CoAP request failed (Code: %d.%02d)",
                 COAP_RESPONSE_GET_CLASS(rcv_code), COAP_RESPONSE_GET_CODE(rcv_code));
        transfer_error = true;
        transfer_complete = true;
    }
    return COAP_RESPONSE_OK; // Acknowledge the received PDU
}


/* --- CoAP Client Task --- */
static void coap_client_task(void *pvParameters)
{
    coap_context_t *ctx = NULL;
    coap_session_t *session = NULL;
    coap_address_t dst_addr;
    static coap_uri_t uri;
    coap_pdu_t *pdu = NULL;

    xEventGroupWaitBits(wifi_event_group, WIFI_CONNECTED_BIT, pdFALSE, pdTRUE, portMAX_DELAY);
    ESP_LOGI(TAG, "Wi-Fi Connected. Starting CoAP client...");

    coap_startup();

    struct addrinfo *res_addr, *ainfo;
    struct addrinfo hints;
    memset(&hints, 0, sizeof(hints));
    hints.ai_socktype = SOCK_DGRAM;
    hints.ai_family = AF_INET;
    if (getaddrinfo(COAP_SERVER_IP_ADDR, COAP_DEFAULT_PORT_STR, &hints, &res_addr) != 0) {
        ESP_LOGE(TAG, "DNS lookup failed for: %s", COAP_SERVER_IP_ADDR);
        goto finish_client;
    }
    ainfo = res_addr;
    coap_address_init(&dst_addr);
    memcpy(&dst_addr.addr.sin, ainfo->ai_addr, sizeof(dst_addr.addr.sin));
    dst_addr.addr.sin.sin_port = htons(atoi(COAP_DEFAULT_PORT_STR));
    freeaddrinfo(res_addr);

    ctx = coap_new_context(NULL);
    if (!ctx) {
        ESP_LOGE(TAG, "Failed to create CoAP context");
        goto finish_client;
    }
    // Register the custom response handler
    coap_register_response_handler(ctx, block_response_handler);

    session = coap_new_client_session(ctx, NULL, &dst_addr, COAP_PROTO_UDP);
    if (!session) {
        ESP_LOGE(TAG, "Failed to create CoAP client session");
        goto finish_client;
    }

    // Prepare PDU for GET request
    pdu = coap_new_pdu(COAP_MESSAGE_CON, COAP_REQUEST_CODE_GET, coap_new_message_id(session));
    if (!pdu) {
        ESP_LOGE(TAG, "Failed to create PDU");
        goto finish_client;
    }

    // Set URI path
    if (coap_string_to_uri((const uint8_t *)COAP_TARGET_URI, strlen(COAP_TARGET_URI), &uri) != 0) {
         ESP_LOGE(TAG, "Cannot parse URI: %s", COAP_TARGET_URI);
         coap_delete_pdu(pdu);
         goto finish_client;
    }
    if (uri.path.length) {
        coap_add_option(pdu, COAP_OPTION_URI_PATH, uri.path.length, uri.path.s);
    }

    // Optionally, suggest an initial Block2 size (e.g., SZX=4 for 256 bytes)
    // libcoap will handle requesting subsequent blocks if the server responds with M=1.
    // coap_block_t block = { .num = 0, .m = 0, .szx = 4 }; // num=0, m=0 (client not sending more), szx for preferred size
    // coap_add_option(pdu, COAP_OPTION_BLOCK2, coap_encode_var_safe(buffer, sizeof(buffer), block_value), buffer);
    // For simplicity, we let libcoap handle the initial Block2 option if it decides to.
    // Usually, for GET, you don't add Block2 to the *first* request unless you want a specific block from the start.
    // libcoap will add it to subsequent requests if the server indicates a block-wise transfer.

    ESP_LOGI(TAG, "Sending CoAP GET request to %s%s", COAP_SERVER_IP_ADDR, COAP_TARGET_URI);
    if (coap_send(session, pdu) == COAP_INVALID_MID) {
        ESP_LOGE(TAG, "Failed to send CoAP request");
        // pdu is freed by coap_send
    }

    // Wait for the transfer to complete or error
    TickType_t start_time = xTaskGetTickCount();
    while(!transfer_complete && (xTaskGetTickCount() - start_time < pdMS_TO_TICKS(30000))) { // 30s timeout
        int result = coap_io_process(ctx, 1000); // Process with 1s timeout for I/O
        if (result < 0) {
            ESP_LOGE(TAG, "coap_io_process error: %d", result);
            transfer_error = true;
            break;
        }
    }

    if (!transfer_complete && !transfer_error) {
        ESP_LOGW(TAG, "Transfer timed out.");
    } else if (transfer_error) {
        ESP_LOGE(TAG, "Transfer failed with an error.");
    } else {
        ESP_LOGI(TAG, "Transfer successful. Total data received: %d bytes.", response_buffer_len);
    }

finish_client:
    if (session) coap_free_session(session);
    if (ctx) coap_free_context(ctx);
    coap_cleanup();
    ESP_LOGI(TAG, "Client task finished.");
    vTaskDelete(NULL);
}


void app_main(void)
{
    ESP_ERROR_CHECK(nvs_flash_init());
    wifi_init_sta();
    ESP_LOGI(TAG, "Target CoAP Server IP: %s", COAP_SERVER_IP_ADDR);
    xTaskCreate(coap_client_task, "coap_client_task", 8192, NULL, 5, NULL);
}

3. Build/Flash/Observe:

  1. Ensure the server (Example 1) is running.
  2. Update COAP_SERVER_IP_ADDR in coap_client_blockwise_main.c.
  3. Build and flash the client.
  4. Observe the client’s serial monitor. It should connect, request /large-text, and print logs for each block received, finally printing the assembled large string.

Note on Block1 (Client Sending Large Data):

Implementing Block1 for sending data from the client (e.g., a large PUT request) follows a similar pattern but uses COAP_OPTION_BLOCK1.

  • The client would use coap_add_data_large_request() or manually construct PDUs with Block1 options and chunks of data.
  • The server’s PUT/POST handler would be called for each block. It would need to assemble these blocks. If the server’s handler uses coap_get_data_large(), libcoap can assist in reassembly.
  • The server responds with 2.31 Continue to request more blocks or a final 2.01/2.04 upon completion.

A full Block1 example would involve:

  • Client: Iteratively sending PDUs, each with a portion of the large data and the appropriate Block1 option (NUM, M, SZX). It would wait for 2.31 Continue from the server before sending the next block.
  • Server: The resource handler for PUT/POST would need to:
    1. Check for the Block1 option in the request.
    2. Get the data chunk using coap_get_data().
    3. Append it to a server-side buffer.
    4. If the M bit in the request’s Block1 option is 1, respond with 2.31 Continue, echoing the received Block1 option (NUM, M=1, SZX).
    5. If M=0, process the complete data and send a final response (e.g., 2.04 Changed), echoing the final Block1 option (NUM, M=0, SZX).

Due to the complexity of managing this state explicitly in the application for both client and server, and libcoap‘s higher-level functions often abstracting parts of it, a simplified example focusing on the concept is more practical for this chapter. libcoap itself has more comprehensive examples (e.g., coap-client -f file -m put ...).

For ESP32, if you use coap_new_pdu() and then coap_add_data() with a large chunk for a PUT/POST, libcoap will automatically attempt to use Block1 if the data exceeds a certain size relative to the session’s MTU or preferred block size. The server side, if using coap_resource_set_request_payload_handling(resource, COAP_PAYLOAD_HANDLER_DEFER_ALL), would get the full payload after libcoap reassembles it. For more direct control or handling data larger than RAM, a custom block handler approach is needed.

Variant Notes

  • ESP32, ESP32-S2, ESP32-S3, ESP32-C3, ESP32-C6 (Wi-Fi variants):
    • All these variants are capable of handling CoAP block-wise transfers. The main limiting factor is available RAM for buffering blocks, especially if the application needs to assemble the entire large payload in memory before processing.
    • libcoap‘s internal buffering and block management are efficient, but for extremely large transfers (e.g., multi-megabyte firmware updates), the application must implement a streaming approach, processing/storing each block as it arrives rather than buffering the whole thing.
  • ESP32-H2 (802.15.4/Thread/Zigbee & Bluetooth LE):
    • CoAP over 6LoWPAN/Thread heavily relies on block-wise transfers due to the much smaller MTUs of 802.15.4 networks. The principles and libcoap mechanisms are the same.
    • Effective block size (SZX) negotiation becomes even more critical to match the smaller underlying datagram capacities. Default SZX values might lead to blocks that are still too large for a single 6LoWPAN frame without further fragmentation at lower layers, which block-wise transfers aim to manage at the CoAP layer.
  • Memory Management:
    • When dealing with block-wise transfers, especially on the receiving end, be mindful of dynamic memory allocation if you are assembling the payload. Free buffers promptly after use.
    • For very large data, consider writing blocks directly to NVS/SPIFFS/SD card as they arrive, rather than trying to hold the entire object in RAM.

Common Mistakes & Troubleshooting Tips

Mistake / Issue Symptom(s) Troubleshooting / Solution
Buffer Overflow Data truncation, corrupted data, crashes (e.g., heap errors on ESP32). Ensure buffers are adequately sized (e.g., using Size1/Size2 options as hints). For very large data, implement block-by-block processing/storage (e.g., to NVS/SPIFFS/SD card) instead of full in-RAM reassembly.
Incorrect Block Option Handling Client/server gets stuck, retransmissions, incomplete transfers, or incorrect data. Rely on libcoap’s automatic handling via functions like coap_add_data_large_response() or coap_add_data_large_request(). If manual control is necessary, thoroughly test the state machine for NUM, M, and SZX.
Server Not Indicating M=0 on Final Block (Block2) Client keeps requesting next block, leading to timeouts or infinite loops. Verify server logic for setting the M bit. coap_add_data_large_response() handles this correctly if given the total data length.
Client Not Requesting Subsequent Blocks (Block2) Client receives first block but stops, transfer never completes. Ensure libcoap’s session is kept alive and coap_io_process() is regularly called. If custom logic, ensure loop correctly increments NUM and sends next request.
Timeout Issues Frequent retransmissions, transfers failing due to timeout errors. Adjust CoAP timing parameters (ACK_TIMEOUT, MAX_RETRANSMIT) via coap_session_set_ack_timeout(), coap_session_set_max_retransmit(). Choose appropriate block sizes (SZX) – smaller blocks for unreliable networks, larger for efficiency.

Exercises

  1. Implement Block1 PUT from Client to Server:
    • Modify the client (Example 2) to send a large string (similar to large_resource_data from Example 1) to a new resource /upload-text on the server using a PUT request and Block1 transfers.
    • Modify the server (Example 1) to add a handler for /upload-text that can receive this large string using Block1. The server should assemble the received blocks and log the complete string.
    • Hint (Client): You’ll need to create a PDU, add the Block1 option, add a chunk of data, and send. Repeat for all chunks, updating the Block1 NUM and M bit.
    • Hint (Server): The PUT handler will be called multiple times. Check the Block1 option, append data to a buffer, and respond 2.31 Continue or 2.04 Changed.
  2. Adjustable Block Size (SZX) for GET:
    • Modify the client (Example 2). Allow the user to specify a preferred SZX value (0-6) for the initial Block2 request via serial input or a Kconfig option.
    • Observe how the server (which might choose its own SZX) and client interact with different requested block sizes. Log the actual SZX used in the transfer.
  3. Simulate Limited Memory Reception:
    • Modify the client’s block_response_handler (Example 2). Instead of assembling the full response in response_buffer, simulate a device with very limited RAM (e.g., can only hold one block at a time).
    • As each block arrives, “process” it (e.g., print its content or calculate a checksum) and then discard it before the next block arrives (conceptually). Log the progress. This exercise highlights the benefit of not needing to buffer the entire payload.

Summary

  • CoAP block-wise transfers are essential for handling data larger than what fits in a single CoAP/UDP message, avoiding IP fragmentation.
  • Block2 Option: Used for transferring large server responses (e.g., GET results). The client requests blocks sequentially.
  • Block1 Option: Used for transferring large client requests (e.g., PUT/POST payloads). The client sends blocks sequentially, and the server acknowledges them.
  • Both options use NUM (block number), M (more bit), and SZX (block size exponent).
  • Size1 and Size2 options can indicate the total payload size.
  • libcoap (used by esp-coap) provides significant support for block-wise transfers, often handling the low-level option management and reassembly/fragmentation.
  • Functions like coap_add_data_large_response (server) simplify serving large resources in blocks. The client-side handling often involves a response handler that assembles incoming blocks.
  • Careful memory management and choice of block size are important on constrained devices like the ESP32.

Further Reading

Related Posts

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top