Chapter 158: J1939 Protocol Implementation with ESP-IDF

Chapter Objectives

By the end of this chapter, you will be able to:

• Understand the fundamentals of the SAE J1939 protocol and its application in heavy-duty vehicles.
• Describe the structure of J1939 messages, including the 29-bit CAN identifier, Parameter Group Numbers (PGNs), and Source Addresses (SAs).
• Configure the ESP32’s TWAI peripheral (or an external CAN controller) for J1939 communication (29-bit identifiers, standard baud rates like 250 kbps or 500 kbps).
• Implement functions to transmit and receive basic J1939 messages (single-frame PGNs).
• Parse received J1939 messages to extract PGN, SA, and data payload.
• Understand the basics of J1939 Transport Protocol (TP) for multi-packet messages (BAM and CM).
• Learn about the J1939 Address Claiming process and its importance.
• Recognize differences in implementing J1939 solutions across ESP32 variants.
• Identify common issues and troubleshooting techniques when working with ESP32 and J1939.

Introduction

While OBD-II (covered in Chapter 156) provides a standardized diagnostic interface primarily for passenger cars and light trucks, the world of heavy-duty vehicles (trucks, buses, agricultural machinery, marine engines) relies on a different, more comprehensive communication standard: SAE J1939. Developed by the Society of Automotive Engineers (SAE), J1939 defines a high-level communication protocol suite that runs on top of the Controller Area Network (CAN) physical layer, specifically CAN 2.0B utilizing 29-bit extended identifiers.

J1939 is crucial for these vehicles as it enables communication between various Electronic Control Units (ECUs) for engine control, transmission management, braking systems, instrument clusters, diagnostics, and much more. Understanding J1939 allows for the development of advanced telematics systems, diagnostic tools, data loggers, and custom interfaces for heavy-duty applications.

The ESP32, with its processing power, connectivity options, and (for most variants) a built-in TWAI (CAN) controller, serves as an excellent platform for creating J1939-interfacing devices. This chapter will guide you through the essentials of the J1939 protocol and demonstrate how to implement J1939 communication using ESP32 and ESP-IDF.

Theory

Overview of SAE J1939

SAE J1939 is a family of standards that define how ECUs (often called “controllers” or “nodes” in J1939 terminology) communicate information on a vehicle network. It’s built upon the CAN bus physical layer (typically CAN 2.0B with 29-bit identifiers) and specifies:

• Message Format: How data is structured within a CAN frame.
• Network Parameters: Standard baud rates (commonly 250 kbps, sometimes 500 kbps).
• Parameter Definitions: A vast set of standardized signals (Parameter Group Numbers or PGNs).
• Network Management: Procedures for address claiming, node identification, and managing communication between nodes.
• Transport Protocols: Mechanisms for sending messages larger than the 8-byte payload limit of a single CAN frame.
• Diagnostic Services: Similar in concept to OBD-II but tailored for heavy-duty systems (J1939-73).

J1939 Message Structure and 29-bit CAN Identifier

J1939 exclusively uses the 29-bit extended CAN identifier. This identifier is not just an arbitrary number; it’s structured to convey specific information about the message:

The 29-bit J1939 CAN ID is composed of the following fields:

• Priority (3 bits): Determines message priority during bus arbitration (0 is highest, 7 is lowest).
• Reserved (1 bit – formerly EDP): Originally Extended Data Page, now typically reserved and set to 0.
• Data Page (DP – 1 bit): Selects the page for PGN definition (0 or 1).
• PDU Format (PF – 8 bits): Defines the Parameter Group Number (PGN) if PF < 240 (PDU1 format, destination-specific). If PF >= 240 (PDU2 format, broadcast), this field, along with PS, forms the PGN.
• PDU Specific (PS – 8 bits):
  • If PF < 240 (PDU1, destination-specific): This field contains the Destination Address (DA) of the intended recipient ECU.
  • If PF >= 240 (PDU2, broadcast): This field is an extension of the PF to form the PGN. The message is broadcast to all nodes.
• Source Address (SA – 8 bits): Identifies the ECU that transmitted the message. Each ECU on the J1939 network must have a unique source address (0-253).

Protocol Data Unit (PDU): A J1939 message is referred to as a PDU.

• PDU1 Format (Destination Specific): When PF < 240. The message is addressed to a specific DA.
  • Example: A request for data from a specific ECU.
• PDU2 Format (Broadcast): When PF >= 240. The message is broadcast to all ECUs on the network.
  • Example: Engine speed broadcast by the engine controller.

Parameter Group Numbers (PGNs)

A Parameter Group Number (PGN) is a unique number that identifies a specific set of parameters or data being communicated. Think of it as a topic or subject of the message. Each PGN defines:

• The data parameters contained within the message.
• The length of each parameter.
• The scaling, offset, and units of each parameter.
• The transmission rate or conditions.

Calculating PGN from CAN ID fields:

• If PF < 240 (PDU1): PGN = (Reserved << 16) | (DP << 16) | (PF << 8) (PS field is DA, not part of PGN)
  • Often simplified as PGN = (DP << 16) | (PF << 8) since Reserved is usually 0.
• If PF >= 240 (PDU2): PGN = (Reserved << 16) | (DP << 16) | (PF << 8) | PS
  • Often simplified as PGN = (DP << 16) | (PF << 8) | PS.

SAE J1939-71 defines thousands of standard PGNs for various vehicle functions (e.g., engine speed, wheel speed, fuel level, transmission status).

Example PGNs (Illustrative):

The data bytes (up to 8 for a standard CAN frame) within a J1939 message are further broken down into Suspect Parameter Numbers (SPNs). Each SPN represents an individual piece of data (e.g., Engine RPM is an SPN within the EEC1 PGN).

Source Addresses (SA) and Address Claiming

Each ECU (Controller Application or CA) on a J1939 network must have a unique 8-bit Source Address (SA) in the range 0-253. Address 254 is the null address (cannot claim), and 255 is the global destination address (for broadcasts that need a DA field, like some TP messages).

Since addresses are not pre-assigned in all cases, J1939 includes an Address Claiming Procedure (J1939-81):

graph TD
    A[Start: ECU Powers Up] --> B{"Choose Preferred SA <br> (e.g., 0 for Engine)"};
    B --> C["Broadcast Address Claim Message (ACL) <br> PGN 60928, with own SA and 64-bit NAME"];
    C --> D{"Wait for Contention Period <br> (e.g., 250ms)"};
    D --> E{Another ACL for same SA Received?};
    E -- No --> F[Success! <br> Address Claimed. <br> Begin Normal Operation];
    E -- Yes --> G{Compare Received NAME <br> with own NAME};
    G -- "Our NAME is higher priority (lower value)" --> H[Defend Address: <br> Re-broadcast our ACL];
    H --> F;
    G -- "Our NAME is lower priority (higher value)" --> I[Contention Lost];
    I --> J{Choose a different SA <br> from configurable list};
    J -- Address Available --> B;
    J -- No Addresses Left --> K["Cannot Claim Address. <br> Operate at Null Address (254)"];
    K --> L[End: Listen-only mode or limited functionality];
    F --> M[End: Normal Operation];
    
    classDef primary fill:#EDE9FE,stroke:#5B21B6,stroke-width:2px,color:#5B21B6;
    classDef success fill:#D1FAE5,stroke:#059669,stroke-width:2px,color:#065F46;
    classDef decision fill:#FEF3C7,stroke:#D97706,stroke-width:1px,color:#92400E;
    classDef process fill:#DBEAFE,stroke:#2563EB,stroke-width:1px,color:#1E40AF;
    classDef check fill:#FEE2E2,stroke:#DC2626,stroke-width:1px,color:#991B1B;

    class A,B,C,H,J,K primary;
    class D,E,G decision;
    class F,M success;
    class I,L check;

1. Preferred Address: Each CA has a preferred SA defined by its function (e.g., engine controller often prefers SA 0).
2. Address Claim Message (ACL – PGN 60928, 0x00EE00): When a CA powers up, it broadcasts an ACL message containing its 64-bit NAME and its preferred SA.
   • The NAME is a unique 64-bit identifier for the CA, composed of fields like manufacturer code, identity number, function instance, etc.
3. Contention:
   • If no other CA is using the preferred SA, the CA successfully claims it.
   • If another CA with a higher priority NAME (lower numerical value) is already using or simultaneously claims the same SA, the CA with the lower priority NAME must choose a different SA (e.g., from a configurable range or by using the null address).
   • If a CA detects another CA claiming an address it already successfully claimed, it will re-broadcast its ACL to defend its address.
4. Cannot Claim Address: If a CA cannot claim any address, it typically uses the null address (254) and can only listen or respond to specific requests.

This dynamic process ensures that all active CAs on the network have unique source addresses.

Transport Protocol (TP) for Messages > 8 Bytes

Classical CAN frames are limited to 8 data bytes. J1939 needs to transmit larger data sets (e.g., diagnostic messages, software updates, vehicle configuration). The J1939 Transport Protocol (J1939-21) handles this:

1. Connection Mode (TP.CM):
   • Used for peer-to-peer communication of large messages.
   • Involves a handshake:
     • RTS (Request to Send – PGN 59392, 0x00EC00): Sender to Receiver. Specifies total message size, number of packets, and PGN of the data being sent.
     • CTS (Clear to Send – PGN 58880, 0x00EB00): Receiver to Sender. Specifies how many packets the sender can send next and the starting packet number.
     • Data Transfer (TP.DT – PGN 60160, 0x00E800): Sender sends data packets (up to 7 data bytes per TP.DT frame + sequence number).
     • EOM ACK (End of Message Acknowledgment – PGN 58368, 0x00E700): Receiver to Sender after all packets are received successfully. Or Abort message if issues.
   • Destination Address (DA) in the CAN ID of TP.CM messages is the specific peer. SA is the sender.
2. Broadcast Announce Message (TP.BAM) / Data Transfer (TP.DT):
   • Used for broadcasting large messages to all nodes (or a group). No handshake like CTS.
   • BAM (PGN 60416, 0x00ECFF if DA is global): Sender broadcasts a BAM message. Specifies total message size, number of packets, and PGN of the data being sent.
   • Data Transfer (TP.DT – PGN 60160, 0x00EBFF if DA is global): Sender then broadcasts all data packets sequentially.
   • Receivers listen for the BAM and then collect the subsequent TP.DT packets. No individual CTS or EOM ACK.
   • The DA in the CAN ID for BAM/DT is typically the global address (255).

Each TP.DT packet contains a sequence number (1-255) as the first data byte, followed by up to 7 bytes of the actual data payload.

ESP32 TWAI Configuration for J1939

To use the ESP32’s built-in TWAI controller for J1939:

• Baud Rate: Typically 250 kbps (TWAI_TIMING_CONFIG_250KBITS()) or sometimes 500 kbps.
• Extended Identifiers: J1939 exclusively uses 29-bit identifiers. Transmitted messages must have the TWAI_MSG_FLAG_EXTD flag set.
• Acceptance Filter: Configure the TWAI acceptance filter to accept relevant 29-bit J1939 messages. This can be broad (accepting a range of source addresses or PDU Formats) or specific. For J1939, you often want to filter based on PGNs or SAs. Since the PGN is embedded within the 29-bit ID, the filter needs to be set accordingly.
  • Example: To accept all messages from a specific Source Address SA_OF_INTEREST, the filter code would have the SA in its lower 8 bits, and the mask would ensure these bits must match.
  • Example: To accept a specific PGN, the filter code would represent the PGN (considering DP, PF, PS fields), and the mask would target these bits.
  • Using TWAI_FILTER_CONFIG_ACCEPT_ALL() and filtering in software is also an option, especially during development, but less efficient for busy buses.

Practical Examples

Warning: Connecting to a live heavy-duty vehicle’s J1939 network requires caution. Ensure your hardware (ESP32 + CAN Transceiver) is correctly wired and functioning. Incorrect connections or faulty software could disrupt vehicle communication. Testing with a J1939 simulator or an isolated bench setup with J1939 ECUs is highly recommended.

Hardware Setup

1. ESP32 Development Board: (e.g., ESP32, ESP32-S2, ESP32-S3, ESP32-C6, ESP32-H2).
2. CAN Transceiver Module: A module compatible with 3.3V logic and the CAN bus voltage levels (e.g., MCP2551, TJA1050, SN65HVD230). J1939 typically uses 5V CAN transceivers, ensure compatibility or use appropriate level shifting if your ESP32 board is 3.3V and transceiver module is 5V logic. Most common transceiver modules handle this.
3. Wiring:
   • ESP32 TWAI_TX -> Transceiver TXD
   • ESP32 TWAI_RX -> Transceiver RXD
   • Transceiver CAN_H/CAN_L -> J1939 Bus CAN_H/CAN_L (often via a Deutsch 9-pin connector on vehicles).
   • Common Ground.
   • Power for ESP32 and transceiver.
4. J1939 Network/Simulator: A live vehicle J1939 bus, a J1939 simulator, or at least one other J1939 node for testing.
5. Termination: Ensure the CAN bus is properly terminated (typically two 120-Ohm resistors at the ends of the bus).

Project Setup (VS Code, ESP-IDF)

1. Create a new ESP-IDF project (e.g., esp32_j1939_node).
2. Configure TWAI GPIO pins in menuconfig or sdkconfig.defaults (e.g., CONFIG_TWAI_TX_GPIO=21, CONFIG_TWAI_RX_GPIO=22).

Example 1: Basic J1939 Configuration and PGN Transmission

This example initializes TWAI for 250 kbps, 29-bit IDs, and transmits a message with an example PGN.

main/main.c:

#include <stdio.h>
#include <string.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "driver/twai.h"
#include "esp_log.h"

static const char *TAG = "J1939_NODE";

// Default TWAI GPIOs - configure via menuconfig
#define TWAI_TX_GPIO CONFIG_TWAI_TX_GPIO
#define TWAI_RX_GPIO CONFIG_TWAI_RX_GPIO

// J1939 Defines
#define J1939_BAUD_RATE_250K     250000
#define MY_SOURCE_ADDRESS        0x25  // Example Source Address for our ESP32 node (37 decimal)

// Function to construct a J1939 29-bit CAN ID
// PGN: Parameter Group Number
// DA: Destination Address (used if PF < 240, otherwise part of PGN or 0xFF for broadcast)
// SA: Source Address
// Priority: 0-7
uint32_t construct_j1939_id(uint8_t priority, uint32_t pgn, uint8_t da_or_pgn_ext, uint8_t sa) {
    uint32_t id = 0;
    uint8_t pf = (pgn >> 8) & 0xFF;
    uint8_t ps_or_da;

    id |= (uint32_t)priority << 26;
    // EDP (bit 25) and DP (bit 24) are part of PGN.
    // PGN mapping:
    // PGN bits 17 (DP), 16 (EDP=0)
    // PGN bits 15-8 (PF)
    // PGN bits 7-0 (PS, if PDU2 format, i.e., PF >= 240)

    if (pf < 240) { // PDU1 format, PS field is DA
        id |= (pgn & 0x03FF00); // DP + PF (PGN bits 17-8)
        ps_or_da = da_or_pgn_ext; // This is the DA
        id |= (uint32_t)ps_or_da; // DA takes the PS field position
    } else { // PDU2 format, PS field is PGN extension
        id |= (pgn & 0x03FFFF); // DP + PF + PS (PGN bits 17-0)
        // da_or_pgn_ext is not used directly in ID here as PS is part of PGN
        // For broadcasts, the "DA" concept is handled by PF >= 240.
        // If a PDU2 PGN needs to be sent to a specific DA (e.g. for TP.CM),
        // that DA is typically put into the PS field of the PGN itself,
        // or the TP mechanism handles addressing.
        // For simple PDU2 broadcasts, PS is part of PGN.
    }
     id |= (uint32_t)sa; // Source address is always the last 8 bits of the CAN ID's arbitration part.
                         // However, the J1939 ID structure places SA at the end of the fields that form the identifier.
                         // The CAN hardware ID is what we are constructing.
                         // Correct construction: Priority | PGN_Fields | SA
                         // PGN_Fields are EDP, DP, PF, PS(DA or PGN ext)
    
    // Let's use a clearer construction based on J1939 structure:
    // P (3 bits), R (1 bit = 0), DP (1 bit), PF (8 bits), PS (8 bits), SA (8 bits)
    // Total 29 bits.
    // R is bit 25, DP is bit 24.
    // PF is bits 23-16.
    // PS is bits 15-8.
    // SA is bits 7-0.

    id = 0; // Reset
    id |= (uint32_t)(priority & 0x07) << 26;
    // id |= (0 << 25); // Reserved bit (EDP) - usually 0
    // id |= ((pgn >> 16) & 0x01) << 24; // DP bit from PGN

    uint8_t pgn_dp = (pgn >> 16) & 0x01; // Extract DP from PGN
    uint8_t pgn_pf = (pgn >> 8) & 0xFF;  // Extract PF from PGN
    uint8_t pgn_ps = pgn & 0xFF;         // Extract PS from PGN (if PDU2)

    id |= (uint32_t)pgn_dp << 24;
    id |= (uint32_t)pgn_pf << 16;

    if (pgn_pf < 240) { // PDU1, PS field in ID is DA
        id |= (uint32_t)da_or_pgn_ext << 8; // DA
    } else { // PDU2, PS field in ID is PGN extension
        id |= (uint32_t)pgn_ps << 8; // PS from PGN
    }
    id |= (uint32_t)sa; // Source Address
    return id;
}


esp_err_t twai_init_j1939(void) {
    twai_general_config_t g_config = TWAI_GENERAL_CONFIG_DEFAULT(TWAI_TX_GPIO, TWAI_RX_GPIO, TWAI_MODE_NORMAL);
    // For J1939, baud rate is commonly 250kbps or 500kbps
    twai_timing_config_t t_config;
    if (J1939_BAUD_RATE_250K == 250000) {
        t_config = TWAI_TIMING_CONFIG_250KBITS();
    } else if (J1939_BAUD_RATE_250K == 500000) { // Placeholder if you want to use 500k
        t_config = TWAI_TIMING_CONFIG_500KBITS();
    } else {
        ESP_LOGE(TAG, "Unsupported J1939 baud rate");
        return ESP_FAIL;
    }
    
    // Accept all 29-bit messages for simplicity in this example.
    // In a real application, configure filters for specific PGNs or SAs.
    // To accept only 29-bit IDs:
    // The filter registers are 32-bit. For 29-bit IDs, they are left-aligned.
    // A mask of 0x1FFFFFFF (left-aligned) would check all 29 bits.
    // For simplicity, accept all:
    twai_filter_config_t f_config = TWAI_FILTER_CONFIG_ACCEPT_ALL();

    if (twai_driver_install(&g_config, &t_config, &f_config) != ESP_OK) {
        ESP_LOGE(TAG, "Failed to install TWAI driver");
        return ESP_FAIL;
    }
    ESP_LOGI(TAG, "TWAI driver installed for J1939 at %d bps", J1939_BAUD_RATE_250K);

    if (twai_start() != ESP_OK) {
        ESP_LOGE(TAG, "Failed to start TWAI driver");
        return ESP_FAIL;
    }
    ESP_LOGI(TAG, "TWAI driver started");
    return ESP_OK;
}

esp_err_t send_j1939_message(uint32_t pgn, uint8_t priority, uint8_t dest_addr, 
                             uint8_t *data, uint8_t data_len) {
    if (data_len > 8) {
        ESP_LOGE(TAG, "Data length exceeds 8 bytes for single frame J1939 message.");
        // Here you would implement TP (Transport Protocol) for longer messages.
        return ESP_ERR_INVALID_ARG;
    }

    twai_message_t message;
    message.identifier = construct_j1939_id(priority, pgn, dest_addr, MY_SOURCE_ADDRESS);
    message.flags = TWAI_MSG_FLAG_EXTD; // J1939 uses 29-bit extended identifiers
    message.data_length_code = data_len;
    memcpy(message.data, data, data_len);

    // Pad remaining data bytes if needed (CAN hardware often pads with zeros or last byte)
    for (int i = data_len; i < 8; ++i) {
        message.data[i] = 0x00; // Or 0xFF as per some J1939 recommendations for fill
    }
    
    ESP_LOGI(TAG, "Sending J1939 Msg: ID=0x%08lX, PGN=%lu, DLC=%d, Data[0]=0x%02X",
             message.identifier, pgn, data_len, data[0]);

    if (twai_transmit(&message, pdMS_TO_TICKS(1000)) == ESP_OK) {
        ESP_LOGI(TAG, "Message queued for transmission");
        return ESP_OK;
    } else {
        ESP_LOGE(TAG, "Failed to queue message for transmission");
        return ESP_FAIL;
    }
}

void j1939_tx_task(void *pvParameters) {
    if (twai_init_j1939() != ESP_OK) {
        ESP_LOGE(TAG, "J1939 TWAI initialization failed. Task stopping.");
        vTaskDelete(NULL);
        return;
    }

    // Example: Transmit Electronic Engine Controller 1 (EEC1 - PGN 61444)
    // This PGN is typically broadcast (PDU2 format, PF=240, PS=0x04 from PGN)
    // For PDU2, the 'dest_addr' field in send_j1939_message is effectively the PS part of PGN.
    // PGN 61444 = 0x00F004. DP=0, PF=0xF0 (240), PS=0x04.
    uint32_t pgn_eec1 = 61444; 
    uint8_t priority_eec1 = 3; // Typical priority for EEC1
    // For PDU2 broadcast, DA field in construct_j1939_id is effectively PGN's PS part.
    // Or, if the PGN itself implies broadcast (PF >= 240), DA is often 0xFF conceptually.
    // The construct_j1939_id function needs to handle this.
    // Let's assume for PDU2, the da_or_pgn_ext parameter to construct_j1939_id is the PS of the PGN.
    uint8_t ps_eec1 = pgn_eec1 & 0xFF;


    uint8_t eec1_data[8] = {
        0x10, // Engine Torque Mode, Actual Engine % Torque High Res etc. (Example values)
        0x20, // Driver's Demand Engine % Torque
        0x87, // Actual Engine % Torque
        0x45, // Engine Speed (Byte 4 & 5, LSB first)
        0x1A, // Engine Speed (0x1A45 = 6725 => 6725 * 0.125 RPM = 840.625 RPM)
        0xFE, // Source Address of Controlling Device for Engine Control
        0x7F, // Engine Starter Mode
        0xFF  // Engine Demand % Torque
    };

    while (1) {
        send_j1939_message(pgn_eec1, priority_eec1, ps_eec1 /* For PDU2, this is PGN's PS */, eec1_data, 8);
        // Modify eec1_data[3] and eec1_data[4] for changing RPM for testing
        eec1_data[3]++; 
        vTaskDelay(pdMS_TO_TICKS(1000)); // Send every 1 second
    }
}

void app_main(void) {
    ESP_LOGI(TAG, "Starting J1939 Node Application");
    xTaskCreate(j1939_tx_task, "j1939_tx_task", 4096, NULL, 5, NULL);
    // Add a receive task here as well for a complete example
}

Build Instructions:

1. Save as main/main.c.
2. Ensure TWAI GPIOs are configured.
3. idf.py build.

Run/Flash/Observe:

1. idf.py -p /dev/ttyUSB0 flash monitor.
2. Connect to a J1939 bus or simulator.
3. Observe the ESP32 transmitting J1939 messages. Use a CAN analyzer tool (like PCAN-View, BusMaster, or another ESP32 running a receiver) to verify the 29-bit ID, PGN, SA, and data.

Example 2: Receiving and Parsing J1939 Messages

This example shows how to receive messages and extract PGN and SA.

// Add this task and helper functions to main.c

// Helper to extract PGN from a raw 29-bit J1939 CAN ID
uint32_t extract_pgn_from_id(uint32_t can_id) {
    uint8_t pf = (can_id >> 16) & 0xFF;
    uint8_t ps = (can_id >> 8) & 0xFF;
    uint8_t dp = (can_id >> 24) & 0x01; // DP is bit 24 of ID
    // uint8_t edp = (can_id >> 25) & 0x01; // EDP/Reserved is bit 25 of ID (usually 0)

    uint32_t pgn = 0;
    if (pf < 240) { // PDU1 format, PS is DA
        pgn = ((uint32_t)dp << 16) | ((uint32_t)pf << 8);
        // The PS (DA) is not part of the PGN itself in PDU1
    } else { // PDU2 format, PS is PGN extension
        pgn = ((uint32_t)dp << 16) | ((uint32_t)pf << 8) | (uint32_t)ps;
    }
    return pgn;
}

// Helper to extract Source Address from a raw 29-bit J1939 CAN ID
uint8_t extract_sa_from_id(uint32_t can_id) {
    return (uint8_t)(can_id & 0xFF);
}

// Helper to extract Destination Address from a PDU1 raw 29-bit J1939 CAN ID
uint8_t extract_da_from_pdu1_id(uint32_t can_id) {
    uint8_t pf = (can_id >> 16) & 0xFF;
    if (pf < 240) {
        return (uint8_t)((can_id >> 8) & 0xFF);
    }
    return 0xFF; // Not a PDU1 message or DA is global
}


void j1939_rx_task(void *pvParameters) {
    // Ensure TWAI is initialized (e.g., by tx_task or call twai_init_j1939() here if tx_task is not running)
    // For this example, we assume twai_init_j1939() has been called.
    // If only running RX, uncomment:
    // if (twai_init_j1939() != ESP_OK) {
    //     ESP_LOGE(TAG, "J1939 TWAI initialization failed for RX. Task stopping.");
    //     vTaskDelete(NULL);
    //     return;
    // }
    ESP_LOGI(TAG, "J1939 RX Task Started. Waiting for messages...");

    twai_message_t rx_message;
    while (1) {
        esp_err_t ret = twai_receive(&rx_message, pdMS_TO_TICKS(portMAX_DELAY)); // Wait indefinitely
        if (ret == ESP_OK) {
            if (rx_message.flags & TWAI_MSG_FLAG_EXTD) { // Check if it's an extended ID
                uint32_t pgn = extract_pgn_from_id(rx_message.identifier);
                uint8_t sa = extract_sa_from_id(rx_message.identifier);
                uint8_t pf = (rx_message.identifier >> 16) & 0xFF;
                uint8_t da = 0xFF; // Default for broadcast
                if (pf < 240) {
                    da = extract_da_from_pdu1_id(rx_message.identifier);
                }

                ESP_LOGI(TAG, "J1939 RX: ID=0x%08lX, PGN=%lu (0x%lX), SA=0x%02X, DA=0x%02X, DLC=%d",
                         rx_message.identifier, pgn, pgn, sa, da, rx_message.data_length_code);
                
                // Log data bytes
                char data_str[3 * TWAI_FRAME_MAX_DLC + 1] = {0};
                for (int i = 0; i < rx_message.data_length_code; i++) {
                    sprintf(data_str + i * 3, "%02X ", rx_message.data[i]);
                }
                ESP_LOGI(TAG, "Data: %s", data_str);

                // Example: Parse EEC1 (PGN 61444) for Engine Speed
                if (pgn == 61444 && rx_message.data_length_code == 8) {
                    uint16_t raw_rpm = (rx_message.data[4] << 8) | rx_message.data[3]; // Bytes 4 and 5 (0-indexed)
                    float engine_speed_rpm = raw_rpm * 0.125f;
                    ESP_LOGI(TAG, "EEC1 - Engine Speed: %.2f RPM", engine_speed_rpm);
                }

            } else {
                ESP_LOGD(TAG, "Received standard CAN frame, ignoring in J1939 context.");
            }
        } else if (ret == ESP_ERR_TIMEOUT) {
            // Should not happen with portMAX_DELAY, but good practice
            ESP_LOGV(TAG, "Timeout receiving J1939 message.");
        } else {
            ESP_LOGE(TAG, "Failed to receive J1939 message: %s", esp_err_to_name(ret));
            // Potentially re-initialize TWAI or handle error
            vTaskDelay(pdMS_TO_TICKS(1000)); // Avoid busy loop on persistent error
        }
    }
}

// In app_main, add:
// xTaskCreate(j1939_rx_task, "j1939_rx_task", 4096, NULL, 5, NULL);

Note: Ensure twai_init_j1939() is called before this task starts, or call it within the task. If both TX and RX tasks are running, one initialization is sufficient.

Example 3: Conceptual J1939 Address Claiming

This is a simplified conceptual outline. A full implementation is complex.

// Conceptual - Add to main.c
#define PGN_ADDRESS_CLAIMED 60928 // 0x00EE00
#define J1939_NULL_ADDRESS  254

// Our node's 64-bit NAME (example, should be unique)
// J1939 NAME structure: (LSB first in memory, but fields have specific bit positions)
// Arbitrary Address Capable (1 bit), Industry Group (3 bits), Vehicle System Instance (4 bits)
// Vehicle System (7 bits), Reserved (1 bit), Function (8 bits)
// Function Instance (5 bits), ECU Instance (3 bits)
// Manufacturer Code (11 bits), Identity Number (21 bits)
const uint8_t MY_J1939_NAME[8] = {
    0x12, 0x34, 0x56, 0x78, // Identity Number (lower 21 bits) & Manufacturer Code (next 11 bits)
    0x01,                   // ECU Instance, Function Instance
    0x80,                   // Function (e.g., 128 for an instrument cluster)
    0x00,                   // Vehicle System, Reserved
    0xA0                    // Vehicle System Instance, Industry Group, Arbitrary Address Capable (e.g., 1)
};

uint8_t current_sa = J1939_NULL_ADDRESS;
bool address_claimed = false;

void send_address_claim(uint8_t sa_to_claim) {
    ESP_LOGI(TAG, "Attempting to claim address 0x%02X", sa_to_claim);
    // PGN_ADDRESS_CLAIMED (0xEE00) is PDU2, so PS is part of PGN (00). DA is global (FF).
    // However, the J1939 spec for PGN 0xEE00 (Address Claim/Cannot Claim)
    // uses PF=0xEE, PS=0x00. The CAN ID's DA field is not used (implicitly global).
    // So, for construct_j1939_id, we can pass 0xFF or 0x00 as dest_addr for PDU2 PGNs.
    // The SA in the ID will be the SA we are trying to claim.
    
    uint32_t acl_id = construct_j1939_id(6, PGN_ADDRESS_CLAIMED, 0x00 /*PS part of PGN_ADDRESS_CLAIMED*/, sa_to_claim);

    twai_message_t message;
    message.identifier = acl_id;
    message.flags = TWAI_MSG_FLAG_EXTD;
    message.data_length_code = 8; // NAME is 8 bytes
    memcpy(message.data, MY_J1939_NAME, 8);

    if (twai_transmit(&message, pdMS_TO_TICKS(100)) == ESP_OK) {
        ESP_LOGI(TAG, "Address Claim message sent for SA 0x%02X", sa_to_claim);
    } else {
        ESP_LOGE(TAG, "Failed to send Address Claim message for SA 0x%02X", sa_to_claim);
    }
}

void process_received_acl(const twai_message_t *rx_msg) {
    uint8_t claimed_sa_in_id = extract_sa_from_id(rx_msg->identifier); // SA in ACL ID is the address being claimed
    const uint8_t* received_name = rx_msg->data;

    ESP_LOGI(TAG, "ACL received: Claimant SA=0x%02X is claiming address 0x%02X in ID", 
        extract_sa_from_id(rx_msg->identifier), // This is the SA of the node sending the ACL
        claimed_sa_in_id); // This is not correct. The SA in the ID *is* the address being claimed.

    // The SA in the CAN ID of an ACL message IS the address being claimed.
    // The source of the message is also that SA.
    uint8_t sa_being_claimed_by_other = extract_sa_from_id(rx_msg->identifier);


    if (address_claimed && sa_being_claimed_by_other == current_sa) {
        // Someone else is claiming our address!
        ESP_LOGW(TAG, "Address contention for SA 0x%02X!", current_sa);
        // Compare NAMEs (lexicographical comparison, lower value wins)
        if (memcmp(MY_J1939_NAME, received_name, 8) < 0) {
            ESP_LOGI(TAG, "Our NAME is higher priority. Defending address 0x%02X.", current_sa);
            send_address_claim(current_sa); // Re-send our claim
        } else {
            ESP_LOGW(TAG, "Their NAME is higher priority. We lost SA 0x%02X.", current_sa);
            address_claimed = false;
            current_sa = J1939_NULL_ADDRESS;
            // TODO: Try claiming a different address or operate at NULL_ADDRESS
        }
    } else if (!address_claimed && sa_being_claimed_by_other == MY_SOURCE_ADDRESS) {
        // We were trying to claim MY_SOURCE_ADDRESS, and someone else also claimed it.
        ESP_LOGW(TAG, "Contention during our initial claim for SA 0x%02X.", MY_SOURCE_ADDRESS);
         if (memcmp(MY_J1939_NAME, received_name, 8) < 0) {
            ESP_LOGI(TAG, "Our NAME is higher priority for SA 0x%02X. Will retry claim or assume success after timeout.", MY_SOURCE_ADDRESS);
            // J1939-81 specifies a timeout (250ms) after which if no conflicting higher-priority claim is seen,
            // the address is considered claimed.
            // For simplicity, we might just assume claim is successful if our NAME is better.
            // Or, if we sent our claim, we wait. If we hear a conflicting claim with worse NAME, we ignore it.
            // If we hear a conflicting claim with better NAME, we lose.
         } else {
            ESP_LOGW(TAG, "Their NAME is higher priority for SA 0x%02X. We cannot claim it now.", MY_SOURCE_ADDRESS);
             // TODO: Try a different address.
         }
    }
}

// In j1939_rx_task, when a message is received:
// if (pgn == PGN_ADDRESS_CLAIMED) {
//     process_received_acl(&rx_message);
// }

// In a startup sequence or main task:
// send_address_claim(MY_SOURCE_ADDRESS);
// vTaskDelay(pdMS_TO_TICKS(300)); // Wait for contention period (J1939-81 specifies ~250ms random delay + listening)
// // After this, if no higher priority claim for MY_SOURCE_ADDRESS was received,
// // we can assume `address_claimed = true; current_sa = MY_SOURCE_ADDRESS;`
// // This simplified logic doesn't fully implement J1939-81 random delays and state machine.

Tip: Full J1939-81 address claiming is a state machine. The above is a highly simplified illustration.

Variant Notes

• ESP32, ESP32-S2, ESP32-S3, ESP32-C6, ESP32-H2:
  • These variants have a built-in TWAI controller that is CAN 2.0B compliant and supports 29-bit extended identifiers, making them suitable for direct J1939 implementation.
  • The TWAI driver (driver/twai.h) is used as shown in the examples.
  • Ensure correct GPIO configuration for TWAI TX/RX.
• ESP32-C3:
  • The ESP32-C3 does not have a built-in hardware TWAI/CAN controller.
  • To implement J1939 on an ESP32-C3, an external CAN controller IC (e.g., MCP2515 for CAN 2.0B) connected via SPI is required.
  • The ESP32-C3 would act as an SPI master to configure and communicate with the external CAN controller. The TWAI driver examples in this chapter will not directly apply. You would need a specific driver for the chosen external CAN IC. The logic for constructing/parsing J1939 messages would remain similar, but the CAN frame transmission/reception would go through the external IC’s driver API.
• General:
  • Always use a CAN transceiver between the ESP32’s (or external CAN controller’s) logic-level signals and the physical J1939 CAN bus.
  • The standard J1939 baud rate is 250 kbps. Some networks might use 500 kbps. Ensure your configuration matches the network.

Common Mistakes & Troubleshooting Tips

Exercises

1. Exercise 1 (Easy): Transmit Engine Temperature PGN
   • Modify Example 1 to transmit the “Engine Temperature 1” (ET1 – PGN 65262, 0x00FEEE). This is a PDU2 broadcast PGN.
   • Data for ET1 (8 bytes):
     • Byte 1: Engine Coolant Temperature (Actual value – 40, unit °C)
     • Byte 2: Fuel Temperature (Actual value – 40, unit °C)
     • Byte 3-4: Engine Oil Temperature (Actual value * 0.03125 – 273, unit K)
     • Byte 5-6: Turbo Oil Temperature (Actual value * 0.03125 – 273, unit K)
     • Byte 7: Engine Intercooler Temperature (Actual value – 40, unit °C)
     • Byte 8: Engine Coolant Temperature (Raw value for extended range display)
   • Send example data (e.g., coolant temp 90°C -> byte 1 = 130).
2. Exercise 2 (Medium): PGN Request and Response
   • Implement functionality to send a J1939 Request PGN (RQST – PGN 59904, 0x00EA00). This PGN is used to request another PGN from a specific DA or all nodes (DA=255).
   • The data for RQST is 3 bytes: the PGN being requested (LSB, middle, MSB).
   • Send a request for PGN 61444 (EEC1) to the global address (255).
   • Modify your j1939_rx_task to listen for and log the response (which would be PGN 61444 from an engine controller).
3. Exercise 3 (Medium): Source Address Filtering
   • Modify twai_init_j1939() to configure the TWAI acceptance filter to only accept messages from a specific Source Address (e.g., SA 0x00 – typical engine controller).
   • Test by having your ESP32 transmit messages from MY_SOURCE_ADDRESS and verify they are not received by its own j1939_rx_task. Then, use another J1939 source (simulator or another ESP32) to send messages from SA 0x00 and verify they are received.
   • Hint: The filter code will have 0x00 in its SA bits, and the mask will ensure these SA bits must match, while other ID bits can be “don’t care” for this specific filter entry.
4. Exercise 4 (Conceptual): BAM Transport Protocol Sender
   • Outline the steps and data structures needed to send a 20-byte message using the J1939 BAM Transport Protocol.
   • Specify:
     • How the BAM control message (PGN 60416) would be formatted (ID, data bytes including total message size, number of packets, PGN of the data being sent).
     • How the subsequent TP.DT data packets (PGN 60160) would be formatted (ID, sequence number in the first data byte, 7 data bytes per packet).
   • You don’t need to write full C code, but describe the logic for segmenting the 20-byte message and constructing each BAM and TP.DT CAN frame.

Summary

• SAE J1939 is the standard CAN-based protocol for communication in heavy-duty vehicles, using 29-bit extended identifiers.
• J1939 messages (PDUs) are identified by Parameter Group Numbers (PGNs), and each node has a unique Source Address (SA).
• The 29-bit CAN ID encodes priority, PGN information (DP, PF, PS), and SA. PDU Format (PF) determines if the message is destination-specific (PDU1) or broadcast (PDU2).
• The ESP32‘s TWAI peripheral can be configured for J1939 (250/500 kbps, 29-bit IDs) on variants with a built-in controller. ESP32-C3 requires an external CAN controller.
• Address Claiming (J1939-81) is a crucial network management process for nodes to acquire unique SAs.
• Transport Protocol (J1939-21) handles messages larger than 8 bytes using BAM (broadcast) or CM (peer-to-peer) mechanisms.
• Practical J1939 implementation involves constructing and parsing CAN IDs, managing PGN data, and potentially handling address claiming and transport protocols.

Further Reading

• SAE J1939 Standards Documents: (Official documents, typically require purchase from SAE International)
  • J1939 – Top Level Document (Recommended Practice for a Serial Control and Communications Vehicle Network)
  • J1939-21 – Data Link Layer (Defines Transport Protocol)
  • J1939-71 – Vehicle Application Layer (Defines PGNs and SPNs)
  • J1939-73 – Application Layer – Diagnostics
  • J1939-81 – Network Management (Defines Address Claiming)
• “A Comprehensible Guide to J1939” by Wilfried Voss: A widely recommended book explaining J1939 in detail.
• Kvaser’s J1939 Introduction: https://www.kvaser.com/about-can/can-standards/j1939-introduction/
• ESP-IDF TWAI Programming Guide: https://docs.espressif.com/projects/esp-idf/en/v5.4/esp32/api-reference/peripherals/twai.html (Ensure your target ESP32 variant is selected).
• Vector Informatik – J1939 Resources: Vector is a major player in automotive networking tools and often has good introductory materials or whitepapers on J1939.