Controller - ESPEasy P2P Networking

.

Controller details

Type: Controller

Name: ESPEasy P2P Networking

Status: NORMAL

GitHub: C013.ino

Maintainer: TD-er

Change log

Changed in version 2.0:

improved Implementation of secure communication and check for valid data.

New in version 1.0:

added Initial release version.

Description

ESPEasy is able to communicate between nodes itself. It is an IANA registered service: espeasy-p2p

  • Service Name: espeasy-p2p
  • Port Number: 8266
  • Transport Protocol: UDP
  • Description: ESPeasy peer-2-peer communication
  • Registration date: 2018-11-28

This protocol is targeted specific for use by ESPeasy to let ESPeasy nodes communicate with each other to create a big swarm of nodes working together without the need for a hosted service like MQTT, which needs a central broker.

It is currently used for:

  • Discovery of nodes
  • Sharing sensor data among nodes

Later updates may add:

  • Distribution of settings
  • Sending commands

Sharing Plugins among Nodes

It is possible to share the data collected by a plugin on one node so it can be used on another node. This data can be used as if it is actually being run on the second node.

For example, a Dallas DS18b20 sensor on Node-1 is shared using the ESPeasy p2p controller. This plugin can then automatically be setup on Node-2 and using the data collected by Node-1.

This is a rather non-intuitive process to setup.

Prerequisites

  • Same UDP port must be setup on both nodes. (preferrably UDP port 8266) This can be done in Tools -> Advanced -> UDP port
  • Nodes must be rebooted after UDP port has changed.
  • Each node must have an unique unit number. This must not be 0 and not 255, but anything inbetween is fine.

How to share a plugin

  • Check to see if all nodes can see eachother. This will be visible on the main page showing a list of all nodes.
  • Enable p2p networking controller on receiving node
  • Make sure the receiving node has the spot free which is being used on the ‘sending’ node (For example slot 12)
  • Enable p2p networking controller on sending node
  • Set the plugin you want to share to use the p2p controller

Any node that is setup to receive data like this will see a plugin being added if the spot in the device list was still free.

Builds made after 2019/08/08 will show in the device overview page from which unit the shared plugin does get its data. This also means the plugin must be removed and re-created if the sending node is changed. (e.g. another node or change of unit number)

In later builds there will be added an option to update this node number.

Some tips on trouble shooting

  • Make sure to reboot the node after changing the UDP port.
  • Make sure all nodes have an unique unit number and share the same UDP port.
  • If you have ArduinoOTA enabled, use another port for ESPeasy p2p UDP (port suggested in most OTA examples use port 8266)
  • Ping the nodes from some other host to keep their WiFi awake.
  • Disable “Eco mode” in the advanced settings.
  • Sharing a plugin to be auto installed on another node is only sent right after the plugin is set to use the p2p controller. So if you don’t see it appear on the other node, save it again on the source node.
  • Make sure the same plugin is available in the build on both nodes. (e.g. both supporting Dallas DS18b20, if that’s the one you want to share)
  • When using the “Guest” feature of an access point, some will not allow direct communication between clients on the same AP. This will also prevent this p2p protocol to share data.
  • If updating from builds before 2019/08/08, you may need to remove and add again a plugin receiving data from a remote node.

Sending & Known Nodes

ESPEasy keeps track of all nodes advertising themselves via Sysinfo messages.

This knowledge is kept in a NodeStruct for at least 10 minutes. If a node is not sending a Sysinfo message in this period, it will be removed from the list.

Data Format Versions

During the IANA port assignment assessment, a number of issues were pointed out by their experts.

  • Versioning
  • Security
  • Data validation
  • Traffic limiting and congestion handling

There are now 2 versions available:

  • Version “0” - No security, no data validation.
  • Version “1” - Introduced in ESPeasy build <???>

Data Format Version 0

Sending and receiving is causing issues when the swarm of nodes increases.

  • All nodes with this service enabled will advertise their presence every 30 seconds via broadcast
  • Nodes can not subscribe to receiving sensor data updates
  • Non broadcast messages are sent to each individual known node, regardless if the receiving node will use the data
  • Sensor Data messages are sent to each individual known node
  • Sensor Info updates are sent to each individual known node when a plugin coupled to this plugin is saved.

Of each known node the following data is kept:

struct NodeStruct
{
  String nodeName;
  byte ip[4];
  uint16_t build;
  byte age;
  byte nodeType;
};

The key to index this NodeStruct is the nodes unit number.

First byte is not 0xFF.

The entire message processed as a command like this:

packetBuffer[len] = 0;
String request = &packetBuffer[0];
struct EventStruct TempEvent;
parseCommandString(&TempEvent, request);
TempEvent.Source = VALUE_SOURCE_SYSTEM;
if (!PluginCall(PLUGIN_WRITE, &TempEvent, request)) {
  ExecuteCommand(VALUE_SOURCE_SYSTEM, &packetBuffer[0]);
}

As can be seen, no checks for size, and it is just expected to be a valid ESPeasy command. Also no check to see if the command is supported by the receiving end and no feedback to the sender.

Binary data is marked with the first byte 0xFF.

On the receiving end, it is packed in an event in the Data field and processed like this:

struct EventStruct TempEvent;
TempEvent.Data = reinterpret_cast<byte*>(&packetBuffer[0]);
TempEvent.Par1 = remoteIP[3];
TempEvent.Par2 = len;
PluginCall(PLUGIN_UDP_IN, &TempEvent, dummyString);
CPluginCall(CPLUGIN_UDP_IN, &TempEvent);

N.B. only the controller C013 implements code for handling UDP data.

Message types supported, determined by the 2nd byte:

  • 1: Sysinfo message
  • 2: Sensor info pull request (not implemented)
  • 3: Sensor info
  • 4: Sensor data pull request (not implemented)
  • 5: Sensor data

There are 2 types of Sysinfo messages, a standard and an extended message. The extended message starts with the same information as the standard one.

Standard Sysinfo message (13 bytes):

  • 2 bytes marker (255 , 1)
  • 6 byte MAC address
  • 4 byte IP address
  • 1 byte unit number

Extended Sysinfo message (13 + 28 = 41 bytes):

  • 2 bytes ESPeasy data version number (LSB, MSB)
  • 25 bytes node name
  • 1 byte node type

The node type is defined as:

  • 1 = “ESP Easy”
  • 17 = “ESP Easy Mega”
  • 33 = “ESP Easy 32”
  • 65 = “Arduino Easy”
  • 81 = “Nano Easy”

Sensor Info messages are just a description of a shared sensor. It contains some information to setup a new sensor on the receiving end.

These messages are just a serialized byte stream of struct C013_SensorInfoStruct .

struct C013_SensorInfoStruct
{
  byte header = 255;
  byte ID = 3;
  byte sourcelUnit;
  byte destUnit;
  byte sourceTaskIndex;
  byte destTaskIndex;
  byte deviceNumber;
  char taskName[26];
  char ValueNames[VARS_PER_TASK][26];
};

These messages are just a serialized byte stream of struct C013_SensorDataStruct .

struct C013_SensorDataStruct
{
  byte header = 255;
  byte ID = 5;
  byte sourcelUnit;
  byte destUnit;
  byte sourceTaskIndex;
  byte destTaskIndex;
  float Values[VARS_PER_TASK];
};

Data Format Version 1

This version remains compatible with version 0 for backwards compatibility. It is using the “next” unused marker.

All messages will have a standard packet data format:

  • 2 bytes Marker (255 , 6)
  • 2 bytes Version => also determines data offset (header length)
  • 2 bytes Message type
  • 2 bytes Size of data block in “N” blocks of 16 bytes
  • 2 bytes Key/group selector
  • 2 bytes Sequence number
  • (16 x N) bytes Data block AES encrypted data (including 2 bytes checksum)
  • 2 bytes Packet checksum

This allows to:

  • Distinguish data format versions
  • Filter on message type before allocating large buffers
  • Use multiple (pre-shared) encryption keys to have several levels of security or just several groups.
  • Validate correct transmission of packet (last 2 checksum bytes) before decrypting data.
  • Allow for larger messages to be sent in sequences. (e.g. firmware upgrades?)
  • Validate sender and content of data block, since it contains a checksum too, which is part of the encrypted data block.

Since AES has a block size of 16 bytes (128 bit), the size of the data block is defined as a block of 16 bytes. This allows up-to 1 MB of messages. (2^16 * 2^4 = 2^20) An UDP datagram sent over IPv4 cannot exceed 65,507 bytes (65,535 - 8 byte UDP header - 20 byte IP header). In IPv6 jumbograms it is possible to have UDP packets of size greater than 65,535 bytes.