Communication - SerialProxy¶

Plugin details¶

Type: Communication

Port Type: Serial

Name: SerialProxy

Status ESP32: COLLECTION C ENERGY

Status ESP8266: COLLECTION C

GitHub: P087_SerialProxy.ino

Maintainer: TD-er

Used libraries: ESPEasySerial

Configuration¶

Name A unique name should be entered here.
Enabled The device can be disabled or enabled. When not enabled the device should not use any resources.

Sensor¶

See: Serial Helper configuration

Filtering¶

This section only shows settings when the plugin is configured and enabled.

RegEx: Specify a regular expression to apply to the received data before it is accepted as acceptable data. This regular expression is based on the Lua script language, and is documented here: http://www.lua.org/manual/5.2/manual.html#6.4.1
Nr Chars use in RegEx: Limit the length of the data to be checked using the RegEx regular expression filter. When set to 0 all data is checked.
Filter Off Window after send: The time in milliseconds to turn off filtering after receiving data that is accepted by the filter. Setting this to 0 will not turn off the filter after sending to the connected device.
Match Type: Select the type of matching that is to be applied:

Regular Match: Use the Regular Expression matching, configured above.
Regular Match inverted: Use the Regular Expression matching but with inverting the result, as configured above.
Global Match: Use the group values defined the Regular Expression above, to compare to the Capture Filter 1..10 definitions explained below.
Global Match inverted: Like Global Match, inverting the outcome.
Filter Disabled: No filtering is applied.

Capture Filter 1..10¶

Each Capture Filter has 3 fields:

Group: Refer to the group index as returned from the Regular Expression, range 0..31.
Comparison: The comparison can be Equals or Not Equals:

Compared value: The literal value that should match the value from the RegEx Group chosen, and compared using the Comparison. Case sensitive.

Statistics¶

Statistic data is only visible if the plugin is configured and enabled.

This shows the latest data received and some statistics.

Data options¶

(These options are only available if build is without LIMIT_BUILD_SIZE set)

Receive binary data: When enabled accepts any character for input. Does not terminate on a CR character received, but on end of data (no more bytes available), or when using the Fixed length input data setting.
Fixed length input data: When 0 this feature is disabled. When set > 0 (max. length 255 characters), reading is stopped after this number of characters. Best used for reading binary data.
Event with hex. data (no prefix): When checked will generate the event with the received data converted to hexadecimal data (length is doubled), without a prefix (often hex data is prefixed with 0x). Data is still filtered (when enabled) and proxied without conversion!

Data Acquisition¶

This group of settings are standard available configuration items.

Single event with all values: When this setting is enabled, all available values will be sent in a single event <TaskName>#All, with all values in order as arguments to the event.
Show derived values: When checked, the Devices overview page, and the /json endpoint (used for updating the Devices overview page) will include any Derived values as defined. See the TaskValueSetDerived and TaskValueSetPresentation commands.
Event & Log derived values: When checked, the Derived values will be generated as Events, to be handled in Rules, and sent to logging devices like the Syslog server and/or SD-card logging.

(The derived values options are only available if String variables feature is included in the build.)

Send to Controller: Select the Controller(s) to send the Values to, either on a TaskRun command applied to the task, or on an Interval time action.

Send to Controller is only visible when one or more Controllers are configured.

Depending on the controller capabilities, some configuration settings may be shown:

../_images/Task_config_page_Controllers_section.png

All configured Controllers are shown here, including the enabled or disabled state (multiple Controllers can be enabled, only a single MQTT Controller can be enabled at one time!).

For each controller the user can select wether the data should be sent on each Interval (or explicit TaskRun).

For the Domoticz controllers the value index (IDX) has to be configured.

For some controllers, like Home Assistant/openHAB, there are extra options available.

Group: This represents the group id to combine all values from multiple tasks into a single grouped-device during MQTT AutoDiscovery. Groups, by design, can span multiple ESPEasy devices, if desired, as long as the Task/Valuename combinations are unique. If a group should only combine Tasks from a single ESPEasy unit, the group id should be unique across multiple ESPEasy units. The group description, default Group <n>, can be adjusted in Home Assistant. If the Group value matches the current Unit nr, the Unit name, %sysname%, is used instead of Group <nr>.
Retained: For MQTT Controllers, this setting can be enabled to send the values for the current task with the Retain flag set. The Publish Retain flag in the Controller settings will override this by sending all task values with Retain flag enabled.
Send derived: This checkbox determines if any configured Derived values should also be sent to the controller (and included in the AutoDiscovery if that’s available and enabled).
Resend MQTT Discovery: When checked, will start a resend of the MQTT Discovery process for this task after a random delay, when Submit is clicked, so any changed settings will be updated in the MQTT broker. This setting is only available if the controller is enabled, the Auto Discovery feature is available and enabled for the controller. This setting is not stored.

Other controllers, like f.e. FHEM HTTP, do not support additional settings besides the checkbox to enable sending the data.

Interval By default, Interval will be set to 60 sec.

Values¶

The Values are not changed during processing, and can be ignored.

Commands available¶

Command	Extra information
`serialproxy_write,'<content>'` `Content`: Data to send out via the serial port. Should use quotes when containing spaces or commas.	Send out data via the configured serial port.
`serialproxy_writemix,'<content to send>'[,...]` `<content to send>`: Text and/or hex byte(s) (having 0x prefix) that will be sent (nearly) unprocessed. Only the regular variable replacements will be applied before sending the content to the serial port.	This command requires quotes to be used if spaces or commas are part of the content. Any data can be sent, even if it can not be typed in a text content, by specifying that as a separate argument: `serialproxy_writemix,'text, optionally including spaces or commas',0xXX,'0xXXxx XX,xx-XX:xx'` `'text, optionally including spaces or commas'`: Any text content to be sent to the serial port. Can contain variables. Quotes are only required if spaces or commas (separators) are used. `0xXX`: A single character in hexadecimal notation (range: 0x00..0xFF), that is appended to the data to send. `'0xXXxx XX,xx-XX:xx'`: A sequence of hexadecimal values (range: 0x00..0xFF), that can be separated by a space, comma, dash, colon, semicolon or period, or are just entered adjecent. Only the first 2 characters should be `0x` or `0X`, the rest is interpreted as hex bytes, and appended to the data to send. Quotes are only required if space or comma separators are used. Using this command, either from rules, via http or mqtt, the text that is provided as content is completely sent to the serial port. No extra data is added, other than any (system) variables that are included, being replaced.
`serialproxy_test,'<content to process>[,...]'` `<content to process>`: Some text to be processed as if it was received via the serial port.	This command is intended for testing the regular expression and filtering. The sentence provided with this command is handed over for processing as if it was received via the serial port, so it will be processed and filtered, and an event will be generated if it’s matched.

Command

Extra information

serialproxy_write,'<content>'

Content: Data to send out via the serial port. Should use quotes when containing spaces or commas.

Send out data via the configured serial port.

serialproxy_writemix,'<content to send>'[,...]

<content to send>: Text and/or hex byte(s) (having 0x prefix) that will be sent (nearly) unprocessed. Only the regular variable replacements will be applied before sending the content to the serial port.

This command requires quotes to be used if spaces or commas are part of the content.

Any data can be sent, even if it can not be typed in a text content, by specifying that as a separate argument: serialproxy_writemix,'text, optionally including spaces or commas',0xXX,'0xXXxx XX,xx-XX:xx'

'text, optionally including spaces or commas': Any text content to be sent to the serial port. Can contain variables. Quotes are only required if spaces or commas (separators) are used.

0xXX: A single character in hexadecimal notation (range: 0x00..0xFF), that is appended to the data to send.

'0xXXxx XX,xx-XX:xx': A sequence of hexadecimal values (range: 0x00..0xFF), that can be separated by a space, comma, dash, colon, semicolon or period, or are just entered adjecent. Only the first 2 characters should be 0x or 0X, the rest is interpreted as hex bytes, and appended to the data to send. Quotes are only required if space or comma separators are used. Using this command, either from rules, via http or mqtt, the text that is provided as content is completely sent to the serial port. No extra data is added, other than any (system) variables that are included, being replaced.

serialproxy_test,'<content to process>[,...]'

<content to process>: Some text to be processed as if it was received via the serial port.

This command is intended for testing the regular expression and filtering.

The sentence provided with this command is handed over for processing as if it was received via the serial port, so it will be processed and filtered, and an event will be generated if it’s matched.

Get Config Values¶

Get Config Values retrieves values or settings from the sensor or plugin, and can be used in Rules, Display plugins, Formula’s etc. The square brackets are part of the variable. Replace <taskname> by the Name of the task.

Config value

Information

[<taskname>#group.<groupnr>]

Get the contents from groupnr after processing received data.

NB: The regular expression parser uses 0-based group numbers (where most other regex parsers use 1-based group numbers!)

[<taskname>#next.<groupvalue>]

Get the value of the next group that holds <groupvalue>.

Example:

Regular expression: ((node)=(%d+);?)((weight)=(%d+);?)((temp%d?)=(%-?%d+);?)((rssi)=(%-?%d+);?)

Received data: node=1;weight=40;temp1=20;rssi=-30

This will result in these groups:

Group	Data
0	node=1
1	node
2	1
3	weight=40
4	weight
5	40
6	temp1=20
7	temp1
8	20
9	rssi=-30
10	rssi
11	-30

Requesting [SerialProxy#next.weight] will return the value 40.

Example configuration¶

A practical example:

Received data via serial:

PID   0xA060
FW    159
SER#  HQ12345E6M7
V     26280
I     750
VPV   31920
PPV   20
CS    3
MPPT  2
OR    0x00000000
ERR   0
LOAD  OFF

(Tokens are tab-separated)

Suppose we want to handle the lines starting with the token V and ERR, and send the values to a MQTT broker.

We start by devising a regular expression (Lua syntax, using % instead of the more common \ escape character) that can handle each line, collecting the tokens into groups:

(%w+#?)%s+(%w+)

A group ( that captures a set of 1 or more alphanumeric characters %w+ and accepts an optional hash-sign #?, end of group ).
1 or more white-space characters %s+ (space, tab).
Another group ( that captures a set of 1 or more alphanumeric characters %w+, end of group ).

The regular expression can be entered into the RegEx field.

If we keep the Match Type setting on Regular Match, received data has to match the regex before it is passed on for processing. This is the correct setting if we want to process all lines received in Rules.

When we want to reduce the CPU load by limiting the processing to only some of the lines of received data, we can use filters. The filters are checked if we change Match Type to Global Match. There are 10 available filter entries, where we can select what values should pass, or not pass by using the Match Type Global Match inverted.

For this example we only want the V and ERR lines to pass, and the first token is captured in group 0, so we set the filters like this:

To process the received data, Rules have to be enabled in the Tools/Advanced page.

The event when receiving the data V 26280 is logged like this:

01:27:15.147 : (308048) Info   : EVENT: SerialProxy#v=`V 26280`

For this example, the Taskname is SerialProxy, so we’ll define a rule to process the Value, default name is v:

On *serialproxy#v Do
  LogEntry,'Match P1: [serialproxy#group.0], P2: [serialproxy#group.1] V: [serialproxy#next.V] ERR: [serialproxy#next.err]'
  If {equals:V:'[serialproxy#group.0]'}=1
    publish,receiver/V,[serialproxy#next.V]
  Endif
  If {equals:ERR:'[serialproxy#group.0]'}=1
    publish,receiver/ERR,[serialproxy#next.ERR]
  Endif
Endon

We’ll discuss the lines one by one:

On *serialproxy#v Do : To have the rules engine handle the non-numeric parameter received in the event, the wildcard character * has to be prefixed to the event name. The taskname and valuename are handled non-case-sensitive.

LogEntry... : For testing purposes it is helpful to log the data, so it can be viewed via the serial console or the Tools/Log page. Here it is used to demonstrate 2 ways of retrieving the processed data from the Regular expression:

Via the [<taskname>#group.<nr>] config value like [serialproxy#group.0], that will return the current value from that group.
Via the [<taskname>#next.<value>] config value, like [serialproxy#next.V], that will return the current value from the group next from a group that contains V.

The second method is especially useful for handling multiple name/value pairs in a single data package (and a different regular expression to capture that).

If {equals:V:'[serialproxy#group.0]'}=1 : Do a case-sensitive string comparison on the value from the first group (0) to equal V.

publish,receiver/V,[serialproxy#next.V] : Publish the received data value from the next group after the V value to a MQTT topic, using the first enabled MQTT Controller.

Endif : End of first If statement.

If {equals:ERR:'[serialproxy#group.0]'}=1 : Do a case-sensitive string comparison on the value from the first group (0) to equal ERR.

publish,receiver/ERR,[serialproxy#group.1] : Publish the received data value from the second group (1) to a MQTT topic, using the first enabled MQTT Controller.

Endif : End of second If statement.

Endon : End of this rule/event handler

For this example I used a rather elaborate way of writing the rule, and this should be used if quite different MQTT topics are to be used, but it can probably be simplified to:

On *serialproxy#v Do
  LogEntry,'Match P1: [serialproxy#group.0], P2: [serialproxy#group.1]'
  publish,receiver/[serialproxy#group.0],[serialproxy#group.1]
Endon

Note

For testing your regular expression and the handling of the rule, have a look at the serialproxy_test command, above.

Change log¶

Changed in version 2.0: …

added 2025-12-23 Add support for binary data, fixed length input and convert event data to hex.

added 2024-02-25 Add support for serialproxy_test command and retrieving the separate groups from parsed regex.

added 2023-03-22 Add support for writing any binary data out via the serial port.

added 2020-02-22