Generic - MQTT Import

.

Plugin details

Type: Generic

Name: MQTT Import

Status: NORMAL

GitHub: P037_MQTTImport.ino

Maintainer: .

Used libraries: .

Introduction

You might want to use MQTT to send commands or values to your ESPEasy unit. To do this you use the plugin MQTT Import.

Only numbers can be stored in variables (Plugin Generic - Dummy Device), so either send numeric values, or use this plugin’s feature of mapping string to numeric values to do the conversion, or define an event handler in Rules to react on a specific value (examples below).

Device configuration

NB: To save space it is possible that not all features are available in all builds. All features are available in normal builds with 2MB or more flash size, max builds, and when self-building in a Custom build. The screenshots are taken including all options.

The options that can be included or excluded are:

  • Parse JSON messages

  • Apply filters

  • Apply mappings

  • Character to replace by comma in event

Initial setup after adding the plugin:

Device setup, initial
  • Name is used to uniquely identify the plugin, and will be used in the standard events generated by the plugin.

  • Enabled should be checked to enable the plugin.

Options

Option: Parse JSON messages

  • Parse JSON messages: when changed to Yes, the page will be reloaded, and the corresponding settings will be made visible, an extra column next to the MQTT Topics, to specify the attribute from any JSON payload received on the topic specified.

When JSON is enabled, regular MQTT Topics will be processed normally.

Device setup, JSON

The JSON Attribute column, when used, can be used to specify what data element to retrieve from a received JSON payload. If the resulting value is numeric it will be stored to the corresponding variable, and the regular event for that will be generated.

When receiving multiple-value attributes, like "svalue":"28.1;17.8;2;1010.3;0", an index can be appended to the attribute like svalue;4 to get the 4th value, in this example 1010.3 (1-based index).

When receiving a JSON payload, an event can (in addition to the standard event when receiving a numeric value) be generated for the received Attribute, or for all attributes if no specific attribute is configured.

The event for such attribute looks like <topic>#<attribute>=<attribute_value>, for example, when, on MQTT Topic zigbee2mqtt/eria_dimswitch_1, receiving this JSON payload: {"action":"on","linkquality":5} will generate these events: zigbee2mqtt/eria_dimswitch_1#action=on and zigbee2mqtt/eria_dimswitch_1#linkquality=5. If Mappings are configured, then the value after applying the mappings will be used!

In addition, for each accepted JSON attribute also an event is generated, like <devicename>#<valuename>=<value>, for example MQTT_Import#Value1=on. If Mappings are used, again the value after mapping is provided.

NB: When receiving non-numeric values, like above, you can use wild-card matching on <topic>* do or explicitly name the event and value to trigger on, for example:

on zigbee2mqtt/eria_dimswitch_1#action=on do
  logentry,"Dimmerswitch 1 action = %eventvalue%"
  gpio,12,1 // Turn light on
endon
on zigbee2mqtt/eria_dimswitch_1#action=off do
  logentry,"Dimmerswitch 1 action = %eventvalue%"
  gpio,12,0 // Turn light off
endon

Using %eventvalue1% (or the shortcut %eventvalue%) will result in the ‘on’ or ‘off’ strings as provided to the event.

Option: Apply filters

  • Apply filters: when changed to Yes, the page will be reloaded and the Filters options will be made visible.

Device setup, filters
  • Filters are a list of Name[;Index] Operand Value sets.

In the current (compile-time) configuration, there are the same amount of filters as there are MQTT Topics, and each Filter is applied to the matching MQTT Topic.

It is possible to reconfigure that to a higher number of filters, but then the filters will be applied to all MQTT Topics, and can not be applied on a specific Topic. Therefore this configuration of 1 Filter per Topic was chosen and documented.

  • Name, with an optional Index, like the JSON Attribute above, specifies what to filter on, the Operand specifies how to compare that to the Value.

  • Operand:

  • Equals

  • Range

  • List

  • Equals: A standard (double, so decimals are supported) comparison, the filter matches if the value is equal, for example:

Device setup, filter, equals
  • Range: Value must be in a range of values. The range can be inclusive, f.e. 2;5 means the value can be between 2 and 5, including 2 and 5. The range can be made exclusive, by reversing the ‘from’ and ‘to’ values, f.e. 5;2 means the value must be less than or equal (<=) to 2 or greater than or equal (>=) to 5. Examples:

Device setup, filter, range
  • List: Value must be one of the values in the list, for example:

Device setup, filter, list

NB: Range and List Values must be semicolon-separated.

Option: Apply mappings

  • Apply mappings: when changed to Yes, the page will refresh and the Mappings options will be made visible.

Device setup, mappings
  • Mappings are a variable length list of Name Operand Value sets.

  • Name: The received value (payload or JSON attribute value), for example the Adurosmart Eria dimmer switch (Zigbee) sends values ‘on’, ‘off’, ‘up’ and ‘down’ for the respective buttons. Normally ESPEasy wouldn’t be able to process those values, but by mapping them to numeric values, the become usable with ESPEasy.

  • Operand:

  • Map

  • Percentage

  • Map: Simple replacement of the input by the Value

  • Percentage: Convert the input to a percentage. Value is the 100% value. This allows to convert f.e. a 0 to 1024 input range to 0 to 100%. If the input exceeds the max. value, the percentage will be > 100, so an input of 2048 for this value will result in 200%. The % sign isn’t part of the output.

Examples:

Device setup, mapping, examples

The number of mappings is currently limited to 25, but only the number of used mappings + 5 empty mappings are displayed on screen to keep the page smaller. Once the on screen list of mappings is full, submitting the page will store the values, and make up to 5 empty mappings available. Unless all 25 are used, of course.

Option: Generate events for accepted topics

  • Generate events for accepted topics: Enabling this option will generate an event for every incoming, and accepted if Filtering is used, non-JSON payload even when that has a non-numeric value. The event generated is: <devicename>#<valuename>=<value>, for example MQTT_Import#Value1=on.

When this option is disabled it has the backward-compatible behavior of discarding that message as invalid.

Option: Limit events being generated

  • Deduplicate events: Receiving many MQTT messages, especially in JSON format, can cause a lot of events to be generated. To somewhat limit the system-load, caused by the amount of events, new events can be de-duplicated while still in the event queue, meaning that an event that is already queued, will not be inserted in the queue, but discarded.

  • Max. # events in event queue: As a protection against event-overflow this configures a check for the queue-length, so if more than the selected number of events is still in the queue, new events will be discarded until some events are processed, and the remaining is less than this count. When set to 0 this check is disabled.

Option: Modify separater character in events

  • To replace by comma in event: Select a character that is to be replaced by a comma, before it is put into the event-queue. There is a limited set of characters that can be replaced, to avoid ending up with malformed events.

This can be used to ‘transform’ the content of a JSON message so the used separator is a comma, for easier use in rules.

Topic Subscriptions

By configuring a MQTT Topic, the plugin will subscribe to that Topic so the MQTT server will send any message for that topic to this ESPEasy unit.

Subscribing to high-trafic topics will cause quite some load on the ESP, so either use an ESP32 unit, or limit trafic by subscribing to more specific topics or use filtering to limit the number of accepted messages, and thus limit the number of events fired. F.e. subscribing to domoticz/out is such a high-trafic topic, when using Domoticz and an MQTT server. Filtering on the idx values that are of interest is then highly advised.

Wildcard MQTT Topic subscriptions (using + or #) can be used, but please be aware this may cause a high load because of a high number of topics accepted. NB: The # wildcard, when used, must be the last element in a topic (according to MQTT specifications).

For systems that use long topics, the extra input field Prefix for all topics is available. The contents of this field will prefixed to all topics (without any extra characters, so slashes should be included as required). This field is optional.

Supported hardware

No ESPEasy related hardware is needed, though a MQTT server (local network or external), and a MQTT Controller configuration is required to be able to use this plugin, as it ‘piggy-backs’ on that connection to subscribe to, and receive, the topics.

Events

Event

Example

MQTTimport#Value1 Will trigger when the value is set from MQTT topic 1 payload.

on MQTTimport#Value1 do
  GPIO,2,%eventvalue% // Turn on based on an MQTT message
endon

When option Generate events for accepted topics is enabled:

on MQTTimport#Value1=on do
  event,switchToggle // See next example
endon

{mqtttopic}#{json_attribute}={attribute_value} Will trigger when the JSON option is available and enabled, and a JSON message is received, for each JSON attribute, or the Attribute specified.

Should best be used combined with the Filtering option.

on domoticz/in#svalue do
  TaskValueSet,2,1,%eventvalue%
endon
on zigbee2mqtt/eria_dimswitch_1#action=on do
  event,switchToggle
endon
on switchToggle do // Sonoff switch here
  if [Relay#State]=1
    gpio,12,0 // Relay off
  else
    gpio,12,1 // Relay on
  endif
endon

Change log

Changed in version 2.0:

added Major overhaul for 2.0 release.

2020-12-19:

added Support for JSON parsing

added Filtering

added Mapping of incoming values

added Generate (optional) events for non-numeric payloads

added Prefix for all topics

New in version 1.0:

added Initial release version.