n

Rigado Edge Connect API Reference

Rigado Edge Connect is an optimized BLE connectivity app that runs on a Rigado Gateway. Edge Connect scans for BLE data and makes it available to BLE Pipelines. A Pipeline is made up of a list of Filters that process event data, filtering, manipulating, and eventually publishing to the cloud or other data destination.

Bluetooth Connections can also be orchestrated using ConnectionSequences, which are comprised of a set of ConnectionActions. Sequences are triggered by a connect filter in a Pipeline.

Information about how to use this configuration can be found in the main Edge Connect Documentation

A number of keys and values in these configuration objects support templating. For more info about how to use templates, see Edge Connect Templates.

Configuration Elements

AcknowledgeConfig

AcknowledgeConfig is used to receive and validate an acknowledgement after writing to a characteristic with the write action. If an acknowledgement fails validation, the write action stops and returns an error.

ackMessageCount: integer (int64) 1

Setting to wait for acknowledgement every N messages

characteristic: string

UUID of characteristic

indication: boolean true

Acknowledgement will be an indication instead of a notification?

keepNotificationsEnabled: boolean false

Keep notifications on characteristic enabled after receiving an acknowledgement?

match: object

Keys to match (all keys) for an inline definition (see MatcherDefinition), or a Template to evaluate and match to a true value.

matchAny: boolean false

Should we match any of the provided matchers (true)? Or all of the matchers (false)

matchers: string[]

The list of top-level matcher IDs to apply to the input map

string
outputMode: string replace

Acknowledgement data output mode "replace", "array" or "append"

service: string

UUID of the service

timeout: string

Acknowledgement timeout

Example
{
  "ackMessageCount": "integer (int64)",
  "characteristic": "2a26",
  "indication": "boolean",
  "keepNotificationsEnabled": "boolean",
  "match": "object",
  "matchAny": "boolean",
  "matchers": [
    "is-nearby",
    "is-my-device-type"
  ],
  "outputMode": "string",
  "service": "375aee4c-6dea-5105-9074-14d5ba44f766",
  "timeout": "string"
}

Cloud

connectors: object

Defines the cloud Connectors to connect to.

A map of unique identifier to a definition of a CloudConnector.

Connectors defined here will be connected to, and can be targeted with publish filters. Depending on CloudConnector implementation and configuration, they may also subscribe and generate cloud events.

Example
{
  "connectors": "object"
}

CloudConnector

Describes the configuration for a cloud connector. CloudConnectors can publish events to the cloud by being targeted with a publish or publishTopic filter in a pipeline.

Some CloudConnector types also support subscribing for cloud commands/events and subscribing on topics.

config: object

Configuration information for the particular type of CloudConnector used. Described in the documentation for each CloudConnector type.

type: string

The unique type of the CloudConnector, must match the type of one of the available CloudConnectors

Example
{
  "service": {
    "cloud": {
      "connectors": {
        "my-mqtt": {
          "type": "mqtt",
          "config": {
            "host": "iothub.myserver.net",
            "port": "8883",
            "scheme": "tcps"
          }
        }
      }
    }
  }
}

CloudConnectorDevices

Device configuration for connectors that support identification of individual devices.

Note: not all cloud connectors support all of these options. Support for each option is documented in the specific cloud connector.

attachAll: boolean false

If set to true, devices simply included in the referenced device group will be attached/subscribed. If set to false (default), devices will not be attached/subscribed until they are determined to be present.

connectMessage: CloudMessage

Message to send, per attached device, to the cloud when a connection is successfully established or re-established

detachMissing: boolean false

If set to true, devices will be detached/unsubscribed if they are removed from the device group, or if they go missing from presence tracking. If set to false (default) devices are only detached/unsubscribed when they are removed from the device group.

deviceIdTemplate: string

A text template for converting the lowercase mac field to any deviceId that is desired by the cloud configuration. This template is used to generate a Device ID that is then used to attach to the cloud and subscribe to topics. If no deviceIdTemplate is supplied, defaults to using the lowercase MAC address of the device.

NOTE: this should not be used if you are using the deviceIdTemplate on the device group that is connected to this. The Device ID should be generated using that template instead.

group: string

Device Group that contains the devices this cloud connector is concerned with. The connector will attach and subscribe to topics for devices contained in this group.

subscribeTopics: string[]

Per device topics to subscribe to. Incoming messages on these topics will arrive in pipelines as cloud type events.

Supports text templating. deviceId is available in the text template.

string
Example
{
  "service": {
    "cloud": {
      "connectors": {
        "my-mqtt": {
          "type": "mqtt",
          "config": {
            "devices": {
              "attachAll": true,
              "detachMissing": true,
              "deviceIdTemplate": "{{ .mac | ToUpper }}",
              "group": "my-sensors",
              "subscribeTopics": [
                "sensors/sensor_{{ .deviceId }}/commands"
              ]
            },
            "host": "iothub.myserver.net",
            "port": "8883",
            "scheme": "tcps"
          }
        }
      }
    }
  }
}

CloudMessage

A statically configured message to send to the cloud. Currently only used by Cloud Connectors that support the connectMessage parameter.

message: object

The message body

topic: string

Topic the message is on

Example
{
  "message": "object",
  "topic": "string"
}

Config

Top level configuration for the Edge Connect Service. Must be wrapped in the service key.

actions: object

Defines the ConnectionActions that can be used in ConnectionSequences.

A map of unique identifier to a ConnectionAction definition.

ConnectionActions defined here can be reused in multiple ConnectionSequences.

api: RestAPI
cloud: Cloud
connections: object

Defines the ConnectionSequences that will handle processing of BLE connections.

A map of unique identifier to a ConnectionSequence definition.

ConnectionSequences are triggered by a connect filter.

eventGenerators: object

Defines the EventGenerators to run.

A map of EventGenerator ID's to EventGeneratorConfig

filters: object

Defines the Filters available for use in Pipelines.

A map of unique identifier to a definition of a Filter.

Filters defined here can be reused in multiple Pipelines.

groups: object

Defines the DeviceGroups that can be referenced by Pipelines.

A map of unique identifier to DeviceGroup definition.

health: Health
imports: Imports
log: Log
matchers: object

Defines the Matchers that can be referenced by Pipelines or the matcher filter.

A map of matcher ID's to key/value pairs, or matcher template strings

metadata: Metadata
metrics: Metrics
minimumVersion: string

Set the minimum version of edge-connect required to run this configuration

pipelines: object

Defines the Pipelines of Filters that events will flow through.

A map of unique identifier to a Pipeline definition.

Example
{
  "service": {
    "metrics": {
      "reporting": {
        "intervalMinutes": 10
      }
    },
    "health": {
      "reporting": {
        "intervalMinutes": 10
      }
    },
    "api": {
      "port": 62307,
      "devmode": true
    },
    "log": {
      "level": "info",
      "printJSONEvents": false,
      "eventHistory": 0,
      "includeUnmatchedHistory": false,
      "stacktrace": false,
      "jdiff": false
    },
    "filters": {
      "eddyf": {
        "type": "eddystone"
      },
      "ruuvif": {
        "type": "ruuvi"
      }
    },
    "pipelines": {
      "eddypipe": {
        "filters": [
          "eddyf",
          "my-mqtt"
        ],
        "eventType": "advertisement",
        "groups": [
          "east-wing-eddystones"
        ]
      },
      "ruuvipipe": {
        "filters": [
          "ruuvif",
          "my-mqtt"
        ],
        "eventType": "advertisement"
      }
    },
    "groups": {
      "east-wing-eddystones": {
        "type": "static",
        "config": {
          "absenceMinutes": 5,
          "exclude": [
            "abcdef7",
            "abcdef8"
          ],
          "include": [
            "abcdef5",
            "abcdef6"
          ]
        }
      }
    },
    "cloud": {
      "connectors": {
        "my-mqtt": {
          "type": "mqtt",
          "config": {
            "base": "",
            "host": "iothub.myserver.net",
            "port": 8883,
            "scheme": "tcps"
          }
        }
      }
    }
  }
}

ConnectionAction

Action to take once BLE Connection is established. Chained together in a ConnectionSequence.

config: object

Configuration information for the particular type of Action used. Described in the documentation for each Action type.

In addition to the config parameters specific to each Action type, you may also specify these debug options in the config map:

  • debug - a bool indicating whether to perform debug logging
  • verbose - a bool indicating whether to perform verbose logging
  • debugInput

    Enable logging a custom map built from the input map; will override the default event logging. Define the map as key pairs (see `inject`), supports dot-notation, templating.

    e.g.: {"importantValue":"{{.nested.important.value}}"}

  • debugOutput

    Enable logging a custom map built from the output map; will override the default event logging.

    Define the map as key pairs (see inject), supports dot-notation, templating.

    e.g.: {"importantValue":"{{.nested.important.value}}"}

type: string

The unique type of the Action, must match the type of one of the available ConnectionActions.

Example
{
  "service": {
    "actions": {
      "read-custom-char": {
        "type": "read",
        "config": {
          "characteristic": "2a26",
          "longRead": false,
          "mode": "append",
          "outputKey": "logs",
          "service": "375aee4c-6dea-5105-9074-14d5ba44f766"
        }
      }
    }
  }
}

ConnectionSequence

Sequence of ConnectionActions to run upon BLE Connection. Triggered by the connect filter.

actions: string[]

List of ConnectionAction identifiers.

References the actions defined on Config.

string
allowEarlyExit: boolean false

Allow for the connection to exit, but proceed with the rest of the pipeline. Default behavior (false) will end the pipeline if the connection ends prematurely. This does not allow errors to proceed, only actions that return nil.

attempts: integer (int64) 5

When set to a value greater than 0, the set of actions for the connection will be attempted the number of specified times. Note that this only applies to when a connection is successfully made to a device. If a device is not present and the connection request times out, no further attempts to connect will occur until an advertisement from the device is seen again. This minimum number of attempts is 1. The maximum is 10.

debug: boolean false

Enable debug logging

verbose: boolean false

Enable verbose logging

Example
{
  "service": {
    "connections": {
      "readBatteryConn": {
        "actions": [
          "read-custom-battery",
          "update-custom-metadata"
        ]
      }
    }
  }
}

ConnectionSlotsConfig

BLE Connection Slots Configuration

connectionSlots: integer (int64) 6

Set the number of BLE connection slots The number of connection slots is the maximum number of concurrent BLE connections supported must be > 0

retryDelay: object 5s

Retry delay when no connection slot is available must be a positive duration string

retryTimeoutMultiplier: integer (int64) 4

Timeout multiplier for waiting for a BLE connection slot to be available The connect filter timeout is multiplied by this value to determine the connection slot timeout value. must be > 0

Example
{
  "connectionSlots": "integer (int64)",
  "retryDelay": "object",
  "retryTimeoutMultiplier": "integer (int64)"
}

ConverterInput

Used by the injectKeys parameter of the typeConvert Filter. Defines the source key, bytes (offset/count), and destination type of a []byte conversion

length: integer (int64) 0

Number of bytes to select starting from offset. Use 0 (default) to mean all bytes after offset.

offset: integer (int64) 0

Offset to select bytes from

source: string

Key containing the bytes to convert (supports dot-notation)

type: string

Type to convert the bytes to, see above for supported types

Example
{
  "length": 9,
  "offset": 3,
  "source": "rawData",
  "type": "base64"
}

DeviceBluetooth

legacySecurity: boolean false
supportsBonding: boolean false

Whether or not the device supports permanent bonds. If this is set, it adds an optional device property 'bluetooth.longTermKey' that can be set with a pre-shared key that allows bonding/pairing with the device.

Example
{
  "legacySecurity": "boolean",
  "supportsBonding": "boolean"
}

DeviceGroup

Describes the configuration for a group of devices. Groups can be used to restrict a Pipeline to only see events from certain devices.

They can also be used to define a group of devices that are expected to be present.

Only types static or persistent are currently supported.

config: object

Configuration information for the particular type of DeviceGroup used. Described in the documentation for each DeviceGroup type.

type: string

The unique type of the DeviceGroup, must match the type of one of the available DeviceGroups

Example
{
  "service": {
    "groups": {
      "my-eddystones": {
        "type": "static",
        "config": {
          "absenceMinutes": 19,
          "exclude": [
            "eddymac4",
            "eddymac5",
            "eddymac6"
          ],
          "include": [
            "eddymac1",
            "eddymac2",
            "eddymac3"
          ]
        }
      }
    }
  }
}

DeviceGroupBattery

DeviceGroupBattery Configuration for DeviceGroup battery level monitoring.

enabled: boolean false
failPercent: number (float) 1

The percentage at which to trigger a Status of Fail

warnPercent: number (float) 20

The percentage at which to trigger a Status of Warn. Anything greater than this will be Status OK

Example
{
  "enabled": "boolean",
  "failPercent": "number (float)",
  "warnPercent": "number (float)"
}

DeviceProp

DeviceProp describes a property that is settable/gettable on a device. See the propSet filter and {{ Device.Props }} template function.

persistent: boolean
Example
{
  "persistent": "boolean"
}

DeviceType

Definition for a particular type of Device that Edge Connect will interact with. DeviceTypes can be used to add properties to a device and can be used to define a device group. A DeviceType minimally has an ID and a way to identify the devices from advertisements using matchers.

allowUnknownProps: boolean false

True if devices of this type allow undefined properties to be set on them

deviceIdTemplate: string {{ .mac }}

The template used to generate the deviceId for devices of this type

id: string

Unique ID of the DeviceType

match: object

Keys to match (all keys) for an inline definition (see MatcherDefinition), or a Template to evaluate and match to a true value.

matchAny: boolean false

Should we match any of the provided matchers (true)? Or all of the matchers (false)

matchers: string[]

The list of top-level matcher IDs to apply to the input map

string
props: object

Properties that can be set/get on devices of this type. See the propSet filter and the Device.Props template function.

Example
{
  "allowUnknownProps": "boolean",
  "deviceIdTemplate": "mydevice_{{ .mac | ToUpper }}",
  "id": "rs40",
  "match": "object",
  "matchAny": "boolean",
  "matchers": [
    "is-nearby",
    "is-my-device-type"
  ],
  "props": "object"
}

DevicesConfig

Configuration of Devices and Device Templates

types: DeviceType

An ordered list of DeviceTypes to try to apply to any incoming events from devices, in order to identify them.

DeviceType
Example
{
  "types": [
    {
      "allowUnknownProps": "boolean",
      "deviceIdTemplate": "mydevice_{{ .mac | ToUpper }}",
      "id": "rs40",
      "match": "object",
      "matchAny": "boolean",
      "matchers": [
        "is-nearby",
        "is-my-device-type"
      ],
      "props": "object"
    }
  ]
}

EncodedField

For use with the decodeLTV and decodeType filter/action. EncodedField describes one field in an EncodedType. A string template can be specified to manipulate the decoded data, this will be output as a string.

If field length is set to 0, a template must be specified in order to compute a field. This can be useful for including items based on the index (id) of the containing EncodedType instance , or for incorporating other computed items like timestamps. In the case of a computed field, type is ignored and the output will be string unless convertTemplate is set a valid type for string conversion.

See the typeConvert filter for details on available types for byte array or string conversions.

key: string

The key to put the decoded data in

length: integer (int64)

The number of bytes that represents this fields value.

template: string

A template string to process the value with after decoding. The full event will be available as well as the decoded value as .item.value and the offset of the containing EncodedType as .item.id.

Note: this means the final value will be a string, unless this is a computed field. To create a computed field, the length must be set to 0, and the type must be a valid type converted from a string. See above in the EncodedField description for more details about computed fields.

type: string

The type of the data. E.g. string,int,etc Supports any of the types supported by the typeConvert filter when converting from []byte

Example
{
  "key": "string",
  "length": "integer (int64)",
  "template": "string",
  "type": "string"
}

EncodedLTV

For use with the decodeLTV filter/action. EncodedLTV describes a type containing multiple fields that can be encoded in a byte stream, identified by a type.

appendToArray: string

If set to a non-empty string, instances of this type will be appended to an array with this key, rather than merged with the main output event.

fields: EncodedField

The fields of the type, in order that they are encoded in the data.

EncodedField
id: integer (uint8)

The unique ID of the type this describes.

Example
{
  "appendToArray": "string",
  "fields": [
    {
      "key": "string",
      "length": "integer (int64)",
      "template": "string",
      "type": "string"
    }
  ],
  "id": "integer (uint8)"
}

EncodedType

For use with the decodeType filter/action. EncodedType describes a type containing multiple fields that can be encoded in a byte stream.

fields: EncodedField

The fields of the type, in order that they are encoded in the data.

EncodedField
Example
{
  "fields": [
    {
      "key": "string",
      "length": "integer (int64)",
      "template": "string",
      "type": "string"
    }
  ]
}

EventGenerator

Describes the configuration for an EventGenerator. For an example of an EventGenerator, see timer.

config: object

Configuration information for the particular type of EventGenerator used. Described in the documentation for each event generator type.

type: string

The unique type of the EventGenerator.

Example
{
  "config": "object",
  "type": "timer"
}

Filter

Filter describes the configuration for one Filter Node in a Pipeline

config: object

Configuration information for the particular type of Filter used. Described in the documentation for each Filter type.

In addition to the config parameters specific to each filter type, you may also specify these debug options in the config map:

  • debug - a bool indicating whether to perform debug logging
  • verbose - a bool indicating whether to perform verbose logging
  • debugInput

    Enable logging a custom map built from the input map; will override the default event logging. Define the map as key pairs (see `inject`), supports dot-notation, templating.

    e.g.: {"importantValue":"{{.nested.important.value}}"}

  • debugOutput

    Enable logging a custom map built from the output map; will override the default event logging.

    Define the map as key pairs (see inject), supports dot-notation, templating.

    e.g.: {"importantValue":"{{.nested.important.value}}"}

type: string

The unique type of the Filter, must match the type of one of the available Filters

Example
{
  "config": "object",
  "type": "inject"
}

Health

reporting: Reporting
watchdogRestartMinutes: integer (int64) 10

The number of minutes with no advertisements processed that will trigger an auto restart of Bluetooth/HCI.

Example
{
  "service": {
    "health": {
      "reporting": {
        "filterId": "my-mqtt",
        "topic": "/health",
        "secFrequency": 900,
        "intervalMinutes": 0
      }
    }
  }
}

Imports

Specify recipes to import and set configuration for the imported recipes.

recipes: object[]

Defines recipes to import Can accept named recipes ( [email protected]) or inline recipe json.

object
variables: object

Define recipe variables to set Configure a recipe variable to a non-default value, the recipe variables are defined in the form recipeName.variableName. If no values are provided here, the recipe default values will be used.

Example
{
  "recipes": [
    "[email protected]"
  ],
  "variables": {
    "myRecipe.connectionTimeout": "20s"
  }
}

KKMEslConfig

Describes configuration for KKM Electronic Shelf Labels. Must be used with the kkm action.

mode: string

Mode specifies the operation to be performed on the display. Valid options are play and download. If download is specified, then playAfterDownload is used to control the behavior upon successful download.

nodeId: int64

NodeId is the id of the node to play or to which the picture //data is downloaded. Generally, valid values are 0,1,2,3.

pictureData: bytes

PictureData is the data for the picture to be downloaded. Required when the mode is download.

pictureId: int64

PictureId is the id of the picture that will be downloaded to the display. If the PictureId already exists on the device the download will be skipped. If PlayAfterDownload is true, the node with the matching PictureId will be played. Required for download mode.

playAfterDownload: bool

PlayAfterDownload denotes that a downloaded image should be played once the download is successful. If not specified, the picture will only be downloaded to the device.

Example
{
  "mode": "play",
  "nodeId": 2
}

KKMSocialBandConfig

Describes configuration for KKM Social Bands. Must be used with the kkm action.

advPower: integer (int64)

Not documented. This is only for testing and development purposes.

alarmAction: integer (int64)

This value specifies the behavior of the band when an alarm occurs. The options are off (0), vibrate (2), audible (8), vibrate and audible (10)

alarmDelay: integer (int64)

This value specifies the amount of time that must pass while two devices are within range of each other before an alarm will activate. Value is seconds

alarmDistance: integer (int64)

This value controls the distance at which the social distance bands will alarm. It is specified in .1 m units. Note that 1.8m is ~6ft.

alarmInterval: integer (int64)

This value specifies the time between alarms when an alarm occurs. Specified in 100ms increments.

keepAfterRead: boolean false

If set to true, keep after read leaves the contact records on the band. Otherwise, they are cleared once they are successfully read from the device.

recordDelay: integer (int64)

This value specifies the amount of time that must pass while two devices are within alarm range of each other before a contact record will be recorded.

Example
{
  "advPower": "integer (int64)",
  "alarmAction": "integer (int64)",
  "alarmDelay": "integer (int64)",
  "alarmDistance": "integer (int64)",
  "alarmInterval": "integer (int64)",
  "keepAfterRead": "boolean",
  "recordDelay": "integer (int64)"
}

Log

Log describes the logging configuration

bleDebug: boolean false

If true, enable debug logging from the BLE layer

eventHistory: integer (int64) -1

Number of pipeline events to keep in memory in a rolling buffer, for debugging purposes. Can be accessed via /pipelines/events API endpoint Set to -1 to disable, which is the default. Maximum value 10000.

excludeEventKeys: string[]

Any event keys provided will have their values replaced with [excluded] when printing the event. These keys will be ignored in any map at any nesting level and dot-notation is ignored.

To exclude nestedMap.key1, add key1 to excludeEventKeys. Note that all other instances of key1 will also be excluded (otherNestedMap.key1, etc).

Does not apply if printJSONEvents is true

string
includeUnmatchedHistory: boolean false

Include pipeline events in history that don't match any pipelines.

jdiff: boolean false

Use jdiff to show only changes in events instead of the whole map when logging pipeline/connection events

level: string info

The log level. Valid options are: debug, info, error, fatal

printJSONEvents: boolean false

If true, events will be printed as indented JSON instead of the native "map:[key:value]" format note: this will have a significant performance penalty, use for debugging only

printTimestamps: boolean false

If true, log messages will contain timestamps.

suppressAdvDecodeErrors: boolean false

If true, advertisement decode errors will be suppressed

Example
{
  "service": {
    "log": {
      "level": "info",
      "printJSONEvents": false,
      "eventHistory": 0,
      "includeUnmatchedHistory": false,
      "stacktrace": false,
      "jdiff": false
    }
  }
}

Matcher

Configures a set of key/value pairs for purposes of matching an input map. Allowable value types are string (match one value), []interface{} (match any of the values).

For example, the config below would match an event with: "key1"=("value1" OR "value2") AND "key2"="value3"

{ "key1": []interface{}{"value1", "value2"}, "key2": "value3", }
object
Example
{
  "rssi": ">-40",
  "serviceData.fd95": ""
}

MatcherSet

Configures a set of key/value pairs as well as a list of top level matchers

match: object

Keys to match (all keys) for an inline definition (see MatcherDefinition), or a Template to evaluate and match to a true value.

matchAny: boolean false

Should we match any of the provided matchers (true)? Or all of the matchers (false)

matchers: string[]

The list of top-level matcher IDs to apply to the input map

string
Example
{
  "match": "object",
  "matchAny": "boolean",
  "matchers": [
    "is-nearby",
    "is-my-device-type"
  ]
}

Metadata

Configures the initial map of metadata that is loaded at startup. Device specific identifiers, api keys, and other application specific items can be defined here for later lookup in Pipelines or ConnectionSequences.

devices: object

Device metadata to initialize

global: object

Global metadata to initialize

persistent: boolean

Store device and global metadata persistently on the file system. Metadata will be reloaded if Edge Connect restarts, including if the gateway is rebooted.

transientKeys: string[]

A list of keys that will always be in memory, and never persisted. This will include their sub-items. For example, if you specify "status" then "status.active" will also be transient.

string
Example
{
  "global": {
    "secretAPIKey": "sdfjhsdfjkhsdfjk"
  },
  "devices": {
    "someMacAddress": {
      "mySensorID": "sensor1234124"
    }
  }
}

Metrics

reporting: Reporting
Example
{
  "service": {
    "metrics": {
      "reporting": {
        "filterId": "my-mqtt",
        "topic": "/metrics",
        "secFrequency": 60,
        "intervalMinutes": 0
      }
    }
  }
}

Pipeline

Pipeline describes the configuration for one event processing pipeline. Pipelines target 'advertisement' events by default, but can target any event type.

debug: boolean false

Enable debug logging

disabled: boolean false

If set to true, this pipeline will be disabled and will not process events.

eventType: object advertisement

The type of events to process in this Pipeline. Specified as a string or list of strings

Available Options:

  • advertisement (default) for BLE advertising events
  • deviceDetected
  • deviceHealth
  • deviceAssignment
  • deviceStatus for events about newly discovered devices, devices that have disappeared, and (optionally) batteryLevel
  • deviceGroup for events when a device is added or removed from a device group
  • cloud for events from cloud subscriptions
  • any match any event type (including custom events)
  • ^regular expression$ use a regular expression to match events (must start with ^ and end with $)
  • custom event type match only the custom event type emitted by a recipe or the emitEvent filter
filters: object[]

The list of Filters, in order, that event data will flow through.

The list can contain a mix of strings, or objects.

If a string is provided, it must match a named filter defined in the filters section of Config.

If an object is provided, it must have one key:value pair. The key must the the type of filter and the value must be the config object for that particular filter.

References the filters defined on Config

object
groups: string[]

The list of DeviceGroups to receive events for. Defaults to an empty list, which means this pipeline will receive all events that match its eventType, regardless of DeviceGroup.

References the groups defined on Config

string
match: object

Keys to match (all keys) for an inline definition (see MatcherDefinition), or a Template to evaluate and match to a true value.

matchAny: boolean false

Should we match any of the provided matchers (true)? Or all of the matchers (false)

matchers: string[]

The list of top-level matcher IDs to apply to the input map

string
verbose: boolean false

Enable verbose logging

Example
{
  "service": {
    "pipelines": {
      "eddypipe": {
        "filters": [
          "eddyf",
          {
            "inject": {
              "keyPairs": {
                "source": "an inline filter!"
              }
            }
          },
          "my-mqtt"
        ],
        "eventType": "advertisement",
        "groups": [
          "east-wing-eddystones"
        ]
      }
    }
  }
}

Processing

Set up devicegroups, pipelines, connections, cloud connectors, filters, connection actions, etc.

actions: object

Defines the ConnectionActions that can be used in ConnectionSequences.

A map of unique identifier to a ConnectionAction definition.

ConnectionActions defined here can be reused in multiple ConnectionSequences.

cloud: Cloud
connections: object

Defines the ConnectionSequences that will handle processing of BLE connections.

A map of unique identifier to a ConnectionSequence definition.

ConnectionSequences are triggered by a connect filter.

devices: DevicesConfig
eventGenerators: object

Defines the EventGenerators to run.

A map of EventGenerator ID's to EventGeneratorConfig

filters: object

Defines the Filters available for use in Pipelines.

A map of unique identifier to a definition of a Filter.

Filters defined here can be reused in multiple Pipelines.

groups: object

Defines the DeviceGroups that can be referenced by Pipelines.

A map of unique identifier to DeviceGroup definition.

matchers: object

Defines the Matchers that can be referenced by Pipelines or the matcher filter.

A map of matcher ID's to key/value pairs, or matcher template strings

metadata: Metadata
minimumVersion: string

Set the minimum version of edge-connect required to run this configuration

pipelines: object

Defines the Pipelines of Filters that events will flow through.

A map of unique identifier to a Pipeline definition.

Example
{
  "actions": "object",
  "cloud": {
    "connectors": "object"
  },
  "connections": "object",
  "devices": {
    "types": [
      {
        "allowUnknownProps": "boolean",
        "deviceIdTemplate": "mydevice_{{ .mac | ToUpper }}",
        "id": "rs40",
        "match": "object",
        "matchAny": "boolean",
        "matchers": [
          "is-nearby",
          "is-my-device-type"
        ],
        "props": "object"
      }
    ]
  },
  "eventGenerators": "object",
  "filters": "object",
  "groups": "object",
  "matchers": "object",
  "metadata": {
    "global": {
      "secretAPIKey": "sdfjhsdfjkhsdfjk"
    },
    "devices": {
      "someMacAddress": {
        "mySensorID": "sensor1234124"
      }
    }
  },
  "minimumVersion": "2021.01-01",
  "pipelines": "object"
}

Recipe

A recipe will typically consume events and emit events for a top-level configuration to consume.

To receive all/multiple from a recipe a pipeline's eventType can be specified as a regular expression (must start with '^' or end with '$')

For example "eventType": "^mysensor\..+" would receive all events emitted by mysensor.

For consistency, all the identifiers (filter, action, pipelines, connections, etc) in a recipe must contain only alphanumeric or '-', '/' characters and start with a lowercase alphanumeric character.

description: string

Recipe description string

emits: object

Declaration of events that the recipe emits map key is the eventType name (accessed via recipeName.eventTypeName) map value is the metadata for the eventType

exports: object

Recipe export declarations map key is the exported name (accessed via recipe.exportedName) map value is the metadata for the exported item

imports: object[]

Recipe imports Specify imports required by the recipe. Imports can be recipe names or inline json.

object
name: string

Recipe name

recipe: Processing
variables: object

Recipe variable declarations map key is the exposed variable name (accessed via recipe.variableName) map value is the metadata for the variable

version: string

Recipe version string Must follow semantic versioning rules

Example
{
  "name": "myRecipe",
  "version": "1.0.0",
  "imports": [
    "[email protected]"
  ],
  "variables": {
    "aggregationInterval": {
      "key": "filters.aggregator.config.interval",
      "description": "set the aggregation interval for advertisements",
      "default": "10s"
    }
  },
  "exports": {
    "hasCheesyEnding": {
      "description": "matcher that matches when the value stored in key 'string' ends in 'cheese'"
    }
  },
  "emits": {
    "aggregateEvent": {
      "description": "aggregated advertisement data"
    }
  },
  "recipe": {
    "filters": {
      "aggregator": {
        "type": "aggregate",
        "config": {
          "interval": "10s",
          "mode": "map"
        }
      },
      "emitter": {
        "type": "emitEvent",
        "config": {
          "eventType": "myRecipe.aggregateEvent",
          "interval": "10s"
        }
      }
    },
    "pipelines": {
      "scanner": {
        "filters": [
          "aggregator",
          "emitter"
        ],
        "eventType": [
          "advertisement"
        ]
      }
    },
    "matchers": {
      "cheesyEnding": {
        "string": ".*cheese$"
      }
    }
  }
}

RecipeEvent

All emitted eventTypes must be declared by the recipe.

description: string

Description of the recipe eventType

Example
{
  "description": "description of the emitted event"
}

RecipeExport

Once exported, the item can be used in the top-level configuration as well as other recipes by recipeName.exportedName. Any processing unit can be exported (filter, action, pipeline, connection, matcher, devicegroup, etc).

The internal lookup is lazy so if you have a filter named myProcessor and an action named myProcessor, and try to export myProcessor an error will be thrown.

description: string

Description of the exported item

Example
{
  "description": "description of the exported item"
}

RecipeVariable

Once a RecipeVariable is declared, the value can be set from the top-level Imports configuration.

default: object

The default value of the variable in the recipe

description: string

The description of the variable

key: object

The path to the values in the recipe, can also give an array of keys

Example
{
  "key": "filters.aggregator.config.interval",
  "description": "set the aggregation interval for advertisements",
  "default": "10s"
}

ReduceConfig

Configuration for processing of the reduce filter.

reduce: string

Reduce template The array of values extracted with the value template will be passed into this template under the key __items (type []interface{}) for processing.

value: string

Value template used to collect an array of values from a collection of maps. If the items in the collection is not a map, then the value will be wrapped in the key item. The array of collected values is passed to the reduce function under key __items.

Example
{
  "reduce": "{{ maxf .__items }}",
  "value": "{{ .rssi }}"
}

Reporting

intervalMinutes: integer (int64) 10

The frequency in minutes in which data will be reported to cloud-based monitoring.

Example
{
  "service": {
    "metrics": {
      "reporting": {
        "filterId": "my-mqtt",
        "topic": "/metrics",
        "secFrequency": 60,
        "intervalMinutes": 0
      }
    },
    "health": {
      "reporting": {
        "filterId": "my-mqtt",
        "topic": "/health",
        "secFrequency": 900,
        "intervalMinutes": 0
      }
    }
  }
}

RestAPI

RestAPI describes the configuration for the API server running inside of Edge Connect

devmode: boolean false

If devmode is set to true, the API server is accessible from any IP, not just localhost. If set to false, the API is only available locally.

port: integer (int64) 62307

The port to host the API server on

Example
{
  "service": {
    "api": {
      "port": 62307,
      "devmode": true
    }
  }
}

Template

Templates are strings used to compute strings or other values. They can be used for a number of key/value pairs in configuration. Refer to each filter / action / cloud connector for places in which templates can be used.

See Edge Connect Templates for more detail.

Example
"object"

Connection Actions

encrypt

Attempts to enable encryption using previously obtained encryption information. If the encryption information is not available, this command will fail.

Output: None

Example
{
  "service": {
    "actions": {
      "encrypt-comm": {
        "type": "encrypt"
      }
    }
  }
}

kkm

Reads/writes data to the specified KKM device. Supported devices types are display and socialDistance. Each type of device has a specific configuration.

device: string

The type of device this action will interact with. Valid values are display and socialDistance.

displayConfig: KKMEslConfig
outputKey: string ""

The key to which output data will be written. If not specified for the socialDistance device, the default output key is contactRecords.

socialBand: KKMSocialBandConfig
Example
{
  "service": {
    "actions": {
      "loop-example": {
        "type": "kkm",
        "config": {
          "device": "display",
          "displayConfig": {
            "mode": "play",
            "nodeId": 2
          },
          "outputKey": ""
        }
      }
    }
  }
}

loop

The loop action allows you to perform a connection sequence in a loop. There are 3 modes of operation for the loop action:

  • In count mode, the connection sequence is executed N times by providing count.
  • In matcher mode, the connection sequence is executed while the matcher is true.
  • In collection mode, the connection sequence is executed with each item from the collection stored in item.id/item.value.

Output: output of the underlying connection sequence

collection: string

Iterate over a collection must be a template that outputs a map[string]interface{} or []interface

condition: PolicySet
count: integer (int64)

Provide the number of iterations to run the loop to enable count mode.

id: string

The ID of the Connection Sequence that should be looped

match: object

Keys to match (all keys) for an inline definition (see MatcherDefinition), or a Template to evaluate and match to a true value.

matchAny: boolean false

Should we match any of the provided matchers (true)? Or all of the matchers (false)

matchers: string[]

The list of top-level matcher IDs to apply to the input map

string
Example
{
  "service": {
    "actions": {
      "loop-example": {
        "type": "loop",
        "config": {
          "id": "read-more-data",
          "match": {
            "more-data": "true"
          }
        }
      }
    }
  }
}

notificationControl

Enable/disable notifications/indications for a characteristic

This action enables/disables notifications for a characteristic. To receive notifications, use the wait action.

characteristic: string

The UUID of characteristic to read supports templating

enable: boolean

Enable or disable notifications/indications for the characteristic

indication: object

If true, indications are enabled for the characteristic boolean or template string

service: string

The UUID of the service to which the characteristic belongs supports templating

Example
{
  "service": {
    "actions": {
      "turn-on-notis": {
        "type": "notificationControl",
        "config": {
          "characteristic": "2a26",
          "enable": false,
          "indication": true,
          "service": "375aee4c-6dea-5105-9074-14d5ba44f766"
        }
      }
    }
  }
}

pair

Attempts to create a pairing with the connected device.

Edge Connect will defer to the pairing parameters specified in the device pairing response. If bonding is specified, the encryption key will be stored locally. If the device is already paired, this action may fail. Legacy and LESC pairings are supported. JustWorks, Passkey, and OOB Data are all supported pairing methods.

To use Passkey, supply the key passkey within the input map. The Passkey must be either a numeric string ("123456") or a number (123456). Valid values are 0-999999, inclusive.

To use OOB Data, specify the key oobData in the input map. The OOB Data must be specified as bytes encoded in a base64 string or a byte array.

Note: The actual pairing method used, regardless of the specified key, is based on the table in the Bluetooth specification found here: Core v5.0 Vol 3, Part H, 2.3.5.1 Tables 2.6, 2.7, and 2.8

The use of Passkey and OOB Data are mutually exclusive. If both are specified, execution will fail with an error.

Output: The input map without changes

Example
{
  "service": {
    "actions": {
      "pair-with-device": {
        "type": "pair"
      }
    }
  }
}

read

Read the value of a characteristic

Output: This action adds outputKey containing the data read from the characteristic as an array of bytes.

Data from this action can be sent to the write action by using an output key of writeData see the write action docs for more details.

characteristic: string

The UUID of characteristic to read.
Supports templating

longRead: bool

Specifies that the reads of this characteristic are longer than the maximum read length for BLE. In this case, the Read Blob operation will be used to read the data.
Supports templating

mode: string

Specifies how the output data should be treated.
If replace, existing data will be replaced when read. outputs ([]byte)
If append, the read data will be appended to an existing key. outputs ([]byte)
If array, each read will be placed into an array of read data. outputs ([][]byte)
A new key will be added if the key does not exist upon the first read in append or array mode.
Supports templating

outputKey: string

The map key for the output data.
Supports dot notation and text templating.

service: string

The UUID of the service to which the characteristic belongs.
Supports templating

Example
{
  "service": {
    "actions": {
      "read-custom-char": {
        "type": "read",
        "config": {
          "characteristic": "2a26",
          "longRead": false,
          "mode": "append",
          "outputKey": "logs",
          "service": "375aee4c-6dea-5105-9074-14d5ba44f766"
        }
      }
    }
  }
}

readBattery

Attempts to read the battery level characteristic from the standard GATT Battery Service. The service UUID is 180f and the value characteristic UUID is 180a.

Output: The battery level as a percentage (0-100) with an output key as specified in the configuration.

outputKey: string batteryLevel

The key name used to store the battery level Supports dot notation and text templating.

Example
{
  "service": {
    "actions": {
      "read-the-battery": {
        "type": "readBattery",
        "config": {
          "outputKey": "batteryLevel "
        }
      }
    }
  }
}

readDeviceInfo

Read all available characteristics of the device information service if available. All data is placed into a map using the specified output key.

Output: Outputs the following keys with their values when the data is present: mfgName, modelNumber, serialNumber, hwRev, swRev, fwRev, systemId, ieeeCertList, pnpId.

outputKey: string

Specifies the key to be used for the output. Supports dot notation and text templating.

Example
{
  "service": {
    "actions": {
      "read-all-the-device-info": {
        "type": "readDeviceInfo",
        "config": {
          "outputKey": "deviceInfo"
        }
      }
    }
  }
}

run

Runs another connection sequence with an optional condition.

The connection sequence specified by the id parameter is run before this action completes and moves to the next action in this sequence.

When the if parameter is supplied the sequence is only run if the matchers in the condition pass. If the matchers do not pass, execution proceeds to the next action subsequent to this one, rather than exit.

id: string

The ID of the connection sequence to run.

if: MatcherSet
Example
{
  "service": {
    "actions": {
      "run-example": {
        "type": "run",
        "config": {
          "id": "read-more-data",
          "if": {
            "match": {
              "needToRead": "true"
            }
          }
        }
      }
    }
  }
}

wait

Wait pauses a set of actions until some behavior occurs. Typical waiting patterns are for a certain mount of time, or to await a notification/indication to occur.

Output: Waiting for a notification will output the input map plus the notification data. Waiting for a certain period of time pass data through to the next action.

characteristic: string

UUID of characteristic to wait for a notification from.
Supports templating

indicationOverride: object

Dynamically override notification or indication modes with this template

interval: positiveDuration

The amount of time to wait for.
When the mode is time, this specifies how long to wait.
When the mode is notification, this specifies how long to wait for the notification. If the notification is not received in that time, a nil result will be placed in the output key.

keepNotificationsEnabled: boolean

When the mode is notification, setting this true will prevent the notifications from being disabled

mode: string

The type of wait to perform. The options are: time, notification, indication

multi: boolean

If true, collect multiple notifications/indications until a timeout is reached. Must recieve at least one notification/indication for success.
Combine with:
outputMode = append/array
mode = notification/indication

outputKey: string

When the mode is notification, this specifies the output keyfor the notification data.
Supports dot notation/templating.

outputMode: string replace

When the mode is notification, this specifies the output mode
replace will replace the contents with the new notification data (outputs []byte)
append will append the new notification data to old notification data (outputs []byte)
array will add the new notification data to an array of notification data (outputs [][]byte)
When mode is append, the last item to be appended is also inserted into the lastItem key.

service: string

The UUID of the service to wait for a notification from.
Supports templating

Example
{
  "service": {
    "actions": {
      "wait-for-something": {
        "type": "wait",
        "config": {
          "interval": 10000000000,
          "keepNotificationsEnabled": false,
          "mode": "time",
          "outputKey": "",
          "outputMode": ""
        }
      }
    }
  }
}

write

Writes data to a characteristic.

If the input event contains any of the following data payload keys with the specified types, data from the event will be converted to bytes, deleted from the event and written to the specified service/characteristic. If no data payload key is present, the data provided in the data configuration item will be written, In the case that data is not set and no data payload key is present, an error will be returned.

Data Payload Keys:

  • writeData: hex string -> []byte or []byte (no conversion)
  • writeBase64: base64 string -> []byte or []interface{} with base64 strings -> [][]byte
  • writeString: ascii string -> []byte

Output: None

acknowledge: AcknowledgeConfig
characteristic: string

UUID of characteristic to read supports templating

data: string

The data to write to the characteristic formatted as a hexadecimal or base64 string. While data is not required, if no data payload key (see above) is present this action will report an error.

header: string

Add a template or hexstring header

headerEveryWrite: boolean false

Add the header to every write in chunked mode

maxWriteSize: integer (int64) 0

Set maximum number of bytes to write at once (0 for disabled)

service: string

UUID of the service to which the characteristic belongs supports templating

withResponse: object true

Should perform write and wait for response? boolean or template

Example
{
  "service": {
    "actions": {
      "write-something": {
        "type": "write",
        "config": {
          "characteristic": "2a26",
          "data": "01ae489c25",
          "service": "375aee4c-6dea-5105-9074-14d5ba44f766"
        }
      }
    }
  }
}

Cloud Connectors

azuredps

This cloud connector provides per-device connections to Azure IoT Hub, with connections provisioned by Azure DPS. Each device that communicates with this cloud connector will get a dedicated connection with authorization from the Azure DPS service.

Cloud to device messages arrive in pipelines as an event with cloud eventType. The payload is included in the payload key. If you want the payload to be the top level event map, use the unwrapCloudPayload option.

DirectMethods also fire as cloud eventType events, but they also include a _requestId key. To respond to the DirectMethod, publish to this cloud connector and make sure to include the _requestId in the event map. The _requestId doesn't need to be in the actual payload published to Azure, just in the event map that reaches the publish filter.

DirectMethods in Edge Connect only support map/object input and map/object response, not primitives.

Regular telemetry publishes (non-direct method responses) support the _properties key in the event map. If this key is included and it contains a map of strings, those key/value pairs will be added to the published message as message properties.

capabilityModelIdKey: string capabilityModelId

The key that contains the Capability Model ID of the device.

credentialLifetime: positiveDuration

credential lifetime (duration string)

debug: boolean false

Enable debug logging

devices: object

Device configuration, primarily for translating MAC address to Device IDs

deviceIdTemplate: string

A text template for generating the Azure deviceId that is desired by the cloud configuration. This template is used to generate a Device ID that is then used to attach to the cloud and subscribe to topics. If no deviceIdTemplate is supplied, defaults to using the 'deviceId' field in the event. If that is not used, the lowercase MAC address of the device is used.

directMethods: object

Direct Methods A map of capabilityModelIds to a list of commands that they support

directMethods: string[]

A list of Direct Methods to subscribe to for all device types

string
host: string

The DPS Registration Host URL

idScope: string

The ID Scope of the Azure DPS Service. A unique value used by the IoT Hub Device Provisioning Service.

nonPersistentMaxCount: integer (int64) 1000

If set to > 0, this will set the maximum number of messages the MQTT memorystore will hold before dropping messages Must be used with persistent=false

persistent: boolean false

If set to true, outbound and inbound MQTT messages are stored on disk instead of in memory, allowing them to be reloaded upon restart or reboot if they were not yet processed.

persistentMaxSize: string 1GiB

If set to > 0, this will set the maximum size (bytes) on disk that the MQTT filestore can consume before dropping messages. Supported suffixes (B, KB, KiB, MB, MiB, GB, GiB,...), max size of 1GiB. Must be used with persistent=true

previewAPI: boolean false

Use the new preview API of DPS

reconnectTimeout: string 1m

How long to wait before retrying to connect a failed/disconnected device to IOT Hub This is used when a device is waiting for approval, gets blocked, or gets otherwise disconnected. Supports durations in the form of 1s 5m 1h etc.

registerOnReconnect: boolean false

Re-register devices with DPS and ask for hub assignment on every reconnect. Enabling this will slow down reconnect performance but can help with device load balancing and failover on IOT Hub. Restarting Edge Connect will always cause all devices to request hub assignment.

sasKey: string

The Shared Access Signature key to use to generate device keys when authenticating with Azure DPS. This is required unless the key dpsDeviceKey is provided for each device in the event map when connecting.

unwrapCloudPayload: boolean false

Put the payload from cloud messages in the top level event map, rather than wrapping them in the "payload" key

verbose: boolean false

Enable verbose logging

Example
{
  "service": {
    "cloud": {
      "connectors": {
        "my-dps": {
          "type": "azuredps",
          "config": {
            "devices": {
              "deviceIdTemplate": "my_device_{{ .mac }}"
            },
            "host": "https://global.azure-devices-provisioning.net",
            "idScope": "0123f0_redacted",
            "sasKey": "abcdf_redacted_abcd"
          }
        }
      }
    }
  }
}

azureiot

This cloud connector provides a connection to the Azure IoT Device cloud.

Supports publishing telemetry and receiving cloud commands, but does not support topic based publishing, or topic based subscriptions.

Cloud to device messages arrive in pipelines as an event with cloud eventType. The payload is included in the payload key. If you want the payload to be the top level event map, use the unwrapCloudPayload option.

DirectMethods also fire as cloud eventType events, but they also include a requestId key. To respond to the DirectMethod, publish to this cloud connector and make sure to include the requestId in the event map. The requestId doesn't need to be in the actual payload published to Azure, just in the event map that reaches the publish filter.

DirectMethods in Edge Connect only support map/object input and map/object response, not primitives.

connectionString: string

The connection string provided by Azure IoT for the connecting device.

credentialLifetime: positiveDuration

credential lifetime (duration string)

debug: boolean false

Enable debug logging

directMethods: string[]

Direct Methods to subscribe to

string
nonPersistentMaxCount: integer (int64) 1000

If set to > 0, this will set the maximum number of messages the MQTT memorystore will hold before dropping messages Must be used with persistent=false

persistent: boolean false

If set to true, outbound and inbound MQTT messages are stored on disk instead of in memory, allowing them to be reloaded upon restart or reboot if they were not yet processed.

persistentMaxSize: string 1GiB

If set to > 0, this will set the maximum size (bytes) on disk that the MQTT filestore can consume before dropping messages. Supported suffixes (B, KB, KiB, MB, MiB, GB, GiB,...), max size of 1GiB. Must be used with persistent=true

reconnectTimeout: string 1m

How long to wait before retrying to connect to IOT Hub Supports durations in the form of 1s 5m 1h etc.

unwrapCloudPayload: boolean false

Put the payload from cloud messages in the top level event map, rather than wrapping them in the "payload" key

verbose: boolean false

Enable verbose logging

Example
{
  "service": {
    "cloud": {
      "connectors": {
        "my-azure": {
          "type": "azureiot",
          "config": {
            "connectionString": "azureiot:connectionstring"
          }
        }
      }
    }
  }
}

azureiotcentral

This cloud connector provides per-device connections to the Azure IoT Central Cloud. Each device that communicates with this cloud connector will get a dedicated connection with authorization from the Azure DPS service.

Commands fire as cloud eventType events, and include a requestId key. To respond to the Command, publish to this cloud connector and make sure to include the requestId in the event map. The requestId doesn't need to be in the actual payload published to Azure, just in the event map that reaches the publish filter.

Commands in Edge Connect only support map/object input and map/object response, not primitives.

capabilityModelIdKey: string capabilityModelId

The key that contains the Capability Model ID of the device.

credentialLifetime: positiveDuration

credential lifetime (duration string)

debug: boolean false

Enable debug logging

devices: object

Device configuration, primarily for translating MAC address to Device IDs

commands: object

Commands A map of capabilityModelIds to a list of commands that they support

deviceIdTemplate: string

A text template for generating the Azure deviceId that is desired by the cloud configuration. This template is used to generate a Device ID that is then used to attach to the cloud and subscribe to topics. If no deviceIdTemplate is supplied, defaults to using the 'deviceId' field in the event. If that is not used, the lowercase MAC address of the device is used.

directMethods: string[]

A list of Direct Methods to subscribe to for all device types

string
idScope: string

The ID Scope of the Azure IOT Central Application. A unique value used by the IoT Hub Device Provisioning Service. Can be found in the Administration/Device Connection page on the IOT Central site.

nonPersistentMaxCount: integer (int64) 1000

If set to > 0, this will set the maximum number of messages the MQTT memorystore will hold before dropping messages Must be used with persistent=false

persistent: boolean false

If set to true, outbound and inbound MQTT messages are stored on disk instead of in memory, allowing them to be reloaded upon restart or reboot if they were not yet processed.

persistentMaxSize: string 1GiB

If set to > 0, this will set the maximum size (bytes) on disk that the MQTT filestore can consume before dropping messages. Supported suffixes (B, KB, KiB, MB, MiB, GB, GiB,...), max size of 1GiB. Must be used with persistent=true

previewAPI: boolean false

Use the new preview API of IOT Central

reconnectTimeout: string 1m

How long to wait before retrying to connect a failed/disconnected device to IOT Central This is used when a device is waiting for approval, gets blocked, or gets otherwise disconnected. Supports durations in the form of 1s 5m 1h etc.

sasKey: string

The Shared Access Signature key to use to generate device keys when authenticating with Azure IOT Central. Can be found in the Administration/Device Connection page on the IOT Central site.

unwrapCloudPayload: boolean false

Put the payload from cloud messages in the top level event map, rather than wrapping them in the "payload" key

verbose: boolean false

Enable verbose logging

Example
{
  "service": {
    "cloud": {
      "connectors": {
        "my-iotcentral": {
          "type": "azureiotcentral",
          "config": {
            "capabilityModelIdKey": "",
            "devices": {
              "deviceIdTemplate": "my_device_{{ .mac }}"
            },
            "idScope": "0123f0_redacted",
            "sasKey": "abcdf_redacted_abcd"
          }
        }
      }
    }
  }
}

digitalTwins

This connector sends sensor telemetry data to Microsoft Azure Digital Twins. All devices, sensors, matchers, and user defined functions must already be setup in Digital Twins. The device must also have a role that allows it to update sensor values. The cloud connector simply authenticates the device (the Gateway) and then routes telemetry data for each sensor over MQTT.

Supports publishing but does not support subscriptions.

baseURL: string

The BaseURL of the digital twins instance to use

debug: boolean false

Enable debug logging

deviceHardwareId: string

The device hardwareId in Digital Twins that matches the Gateway running Edge Connect and for which the SAS token is issued.

nonPersistentMaxCount: integer (int64) 1000

If set to > 0, this will set the maximum number of messages the MQTT memorystore will hold before dropping messages Must be used with persistent=false

persistent: boolean false

If set to true, outbound and inbound MQTT messages are stored on disk instead of in memory, allowing them to be reloaded upon restart or reboot if they were not yet processed.

persistentMaxSize: string 1GiB

If set to > 0, this will set the maximum size (bytes) on disk that the MQTT filestore can consume before dropping messages. Supported suffixes (B, KB, KiB, MB, MiB, GB, GiB,...), max size of 1GiB. Must be used with persistent=true

sasToken: string

A Shared Access Signature that has been generated for this deviceId

sensorHardwareIdKey: string

A key in the incoming data that maps to the sensor hardwareId for the particular sensor in Digital Twins

verbose: boolean false

Enable verbose logging

Example
{
  "service": {
    "cloud": {
      "connectors": {
        "my-dt": {
          "type": "digitalTwins",
          "config": {
            "baseURL": "mydigitaltwinsinstance.westcentralus.azuresmartspaces.net",
            "deviceHardwareId": "D149731826-00310",
            "sasToken": "SharedAccessSignature id=D149731826-00310&se=44456789124&kv=1&sig=djmMcgXRRDBdvIIJZCogvxRynxUniyCwGlmUF7vgB6TlLvk%3D",
            "sensorHardwareIdKey": "mac"
          }
        }
      }
    }
  }
}

googlecloudiot

This cloud connector provides a connection to the Google Cloud IoT Core service.

Supports publishing on topics and subscribing to topics. Also supports Device configuration via linking to a Device Group.

For devices to be attached to a Gateway successfully, they first need to be "bound" in the Google Cloud IOT Core API/settings. At startup, or when dynamically attached, an "attach" message is sent to Google to let it know the device is online. This will fail if the device is not already "bound" on the Google side.

Cloud to device messages arrive in pipelines as a cloud eventType. The topic is included in the topic key and the payload is included in the payload key. If you want the payload to be the top level event map, use the unwrapCloudPayload option.

connectMessage: CloudMessage
debug: boolean false

Enable debug logging

devices: CloudConnectorDevices
gatewayId: string

ID of this Gateway in Google Cloud. Required by Google to start with a lowercase character, so if the Gateway serial number is used, make sure to lowercase it.

nonPersistentMaxCount: integer (int64) 1000

If set to > 0, this will set the maximum number of messages the MQTT memorystore will hold before dropping messages Must be used with persistent=false

persistent: boolean false

If set to true, outbound and inbound MQTT messages are stored on disk instead of in memory, allowing them to be reloaded upon restart or reboot if they were not yet processed.

persistentMaxSize: string 1GiB

If set to > 0, this will set the maximum size (bytes) on disk that the MQTT filestore can consume before dropping messages. Supported suffixes (B, KB, KiB, MB, MiB, GB, GiB,...), max size of 1GiB. Must be used with persistent=true

privateKey: string

Private Key that matches the Public Key for this Gateway that was provided to Google.

projectId: string

Google Project ID

region: string

Google Region the project is in

registryId: string

Google IOT Registry ID

subscribeTopics: string[]

Topics to subscribe to. Incoming messages on these topics will arrive in pipelines as cloud type events.

Supports text templating.

string
unwrapCloudPayload: boolean false

Put the payload from cloud messages in the top level event map, rather than wrapping them in the "payload" key

verbose: boolean false

Enable verbose logging

Example
{
  "service": {
    "cloud": {
      "connectors": {
        "googlecloud": {
          "type": "googlecloudiot",
          "config": {
            "gatewayId": "gatewaySerialLowercased",
            "privateKey": "....",
            "projectId": "myproject",
            "region": "uswest",
            "registryId": "myregistry"
          }
        }
      }
    }
  }
}

http

This cloud connector represents an HTTP destination using the URL and method specified.

Support publishing but no subscriptions.

debug: boolean false

Enable debug logging

headers: object

Headers contains a map of key and value pairs to place in the headers section of the HTTP request. Typically, these headers would include things like an API key or other authentication data. Supports text templating.

method: string

Tells the filter which HTTP command to use to send data. Valid options are POST and PUT.

persistent: boolean false

Enable persistent storage for failed requests

retryDelayMultiplier: number (double) 1.1

Retry delay multiplier, the retry delay will be multiplied by this value on each request error

retryDisable: boolean false

Disable retrying a request on error

retryInitialDelay: string 1s

Retry initial delay

retryMaxAttempts: integer (int64)

Maximum retry attempts If either RetryMaxAttempts or RetryTimeout is satisfied the request will be discarded

retryMaxDelay: string 30m

Maximum delay between any retry

retryOfflineDelay: string 1m

Retry delay (fixed) when server is unreachable

retryTimeout: string 30m

How long to keep retrying a request when getting an error If either RetryMaxAttempts or RetryTimeout is satisfied the request will be discarded

timeout: integer (int64) 30

A single HTTP request attempt will timeout after this length of time. The timeout is specified as an integer number of seconds. The default timeout is 30 seconds.

url: string

The HTTP url to send data Supports text templating.

verbose: boolean false

Enable verbose logging

Example
{
  "service": {
    "cloud": {
      "connectors": {
        "my-http": {
          "type": "http",
          "config": {
            "headers": {
              "AuthToken": "something-secret"
            },
            "method": "POST",
            "url": "http://myserver.com/path"
          }
        }
      }
    }
  }
}

mqtt

This cloud connector provides a connection to an MQTT broker.

Support publishing on topics and subscriptions on topics.

Cloud to device messages arrive in pipelines as an event with cloud eventType. The payload is included in the payload key. If you want the payload to be the top level event map, use the unwrapCloudPayload option.

base: string

The base address of the broker. This is required by some brokers, such as AWS IoT

clientId: string the hostname (typically the serial number of the gateway)

The ClientID to use in the MQTT connection.

connectMessage: CloudMessage
credentials: object

A set of base64 encoded TLS credentials.

ca: string

Certificate authority (may be optional as some certificates already contain this information)

cert: string

TLS certificate

key: string

TLS private key

debug: boolean false

Enable debug logging

devices: CloudConnectorDevices
host: string

The host address of the broker

nonPersistentMaxCount: integer (int64) 1000

If set to > 0, this will set the maximum number of messages the MQTT memorystore will hold before dropping messages Must be used with persistent=false

password: string

Broker Password

persistent: boolean false

If set to true, outbound and inbound MQTT messages are stored on disk instead of in memory, allowing them to be reloaded upon restart or reboot if they were not yet processed.

persistentMaxSize: string 1GiB

If set to > 0, this will set the maximum size (bytes) on disk that the MQTT filestore can consume before dropping messages. Supported suffixes (B, KB, KiB, MB, MiB, GB, GiB,...), max size of 1GiB. Must be used with persistent=true

port: integer (int64)

The port of the broker

publishQos: integer (uint8) 1

MQTT QoS level. Defaults to 1 as that is most widely supported. Valid options are 0,1,2. Note: some brokers (such as AWS IoT Core) do not support QoS 2.

scheme: string

The communication protocol of the broker. Options are: tcp, tcps, wss, ssl

subscribeTopics: string[]

Topics to subscribe to. Incoming messages on these topics will arrive in pipelines as cloud type events.

Supports text templating.

string
unwrapCloudPayload: boolean false

Put the payload from cloud messages in the top level event map, rather than wrapping them in the "payload" key

username: string

Broker Username

verbose: boolean false

Enable verbose logging

Example
{
  "service": {
    "cloud": {
      "connectors": {
        "my-mqtt": {
          "type": "mqtt",
          "config": {
            "base": "",
            "host": "iothub.myserver.net",
            "port": 8883,
            "scheme": "tcps"
          }
        }
      }
    }
  }
}

websocket

This cloud connector represents a websocket connection

binaryMode: boolean false

Enable binary mode for messages to cloud If false, messages will be sent to the cloud in text mode

debug: boolean false

Enable debug logging

headers: object

Headers contains a map of key and value pairs to place in the headers section of the HTTP request. Typically, these headers would include things like an API key or other authentication data.

persistent: boolean false

Enable persistent storage for failed requests

retryDelayMultiplier: number (double) 1.1

Retry delay multiplier, the retry delay will be multiplied by this value on each request error

retryDisable: boolean false

Disable retrying a request on error

retryInitialDelay: string 1s

Retry initial delay

retryMaxAttempts: integer (int64)

Maximum retry attempts If either RetryMaxAttempts or RetryTimeout is satisfied the request will be discarded

retryMaxDelay: string 30m

Maximum delay between any retry

retryOfflineDelay: string 1m

Retry delay (fixed) when server is unreachable

retryTimeout: string 30m

How long to keep retrying a request when getting an error If either RetryMaxAttempts or RetryTimeout is satisfied the request will be discarded

url: string

The websocket url to connect to

verbose: boolean false

Enable verbose logging

Example
{
  "service": {
    "cloud": {
      "connectors": {
        "my-websocket": {
          "type": "websocket",
          "config": {
            "headers": {
              "AuthToken": "something-secret"
            },
            "url": "wss://server.com/xyz"
          }
        }
      }
    }
  }
}

Device Group Types

persistent

Persistent device groups operate the same as static device groups, with the exception that dynamically added devices persist across reboots. This makes persistent groups better suited for dynamic provisioning, while static groups are better suited for pre-configured device lists.

absenceMinutes: integer (int64) 5

The number of minutes to tolerate a device being missing before reporting it

batteryEvents: DeviceGroupBattery
debug: boolean false

Enable debug logging

deviceTypes: string[]

Device Types to include in this device group.

string
disableMonitoring: boolean false

Disable sending monitoring events for this group to Edge Direct

exclude: string[]

Deprecated: The list of mac addresses to exclude from the device group.

string
excludeAllByDefault: boolean true

Deprecated: If set, a device is excluded unless it is in the include list.

include: string[]

The list of mac addresses to include in the device group. Overrides the exclude list.

string
includeMatchers: MatcherSet
match: object

Keys to match (all keys) for an inline definition (see MatcherDefinition), or a Template to evaluate and match to a true value.

matchAny: boolean false

Should we match any of the provided matchers (true)? Or all of the matchers (false)

matchers: string[]

The list of top-level matcher IDs to apply to the input map

string
presenceMatchers: MatcherSet
verbose: boolean false

Enable verbose logging

Example
{
  "service": {
    "groups": {
      "my-eddystones": {
        "type": "persistent",
        "config": {
          "absenceMinutes": 19,
          "exclude": [
            "eddymac4",
            "eddymac5",
            "eddymac6"
          ],
          "include": [
            "eddymac1",
            "eddymac2",
            "eddymac3"
          ]
        }
      }
    }
  }
}

static

A static Device Group tracks devices based on an include list.

If an includeList is provided, devices must match the include list.

If both are omitted, all devices are matched to this group.

Any device in the group is considered present if it has been seen within absenceMinutes.

Any device in the group that goes missing for absenceMinutes will be considered not present.

Any time a new device is discovered, or the presence changes from true to false or vice versa, a Device Status event will be fired. See Pipeline for information on how to subscribe to Device Status events.

Devices in the include list that are not seen after absenceMinutes from startup are considered not present, and a device status event will be fired with present=false.

absenceMinutes: integer (int64) 5

The number of minutes to tolerate a device being missing before reporting it

batteryEvents: DeviceGroupBattery
debug: boolean false

Enable debug logging

deviceTypes: string[]

Device Types to include in this device group.

string
disableMonitoring: boolean false

Disable sending monitoring events for this group to Edge Direct

exclude: string[]

Deprecated: The list of mac addresses to exclude from the device group.

string
excludeAllByDefault: boolean true

Deprecated: If set, a device is excluded unless it is in the include list.

include: string[]

The list of mac addresses to include in the device group. Overrides the exclude list.

string
includeMatchers: MatcherSet
match: object

Keys to match (all keys) for an inline definition (see MatcherDefinition), or a Template to evaluate and match to a true value.

matchAny: boolean false

Should we match any of the provided matchers (true)? Or all of the matchers (false)

matchers: string[]

The list of top-level matcher IDs to apply to the input map

string
presenceMatchers: MatcherSet
verbose: boolean false

Enable verbose logging

Example
{
  "service": {
    "groups": {
      "my-eddystones": {
        "type": "static",
        "config": {
          "absenceMinutes": 19,
          "exclude": [
            "eddymac4",
            "eddymac5",
            "eddymac6"
          ],
          "include": [
            "eddymac1",
            "eddymac2",
            "eddymac3"
          ]
        }
      }
    }
  }
}

Use as Filter or Action

append

This filter appends the data in the item key onto the array in the array key. If the array key does not exist, a new array is created, with the item value as the initial element.

Output: The input event with the source data appended to the target key.

array: string

The key of the array to append the item to. Supports dot notation and text templating.

deleteItem: boolean

If true, delete the original item key/value after appending to the array.

item: string

The key of the item to append to the array. Supports dot notation and text templating.r

Example
{
  "service": {
    "filters": {
      "my-append": {
        "type": "append",
        "config": {
          "array": "array",
          "item": "item"
        }
      }
    }
  }
}

cipherDecrypt

Decrypts encrypted data in the input into decrypted bytes stored in the output key. If decryption fails or the input is invalid, the filter will error and execution will not proceed.

additionalData: bytes

Additional data bytes - only used with AEAD ciphers

cipher: string

Cipher name and mode
supported:

  • aes-ecb
  • aes-cbc (requires IV)
  • aes-ctr (requires IV)
  • aes-cfb (requires IV)
  • aes-ofb (requires IV)
  • aes-gcm (AEAD)
  • aes-eax (AEAD)

input: bytes

Input bytes - encrypt: the un-encrypted bytes, decrypt: the encrypted bytes

iv: bytes

Initialization vector bytes - only used with ciphers that require an IV

key: bytes

Cipher key bytes

nonce: bytes

Nonce bytes - only used with AEAD ciphers

output: string
Example
{
  "service": {
    "filters": {
      "decrypter": {
        "type": "cipherDecrypt",
        "config": {
          "additionalData": null,
          "cipher": "aes",
          "input": "{{.ciphertext}}",
          "iv": null,
          "key": "{{.key}}",
          "nonce": null,
          "output": "plaintext"
        }
      }
    }
  }
}

cipherEncrypt

Encrypts data in the input into encrypted bytes stored in the output key. If encryption fails or the input is invalid, the filter will error and execution will not proceed.

additionalData: bytes

Additional data bytes - only used with AEAD ciphers

cipher: string

Cipher name and mode
supported:

  • aes-ecb
  • aes-cbc (requires IV)
  • aes-ctr (requires IV)
  • aes-cfb (requires IV)
  • aes-ofb (requires IV)
  • aes-gcm (AEAD)
  • aes-eax (AEAD)

input: bytes

Input bytes - encrypt: the un-encrypted bytes, decrypt: the encrypted bytes

iv: bytes

Initialization vector bytes - only used with ciphers that require an IV

key: bytes

Cipher key bytes

nonce: bytes

Nonce bytes - only used with AEAD ciphers

output: string
Example
{
  "service": {
    "filters": {
      "encrypter": {
        "type": "cipherEncrypt",
        "config": {
          "additionalData": null,
          "cipher": "aes",
          "input": "{{.plaintext}}",
          "iv": null,
          "key": "{{.key}}",
          "nonce": null,
          "output": "ciphertext"
        }
      }
    }
  }
}

decodeLTV

This filter decodes binary L(ength) (T)type (V)alue data into a map. Data can be in the format of a singular LTV or repeated LTVLTVLTVLTV.

For streams of primitive types in predictable order, the typeConvert filter should be used instead. The intent of this filter is to read streams of complex types that have multiple fields. Each LTV instance can be in unpredictable order, but the fields inside of the LTV must be of predictable order.

Each LTV in the byte stream represents one instance of an EncodedLTV. The length byte of the object is read, then the type byte. The type is used to lookup an EncodedLTV definition, which then describes how to read each field of the type. By default, each field from each type is merged into one final output event, with all keys in it. If there are to be multiple instances of a given EncodedLTV, they can be collected into an Array using the appendToArray property on EncodedLTV.

See the documentation for EncodedLTV and EncodedField for more details. To see the available primitive types available see the documentation for the typeConvert filter.

dataKey: string data

The key to read the data from.

lengthInclusive: boolean false

LengthInclusive describes whether the length includes the length byte itself

outputKey: string decoded

The key to store decoded data in

typeInclusive: boolean false

TypeInclusive describes whether the length includes the type byte

types: object

A map of friendly names to EncodedType descriptions of the types that appear in the data.

Example
{
  "service": {
    "filters": {
      "decode-my-data": {
        "type": "decodeLTV",
        "config": {
          "dataKey": "data",
          "lengthInclusive": true,
          "outputKey": "decoded",
          "typeInclusive": true,
          "types": {
            "event": {
              "appendToArray": "events",
              "fields": [
                {
                  "key": "eventType",
                  "length": 4,
                  "type": "intLE"
                },
                {
                  "key": "eventData",
                  "length": 8,
                  "type": "intLE"
                }
              ],
              "id": 3
            },
            "metadata": {
              "fields": [
                {
                  "key": "serialNumber",
                  "length": 4,
                  "type": "intLE"
                },
                {
                  "key": "deviceName",
                  "length": 16,
                  "type": "string"
                }
              ],
              "id": 1
            },
            "telemetry": {
              "fields": [
                {
                  "key": "xPos",
                  "length": 8,
                  "type": "floatLE"
                },
                {
                  "key": "yPos",
                  "length": 8,
                  "type": "floatLE"
                }
              ],
              "id": 2
            }
          }
        }
      }
    }
  }
}

decodeType

This filter decodes binary data of a single fixed type into a map. Data can be in the format of a singular value or repeated values of the same type.

This filter is very similar to the decodeLTV filter, but works with one fixed type. The data stream cannot include length/type information, but rather contains repeated instances of the EncodedType, with only the field data present.

By default, values are collected into an array in the outputKey field. If outputMap is set to true, then the fields will be placed in a map inside the outputKey, with subsequent instances of fields replacing the values of previous fields.

See the documentation for EncodedType and EncodedField for more details. To see the available primitive types available see the documentation for the typeConvert filter.

dataKey: string data

The key to read the data from.

outputKey: string decoded

The key to store decoded data in

outputMap: boolean false

If set to true, the object(s) that are decoded will be merged into one output map, rather than appended to an array.

type: EncodedType
Example
{
  "service": {
    "filters": {
      "decode-my-fixed-data": {
        "type": "decodeType",
        "config": {
          "dataKey": "data",
          "outputKey": "decoded",
          "type": {
            "fields": [
              {
                "key": "serialNumber",
                "length": 4,
                "type": "intLE"
              },
              {
                "key": "deviceName",
                "length": 16,
                "type": "string"
              }
            ]
          }
        }
      }
    }
  }
}

delete

This filter deletes the specified keys from the input map.

Output: The output map is a copy of the input map with the specified keys deleted.

keys: string[]

Keys to delete

string
Example
{
  "service": {
    "filters": {
      "delete-bad-keys": {
        "type": "delete",
        "config": {
          "keys": [
            "extraneous-key",
            "and-another-one-too"
          ]
        }
      }
    }
  }
}

deviceReport

This filter fetches information about devices in device groups and seen nearby.

groups: stringArray

Restrict report to only these device groups. If empty all device groups will be reported including devices that have been detected but that are not in groups. Can be a device group ID (string), regular expression (string starting with ^ or ending with $) or an array of device group IDs or regular expressions

includeProps: bool

Include the props for each device in the report. This has performance impact, especially for persistent props. Props are include in each device in the props key.

includeUnknownDevices: bool

Includes devices that don't match a declared device type and aren't in a device group.They are not included by default.

outputAsArray: bool

Output report as an array instead of a map?

outputKey: string

Output key for report

Example
{
  "service": {
    "filters": {
      "dg-info-all": {
        "type": "deviceReport",
        "config": {
          "groups": "^.*\\.provisioned",
          "outputAsArray": true,
          "outputKey": "provisionedDevices"
        }
      }
    }
  }
}

emitEvent

This filter is used to emit a new user defined event to feed back into pipeline processing. The new emitted event can then be matched to pipelines based on the eventType. The current pipeline that contains this filter will continue execution with the unchanged input event.

eventType: string

Define the emitted event type.

payloadKey: string

Define the payload key in the current event to emit in the new event. The key must contain a value of type map[string]interface{} to be emitted. Supports templating, dot-notation. If payloadKey is empty, the whole event map will be emitted in the new event. If an error is encountered (bad key, type, template error), no new event is emitted.

Example
{
  "service": {
    "filters": {
      "oven-preheated-notifier": {
        "type": "emitEvent",
        "config": {
          "eventType": "ovenPreheated",
          "payloadKey": "oven.status"
        }
      }
    }
  }
}

hash

This filter calculates, appends or validates hash values Supported algorithms: md5 sha1 sha224 sha256 sha384 sha512 crc32-ieee crc64-ecma crc64-iso crc16-ccitt-false crc16-ccitt

Supported types for inputKey: string []byte

Output: In append mode, the outputKey will contain the data from inputKey with the hash value appended In compute mode, the outputKey will contain the hash value calculated from data in the inputKey In validate mode, the outputKey will contain the data in the inputKey with the hash removed In hmac mode, the outputKey will contain the hmac calculated from the hmacKey and inputKey In expect mode, the hash value calculated from data in the inputKey will be compared to the value specified in the expect field, if the hashes match the event will be emitted, otherwise nil is emitted.

Endianness: when computing a hash the input endianness must be little endian when reading/writing a hash value as []byte, the default endianness is little endian when reading/writing a hash value as string, the default endianness is little endian (except for crc16)

algorithm: string

Set hash algorithm

append: boolean

Enable append mode

compute: boolean

Enable compute mode

expect: string

Enable expect mode

hashBytesBigEndian: boolean false

Handle hash array as big endian

hashStringBigEndian: boolean false

Handle hash strings as big endian Default is false for all algorithms except crc16

hmac: boolean false

enable HMAC mode

hmacKey: string

Set the HMAC key (supports templating, hexstring or base64) Must be set if hmac == true

inputKey: string

Define the input key Supports dot notation and text templating.

outputKey: string

Define the output key Supports dot notation and text templating.

outputSize: integer (int64) 0

Set output size If default hash output size < outputSize, will pad hash output with 0's If default hash output size > outputSize, will truncate hash output if outputSize == 0, default for hash is used

verify: boolean

Enable verify mode (assumes input has hash appended)

Example
{
  "service": {
    "filters": {
      "decode-resp": {
        "type": "hash",
        "config": {
          "algorithm": "crc16-ccitt-false",
          "append": false,
          "compute": false,
          "inputKey": "dataToVerify",
          "outputKey": "data",
          "verify": true
        }
      }
    }
  }
}

history

This filter collects historical values for a device. Events for a device are identified by the key specified in the configuration The value for an event is determined by the value template.

The history filter will emit a history event (device event with history added in (optional)) if the condition evaluates to true.

The options for condition are:

  • always (default) returns true.
  • change returns true if the device event yields a value that is not the same as the previous value, a history event will be emitted.
  • threshold returns true if the device event yields a value that exceeds any defined threshold.
  • thresholdZone returns true if the device event yields a value that exits a device's threshold zone.

A custom condition can be defined with a template which produces a value that can be converted to a boolean. The history data for the device can be accessed via the following keys in the template:

  • .__history ([]interface{}) history values (including current value) sorted by ascending time
  • .__historyCurrent (interface{}) the current value
  • .__historyPrevious (interface{}) the previous value

threshold / thresholdZone conditions can define high and low thresholds and hysteresis.
The threshold zones are the following:

  • unknown new device
  • ok no threshold exceeded.
  • low low threshold has been exceeded, and value has not increased above the low hysteresis value.
  • high high threshold has been exceeded, and value has not decreased below the high hysteresis value.

The threshold hysteresis values are used to debounce the threshold zone transitions, note that the hysteresis values default to 0.
To exit a threshold exceeded zone, the current value must meet the following criteria:

  • high v < (thresholdHigh - thresholdHysteresisHigh)
  • low v > (thresholdLow + thresholdHysteresisLow)

Output: If condition evaluates to:

  • error nil
  • false nil
  • true the input event with the history ([]interface{}) stored in outputKey
condition: string always

Set the condition to determine when an event will be emitted for a device. options: always, condition, template (see overview)

deviceId: string mac

DeviceId to uniquely identify device supports templating

historyCount: integer (int64) 1000

Set history length by count. If the limit is reached, old values will be replaced by new values. note: if used with historyDuration, sets max number of history values saved over the duration, will default. if used without historyDuration, sets max number of history values, will not default

historyDuration: positiveDuration

Set history length by duration string (also see historyCount)

historyTimeout: positiveDuration

History timeout which will enable reporting history at an interval if condition is not matched.If set, the historyStateChanged key will be added to the history output which will be set to true when condition is matched, and false when a history timeout occurs.

maxDeviceCount: integer (int64) 1000

Maximum number of unique devices to track. Once the limit is reached, the oldest devices will be replaced by new devices

omitHistory: boolean false

Omit history data when emitting history event (passthrough device event only)

outputKey: string "history"

Key to output history in event map supports templating

thresholdHigh: number (double)

Threshold values for threshold condition, one or both must be defined when threshold condition is selected. If the value from a device event exceeds either of these thresholds a device history event will be emitted

thresholdHysteresisHigh: number (double)
thresholdHysteresisLow: number (double) 0

Threshold hysteresis values.

thresholdLow: number (double)
value: string

Value to track history for supports templating, dot notation

Example
{
  "service": {
    "filters": {
      "rssi-changed": {
        "type": "history",
        "config": {
          "condition": "change",
          "deviceId": "mac",
          "historyCount": 10,
          "historyDuration": 60000000000,
          "value": "{{.rssi}}"
        }
      }
    }
  }
}

httpFetch

This filter performs an HTTP request against a specified endpoint.

Output: The filter outputs the HTTP response body to the defined outputKey.

The HTTP response body will be converted based on the content-type header. application/json -> map[string]interface{} text/plain -> string all other types -> []byte

body: string none (empty body)

The HTTP body to use in the request (template)

bodyFormatter: string json

The formatter used to encode the HTTP request body

cacheMaxItems: integer (int64) 64

Max Cache size in items

contentTypeOutputKey: string none (HTTP response Content-Type will not be output in the event)

Defines the output key for the HTTP response Content-Type in the event map.

headers: object

Headers contains a map of key and value pairs to place in the headers section of the HTTP request. Typically, these headers would include things like an API key or other authentication data. Supports text templating.

interval: string 1s

The HTTP request cache time to live (duration string) Sets the lifetime for a cached HTTP response. Setting to a 0 or negative duration value will disable caching.

method: string GET

The HTTP body Supports text templating.

outputKey: string httpResponse

Defines the output key for the HTTP response body in the event map.

passthroughHTTPError: boolean false

Emit an event when an HTTP response contains a non-successful HTTP status (see successStatuses). note: all other errors will cause a nil event to be emitted (server unreachable, etc).

persistent: boolean false

Enable persistent cache

statusKey: string httpStatus

Defines the output key for the HTTP status code in the event map

successfulStatuses: integer[] 200, 201, 202

Define the httpStatus codes that are considered successful. A response will only be cached if the status code is in this list.

integer (int64)
timeout: positiveDuration

HTTP request timeout (duration string)

tlsSkipVerify: boolean false

Skip TLS verification, for development only

url: string

The HTTP url to send data Supports text templating.

Example
{
  "service": {
    "filters": {
      "httpGet": {
        "type": "httpFetch",
        "config": {
          "headers": {
            "AuthToken": "something-secret"
          },
          "interval": "25s",
          "outputKey": "my-output-key",
          "url": "https://myserver.net/data"
        }
      }
    }
  }
}

inject

This filter injects keys into the output event map. The injected keys are specified as key/value pairs using the top level keyPairs parameter.

Templating is supported, see Edge Connect Templates for more detail.

There are two reserved key pairs: serial and timestamp. Adding either of these keys as with blank values will insert the serial number and timestamp (using the value to select the output format) into the output event map. The output event map will contain all keys and values from the input. If a key name collision occurs, the injected key will be ignored.

Supported timestamp values:

  • "s": epoch seconds
  • "ms": epoch milliseconds
  • "us": epoch microseconds
  • "ns": epoch nanoseconds
  • standard golang time formats (see https://golang.org/pkg/time/#pkg-constants) use the constant name (case-sensitive) for example:
    • "RFC3339": RFC3339 date string
    • "UnixDate": Unix date string

Output: Contains all keys from the input and the injected keys.

keyPairs: object

Defines a set of keys and values to add to the event map output. Supports dot notation and text templating.

replaceExisting: boolean false

Replace key value pairs that already exist, instead of ignoring them

Example
{
  "service": {
    "filters": {
      "my-inject": {
        "type": "inject",
        "config": {
          "keyPairs": {
            "customSerialUnderscoreMac": "{{ .serial }}_{{ .mac }}}",
            "my-key": "my-value",
            "serial": ""
          }
        }
      }
    }
  }
}

matcher

This filter applies matchers to the event map. An inline matcher may also be defined with the match key (see MatcherConfig for details).

At least one matcher or inline matcher must be defined. All matchers are applied to the input map to determine if the event map is a match.

Output: If all matchers match the event map, the event map is emitted, otherwise nil is emitted.

match: object

Keys to match (all keys) for an inline definition (see MatcherDefinition), or a Template to evaluate and match to a true value.

matchAny: boolean false

Should we match any of the provided matchers (true)? Or all of the matchers (false)

matchers: string[]

The list of top-level matcher IDs to apply to the input map

string
Example
{
  "service": {
    "filters": {
      "my-matcher": {
        "type": "matcher",
        "config": {
          "match": {
            "key1": [
              "thisValue",
              "orThisValue"
            ],
            "key2": "onlyThisValue"
          },
          "matchers": [
            "match-command1",
            "match-command2"
          ]
        }
      }
    }
  }
}

metadataClear

Clears metadata on input. In global mode, any input map will clear the specified global metadata. In device mode, any input map will clear the specified metadata from the device.

Output: unmodified input

allDevices: boolean false

Perform device metadata operations on all devices?

global: boolean false

Perform global metadata operations?

keys: string[] empty

The metadata keys to delete, all keys are deleted if empty

string
Example
{
  "service": {
    "filters": {
      "clear-batt-metadata": {
        "type": "metadataClear",
        "config": {
          "keys": [
            "batteryLevel"
          ]
        }
      }
    }
  }
}

metadataGet

Adds metadata on to an input map

In global mode, the specified global metadata will be added to the input map.

In device mode, the device specific metadata will be added to the input map.

Output: Input map with requested metadata added

global: boolean

Perform global metadata operations?

keys: string[]

The metadata data keys to add, all keys are added to the input map if empty.

string
Example
{
  "service": {
    "filters": {
      "add-batt-metadata": {
        "type": "metadataGet",
        "config": {
          "keys": [
            "batteryLevel"
          ]
        }
      }
    }
  }
}

metadataSet

Sets metadata on input.

In global mode, the specified keys from an input map will be set in global metadata.

In device mode, the specified keys from the input will be set in device specific metadata.

Output: unmodified input

global: boolean

Perform global metadata operations?

keys: string[]

The metadata keys to filter on for a given device, all keys are used if empty

string
Example
{
  "service": {
    "filters": {
      "store-batt-metadata": {
        "type": "metadataSet",
        "config": {
          "keys": [
            "batteryLevel"
          ]
        }
      }
    }
  }
}

ntp

This filter starts an internal NTP client and will inject NTP response data into an input event. Set getMaxOffset to add the maximum clock offset from the history window to the input event Set getHistory to add the full NTP request history for the given history window to the input event

At least one of getMaxOffset, getHistory must be set to true.

Output: Input map with NTP maximum clock offset or history injected

getHistory: boolean false

Add ntp sync history for the history window to input event?

getLastSuccess: boolean false

Add last successful ntp sync into input event?

getMaxOffset: boolean false

Add maximum clock offset from history window to input event?

historyWindow: string 1h

How far back to look in the NTP sync history when processing an event historyCount = (historyWindow/syncInterval) + 1 historyCount must be <= 1000

host: string 3.us.pool.ntp.org

Which NTP server to query

syncInterval: string 1m

NTP server query interval

Example
{
  "service": {
    "filters": {
      "ntp-monitor": {
        "type": "ntp",
        "config": {
          "getMaxOffset": true,
          "historyWindow": "30m",
          "host": "time.google.com",
          "syncInterval": "30s"
        }
      }
    }
  }
}

propSet

Sets props on the current Device. These props must either be defined in the DeviceType for this device, or the DeviceType must support unknown props. See DeviceType for more details. To fetch these props, use the Device.Props template function.

keyPairs: object

Defines a set of keys and values to set in the Device Props. Supports dot notation and text templating.

Example
{
  "service": {
    "filters": {
      "set-the-props": {
        "type": "propSet",
        "config": {
          "keyPairs": {
            "location": "upstairs",
            "room": "kitchen",
            "serial": 12345
          }
        }
      }
    }
  }
}

publish

Publish to a Cloud Connector. Publishes the whole event unless payloadKey or payloadsKey is specified. For topic based publishing, use the topic or payloadsKey options.

attempts: integer (int64) 1

How many attempts to make when publishing a payload. Will only attempt more than once when a retryable error is returned.

connector: string

ID of the Cloud Connector to publish to, defined in the Cloud Connectors Config.

format: string json

The format of the data to publish. Supported options are json and cbor.

maxPayloadSize: integer (int64) 0

Maximum payload size, if <= 0 there is no limit.

payloadKey: string

The key of the payload to send. If left empty, the whole event map is sent.

payloadsKey: string

This key must point to a map of topic strings to payloads. Each entry in that map will be published to its respective topic.

retryDelay: positiveDuration

How long to wait before attempting a retry

timeout: positiveDuration

Timeout for publishing (negative for no timeout)

topic: string

The topic on which to publish outgoing messages. Supports text templating.

Example
{
  "service": {
    "filters": {
      "publishPayload": {
        "type": "publish",
        "config": {
          "connector": "my-cloud",
          "payloadKey": "payload",
          "topic": "my-topic"
        }
      }
    }
  }
}

reduce

This filter is used to reduce a collection of maps into a single value. For example, to calculate the mean rssi for an aggregated set of advertisement events. This filter is generally paired with the aggregate filter preceding it, or after collecting of notifications from a device.

Key concepts: value template The value template is used to get the relevant value out of each individual map for reducing. This value is then stored in an array that is passed to the reduce template. reduce template The reduce template is used to calculate a single value from the array of outputs of the value template.

Depending on the type of collection being processed, the data will be handled differently: array mode: all events in the array as a single group, one value outputted map mode: all events in the map are treated as a single group, one value outputted mapArray mode: events in each sub-array are treated as a single group, one value per key in the input map

key: string

Key that contains the collection of data to reduce. If set, this key will be used to locate the collection in the event. If unset, the whole event will be treated as the collection. supports templating, dot notation

values: object

Map of values to reduce/process from each aggregate. Each entry is a key value pair of the new output mapped to the configuration of the reducer to generate that output.

As an example, this configuration would get the largest rssi from the aggregate and store it in key max-rssi.

Example
{
  "service": {
    "filters": {
      "rssi-calculations": {
        "type": "reduce",
        "config": {
          "values": {
            "average-rssi": {
              "reduce": "{{ meanf .__items }}",
              "value": "{{ .rssi }}"
            }
          }
        }
      }
    }
  }
}

remap

This filter remaps keys into the output. The keys to be remapped are specified as key/value pairs of "old key" to "new key" using the top level "keyPairs" key. Keys that are not found will be ignored.

Output: Contains all keys from the input, with specified keys remapped.

keyPairs: object

Defines a mapping of input keys to output keys for remapping. Supports dot notation and text templating.

Example
{
  "service": {
    "filters": {
      "my-remap": {
        "type": "remap",
        "config": {
          "keyPairs": {
            "oldKey1": "newKey1",
            "oldKey2": "newKey2"
          }
        }
      }
    }
  }
}

select

This filter selects only the specified keys from the input map.

Output: The output map contains only the specified keys from the input map, if none of the keys are present, an empty map is returned.

keys: string[]

Keys to select Supports dot notation and text templating.

string
Example
{
  "service": {
    "filters": {
      "my-select": {
        "type": "select",
        "config": {
          "keys": [
            "rssi",
            "mac",
            "my.nested.key"
          ]
        }
      }
    }
  }
}

sort

This filter is used to sort a collection of maps by a specific value.

Key concepts: value The value template is used to get a relevant value out of an individual map for sorting. The value template is executed on each event in a group of events and the output is stored in an array that is passed to the sorter. sorter The sorter will take the array of outputs from the value template and use it to sort the maps that the values were extracted from. The sorter can only handle numeric and string types, and all types in an array must match.

Depending on the collection being processed, the data will be handled differently: array mode: all events in the array as a single group, one value outputted map mode: all events in the map are treated as a single group, one value outputted mapArray mode: events in each sub-array are treated as a single group, one value per key in the input map

descending: boolean false

If true items will be sorted in descending order

key: string

If set, this key will be used to locate the data set in the event. If unset, the whole event will be treated as the data set. supports templating, dot notation

maxEntries: integer (int64) 0

If >0, the first N items will be returned in the sorted array. If there are less than N items in the sorted array, all items are returned. If <=0, all items will be returned.

outputKey: string sorted

Set the output key for the sorted data. If omitted, defaults to sorted

value: string

Value template used to collect an array of values from a collection of maps. This array of collected values is used to sort the underlying maps.

Example
{
  "service": {
    "filters": {
      "sort-by-rssi": {
        "type": "sort",
        "config": {
          "descending": true,
          "maxEntries": 2,
          "value": "{{ .rssi }}"
        }
      }
    }
  }
}

transform

This filter transforms values using templates and a limited set of available functions.

keyPairs: object

A map of keys to transformation templates. The value of each key is looked up, then transformed using the template.

See Edge Connect Templates for more detail.

For example, to transform a mac address to upper case with colons, you would map the mac key to the template string:

{{ .mac | ToUpper | AddColonsToMac }}

Also supports dot notation in key or value.

Example
{
  "service": {
    "filters": {
      "my-transform": {
        "type": "transform",
        "config": {
          "keyPairs": {
            "mac": "{{ .mac | ToUpper | AddColonsToMac }}"
          }
        }
      }
    }
  }
}

typeConvert

This filter converts map keys from one type to another. Additionally, you can convert and inject, where a converted value is injected into the map, this allows selecting offset/length. The key can be specified using dot-notation.

Supported conversions:

  • any type to:
    • string (using fmt.Sprintf())
  • []byte to:
    • base64: standard base64 encoding
    • hex: ascii hex (lowercase)
    • intLE, intBE: integer (greedy conversion, will consume 1,2,4 or 8 bytes ignoring trailing bytes)
    • uintLE, uintBE: unsigned integer (greedy conversion, will consume 1,2,4 or 8 bytes ignoring trailing bytes)
    • floatLE, floatBE: floating point (greedy conversion, will consume 4 or 8 bytes ignoring trailing bytes)
    • fix88LE, fix88BE: fixed point 8.8
  • string to:
    • int: string to int64
    • uint: string to uint64
    • float: string to float64
    • []byte: base64/base64url/asciihex string to []byte

Output: If keys are defined in convertKeys:

  • The filter will output a map with the specified keys in converted to the specified type
  • If a single conversion fails, the remaining conversions will be processed
  • If you wish to make a more specific conversion, use injectKeys and set the same source/destination keys

If keys are defined in injectKeys:

  • The filter will output a map with the specified keys injected
  • If a single inject fails, the other injections will be processed
  • injectKeys will only handle a []byte input so there is no interface{}->string allowed
  • You cannot inject into a map that does not exist

The actions are handled in the following order:

  1. process all injectKeys
  2. process all convertKeys

In the case that all operations fail, the input map will be passed through unchanged

convertKeys: object

Keys to convert Specify key-type conversions to perform Keys supports dot notation and text templating.

injectKeys: object

Keys to inject Specify conversion-injections to perform. Keys supports dot notation and text templating.

Example
{
  "service": {
    "filters": {
      "convert-it": {
        "type": "typeConvert",
        "config": {
          "convertKeys": {
            "theKeyToConvert": "hex"
          },
          "injectKeys": {
            "theKeyToInject": {
              "length": 8,
              "offset": 1,
              "source": "nestedMap.keyWithBytesToConvert",
              "type": "intLE"
            }
          }
        }
      }
    }
  }
}

unwrap

This filter unwraps a key in the incoming data. For example, if the incoming data is "{"payload":{"key":"value"}}, and the key specified is set to "payload", the output will be "{"key":"value"}".

Any conflicting keys are replaced. The unwrapped keys are always placed at the top level, even if dot notation is used.

Any keys

Output: The input data with the specified key unwrapped.

key: string

The key to unwrap. Must target a map.

Example
{
  "service": {
    "filters": {
      "my-wrap": {
        "type": "unwrap",
        "config": {
          "key": "payload"
        }
      }
    }
  }
}

wrap

This filter wraps the incoming data with the provided key value. For example, if the incoming data is "{"key":"value"}, and the key is set to "payload", the output will be "{"payload":{"key":"value"}}".

Output: The wrapped input data.

key: string

The key used to wrap the incoming data set. Supports dot notation and text templating.

keysToWrap: string[]

The particular keys to wrap. If left empty, all keys will be wrapped. Supports dot notation and text templating.

string
Example
{
  "service": {
    "filters": {
      "my-wrap": {
        "type": "wrap",
        "config": {
          "key": "payload"
        }
      }
    }
  }
}

Dynamic Types

bool

A boolean value that can be specified as a static boolean or as a template which computes to a boolean. If a value is not specified. The static values can be 0, 1, true, false, "t", "true", "f", or "false".

Example
{
  "boolExample": true,
  "templateExample": "{{ .payload.aBoolValue }}"
}

bytes

An array of bytes that can be specified as a static hex/base64 string, or as a template which computes to a byte array or string. If the template computes to a string, this type will attempt to convert it from a hex string or base64 string into an array of bytes.

Example
{
  "base64Example": "YVN0cmluZ09mQnl0ZXM=",
  "templateExample": "{{ .payload.someBytes }}"
}

int64

An int64 that can be specified as a static number, a static string that is convertible to a number, or as a template which computes to a number.

Example
{
  "int64Example": 33,
  "templateExample": "{{ .payload.nodeId }}"
}

positiveDuration

A PositiveDuration value that can be specified as a static duration string, static number of seconds (if no units are specified) or as a template which computes to a duration string or number.

Example
{
  "DurationExample": 10000000000,
  "templateExample": "{{ .payload.aDurationValue }}"
}

string

A string that can be specified as a static string or as a template which computes to a string.

Example
{
  "stringExample": "someString",
  "templateExample": "{{ .payload.value }}"
}

stringArray

A string array that can be specified as a static string, a static array of strings, or as a template which computes to a string array.

Example
{
  "arrayExample": [
    "oneValue",
    "twoValue"
  ],
  "stringExample": "oneValue",
  "templateExample": "{{ .payload.values }}"
}

Event Generators

timer

The timer eventGenerator allows generation of periodic or oneshot timer events.

On a timer tick, a timer event will be output with the following keys: { "eventGenerator": "my-5s-ticker", (string) "eventType": "5s-tick", (string) "time": "2020-09-03T23:53:51.155076768Z", (time.Time) "tickCount": 76, (int) "interval": 5000000000, (time.Duration) }

Output: eventMap if defined, otherwise a timer event as described above

debug: boolean false

Enable debug logging

eventType: string

Timer event type

interval: positiveDuration

Timer tick interval (positive duration string)

keyPairs: object

Key pairs to emit on timer tick event. Supports templating, dot notation

oneshot: boolean false

Timer will ony fire one time, not repeatedly

verbose: boolean false

Enable verbose logging

Example
{
  "service": {
    "eventGenerators": {
      "exampleTimer": {
        "type": "timer",
        "config": {
          "eventType": "heartbeat",
          "interval": 1000000000,
          "keyPairs": {
            "timestamp": "{{ return Timestamp }}"
          }
        }
      }
    }
  }
}

Pipeline Filters

aggregate

This filter aggregates incoming data. On a defined interval or maximum data count, it will emit all aggregated advertisements. There are three modes in which this filter can operate: array, map, and mapArray.

In the array mode, the filter will aggregate all incoming messages into a single array.

In map mode, the filter will keep only the last message from each device.

In mapArray mode, the filter will keep all messages for each device as separate arrays of data keyed on all present devices.

Data is published from this filter either on a defined interval or when a specified maximum size is reached. At least one of these options must be specified in the configuration. If both are specified, whichever condition that occurs first will cause data to be published.

Outputs: All data which is kept will be passed through.

arrayMax: integer (int64) 500

Maximum length of the storage array. Only applies in array and mapArray modes.

arrayOutputKey: string "aggregate"

When in array mode, or map mode with convertMapToArray set to true, wrap the output array in this key.

convertMapToArray: boolean false

When in map mode, convert the output map into an array of the entries instead of a map. This allows for capturing one unique value per Key, but outputting them in array form.

deleteKey: boolean false

Deletes key from the aggregated object(s)

disableEmpty: boolean false

Disable emitting empty aggregates

interval: string

The interval specifies how often this filter should publish aggregated data. It is specified as a string of number plus unit. Valid units are s for seconds, m for minutes, and h for hours.

joinMaps: boolean false

Join maps in map mode. If true, joins the current event with the previous event for a device.

key: string mac

The key to use for aggregation in map modes. Used if map or mapArray mode are used.

mapMax: integer (int64) 20000

Maximum length of the storage map. Only applies to map and mapArray modes.

maxCount: integer (int64) 0

Maximum number of events to hold in the aggregator. Applies to all modes. If > 0, overrides mapMax and arrayMax. If <= 0 (default), mapMax and arrayMax will be used.

mode: string

The operation mode for the aggregate filter. The valid options are: array, map, and mapArray.

Example
{
  "service": {
    "filters": {
      "my-agg": {
        "type": "aggregate",
        "config": {
          "arrayMax": 0,
          "disableEmpty": false,
          "interval": "25s",
          "key": "mac",
          "mapMax": 900,
          "maxCount": 0,
          "mode": "map"
        }
      }
    }
  }
}

chunk

This filter iterates through an aggregate and breaks it into chunks of N items.

If key is non-empty (the aggregate is nested), the emitted events will be the input event with the aggregate value replaced with a chunked aggregate.

Supported aggregate types: []interface{} []map[string]interface{} map[string]interface{}

key: string

The key containing the values to chunk, if blank the whole event map will be chunked.
Supports dot notation and text templating.

size: int64

Maximum number of items to emit in each chunk

Example
{
  "service": {
    "filters": {
      "25-chunker": {
        "type": "chunk",
        "config": {
          "key": "values",
          "size": 25
        }
      }
    }
  }
}

connect

This filter triggers a BLE connection and a subsequent ConnectionSequence of Actions to be run.

Output: The event map emitted by the ConnectionSequence after it has run.

cacheGattDb: boolean false

Cache the GATT database of this device when true

id: string

The ID of the Connection Sequence that should be triggered every time this filter is reached

interval: string

The interval on which connection sequences should run. The minimum connection interval is a function of the connection timeout with a hard minimum of 1 minute. The connection interval cannot be less than 4 times the timeout. Setting an interval of 0 will enable on-demand connections. For example, to achieve a 1 minute connection interval, the timeout must be set to 15 seconds.

timeout: object 10

Set the length of time to wait for a successful connection in seconds.

Example
{
  "service": {
    "filters": {
      "my-connect": {
        "type": "connect",
        "config": {
          "cacheGattDb": false,
          "id": "read-battery-life",
          "interval": "30m",
          "timeout": 20
        }
      }
    }
  }
}

deviceGroupIterator

This filter iterates through members of a device group and outputs a new event for each device consisting of the input map with the Device ID for each device injected into item.deviceId.

Using this filter will essentially cause a pipeline to fan-out and execute the following steps in the pipeline for each device in the specified device group.

groups: stringArray

The device group(s) to iterate over. Can be a single Device Group ID (string) or an array of Device Group IDs.

includeAbsentDevices: bool

Also include devices that are not currently "present" but are members of the device group

Example
{
  "service": {
    "filters": {
      "iterate-array": {
        "type": "deviceGroupIterator",
        "config": {
          "groups": "my-devices"
        }
      }
    }
  }
}

discoverDevice

This filter adds any device that flows through it to a Device Group. See the documentation for Device Groups for more information.

If the deviceId field is in the input map, that value will be used as the unique ID of the device added. Otherwise, the Device Group's Device ID Template will be used. This defaults to the mac field, but can be templatized to extract other fields out of the event map or compute a deviceId.

If the field addDevices is present, then that is expected to be a list of device IDs to add, and deviceId is ignored. These must be exact device IDs, and are not fed through the Device Group's Device ID Template.

If the field replaceDevices is present, the devices in the device group are completely replaced with the members of the replaceDevices array. These must be exact device IDs, and are not fed through the Device Group's Device ID Template.

Output: Contains all keys from the input

groupId: string

The ID of the Device Group for which you want to change membership. Supports templating.

Example
{
  "service": {
    "filters": {
      "discover-for-my-group": {
        "type": "discoverDevice",
        "config": {
          "groupId": "my-ibeacons"
        }
      }
    }
  }
}

eddystone

This filter processes Eddystone based beacon data. It handles processing URL, URI, and TLM Eddystone payloads. Each type of Eddystone data is treated as a separate advertising payload.

Output: All available Eddystone fields are emitted from this filter as well as mac address and RSSI.

No configuration is required for this filter.

ignoreEID: boolean false

Ignore EID advertising payloads

ignoreTLM: boolean false

Ignore TLM advertising payloads

ignoreUID: boolean false

Ignore UID advertising payloads

ignoreURL: boolean false

Ignore URL advertising payloads

passthrough: boolean false

Passthrough non-eddystone events

Example
{
  "service": {
    "filters": {
      "my-eddy": {
        "type": "eddystone",
        "config": {
          "ignoreURL": true
        }
      }
    }
  }
}

enocean

This filter parses enocean sensor data to a map

Output: The filter outputs all available data from the enabled enocean sensors and adds the mac address and rssi of the tag.

types: string[]

Enabled Sensor Types (available: emdcb, emswb)

string
Example
{
  "service": {
    "filters": {
      "my-enocean": {
        "type": "enocean",
        "config": {
          "types": [
            "emdcb",
            "emswb"
          ]
        }
      }
    }
  }
}

fwupdate

Update firmware on a device.

Supported Modes: thingsee (based on nordic) default activation via write 0xff -> f26fb784a2ae54c9b4c6950714c9789e/f26f7fffa2ae54c9b4c6950714c9789e. nordic (nordic bootloader) default activation via nordic buttonless dfu service

A fwupdate operation has the following steps:

  • match target device (via target, or thingsee filter)
  • connect and run activation connection sequence
  • disconnect
  • post activation delay
  • connect and run updater connection sequence
  • disconnect
activationDisable: boolean false

Disable activation sequence

activationSequence: string none, will use default for `type`

Custom activation sequence id

activationTimeout: positiveDuration

Set activation connection timeout

firmwareBytes: string

Select source of the firmware bytes (must be template)

postActivationDelay: positiveDuration

Set post activation delay

target: MatcherSet
type: string thingsee

DFU type set the dfu type

updaterTimeout: positiveDuration

Set updater connection timeout

Example
{
  "service": {
    "filters": {
      "my-fwupdate": {
        "type": "fwupdate"
      }
    }
  }
}

ibeacon

This filter parses iBeacon payloads and outputs all available iBeacon fields.

Output: All available iBeacon fields and their values.

passthrough: boolean false

Passthrough non-iBeacon events

uuidFilter: string

If specified, only iBeacon UUIDs which pass the filter will be processed further. This filter is specified as a regular expression.

Example
{
  "service": {
    "filters": {
      "my-ibeacon": {
        "type": "ibeacon",
        "config": {
          "uuidFilter": "aabbccddeeff1122334455667788990[123]"
        }
      }
    }
  }
}

isensor

This filter parses door sensor data from the iSensor brand door open/closed sensor.

Output: The current state of the sensor along with mac address, RSSI, and manufacturer id. The fields are keyed as mac, rssi, mfgid, and open. If open is true, the door is open.

No configuration is required for this filter.

devices: string[]

Defines a list of MFG IDs from which data should be collected. This list is added to the built in list. The IDs are specified as uint16 values in big-endian string hex format

string
Example
{
  "service": {
    "filters": {
      "my-isensor": {
        "type": "isensor",
        "config": {
          "devices": [
            "1095"
          ]
        }
      }
    }
  }
}

iterator

This filter iterates through an aggregate and outputs the input map with the iterated value injected into item.value and item.id (key or idx).

Supported aggregate types: []interface{} map[string]interface{}

forEach: string[]

ForEach can be used to modify every element in an iterable, rather than split it (the default behavior). If forEach is set, the filters listed in forEach will be run on each element of the iterable, rather than emitting each element as a new event.

string
key: string

The key containing the values to iterate over Supports dot notation and text templating.

max: integer (int64) 1000

Maximum number of items to emit from iteration

Example
{
  "service": {
    "filters": {
      "iterate-array": {
        "type": "iterator",
        "config": {
          "key": "arrayOfThings",
          "max": 0
        }
      }
    }
  }
}

minew

This filter parses minew sensor data to a map

Output: The filter outputs all available data from the enabled minew sensors and adds the mac address and rssi of the tag.

authKey: string

Authentication Key for authenticated S1 payloads

types: string[]

Enabled Sensor Types (available: s1,e6)

string
Example
{
  "service": {
    "filters": {
      "my-minew": {
        "type": "minew",
        "config": {
          "types": [
            "s1"
          ]
        }
      }
    }
  }
}

occupancy

This filter provides occupancy state messages from a Haltian Thingsee BLE occupancy sensor. There are 2 versions of the thingsee firmware (v1, v2) which determine how occupancy state is calculated.

The v1 sensor has one counter that increments everytime movement is detected, on overflow the count is reset to zero. In raw mode, the output contains the mac address, RSSI, and the count value from the device. In regular mode, the output replaces the count with state where true signifies occupied.

The v2 sensor is interval based and provides a motion count over the last 10 intervals with an incrementing interval counter. There is only one mode for handling v2 sensors.

v2 occupancy state is calculated by summing motion counts from the last N intervals (un/occupiedCount). If the sum is equal or greater than the threshold (un/OccupiedThreshold) the state will be changed accordingly with the occupied state taking precedence in a tie. Using the interval counter, incoming data is aligned with the history in the occupancy filter.

v2 output includes: occupied: true/false occupied state history: counts from the last historyCount intervals, -1 is empty lastSeen: last time sensor was seen lastChanged: last time sensor changed occupied state batteryLevel: battery level in % fwVersion: firmware version interval: interval length in seconds counter: incrementing interval counter

Output: see above

historyLength: integer (int64) 120

History length (in intervals) (v2 only)

mode: string regular

The mode in which to operate (v1 only). Options are 'regular' and 'raw'.

occupiedCount: integer (int64) 3

Number of intervals to sum the occupied count over (>0) (v2 only)

occupiedThreshold: integer (int64) 3

Threshold for occupied count to trigger occupied state (>0) (v2 only)

outputHistory: boolean false

Output history (v2 only)

unoccupiedCount: integer (int64) 12

Number of intervals to sum the unoccupied count over (>0) (v2 only)

unoccupiedThreshold: integer (int64) 1

Threshold for unoccupied count to trigger unoccupied state (>0) (v2 only)

updateIntervalSec: integer (int64) 600

Update interval in seconds if there is no state change (>=0, 0=always update) (v2 only)

Example
{
  "service": {
    "filters": {
      "my-occ": {
        "type": "occupancy",
        "config": {
          "mode": "raw"
        }
      }
    }
  }
}

removeDevice

This filter removes any device that flows through it from a Device Group. See the documentation for Device Groups for more information.

If the deviceId field is in the input map, that value will be used as the unique ID of the device added. Otherwise, the Device Group's Device ID Template will be used. This defaults to the mac field, but can be templatized to extract other fields out of the event map or compute a deviceId.

If the field removeDevices is present, then that is expected to be a list of device IDs to remove, and deviceId is ignored. These must be exact device IDs, and are not fed through the Device Group's Device ID Template.

Output: Contains all keys from the input

groupId: string

The ID of the Device Group for which you want to change membership. Supports templating.

Example
{
  "service": {
    "filters": {
      "remove-from-my-group": {
        "type": "removeDevice",
        "config": {
          "groupId": "my-ibeacons"
        }
      }
    }
  }
}

ruuvi

This filter parses Ruuvi Tag broadcasts that send out Ruuvi v3 and v5 custom formats. The available data fields are: temperature, pressure, humidity, accelerometer, battery, tx power, movement count, and sequence number.

Output: The filter outputs all available data from the Ruuvi tag and adds the mac address and rssi of the tag.

No configuration is required for this filter.

Example
{
  "service": {
    "filters": {
      "my-ruuvi": {
        "type": "ruuvi"
      }
    }
  }
}

throttle

This filter throttles emission of events by counter or timeout. Events are identified by the key specified in the configuration

In counter mode: Every N-th event matching the id will be emitted (where N = count).

In timeout mode: An event will be emitted if the timeout has elapsed since the last event emitted matching the id. There is no caching of the input events, if no new event is processed, no event will be emitted when the timeout has elapsed.

Output: If unthrottled: a copy of the input event If throttled: nil

count: integer (int64)

Throttle with a counter

ignoreFirstEvent: boolean false

Ignore the first event, regardless of count/timeout - wait until count/timeout is reached before emitting.

key: string "mac"

Keys to throttle on

maxCount: integer (int64) 1000

Maximum number of id's to track, after hitting this limit, unknown id's will be ignored.

timeout: positiveDuration

Throttle with a timeout

Example
{
  "service": {
    "filters": {
      "send-every-30s": {
        "type": "throttle",
        "config": {
          "key": "mac",
          "timeout": 30000000000
        }
      }
    }
  }
}