4.6. Metadata

This document describes the metadata functionality of Edge Connect.

4.6.1. Types

Two distinct types of metadata are supported in Edge Connect, device and global.

All metadata:

  • Is keyed with a string and can accept any value type (arrays, maps, etc are permitted)
  • Persists for the lifetime of the Edge Connect process unless cleared by metadataClear action/filter.
  • Optionally can be persisted to disk using the persistent configuration option, in order to survive reboots/updates.

Global Metadata

Global metadata is used to store data associated with the instance of Edge Connect.

Use cases:

  • Gateway location data
  • Gateway identification data
  • Authentication data (API keys, etc)

Device Metadata

Device metadata is used to store data associated with a particular device.

A device is identified by the Bluetooth MAC address, which must be present as the mac key in an event.

Use cases:

  • Identification data (map a physical device to an ID)
  • Authentication data (cloud or device)
  • Device data (battery level, sensor data, last refreshed, etc)

4.6.2. Operations

The supported metadata operations are “Get”, “Set” and “Clear” which can be performed on global metadata, all device metadata, or a single device’s metadata.

To configure a metadata operation you can control the operation with the following:

  • global: If set to true global metadata operations will be performed.
  • keys: If non-empty will set the metadata keys to operate on. If empty, all available metadata keys are operated on. (default: empty).

Key dot-notation is allowed in keys.

Get (metadataGet)

The Get operation is used to get metadata and add it into the input map.

Global Mode (global==true)

  • if keys is non-empty, the keys in keys are retrieved (if present) from global metadata and added to the input map.
  • If keys is empty, all global metadata keys are added to the input map.

Device Mode (global==false)

  • The device metadata is looked up using the MAC.
  • if keys is non-empty, the keys in keys are retrieved (if present) from the device’s metadata and added to the input map.
  • If keys is empty, all keys from the device’s metadata are added to the input map.

Set (metadataSet)

The Set operation is used to take data from the input map and store it in metadata. When a Set is performed on a metadata key that exists, the previous value is overwritten.

Global Mode (global==true)

  • if keys is non-empty, the keys in keys are retrieved (if present) from the input map and stored into global metadata.
  • If keys is empty, all keys from the input map are stored into global metadata.

Device Mode (global==false)

  • The device metadata is looked up using the MAC.
  • if keys is non-empty, the keys in keys are retrieved (if present) from the input map and stored in the device’s metadata.
  • If keys is empty, all keys from the input map are stored into the device’s metadata.

Clear (metadataClear)

The Clear operation is used to clear metadata when an input map is received.

In addition to the global setting, there is an allDevices setting that allows performing device metadata operations on all device metadata.

Global Mode (global==true)

  • if keys is non-empty, the keys in keys are retrieved (if present) from the input map and stored into global metadata.
  • If keys is empty, all keys from the input map are stored into global metadata.

All Devices Mode (global==false and allDevices==true)

  • if keys is non-empty, the keys in keys are deleted from every device’s metadata stored in device metadata.
  • If keys is empty, all devices are deleted from device metadata.

Single Device Mode (global==false and allDevices==false)

  • The device metadata is looked up using the MAC.
  • if keys is non-empty, the keys in keys are deleted from device’s metadata.
  • If keys is empty, the device is deleted from device metadata.

4.6.3. Usage

Metadata operations are permitted in an Edge Connect pipeline as well as in a connection sequence. Naming of pipeline filters and connection actions have the same name and configuration for a given operation (see Operations).

In a Pipeline

To perform metadata operations in a data pipeline, add one of the metadata filters to your pipeline:

  • metadataGet
  • metadataSet
  • metadataClear

In a Connection

To perform metadata operations in a connection sequence, add one of the metadata actions to your connection sequence:

  • metadataGet
  • metadataSet
  • metadataClear

4.6.4. Initialization

On Edge Connect startup, global and device metadata are empty. If metadata needs to be initialized at startup, populate service.metadata in the Edge Connect configuration.

Key dot-notation is allowed at the top-level of service.metadata.global and service.metadata.devices.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
{
    "apps": {
        "rigado-edge-connect": {
            "values": {
                "service": {
                    "metadata": {
                        "global": {
                            "apiKey": "secretStuff9000",
                            "location.lat": "12",
                            "location.lon": "34"
                        },
                        "devices": {
                            "9454930000001": {
                                "key": "secret88"
                            },
                            "9454930000002": {
                                "key": "secret99"
                            }
                        }
                    }
                }
            }
        }
    }
}

4.6.5. Example Configuration

The following example configuration connects to two Thingy52 sensors and collects the battery level and sends the data to MQTT every 2 minutes.

Gateway location metadata is initialized in service.metadata.global and then attached to a device with the get-global-metadata filter.

Device alias metadata is initialized in service.metadata.devices per device.

Device battery metadata is set in the thingy connection sequence by the set-device-metadata action.

Device metadata is then attached to an advertisement with get-device-metadata filter.

The device metadata key battery is cleared for all devices when the aggregator fires an event so only recent battery levels are published.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
{
    "apps": {
        "rigado-edge-connect": {
            "values": {
                "service": {
                    "metadata": {
                        "global": {
                            "location": "desk"
                        },
                        "devices": {
                            "c4efb9686279": {
                                "alias": "chair"
                            },
                            "e91ae5a6831f": {
                                "alias": "laptop"
                            }
                        }
                    },
                    "cloud": {
                        "connectors": {
                            "mqtt": {
                                "type": "mqtt",
                                "config": {
                                    "host": "192.168.1.1",
                                    "port": 1883,
                                    "scheme": "tcp"
                                }
                            }
                        }
                    },
                    "filters": {
                        "connect": {
                            "type": "connect",
                            "config": {
                                "id": "thingy",
                                "interval": "1m",
                                "timeout": 10
                            }
                        },
                        "select": {
                            "type": "select",
                            "config": {
                                "keys": [
                                    "mac",
                                    "rssi",
                                    "name"
                                ]
                            }
                        },
                        "get-device-metadata": {
                            "type": "metadataGet",
                            "config": {
                                "keys": [
                                    "battery",
                                    "alias"
                                ]
                            }
                        },
                        "get-global-metadata": {
                            "type": "metadataGet",
                            "config": {
                                "global": true,
                                "keys": [
                                    "location"
                                ]
                            }
                        },
                        "aggregate": {
                            "type": "aggregate",
                            "config": {
                                "mode": "map",
                                "key": "mac",
                                "interval": "2m"
                            }
                        },
                        "clear-device-metadata": {
                            "type": "metadataClear",
                            "config": {
                                "allDevices": true,
                                "keys": [
                                    "battery"
                                ]
                            }
                        },
                        "publish": {
                            "type": "publishTopic",
                            "config": {
                                "connector": "mqtt",
                                "topic": "test/thingy"
                            }
                        }
                    },
                    "groups": {
                        "thingys": {
                            "type": "static",
                            "config": {
                                "include": [
                                    "c4efb9686279",
                                    "e91ae5a6831f"
                                ]
                            }
                        }
                    },
                    "pipelines": {
                        "connectThingy": {
                            "groups": [
                                "thingys"
                            ],
                            "filters": [
                                "connect"
                            ]
                        },
                        "publishThingy": {
                            "groups": [
                                "thingys"
                            ],
                            "filters": [
                                "select",
                                "get-device-metadata",
                                "get-global-metadata",
                                "aggregate",
                                "clear-device-metadata",
                                "publish"
                            ]
                        }
                    },
                    "actions": {
                        "read-battery": {
                            "type": "readBattery",
                            "config": {
                                "outputKey": "battery"
                            }
                        },
                        "set-device-metadata": {
                            "type": "metadataSet",
                            "config": {
                                "keys": [
                                    "battery"
                                ]
                            }
                        }
                    },
                    "connections": {
                        "thingy": {
                            "actions": [
                                "read-battery",
                                "set-device-metadata"
                            ]
                        }
                    }
                }
            }
        }
    }
}