Spreading Jam

If you want to spread (really, deploy) some Jam, there

In LoRaWAN Networks

graph LR
    subgraph proto[Protobuf Payloads]
        n1(LoRa Node 1) -.- op[Outpost Gateway]
        n2(LoRa Node 2) -.- op[Outpost Gateway]
        n3(LoRa Node 3) -.- op[Outpost Gateway]
        op[Outpost Gateway] --> aws[AWS Wireless]
    end
        aws --> jam --> rp[RedPanda]
    subgraph json[JSON Payloads]
        rp
        jam[Jamformer] --> qdb[(QuestDB)]
    end

graph LR
    subgraph proto[Protobuf Payloads]
        n1(LoRa Node 1) -.-> op[Outpost Gateway]
        n2(LoRa Node 2) -.-> op
        n3(LoRa Node 3) -.-> op
        op --> aws[AWS IoT Wireless]
    end
    subgraph transfom[Encode/Decode]
        aws --> jam[Jam Service]
    end
    subgraph json[JSON Payloads]
        jam --> rp[RedPanda]
        jam --> qdb[(QuestDB)]
    end 
        jam --> store[JamStore]

In Cellular Networks

Let's face it. For IoT networks, cellular data is still expensive.

graph LR
    subgraph proto[Protobuf Payloads]
        n1(Cellular Node 1) -.- aws[MQTT Broker]
        n2(Cellular Node 2) -.- aws
        n3(Cellular Node 3) -.- aws
        aws --> jam[Jamformer]
    end
    subgraph json[JSON Payloads]
        jam --> RedPanda
        jam --> qdb[(QuestDB)]
    end

Why Protobuf

Protobuf is a well established standard for producing compact binary encoded messages with wide language support. When compared with sending JSON strings over the wire, the network savings are considerable.

Compact Messages

Length: 175

{
    "device_id": "129996789012", 
    "battery_soc": 1.11, 
    "battery_volts": 3.74,
    "humidity": 0.51,
    "moisture": 25.0,
    "temperature": 29.3,
    "lat": 51.507351,
    "lng": -0.127758,
    "fw": 1
}

Length: 80

CgwxMjk5OTY3ODkwMTIVexSOPx0pXG9AJVyPAj8tAADIQTVmZupBOeRLqODwwElAQf+z5sdfWsC/SAE=

Almost 60% compression for the same message.

Decoding

Next, if we decode the above we get almost the same JSON/Dict object back.

{
    'device_id': '129996789012',
    'battery_soc': 1.11, 
    'battery_volts': 3.74, 
    'humidity': 0.51, 
    'moisture': 25.0, 
    'temperature': 29.3, 
    'lat': 51.507351, 
    'lng': -0.127758, 
    'fw': 'UPDATING'
}

Caution

Notice how the value was decoded to UPDATING as opposed to the original input of 1. That's because fw is defined as a Protobuf enum. This is configurable using the use_integers_for_enums=False setting when decoding. Learn more.

Strong Schemas

When you treat .proto files as your strict schema definitions...

Components

LoraFormer the service is comprised of 3 major functions.

LoraFormer Transformer

A service for reliably and scalably encoding and decoding (together, transforming) to and from Protobuf messages into a variable of other message formats (JSON primarily).

LoraFormer Registry

A source of truth for all Protobuf definitions.

See Registry for more information.

Buf Schema Registry

We are actively exploring using Buf Schema Registry as a replacement for our homegrown Registry.