Platform Overview

MeteringLab is a self-hosted Advanced Metering Infrastructure (AMI) platform for water utilities. It connects physical water meters — whether they speak LoRaWAN, NB-IoT, or are polled via DLMS — to a full data pipeline: readings storage, alarm management, OTA firmware updates, DLMS/COSEM protocol translation, and billing-ready interval data.

Everything runs on a single Linux server in Docker Compose. There are no cloud dependencies. The architecture is explicitly designed to separate into independent microservices so it can scale horizontally when needed — but operates efficiently as a single-server deployment today.

Why this platform exists

Traditional AMI systems are locked to a single vendor, a single protocol, or a hosted cloud service. This platform is built to be:

• Vendor-agnostic — supports Diehl, Kamstrup, Axioma, Itron, Arad, Sagemcom, and any meter with a codec
• Protocol-agnostic — LoRaWAN and NB-IoT on the radio side; DLMS/COSEM on the utility-software side
• Self-hosted — no SaaS fees, no data leaving your infrastructure
• Open standards — MQTT, REST/JSON, IEC 62056 DLMS, OMA LwM2M
• Production-ready path — multi-tenant from row zero, designed for scale-out

System Architecture

The platform is composed of containerized services that communicate exclusively over the network — never via shared memory or direct database access across service boundaries.

  ┌─────────────────────────────────────────────────────────────────────────┐
  │                         PHYSICAL LAYER                                  │
  │   LoRaWAN meter ──► Kerlink Gateway ──► Basic Station WSS (port 3002) │
  │   NB-IoT meter  ──────────────────────► Leshan CoAP (UDP 5683/5684)   │
  └────────────────────────────┬────────────────────────┬───────────────────┘
                               │                        │
                    ┌──────────▼──────────┐  ┌─────────▼──────────┐
                    │   ChirpStack NS     │  │   Leshan LwM2M     │
                    │   (port 8080)       │  │   (port 8090)      │
                    └──────────┬──────────┘  └─────────┬──────────┘
                               │ MQTT                   │ SSE stream
                               │ chirpstack/…/event/up  │
                    ┌──────────▼──────────┐  ┌─────────▼──────────┐
                    │   HES MQTT          │  │   LwM2M Bridge     │
                    │   ChirpStack bridge │  │   (port 8300)      │
                    └──────────┬──────────┘  └─────────┬──────────┘
                               │                        │
                               └────────────┬───────────┘
                                            │ MQTT hes/reading/{tenant}/{eui}
                                 ┌──────────▼──────────────────────────┐
                                 │          HES (port 8100)            │
                                 │  Codec decode → canonical reading    │
                                 │  TimescaleDB (meter_readings table)  │
                                 │  Alarm engine, FUOTA, Webhooks       │
                                 └──────┬──────────────────────┬────────┘
                                        │ MQTT                 │ HTTP poll
                                        │ hes/reading/#        │ gurux-dlms
                              ┌─────────▼─────────┐  ┌────────▼────────┐
                              │   MDM (port 8200) │  │ DLMS concentr.  │
                              │   VEE · Billing   │  │ (port 4059 TCP  │
                              │   NRW · Hourly    │  │  port 8400 HTTP)│
                              └───────────────────┘  └─────────────────┘
                                        │
                              ┌─────────▼─────────┐
                              │   Grafana (3003)  │
                              │   TimescaleDB src  │
                              └───────────────────┘

Network Server Layer

The network server layer handles the radio protocol — it knows packets, frequencies, spreading factors, and device activation. It does not know what a water meter is.

ChirpStack (LoRaWAN)

ChirpStack v4 runs as the LoRaWAN Network Server (NS), Application Server (AS), and Join Server (JS) in one stack. Kerlink gateways connect to it via Basic Station over WSS on port 3002. When a meter uplink arrives, ChirpStack decodes the LoRaWAN MAC layer and publishes the raw payload to MQTT topic chirpstack/application/{app_id}/device/{devEui}/event/up.

Leshan (NB-IoT / LwM2M)

Eclipse Leshan is the OMA LwM2M server for NB-IoT devices. Meters register over CoAP (UDP 5683) or CoAPS with DTLS (UDP 5684). Leshan exposes a REST API and Server-Sent Events (SSE) stream that the LwM2M bridge subscribes to. When a meter sends an observe/notify for volume or flow objects, Leshan delivers the event via SSE and the bridge translates it.

Head-End System (HES)

HES is the brain of the platform. It subscribes to both ChirpStack MQTT and LwM2M Bridge MQTT, applies the correct vendor codec to decode raw payloads into a canonical reading format, and stores the result in TimescaleDB.

Key responsibilities:

• Codec registry — maps codec_id to a decode function per vendor
• Canonical model — volume_m3, flow_lph, battery_pct, rssi_dbm, snr_db, alarms[]
• Alarm engine — raises/clears alarms, stores history, fires webhooks
• FUOTA — Firmware Update Over The Air session management (Class A LoRaWAN, LwM2M /5)
• Webhooks — HTTP POST callbacks for reading.received, alarm.raised, alarm.cleared
• API keys — scoped read/write/admin tokens for programmatic access
• DLMS polling — acts as a DLMS client, reads from the virtual concentrator, stores poll records

DLMS Virtual Concentrator

This service makes NB-IoT/LoRaWAN meters appear to any external DLMS-speaking software (another HES, a utility billing system, a SCADA) as if they were native DLMS/COSEM meters.

It subscribes to hes/reading/# MQTT and caches the latest reading for each meter in memory. When a DLMS client connects on TCP port 4059, it performs the standard DLMS TCP Wrapper (IEC 62056-47) handshake — AARQ/AARE — and then serves GET requests for COSEM objects using the cached reading values.

COSEM objects exposed:

Meter Data Management (MDM)

MDM consumes canonical HES events and turns them into billing-ready data products. It never talks to the Network Server — it only knows about canonical readings.

• Hourly/daily aggregation — consumption m³, average flow L/h, quality flag per interval
• VEE — Validation (range checks, rate-of-change), Estimation (fill gaps with averages), Editing (manual corrections)
• Non-Revenue Water (NRW) — per District Metering Area (DMA): inflow − outflow − billed = NRW alert
• Billing export — CSV/JSON intervals per device for billing system ingestion

Device EUIs Explained

Every meter in the system is uniquely identified by its Device EUI (Extended Unique Identifier) — a 64-bit (8-byte, 16 hex character) address assigned by the manufacturer and burned into the hardware.

EUI in LoRaWAN

In LoRaWAN, three EUI-64 values identify a device:

• DevEUI — the device's unique address (like a MAC address)
• AppEUI / JoinEUI — identifies the application/join server
• DevAddr — a temporary 32-bit network address assigned after OTAA join

ChirpStack uses the DevEUI as the canonical device identifier. It appears in MQTT topics as devEui (lowercase hex).

EUI in NB-IoT / LwM2M

NB-IoT devices don't have a LoRaWAN EUI. They identify themselves to Leshan using an endpoint name — typically their IMEI (urn:imei:351234567890123) or a manufacturer-assigned EUI-64 formatted as hex. The LwM2M bridge maps this to the device_eui field used everywhere in the platform.

EUI as the universal key

Once inside the platform, device_eui is the foreign key that links every record across every service:

  device_eui = "deadbeef00000001"

  HES:  devices table         → codec_id, tenant_id, vendor, model
  HES:  meter_readings        → volume_m3, flow_lph, battery_pct, alarms[]
  HES:  alarms                → alarm_type, raised_at, cleared_at
  HES:  dlms_poll_records     → logical_address, polled_at, volume_m3
  MDM:  hourly_intervals      → hour_start, consumption_m3, vee_status
  MDM:  daily_intervals       → date, total_m3, peak_flow_lph
  DLMS: dlms_devices          → logical_address (WPORT for TCP)
  MQTT: hes/reading/{tenant}/{device_eui}

DLMS Logical Address (WPORT)

DLMS/COSEM TCP identifies devices by a logical address (WPORT) — a 16-bit integer in the TCP Wrapper header. The DLMS concentrator maintains a mapping table: device_eui → logical_address. When a DLMS client sends a request to WPORT 2001, the concentrator looks up which EUI maps to that address and serves that meter's cached reading.

You register mappings via the DLMS API:

POST https://dlms.meteringlab.com/devices
{
  "device_eui": "deadbeef00000001",
  "tenant_id": "00000000-0000-0000-0000-000000000001",
  "logical_address": 2001,
  "description": "Diehl IZAR — Site A, Meter 01"
}

Complete Data Flow

LoRaWAN Path

1. Meter transmits uplink The water meter wakes up (Class A), transmits a LoRa radio packet on a configured frequency/spreading-factor, and goes back to sleep. LoRaWAN frames are encrypted at the MAC layer.
2. Gateway receives & forwards The Kerlink gateway receives the packet on all channels simultaneously, timestamps it, and forwards it to ChirpStack via Basic Station WebSocket (WSS port 3002). Multiple gateways can receive the same uplink — ChirpStack deduplicates.
3. ChirpStack decodes LoRaWAN MAC ChirpStack decrypts the LoRaWAN frame, verifies the MIC (message integrity code), handles OTAA join if first time, extracts the raw application payload (base64), and publishes it to MQTT:
   chirpstack/application/{app_id}/device/{devEui}/event/up
4. HES ChirpStack bridge receives The HES MQTT subscriber picks up the message. It extracts devEui, data (base64 payload), fPort, and RF metrics (rssi, snr) from the JSON.
5. Codec decodes vendor payload HES looks up the device's codec_id from the database, loads the corresponding codec module (e.g., diehl_izar_v1), and calls decode(base64_payload). The codec returns a canonical dict: {volume_m3, flow_lph, battery_pct, alarms, timestamp}.
6. Canonical reading stored HES writes the reading to TimescaleDB's meter_readings hypertable (partitioned by time). Duplicates are caught by (device_eui, fcnt) unique constraint.
7. Published to MQTT HES publishes the canonical reading to hes/reading/{tenant_id}/{device_eui}. All downstream subscribers (DLMS concentrator, MDM, Grafana Live) receive it simultaneously.
8. Alarm engine evaluates Alarms from the codec's alarms[] list are compared against the device's active alarm set. New alarms are raised; cleared alarms are resolved. Webhooks fire for each change.
9. MDM aggregates MDM's background job processes the reading into hourly and daily intervals, runs VEE checks, and updates the DMA NRW calculation.
10. DLMS concentrator caches The DLMS concentrator's MQTT subscriber updates its in-memory cache. The next DLMS poll from any utility software will read the fresh value.

NB-IoT / LwM2M Path

1. Meter registers with Leshan The NB-IoT meter powers on, attaches to the cellular network, and sends a CoAP REGISTER to Leshan (UDP 5683 or DTLS 5684). It advertises its LwM2M objects: /3 (device), /3303 (temperature), /3311 (flow), and a custom volume object.
2. LwM2M bridge subscribes via SSE The bridge maintains an open HTTP SSE connection to Leshan's /event endpoint. It sets up OBSERVE requests on the meter's volume and flow objects so Leshan forwards changes as they happen.
3. Notify event arrives When the meter sends a CoAP NOTIFY (after a value change or reporting interval), Leshan delivers it via SSE to the bridge. The bridge reads the LwM2M resource value (raw number).
4. Bridge normalizes and publishes The bridge maps LwM2M object IDs to canonical fields and publishes to hes/reading/{tenant}/{device_eui} — the same MQTT topic as LoRaWAN. From this point, the path is identical.

Vendor Codec Registry

Each vendor encodes meter data differently in their LoRaWAN payload. Codecs are pure Python functions that take raw bytes and return a canonical dict.

Adding a new codec

1. Create the codec file Add services/hes/src/codecs/my_vendor.py with a decode(payload: bytes) -> dict | None function. Return only canonical keys: timestamp, volume_m3, flow_lph, battery_pct, rssi_dbm, snr_db, alarms.
2. Register it Add an entry to _REGISTRY in services/hes/src/codecs/__init__.py:
   "my_vendor_v1": "src.codecs.my_vendor"
3. Provision the device Insert the device into the HES database with codec_id = "my_vendor_v1". Uplinks will be decoded automatically from that point.
4. Rebuild HES Run docker compose build hes && docker compose up -d hes to load the new codec.

DLMS/COSEM Deep Dive

DLMS (Device Language Message Specification) / COSEM (Companion Specification for Energy Metering) is the IEC 62056 standard protocol used by most utility HES software to read smart meters. This platform implements a virtual concentrator that bridges IoT protocols to DLMS.

Why DLMS matters

Many utility billing systems, SCADA platforms, and regulatory systems require meter data via DLMS. By presenting IoT meters as DLMS devices, this platform integrates with existing utility software without requiring any changes to those systems.

TCP Wrapper (IEC 62056-47)

DLMS over TCP uses an 8-byte header prepended to each frame:

Offset  Length  Field
  0       2     Version (always 0x0001)
  2       2     Source WPORT (client logical address)
  4       2     Destination WPORT (server = meter logical address)
  6       2     Data length (bytes that follow)

The destination WPORT is the meter's logical address — the same value registered in the DLMS concentrator's device mapping table.

DLMS Session (AARQ/AARE)

Before reading data, a DLMS client must establish an association:

1. AARQ — Client sends an Association Request (AARQ) PDU. This identifies the client, requests authentication (None in this deployment), and negotiates PDU size.
2. AARE — Server responds with Association Response (AARE). Result 0x00 = accepted. Session is now open.
3. GET requests — Client reads COSEM attributes using Get-Request PDUs, specifying the class ID and OBIS code. Server returns the value in a Get-Response PDU.
4. Disconnect — Client sends a disconnect request to cleanly close the session.

gurux-dlms library

Both the DLMS server (concentrator) and client (HES poll) use the open-source gurux-dlms 1.0.197 Python library. Several bugs in that version required workarounds:

• GXServerReply was missing setReply() and getConnectionInfo() — added via subclass
• MeterServer was missing notifyRead(), notifyConnected(), getTransaction(), setTransaction() — added as stubs
• Object DataType must be explicitly set via setDataType(2, DataType.FLOAT64) — otherwise gurux encodes as NONE and the client gets null
• Values must be plain Python float — GXFloat64 objects fail in struct.pack("d", ...)
• notifyRead() is the correct callback to refresh cached values before getValue(), NOT onPreGet (which gurux Python never calls)

Onboarding a New Meter

Adding a new physical meter to the platform takes under 5 minutes. Here is the full procedure.

LoRaWAN meter

1. Register in ChirpStack Open ns.meteringlab.com, create a device profile (matching the meter's LoRaWAN version and codec), then create a device with the correct DevEUI, AppEUI, and AppKey. The meter will OTAA-join on next transmission.
2. Insert into HES database The HES API does not have a device creation endpoint — devices are provisioned directly into the DB:
   

   docker exec ami-platform-timescaledb-1 psql -U ami ami -c \
   "INSERT INTO devices (id,tenant_id,device_eui,vendor,model,codec_id,created_at,is_active)
    VALUES (gen_random_uuid(),
            '00000000-0000-0000-0000-000000000001',
            'YOUR_16_HEX_EUI',
            'diehl', 'IZAR RC IoT', 'diehl_izar_v1',
            now(), true)
    ON CONFLICT (device_eui) DO NOTHING;"

3. Register DLMS mapping Choose a unique logical address (WPORT) between 1000–65534 and register:
   

   curl -X POST https://dlms.meteringlab.com/devices \
     -H "Content-Type: application/json" \
     -d '{
       "device_eui": "YOUR_16_HEX_EUI",
       "tenant_id":  "00000000-0000-0000-0000-000000000001",
       "logical_address": 2001,
       "description": "Site A - Meter 01"
     }'

4. Verify readings arrive Open hes.meteringlab.com and watch the Devices tab. The meter should appear with a "last seen" timestamp within one reporting interval.
5. Check DLMS Open dlms.meteringlab.com → Live Cache tab. The meter's volume and flow should appear. The HES will automatically poll it every 5 minutes.

NB-IoT / LwM2M meter

1. Meter registers with Leshan Configure the meter's LwM2M bootstrap or direct server URI to point to coap://meteringlab.com:5683 (or coaps:// with DTLS on 5684). The meter will self-register.
2. Note the endpoint name Open leshan.meteringlab.com and find the registered endpoint. This is the string used as device_eui in the platform.
3. Register in LwM2M bridge POST to the bridge API with the endpoint name (treated as EUI):
   

   curl -X POST https://lwm2m.meteringlab.com/devices \
     -H "Content-Type: application/json" \
     -d '{
       "endpoint_name": "urn:imei:351234567890123",
       "tenant_id": "00000000-0000-0000-0000-000000000001"
     }'

4. Continue as LoRaWAN Steps 2–5 from the LoRaWAN procedure above apply identically — the HES and DLMS concentrator don't distinguish by radio technology.

Testing & Validation

Single-meter E2E test

Verifies the full pipeline: MQTT reading → DLMS concentrator cache → HES DLMS poll → DB record.

bash /root/ami-platform/scripts/e2e_dlms_test.sh

Multi-vendor test (all 7 codecs)

Encodes real vendor payloads for all 7 supported meters, injects them via ChirpStack MQTT, and verifies DLMS poll records match — 28 checks total.

bash /root/ami-platform/scripts/e2e_multi_meter_test.sh

Simulating a reading via MQTT

Publish directly to the canonical reading topic to test any downstream service:

docker exec ami-platform-mosquitto-1 mosquitto_pub \
  -t "hes/reading/00000000-0000-0000-0000-000000000001/YOUR_EUI" \
  -m '{
    "device_eui":  "YOUR_EUI",
    "tenant_id":   "00000000-0000-0000-0000-000000000001",
    "timestamp":   "2026-04-19T12:00:00Z",
    "volume_m3":   1234.567,
    "flow_lph":    45.0,
    "battery_pct": 88.0,
    "rssi_dbm":    -80,
    "snr_db":      7.5,
    "alarms":      []
  }'

Triggering an immediate DLMS poll

curl -X POST "https://hes.meteringlab.com/dlms/poll-now?tenant_id=00000000-0000-0000-0000-000000000001"

Checking poll records

curl "https://hes.meteringlab.com/dlms/polls?tenant_id=00000000-0000-0000-0000-000000000001&limit=10"

All Default Credentials

Port Reference