Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions docs/LEARNING_LOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,57 @@ This file should be updated by Codex after each meaningful change.
### What to learn next
```

## 2026-05-25 - MQTT Read-Only Adapter Foundation

### What changed

Added a read-only MQTT adapter foundation that loads enabled MQTT connection
profiles, consumes configured topic filters through an injected message source,
and emits normalized `process.measurement.recorded` FactoryEvents from
Sparkplug-style JSON demo payloads.

### Why it matters

This implements the next protocol adapter step after OPC-UA while preserving
the same safety boundary: configured source reads flow into FactoryEvents, but
the adapter does not publish commands, browse arbitrary topics, perform
writeback, or add UI ingestion controls.

### How it works

`load_enabled_mqtt_profiles()` reads the local connection profile store and
filters to enabled MQTT profiles. `run_mqtt_read_only_adapter()` passes each
profile to an `MqttMessageSource`, which reads messages from the profile's
configured topic filters. The adapter validates topic filters, parses
Sparkplug-style JSON with a `metrics` array, maps numeric metrics into process
measurement events, and records the profile ID, topic, metric name, adapter
name, and poll index in source metadata. Broker-unavailable, malformed payload,
and unmapped topic states raise clear adapter errors.

### How to run it

The foundation is currently exercised from Python code and tests. It reads the
same local profile store used by the connection profile API and writes to the
same event store interface used by ingestion.

### How to test it

```bash
.venv/bin/python -m pytest services/ingestion/tests/test_mqtt_read_only_adapter.py
```

### Key files

- `services/ingestion/factory_ingestion/mqtt_read_only_adapter.py`
- `services/ingestion/tests/test_mqtt_read_only_adapter.py`
- `services/ingestion/README.md`

### What to learn next

Add a real broker-backed message source once connector runtime orchestration and
secret resolution are defined, then keep publish/writeback behavior out of that
runtime unless a dedicated ADR explicitly approves it.

## 2026-05-24 - OPC-UA Read-Only Adapter Foundation

### What changed
Expand Down
44 changes: 44 additions & 0 deletions services/ingestion/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,50 @@ The test suite covers fake reader behavior, bad node and unavailable server
errors, FactoryEvent source metadata, and a practical integration test against
the local simulator-backed OPC-UA server.

## MQTT Read-Only Adapter Foundation

The read-only MQTT adapter foundation reads enabled MQTT
`ProtocolConnectionProfile` records from the local connection profile store,
uses an injected message source to consume configured topic filters, and emits
normalized `process.measurement.recorded` FactoryEvents into an event store.

This foundation does not add a broker runtime, command publisher, arbitrary
topic discovery, writeback path, or UI ingestion controls. Until a connector ADR
explicitly expands the behavior, the adapter is limited to:

- enabled profiles where `protocol = "mqtt"`;
- the configured broker URL in `endpoint`;
- the configured MQTT client ID, topic filters, QoS, TLS flag, and secret or
certificate references already present in the profile;
- read-only consumption from configured topic filters;
- Sparkplug-style JSON demo payloads with a non-empty `metrics` list.

The first supported payload shape is a JSON object with optional `context` and a
`metrics` array. Each metric object must include `name` and a numeric `value`.
Optional metric fields include `signal_id`, `signal_name`, `unit`, `quality`,
and `context`. Context from the payload and metric is used to populate the
FactoryEvent context; adapter defaults fill any missing site, area, line, asset,
batch, work order, or unit values.

The adapter captures source metadata in each emitted event:

- `source.system` is `mqtt:<connection-profile-id>`;
- `source.adapter` is `mqtt-read-only-adapter`;
- `source.source_event_id` includes the connection profile ID, topic, metric
name, and poll index;
- `payload.tag_name` stores `<topic>:<metric-name>`.

Run the focused tests:

```bash
.venv/bin/python -m pytest services/ingestion/tests/test_mqtt_read_only_adapter.py
```

The test suite covers fake MQTT messages, topic wildcard matching, profile
configuration access, Sparkplug-style JSON payload mapping, broker-unavailable
errors, malformed payload errors, unmapped topic errors, and the absence of a
publish/writeback surface.

## Accepted Event Storage

Accepted events are written to the local JSONL event store:
Expand Down
Loading
Loading