Skip to content

feat: separate "Last Status Update" from "Last Packet Observation" for observers (v3 rebase)#969

Merged
KpaBap merged 7 commits intomasterfrom
feat/observer-last-packet-at-v3
May 2, 2026
Merged

feat: separate "Last Status Update" from "Last Packet Observation" for observers (v3 rebase)#969
KpaBap merged 7 commits intomasterfrom
feat/observer-last-packet-at-v3

Conversation

@Kpa-clawbot
Copy link
Copy Markdown
Owner

Rebased version of #968 (which was itself a rebase of #905) — resolves merge conflict with #906 (clock-skew UI) that landed on master.

Conflict resolution

public/observers.js — master (#906) added "Clock Offset" column to observer table; #968 split "Last Seen" into "Last Status" + "Last Packet" columns. Combined both: the table now has Status | Name | Region | Last Status | Last Packet | Packets | Packets/Hour | Clock Offset | Uptime.

What this PR adds (unchanged from #968/#905)

  • last_packet_at column in observers DB table
  • Separate "Last Status Update" and "Last Packet Observation" display in observers list and detail page
  • Server-side migration to add the column automatically
  • Backfill heuristic for existing data
  • Tests for ingestor and server

Verification

  • All Go tests pass (cmd/server, cmd/ingestor)
  • Frontend tests pass (test-packets.js, test-hash-color.js)
  • Built server, hit /api/observerslast_packet_at field present in JSON
  • Observer table header has all 9 columns including both Last Packet and Clock Offset

Prior PRs

Supersedes #968. Closes #905.

you added 7 commits May 2, 2026 17:58
Add a new 'last_packet_at' column to the observers table that is only
bumped when an actual packet observation lands (InsertTransmission path),
while 'last_seen' continues to be bumped on both status updates and packets.

This allows the UI to distinguish between an observer that is alive
(sending status pings) vs one that is actively forwarding packets.

Schema migration backfills last_packet_at = last_seen for observers
with packet_count > 0. Server API now returns last_packet_at in the
Observer JSON response.
…vers

- observers.js: rename 'Last Seen' column to 'Last Status', add 'Last Packet'
  column with a warning badge when no packets observed or packets lag behind
  status by >10min
- observer-detail.js: add 'Last Status Update' and 'Last Packet Observation'
  stat cards with relative + absolute timestamps
- Ingestor: verify last_packet_at is NULL after UpsertObserver (status path),
  set after InsertTransmission, and unchanged by subsequent UpsertObserver calls
- Server: verify last_packet_at reads back through GetObservers and GetObserverByID
The addition of the Last Packet column brings the table to 8 columns.
The previous min-width of 640px was tight for 7 columns; 720px prevents
cramped rendering and ensures the horizontal scroll trigger is appropriate
on narrow viewports.
…istic, test comment

1. Migration ALTER error no longer swallowed: check error from ALTER TABLE
   and return if it fails (unless column already exists). Migration is not
   marked complete on failure.

2. Backfill heuristic fixed: use observations table JOIN instead of
   packet_count > 0, since UpsertObserver sets packet_count = 1 on INSERT
   even for status-only observers.

3. Test clarifying comment: document that InsertTransmission uses
   data.Timestamp (not time.Now()) as source-of-truth for last_packet_at,
   so the hardcoded assertion is correct.
Mirror the ensureObserverInactiveColumn pattern (PR #961) for the
last_packet_at column added by the ingestor migration. Without this,
the server SELECTs last_packet_at but never adds it — causing 500
errors on /api/observers when running against DBs the ingestor has
not yet touched (e.g. the e2e fixture).

Adds TestEnsureLastPacketAtColumn for correctness + idempotency.
@KpaBap KpaBap merged commit 3364eed into master May 2, 2026
6 checks passed
@KpaBap KpaBap deleted the feat/observer-last-packet-at-v3 branch May 2, 2026 19:03
Elektr0Vodka added a commit to Cornmeister/CoreScope that referenced this pull request May 3, 2026
…lawbot#969)

Extend detectSchema() to probe PRAGMA table_info(observers) at startup
and set hasObserverLastPacket. GetObservers and GetObserverByID branch
on that flag so the server works against DBs that predate the ingestor
migration, avoiding a "no such column" crash on observer endpoints.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants