Skip to content

Fix Source.socat_id type: int -> UUID, matching socat's real IDs - #12

Merged
JBorrow merged 1 commit into
mainfrom
fix-socat-id-uuid-type
Jul 22, 2026
Merged

Fix Source.socat_id type: int -> UUID, matching socat's real IDs#12
JBorrow merged 1 commit into
mainfrom
fix-socat-id-uuid-type

Conversation

@axf295

@axf295 axf295 commented Jul 22, 2026

Copy link
Copy Markdown

socat's source IDs have been UUIDs for as long as this repo's own integrations/socat.py has existed (get_box_fixed() returns RegisteredFixedSource.source_id: uuid.UUID, and upsert_sources() already passes that straight through to backend.get_by_socat_id() and Source(socat_id=...)) -- but Source.socat_id, SourceMetadata.socat_id, the ProvidesSourceStorage.get_by_socat_id Protocol, the Postgres sources.socat_id column (INTEGER UNIQUE), and the Postgres/parquet storage implementations were all still typed/declared as int. Any real socat crossmatch (a UUID) hitting this path would fail: Postgres would reject the UUID value for an integer column, and the parquet backend's table["socat_id"] == socat_id comparison would never match after an int-typed column got created from UUID objects.

Fixed end to end: models/source.py, storage/prototype/source.py (Protocol), storage/postgres/{schema,source}.py, and storage/parquet/source.py (which additionally needed the same str-round-trip treatment already given to source_id, since parquet has no native UUID type -- with explicit None-handling, since socat_id is optional and pandas represents a missing value in an otherwise-string column as NaN, not None).

tests/test_integrations/test_socat.py::test_insert already exercised this exact path with real UUIDs (via socat.client.mock.Client), but only through the session-scoped backend fixture, which requires Docker/testcontainers for every parametrization including "parquet" (the shared postgres_database/timescale_database fixtures are unconditional function arguments, and setup_test_data is autouse=True, so no test in this suite can currently run without Docker). Verified the fix with an equivalent standalone script directly against generate_pandas_backend() (no Docker), confirming create -> upsert -> get_by_socat_id round-trips correctly by UUID equality, and that a second upsert_sources() call is idempotent. Also added tests/test_integrations/test_socat_parquet.py, which exercises the same path by building a Pandas backend directly rather than going through the Docker-gated backend fixture -- it will run standalone once Docker is available (e.g. in CI), even though this sandboxed environment couldn't execute it directly (the autouse setup_test_data fixture still pulls in the Docker-gated backend fixture for every test in this directory regardless of what an individual test declares).

socat's source IDs have been UUIDs for as long as this repo's own
integrations/socat.py has existed (get_box_fixed() returns
RegisteredFixedSource.source_id: uuid.UUID, and upsert_sources() already
passes that straight through to backend.get_by_socat_id() and
Source(socat_id=...)) -- but Source.socat_id, SourceMetadata.socat_id,
the ProvidesSourceStorage.get_by_socat_id Protocol, the Postgres
sources.socat_id column (INTEGER UNIQUE), and the Postgres/parquet
storage implementations were all still typed/declared as int. Any real
socat crossmatch (a UUID) hitting this path would fail: Postgres would
reject the UUID value for an integer column, and the parquet backend's
`table["socat_id"] == socat_id` comparison would never match after an
int-typed column got created from UUID objects.

Fixed end to end: models/source.py, storage/prototype/source.py
(Protocol), storage/postgres/{schema,source}.py, and
storage/parquet/source.py (which additionally needed the same
str-round-trip treatment already given to source_id, since parquet has
no native UUID type -- with explicit None-handling, since socat_id is
optional and pandas represents a missing value in an otherwise-string
column as NaN, not None).

tests/test_integrations/test_socat.py::test_insert already exercised
this exact path with real UUIDs (via socat.client.mock.Client), but
only through the session-scoped `backend` fixture, which requires
Docker/testcontainers for every parametrization including "parquet"
(the shared `postgres_database`/`timescale_database` fixtures are
unconditional function arguments, and setup_test_data is autouse=True,
so no test in this suite can currently run without Docker). Verified
the fix with an equivalent standalone script directly against
generate_pandas_backend() (no Docker), confirming create -> upsert ->
get_by_socat_id round-trips correctly by UUID equality, and that a
second upsert_sources() call is idempotent. Also added
tests/test_integrations/test_socat_parquet.py, which exercises the
same path by building a Pandas backend directly rather than going
through the Docker-gated `backend` fixture -- it will run standalone
once Docker is available (e.g. in CI), even though this sandboxed
environment couldn't execute it directly (the autouse setup_test_data
fixture still pulls in the Docker-gated `backend` fixture for every
test in this directory regardless of what an individual test declares).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@JBorrow
JBorrow merged commit b70abeb into main Jul 22, 2026
3 checks passed
@JBorrow
JBorrow deleted the fix-socat-id-uuid-type branch July 22, 2026 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants