Skip to content

adaptive_export passthrough fails on http2_messages.beta every cycle when the beta table is not registered in Vizier #60

Description

@entlein

Summary

In passthrough mode (ADAPTIVE_PASSTHROUGH=1) adaptive_export iterates its builtin
table list and queries each one via PxL. http2_messages.beta fails PxL
compilation every cycle
with Table 'http2_messages.beta' not found, so HTTP/2 +
gRPC is never exported on Viziers where that beta table is not registered, and a
warning is logged on every passthrough pass per node.

Evidence

Rig: k3s, Vizier 0.14.19-aeprod-clean4 (registered, healthy). AE log, every passthrough cycle:

level=warning msg="ADAPTIVE_PASSTHROUGH: pixie query failed"
  error="pixieapi: PxL compilation: Compilation failed: \n2:18 Table 'http2_messages.beta' not found."
  table=http2_messages.beta

Live px run probes against the same Vizier:

table queried result
http2_messages.beta Table not found
http2_messages Table not found
http2_events Table not found
grpc_events Table not found

Every other builtin table compiles fine (http_events, dns_events, conn_stats,
redis_events, mysql_events, pgsql_events, cql_events, mongodb_events,
amqp_events, mux_events, tls_events, and notably kafka_events.beta — the other
dotted beta table — all work).

The table name is correct (not a typo, not missing from the DDL)

Stirling defines the table verbatim as http2_messages.beta:

src/stirling/source_connectors/socket_tracer/http2_messages_table.h:56

DataTableSchema("http2_messages.beta", "HTTP2 messages events", kHTTP2MessagesElements);

And adaptive_export already includes it in both the builtin list and the forensic_db DDL:

  • src/vizier/services/adaptive_export/internal/pxl/tables.go:63{Name: "http2_messages.beta", Protocol: "HTTP/2 + gRPC"}
  • src/vizier/services/adaptive_export/internal/clickhouse/schema.sql:111CREATE TABLE IF NOT EXISTS forensic_db.http2_messages.beta (...)

So the ClickHouse side already exists and the name matches the Stirling source. This is
not a missing-DDL or wrong-name issue.

Root cause

http2_messages.beta is a beta data table that is not present in this Vizier's schema
(the http2 source / beta table is not registered by default — unlike kafka_events.beta,
which is). AE queries it unconditionally and emits a hard warning every cycle.

Impact

  • HTTP/2 + gRPC traffic is not exported when the beta table is not registered in Vizier.
  • One warning per passthrough cycle per node (log noise).
  • Non-fatal (the query failure is handled as a warning; other tables continue).

Proposed fix (pick one)

  1. Probe the Vizier schema once (or on first failure) and skip builtin tables that are
    absent from it — graceful, recommended.
  2. Gate http2_messages.beta behind a config flag (e.g. ADAPTIVE_ENABLE_HTTP2_BETA,
    default off) so it is only queried where the beta table is enabled.
  3. Downgrade the per-cycle "table not found" warning to debug for known-absent tables, and
    document the Vizier flag required to register the http2 beta table.

The name correctness and DDL presence are confirmed; the open decision is which of the
above to implement.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions