feat(sources): add an optional Section field to data sources

[alex-fedotyev]

Add an optional section label to data sources. A source can be tagged with a free-text section from the source settings form; the value is persisted and returned by the read-only GET /api/v2/sources endpoint. Sources without a section are unchanged.

This is the first PR for HDX-4507 (grouping the data source selector by section, with tag-style search). It lands the data model and the read API surface; the selector grouping and search UI follow in later PRs.

Summary

• Add section (optional, max 256 chars) to the shared BaseSourceSchema, so all five source kinds inherit it.
• Persist section on the Mongoose source model. The base schema runs strict: true, so an unlisted field is dropped on write; the model needs section explicitly.
• Add a "Section" input to the source settings form, directly below "Name".
• Document section on the four external source schemas in the GET /api/v2/sources OpenAPI spec and regenerate openapi.json.

Why

section lives on BaseSourceSchema rather than per-kind so every writer (the form, the internal API, any future migration) inherits the same optional field and the same 256-char cap in one place. The external API already re-parses each source through SourceSchema before responding, so once section is on the schema it rides through to GET /api/v2/sources on its own; the only external-API work needed was documenting it in the OpenAPI spec and regenerating openapi.json. The endpoint is read-only (no external write route for sources), so section is read-only externally.

The field is optional, so existing sources, and any source created without one, keep today's behavior: section is absent from the response, not null.

Test plan

• make ci-lint (lint, type check, and lint:openapi, which regenerates the spec and confirms openapi.json is in sync)
• make ci-unit
• External API integration test through the real GET /api/v2/sources handler: a source created with a section returns it; a source created without one omits it. Extends packages/api/src/routers/external-api/__tests__/sources.test.ts (full file is 22/22 green).
• Drove the source settings form with Playwright and captured the new Section input in both light and dark themes; it renders between Name and Source Data Type and matches the existing Name row.

[ui-states: allow] The Section input is a plain optional text field with no empty/loading/error states.
[viewport: allow] Standard full-width form row, identical layout to the existing Name field.

What's not in this PR (follow-up, HDX-4507)

• Grouping and tag-style search over section in the source selector (the selector UI).
• MCP describeSource exposure of section.
• Customer docs for the new field.

[changeset-bot]

🦋 Changeset detected

Latest commit: 9446407

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages

Name	Type
@hyperdx/common-utils	Patch
@hyperdx/api	Patch
@hyperdx/app	Patch
@hyperdx/otel-collector	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hyperdx-oss	Ready	Preview, Comment	Jun 10, 2026 10:14pm
hyperdx-storybook	Ready	Preview, Comment	Jun 10, 2026 10:14pm

[github-actions]

🔴 Tier 4 — Critical

Touches auth, data models, config, tasks, OTel pipeline, ClickHouse, or CI/CD.

Why this tier:

• Critical-path files (1):
  • packages/api/src/routers/external-api/v2/sources.ts
• Cross-layer change: touches frontend (packages/app) + backend (packages/api) + shared utils (packages/common-utils)

Review process: Deep review from a domain expert. Synchronous walkthrough may be required.
SLA: Schedule synchronous review within 2 business days.

Stats

• Production files changed: 5
• Production lines changed: 54 (+ 48 in test files, excluded from tier calculation)
• Branch: alex/HDX-4507-source-section-field
• Author: alex-fedotyev

To override this classification, remove the review/tier-4 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

[greptile-apps]

Greptile Summary

This PR adds an optional section label to all data source types, wiring it through the shared Zod schema, the Mongoose model, the source settings form, and the external GET /api/v2/sources OpenAPI spec. Previous review feedback (dropping nullable: true from the spec and adding maxLength={256} to the form input) has been addressed in this revision.

• Schema & model: section: z.string().max(256).optional() is added to BaseSourceSchema, and section: String is added to the Mongoose base schema so all five source kinds inherit and persist the field without any other schema changes.
• Form: A Section input row is inserted between Name and Source Data Type in SourceForm.tsx, with maxLength={256} matching the Zod cap.
• External API: section is documented in all four source-kind JSDoc blocks and in the regenerated openapi.json, correctly marked as optional (not nullable), consistent with the Zod definition.

Confidence Score: 5/5

Safe to merge; the change is additive and backward-compatible with no impact on existing sources.

The change is purely additive — a new optional field on the base schema, model, and form. Existing sources without a section are unaffected, the external API remains read-only, and the two new integration tests cover the key read paths.

No files require special attention.

Important Files Changed

Filename	Overview
packages/common-utils/src/types.ts	Adds section: z.string().max(256).optional() to BaseSourceSchema; all five source kinds inherit it automatically.
packages/api/src/models/source.ts	Adds section: String to the Mongoose base schema so the field is persisted; no DB-level length constraint, but Zod enforces max 256 at the application layer.
packages/app/src/components/Sources/SourceForm.tsx	Inserts a Section FormRow between Name and Source Data Type; maxLength={256} is present, matching the Zod schema cap.
packages/api/src/routers/external-api/v2/sources.ts	Adds section JSDoc to all four external source schemas (log, trace, metric, session); nullable: true is correctly absent.
packages/api/src/routers/external-api/tests/sources.test.ts	Adds two new integration tests: one verifying section is returned when set, one verifying section is absent when not set.
packages/api/openapi.json	Regenerated spec adds the optional section string field to all four source schemas without nullable: true; consistent with Zod definition.
.changeset/source-section-field.md	Patch-level changeset for common-utils, api, and app packages; description accurately reflects the change.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Source Settings Form] -->|section input maxLength=256| B[useForm submit]
    B -->|internal write API| C[Mongoose sourceBaseSchema\nsection: String]
    C -->|strict: true drops unknown fields\nsection is explicit so it persists| D[(MongoDB)]
    D -->|GET /api/v2/sources| E[formatExternalSource]
    E -->|SourceSchema.safeParse\nsection: z.string.min1.max256.optional| F[External API response]
    F -->|section present if set, absent otherwise| G[API consumer]

Loading

_{Reviews (3): Last reviewed commit: "chore(sources): remove implementation-co..." | Re-trigger Greptile}

[github-actions]

E2E Test Results

✅ All tests passed • 198 passed • 3 skipped • 1297s

Status	Count
✅ Passed	198
❌ Failed	0
⚠️ Flaky	4
⏭️ Skipped	3

Tests ran across 4 shards in parallel.

View full report →

[karl-power]

LGTM. One nice follow up would be to suggest existing sections when inputting one for a source.

[karl-power]

This comment is superfluous and describes the implementation context rather than wider functionality. I would remove it.

[alex-fedotyev]

Good catch. Removed in 9446407. The push reset the review, so it will need another approval when you have a moment.