Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/public/getting-started/first-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,11 +79,11 @@ VectorFlow validates connection compatibility in real time. You cannot connect a
{% endstep %}

{% step %}
### Configure component keys
### Name your components

Each node has a **Component Key** in the detail panel (e.g., `demo_logs_0`). This key becomes the component ID in the generated Vector configuration. You can rename keys to something more descriptive like `demo_source`, `add_timestamp`, and `debug_output`.
Each node has a **Name** field in the detail panel. By default, new components are named after their type (e.g., "Demo Logs"). You can rename them to something more descriptive like "Demo Source", "Add Timestamp", and "Debug Output".

Keys must contain only letters, numbers, and underscores.
Renaming a component only requires saving the pipeline -- it does not require a redeploy. The backend component ID is auto-generated and shown as a read-only field below the name.
{% endstep %}

{% step %}
Expand Down
3 changes: 2 additions & 1 deletion docs/public/reference/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,8 @@ Each node in the `saveGraph` input:
| Field | Type | Description |
|-------|------|-------------|
| `id` | `string?` | Optional ID (auto-generated if omitted) |
| `componentKey` | `string` | Unique identifier within the pipeline (e.g., `my_syslog_source`). Must match `^[a-zA-Z_][a-zA-Z0-9_]*$` |
| `componentKey` | `string` | Auto-generated unique identifier within the pipeline (e.g., `syslog_k7xMp2nQ`). Must match `^[a-zA-Z_][a-zA-Z0-9_]*$` |
| `displayName` | `string?` | Optional human-readable name for the component (e.g., "Syslog Source") |
| `componentType` | `string` | Vector component type (e.g., `syslog`, `remap`, `aws_s3`) |
| `kind` | `"SOURCE" \| "TRANSFORM" \| "SINK"` | Component category |
| `config` | `object` | Component configuration fields |
Expand Down
6 changes: 4 additions & 2 deletions docs/public/reference/pipeline-yaml.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,12 +62,14 @@ sinks:

### Component keys

Each node in the visual editor has a **component key** -- a unique identifier within the pipeline. Component keys must:
Each node in the visual editor has a **component key** -- a unique, auto-generated identifier within the pipeline (e.g., `http_server_k7xMp2nQ`). Component keys are generated when a node is added and never change, even if you rename the component in the editor.

Component keys must:
- Start with a letter or underscore
- Contain only letters, numbers, and underscores
- Be between 1 and 128 characters

These keys become the YAML block names under `sources`, `transforms`, or `sinks`.
These keys become the YAML block names under `sources`, `transforms`, or `sinks`. The human-readable **Name** field in the editor is separate from the component key and does not affect the generated YAML.

### Connections via `inputs`

Expand Down
3 changes: 2 additions & 1 deletion docs/public/user-guide/pipeline-editor.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,8 @@ The panel has two tabs:
The **Config** tab shows:

- **Component name and kind** -- The display name, a badge indicating source/transform/sink, and a delete button.
- **Component Key** -- A unique identifier for this component within the pipeline (e.g. `traefik_logs`). Must contain only letters, numbers, and underscores.
- **Name** -- A human-readable label for the component (e.g. "Traefik Logs"). Changing the name requires saving, but does not require a redeploy.
- **Component ID** -- An auto-generated unique identifier used in the backend configuration (read-only). This key is set when the node is created and never changes.
- **Enabled toggle** -- Disable a component to exclude it from the generated configuration without removing it from the canvas.
- **Type** -- The Vector component type (read-only).
- **Configuration form** -- Auto-generated form fields based on the component's configuration schema. Required fields are marked, and each field has contextual help.
Expand Down
2 changes: 1 addition & 1 deletion docs/public/user-guide/templates.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ VectorFlow creates a new pipeline in the current environment with the template's

Templates save the complete pipeline graph:

- **Nodes** -- Every source, transform, and sink component, including its component type, component key, and configuration.
- **Nodes** -- Every source, transform, and sink component, including its component type, component key, display name, and configuration.
- **Edges** -- All connections between components, preserving the data flow topology.
- **Layout** -- The X/Y positions of nodes on the canvas, so the visual layout is preserved.

Expand Down
103 changes: 103 additions & 0 deletions docs/superpowers/specs/2026-03-10-friendly-component-ids-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# Friendly Component IDs

Decouple pipeline component identity from user-facing display names. Component keys become immutable UUID-based identifiers used only in the backend (YAML, metrics, agent protocol). A new `displayName` field provides the human-readable label users see and edit in the GUI.

## Problem

Today, `componentKey` serves as both the backend identifier (YAML keys, metrics matching, event sampling) and the user-facing label. Renaming a component changes the key, which requires a redeploy and orphans in-flight metrics. Users working in the GUI shouldn't need to trigger infrastructure changes just to rename a component.

## Design

### Data Model

Add a nullable `displayName` column to `PipelineNode`:

```prisma
model PipelineNode {
// existing fields...
componentKey String // Immutable. Format: {type}_{nanoid(8)}
displayName String? // User-facing cosmetic name
// ...
}
```

- `componentKey` -- generated once at node creation, never modified. Format: `{componentType}_{nanoid(8)}` using a custom alphanumeric alphabet (`0-9A-Za-z`, no hyphens) to remain compatible with existing componentKey regex `/^[a-zA-Z_][a-zA-Z0-9_]*$/` and YAML key requirements. Examples: `http_server_k7xMp2nQ`, `remap_vT3bL9wR`.
- `displayName` -- nullable, editable anytime. Defaults to `componentDef.displayName` (e.g., "HTTP Server") on node creation.
- Display logic throughout the app: `displayName ?? componentKey`.

### Component Key Generation & Immutability

**Node creation** (flow-store `addNode`):
- `componentKey = {componentDef.type}_{nanoid(8)}`
- `displayName = componentDef.displayName`

**Copy/paste & duplicate**: new UUID key generated, display name copied as-is (duplicates are fine -- display names are cosmetic).

**Immutability enforcement**:
- Remove `updateNodeKey` action from flow store.
- No user input path for `componentKey` -- generated once, stored, done.

**Validation**:
- `displayName`: permissive -- allow any printable characters (letters, numbers, spaces, hyphens, underscores, slashes, etc.) since some `componentDef.displayName` values may contain characters like `/`. Max 64 chars.
- `componentKey`: no user validation needed (system-generated).

### GUI Changes

**Node components** (source-node, transform-node, sink-node):
- Render `displayName ?? componentKey` as the node label.

**Detail panel**:
- Current "Component Key" input becomes "Name", bound to `displayName`.
- Remove "Letters, numbers, and underscores only" hint.
- Add a small read-only "Component ID" field showing the UUID key (subtle, for debugging).
- Editing the name sets `isDirty: true` (requires save) but does not require redeploy.

**Edge connections**: unaffected (use React Flow node `id`, not `componentKey`).

**Metrics overlay**: unaffected (matched by `componentKey` which is unchanged).

### Backend & Persistence

**Save pipeline**: persists both `componentKey` and `displayName`. Key written once on creation, never updated. Display name updates are a normal save.

**Deploy pipeline**: `generateVectorYaml()` uses `componentKey` for YAML keys. No changes.

**Metrics router**: matching logic (`componentId === pn.componentKey`) unchanged. Include `displayName` in frontend responses for GUI labeling.

**Event sampling**: uses `componentKey` throughout. No changes.

**Pipeline versions**: `nodesSnapshot` naturally includes `displayName` as part of node data.

**GitOps**: YAML uses UUID-based keys. Import leaves `displayName` null (falls back to key). No changes to git-sync or webhook handler.

### Migration & Backwards Compatibility

**Database migration**: `ALTER TABLE PipelineNode ADD COLUMN displayName TEXT` (nullable, no default).

**Existing pipelines**: no data migration. Old components keep `{type}_{timestamp}` keys. `displayName` is `NULL` -- GUI shows `componentKey` as fallback. Users can optionally set display names on old components (just requires save).

**New components**: get `{type}_{nanoid(8)}` keys and `displayName = componentDef.displayName`.

**Mixed pipelines**: old timestamp keys and new UUID keys coexist. The system treats keys as opaque strings.

**Not changing**: pipeline names, agent/heartbeat protocol, metrics store structure, YAML generation logic (beyond key format for new nodes), GitOps sync logic. Note: `displayName` is intentionally not written into YAML output.

### Implementation Notes

Files requiring `displayName` to be threaded through:

- **Prisma schema**: add `displayName String?` to `PipelineNode` model
- **FlowNodeData type** (`flow-store.ts`): add `displayName` to the interface
- **ClipboardData type** (`flow-store.ts`): include `displayName` for copy/paste
- **computeFlowFingerprint** (`flow-store.ts`): include `displayName` so renames trigger "unsaved changes"
- **nodeSchema** (`pipeline.ts`): add `displayName: z.string().nullable().optional()`
- **templateNodeSchema** (`template.ts`): same as above
- **dbNodesToFlowNodes** (`pipelines/[id]/page.tsx`): map `displayName` from DB row to node data
- **pasteFromSession** (`flow-store.ts`): generate fresh nanoid key (not collision-appended timestamp), copy displayName as-is
- **copyPipelineGraph** (`copy-pipeline-graph.ts`): preserve componentKey and displayName on clone/promote
- **Metrics router responses**: include `displayName` alongside `componentKey` in `getComponentMetrics` and `getNodePipelineRates` responses

### Known Trade-offs

- **GitOps full-graph replacement**: bidirectional GitOps does delete-and-recreate on import, so display names set in the GUI will be lost if the pipeline is overwritten from Git. Acceptable given the current architecture.
- **No uniqueness enforcement on displayName**: two nodes can share the same display name. Users distinguish them by expanding the Component ID debug field if needed.
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
"dotenv": "^17.3.1",
"js-yaml": "^4.1.1",
"lucide-react": "^0.575.0",
"nanoid": "^5.1.6",
"next": "16.1.6",
"next-auth": "5.0.0-beta.30",
"next-themes": "^0.4.6",
Expand Down
10 changes: 10 additions & 0 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
-- AlterTable
ALTER TABLE "PipelineNode" ADD COLUMN "displayName" TEXT;
1 change: 1 addition & 0 deletions prisma/schema.prisma
Original file line number Diff line number Diff line change
Expand Up @@ -363,6 +363,7 @@ model PipelineNode {
pipelineId String
pipeline Pipeline @relation(fields: [pipelineId], references: [id], onDelete: Cascade)
Comment on lines 363 to 364
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing Prisma migration file

displayName String? was added to the PipelineNode model in the schema, but no corresponding migration was generated and committed under prisma/migrations/.

Per the project conventions, npx prisma migrate dev --name add_display_name must be run and the resulting migration file committed. Without it:

  • prisma migrate deploy (which the Docker entrypoint runs automatically on startup) finds no pending migration, so the displayName column is never created in the database.
  • Every query or upsert that touches displayName (save pipeline, rollback, copy-graph, metrics read, etc.) will fail at runtime with a database error about an unknown column.

A migration file named something like 20260310000000_add_pipeline_node_display_name containing:

ALTER TABLE "PipelineNode" ADD COLUMN "displayName" TEXT;

must be added before this is merged.

Prompt To Fix With AI
This is a comment left during a code review.
Path: prisma/schema.prisma
Line: 363-364

Comment:
**Missing Prisma migration file**

`displayName String?` was added to the `PipelineNode` model in the schema, but no corresponding migration was generated and committed under `prisma/migrations/`. 

Per the project conventions, `npx prisma migrate dev --name add_display_name` must be run and the resulting migration file committed. Without it:
- `prisma migrate deploy` (which the Docker entrypoint runs automatically on startup) finds no pending migration, so the `displayName` column is never created in the database.
- Every query or upsert that touches `displayName` (save pipeline, rollback, copy-graph, metrics read, etc.) will fail at runtime with a database error about an unknown column.

A migration file named something like `20260310000000_add_pipeline_node_display_name` containing:
```sql
ALTER TABLE "PipelineNode" ADD COLUMN "displayName" TEXT;
```
must be added before this is merged.

How can I resolve this? If you propose a fix, please make it concise.

componentKey String
displayName String?
componentType String
kind ComponentKind
config Json
Expand Down
3 changes: 3 additions & 0 deletions src/app/(dashboard)/pipelines/[id]/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ function dbNodesToFlowNodes(
dbNodes: Array<{
id: string;
componentKey: string;
displayName: string | null;
componentType: string;
kind: string;
config: unknown;
Expand All @@ -82,6 +83,7 @@ function dbNodesToFlowNodes(
configSchema: {},
},
componentKey: n.componentKey,
displayName: n.displayName ?? undefined,
config: (n.config as Record<string, unknown>) ?? {},
disabled: n.disabled ?? false,
},
Expand Down Expand Up @@ -299,6 +301,7 @@ function PipelineBuilderInner({ pipelineId }: { pipelineId: string }) {
nodes: state.nodes.map((n) => ({
id: n.id,
componentKey: (n.data as Record<string, unknown>).componentKey as string,
displayName: (n.data as Record<string, unknown>).displayName as string | undefined,
componentType: ((n.data as Record<string, unknown>).componentDef as { type: string }).type,
kind: (n.type?.toUpperCase() ?? "SOURCE") as "SOURCE" | "TRANSFORM" | "SINK",
config: ((n.data as Record<string, unknown>).config as Record<string, unknown>) ?? {},
Expand Down
53 changes: 23 additions & 30 deletions src/components/flow/detail-panel.tsx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"use client";

import { useCallback, useEffect, useMemo, useState } from "react";
import { useCallback, useMemo } from "react";
import { Copy, Trash2, Lock, Info, MousePointerClick, Book } from "lucide-react";
import { useFlowStore } from "@/stores/flow-store";
import { SchemaForm } from "@/components/config-forms/schema-form";
Expand Down Expand Up @@ -118,20 +118,16 @@ export function DetailPanel({ pipelineId, isDeployed }: DetailPanelProps) {
const nodes = useFlowStore((s) => s.nodes);
const edges = useFlowStore((s) => s.edges);
const updateNodeConfig = useFlowStore((s) => s.updateNodeConfig);
const updateNodeKey = useFlowStore((s) => s.updateNodeKey);
const updateDisplayName = useFlowStore((s) => s.updateDisplayName);
const toggleNodeDisabled = useFlowStore((s) => s.toggleNodeDisabled);
const removeNode = useFlowStore((s) => s.removeNode);

const selectedNode = selectedNodeId
? nodes.find((n) => n.id === selectedNodeId)
: null;

const storeKey = (selectedNode?.data as { componentKey?: string })?.componentKey ?? "";
const [displayKey, setDisplayKey] = useState(storeKey);

useEffect(() => {
setDisplayKey(storeKey);
}, [storeKey]);
const componentKey = (selectedNode?.data as { componentKey?: string })?.componentKey ?? "";
const currentDisplayName = (selectedNode?.data as { displayName?: string })?.displayName ?? "";

const upstream = useMemo(
() =>
Expand All @@ -150,22 +146,14 @@ export function DetailPanel({ pipelineId, isDeployed }: DetailPanelProps) {
[selectedNodeId, updateNodeConfig],
);

const handleKeyChange = useCallback(
const handleNameChange = useCallback(
(raw: string) => {
if (selectedNodeId) {
const sanitized = raw
.replace(/\s+/g, "_")
.replace(/[^a-zA-Z0-9_]/g, "")
.replace(/^(\d+)/, "_$1");
if (sanitized) {
setDisplayKey(raw);
updateNodeKey(selectedNodeId, sanitized);
} else {
setDisplayKey(storeKey);
}
const trimmed = raw.slice(0, 64);
updateDisplayName(selectedNodeId, trimmed);
}
},
[selectedNodeId, updateNodeKey, storeKey],
[selectedNodeId, updateDisplayName],
);

const handleDelete = useCallback(() => {
Expand Down Expand Up @@ -227,7 +215,8 @@ export function DetailPanel({ pipelineId, isDeployed }: DetailPanelProps) {

const { componentDef, config, disabled, isSystemLocked } = selectedNode.data as {
componentDef: VectorComponentDef;
componentKey: string; // used via displayKey/storeKey above
componentKey: string;
displayName?: string;
config: Record<string, unknown>;
disabled?: boolean;
isSystemLocked?: boolean;
Expand Down Expand Up @@ -293,18 +282,22 @@ export function DetailPanel({ pipelineId, isDeployed }: DetailPanelProps) {
</div>
</CardHeader>
<CardContent className="space-y-4">
{/* Component Key */}
{/* Name */}
<div className="space-y-2">
<Label htmlFor="component-key">Component Key</Label>
<Label htmlFor="display-name">Name</Label>
<Input
id="component-key"
value={displayKey}
onChange={(e) => handleKeyChange(e.target.value)}
id="display-name"
value={currentDisplayName}
onChange={(e) => handleNameChange(e.target.value)}
disabled={isSystemLocked}
placeholder="Component name"
/>
<p className="text-xs text-muted-foreground">
Letters, numbers, and underscores only (e.g. traefik_logs)
</p>
</div>

{/* Component ID (read-only) */}
<div className="space-y-1">
<Label className="text-xs text-muted-foreground">Component ID</Label>
<p className="text-xs font-mono text-muted-foreground select-all">{componentKey}</p>
</div>

{/* Enabled toggle */}
Expand Down Expand Up @@ -433,7 +426,7 @@ export function DetailPanel({ pipelineId, isDeployed }: DetailPanelProps) {
<TabsContent value="live-tail" className="min-h-0 flex-1 overflow-y-auto">
<LiveTailPanel
pipelineId={pipelineId}
componentKey={storeKey}
componentKey={componentKey}
isDeployed={isDeployed}
/>
</TabsContent>
Expand Down
1 change: 1 addition & 0 deletions src/components/flow/save-template-dialog.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ export function SaveTemplateDialog({ open, onOpenChange }: SaveTemplateDialogPro
id: n.id,
componentType: ((n.data as Record<string, unknown>).componentDef as VectorComponentDef).type,
componentKey: (n.data as Record<string, unknown>).componentKey as string,
displayName: (n.data as Record<string, unknown>).displayName as string | undefined,
kind: (n.type ?? "source") as "source" | "transform" | "sink",
config: ((n.data as Record<string, unknown>).config as Record<string, unknown>) ?? {},
positionX: n.position.x,
Expand Down
5 changes: 3 additions & 2 deletions src/components/flow/sink-node.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import { nodeStatusVariant } from "@/lib/status";
type SinkNodeData = {
componentDef: VectorComponentDef;
componentKey: string;
displayName?: string;
config: Record<string, unknown>;
metrics?: NodeMetricsData;
disabled?: boolean;
Expand All @@ -23,7 +24,7 @@ type SinkNodeData = {
type SinkNodeType = Node<SinkNodeData, "sink">;

function SinkNodeComponent({ data, selected }: NodeProps<SinkNodeType>) {
const { componentDef, componentKey, metrics, disabled } = data;
const { componentDef, componentKey, displayName, metrics, disabled } = data;
const Icon = useMemo(() => getIcon(componentDef.icon), [componentDef.icon]);

return (
Expand Down Expand Up @@ -52,7 +53,7 @@ function SinkNodeComponent({ data, selected }: NodeProps<SinkNodeType>) {

{/* Body */}
<div className="space-y-2 px-3 py-2.5">
<p className="truncate text-xs font-medium text-foreground">{componentKey}</p>
<p className="truncate text-xs font-medium text-foreground">{displayName || componentKey}</p>

{metrics && (
<p className="truncate text-xs font-mono text-purple-400">
Expand Down
Loading
Loading