diff --git a/README.md b/README.md index 325c602..229de59 100644 --- a/README.md +++ b/README.md @@ -94,7 +94,7 @@ exposes experimental skills. Stable skills shipped from [`skills/`](./skills/): -- **databricks-core** — CLI, authentication, profile selection, data exploration. Parent skill for all product skills. +- **databricks-core** — CLI, authentication, profile selection, data exploration. Companion skill for Databricks product skills. - **databricks-apps** — Build full-stack TypeScript apps on Databricks using AppKit. - **databricks-app-design** — Design the UX of data apps: dashboards, KPI pages, reports, charts, and Genie/chat surfaces, mapped to AppKit components. - **databricks-dabs** — Declarative Automation Bundles (formerly Asset Bundles) for deploying and managing Databricks resources. @@ -190,20 +190,22 @@ skill-name/ ### Adding New Skills -For a narrower variation of an existing skill, create a subskill that declares -its parent via frontmatter. This is how the stable skills are organized today -— each product skill sets `parent: databricks-core`. +For a narrower variation of an existing skill, create a companion skill that +explicitly tells the agent to load/read the related base skill. Do not depend on +`parent:` frontmatter for inheritance: not every agent client loads parent +skills automatically, and Codex only uses `name` and `description` for skill +triggering. Put requirements such as Databricks CLI versions in the Markdown +body, not custom frontmatter like `compatibility:`. ```markdown --- name: "databricks-apps-chatbots" description: "Databricks apps with chatbot features" -parent: databricks-apps --- # Chatbot Apps -**FIRST**: Use the parent `databricks-apps` skill for app development basics. +**FIRST**: Also load/read the `databricks-apps` skill for app development basics. Then apply these patterns: - Pattern 1 diff --git a/skills/databricks-app-design/SKILL.md b/skills/databricks-app-design/SKILL.md index e6f07a4..d2652b5 100644 --- a/skills/databricks-app-design/SKILL.md +++ b/skills/databricks-app-design/SKILL.md @@ -1,6 +1,5 @@ --- name: databricks-app-design -parent: databricks-apps description: Design the UX of Databricks data apps — dashboards, KPI pages, reports, charts, tables, and Genie/chat data assistants — mapped to concrete AppKit components. Use when BUILDING or reviewing any UI that displays data or answers data questions: choosing genre, layout, charts, KPIs, semantic color, required states (loading/empty/error), IBCS notation, and AI-result trust (showing generated SQL/sources for Genie/chat). NOT for authoring managed AI/BI (Lakeview) dashboards (→ databricks-aibi-dashboards), non-data frontend (forms, settings, auth, marketing), or scaffolding/build/deploy (→ databricks-apps). Complements databricks-apps; use it alongside whenever the app has a dashboard, chart, table, KPI, report, or Genie/chat/AI surface. metadata: version: 0.1.0 @@ -20,7 +19,7 @@ Design advice that doesn't name a real component is incomplete. Always end at a ## When to use / when NOT - USE for: dashboard/overview/KPI pages, reports, metric/ontology pages, variance analysis, and Genie/NL data surfaces — design *or* critique. - Do NOT use for: authoring managed **AI/BI (Lakeview) dashboards** (→ `databricks-aibi-dashboards`), generic frontend (forms, auth, settings, marketing), or scaffolding/build/deploy (→ `databricks-apps`). If a request is "add a form", "deploy this", or "build a Lakeview / AI-BI dashboard", this skill should not fire. -- Relationship: `databricks-apps` builds/runs the app; this skill decides what the data screens should look like and which primitives realize them. +- Relationship: load `databricks-apps` alongside this skill when building or changing a Databricks App. `databricks-apps` builds/runs the app; this skill decides what the data screens should look like and which primitives realize them. ## Workflow 1. **Frame** — audience, the decision/question, refresh cadence, device, primary task. One sentence. diff --git a/skills/databricks-apps/SKILL.md b/skills/databricks-apps/SKILL.md index 6f1c694..79bc656 100644 --- a/skills/databricks-apps/SKILL.md +++ b/skills/databricks-apps/SKILL.md @@ -1,15 +1,15 @@ --- name: databricks-apps description: "Build apps on Databricks Apps platform. Use when asked to create dashboards, data apps, analytics tools, or visualizations. Evaluates data access patterns (analytics vs Lakebase synced tables) before scaffolding. Invoke BEFORE starting implementation." -compatibility: Requires databricks CLI (>= v0.294.0) metadata: version: "0.1.2" -parent: databricks-core --- # Databricks Apps Development -**FIRST**: Use the parent `databricks-core` skill for CLI basics, authentication, and profile selection. +Requires Databricks CLI v0.294.0 or newer. + +**FIRST**: Also load/read the `databricks-core` skill for CLI basics, authentication, and profile selection. Do not rely on skill frontmatter to load it automatically. **For data UI design (required for any data-displaying app)**: if the app shows ANY data — a dashboard, KPI/overview page, report, chart, table, query results, OR a **conversational / chat / Genie natural-language assistant** — you MUST use the `databricks-app-design` skill (alongside this one) to decide layout, charts, KPIs, semantic color, required states, and AI-result trust, and map them to AppKit components. This includes chat/Genie apps, not just dashboards — if in doubt, use it. @@ -41,7 +41,7 @@ Build apps that deploy to Databricks Apps platform. - **Smoke test selectors**: use only Playwright locator APIs — `getByRole`, `getByText`, `getByPlaceholder`, `getByLabel`. `getByLabelText` does not exist in Playwright (it is a React Testing Library method) and throws `TypeError` at runtime. See [testing guide](references/testing.md) or `npx playwright codegen`. - **Smoke test data**: keep result sets under the 1 MB analytics-event payload cap. Queries returning thousands of rows cause `INVALID_REQUEST: Event exceeds max size of 1048576 bytes` and `net::ERR_ABORTED`, leaving every asserted UI element absent. Use `LIMIT` or an aggregated query (e.g. `COUNT(*) GROUP BY status`) — never raw row dumps. - **AppKit version**: never override the `@databricks/appkit` or `@databricks/appkit-ui` version in `package.json` — `databricks apps init` sets the correct version. Do not run `npm install @databricks/appkit@` unless explicitly asked by the user. If you need a different version, re-scaffold with `databricks apps init --version `. -- **Authentication**: covered by parent `databricks-core` skill. +- **Authentication**: covered by the companion `databricks-core` skill. - **AppKit API surface**: before writing code that calls AppKit APIs (`createApp`, plugin shapes, `useAnalyticsQuery`, etc.), run `npx @databricks/appkit docs
` and use the actual signature. Training data has stale shapes; a single invented signature fails `tsc --noEmit` during validate. The docs ship with the installed AppKit and are the authoritative source. - **TypeScript casts**: never use `as unknown as ` double-assertions — `appkit lint` enforces `no-double-type-assertion` and one violation fails the entire validate step. Instead: narrow with Zod (`z.infer`), use a runtime type guard, or write a typed mapper function. If a query result needs reshaping, type the row schema via queryKey types rather than casting. @@ -61,7 +61,7 @@ Build apps that deploy to Databricks Apps platform. ## Data Discovery -Before writing any SQL, use the parent `databricks-core` skill for data exploration — search `information_schema` by keyword, then batch `discover-schema` for the tables you need. Do NOT skip this step. +Before writing any SQL, use the companion `databricks-core` skill for data exploration — search `information_schema` by keyword, then batch `discover-schema` for the tables you need. Do NOT skip this step. **State Storage Guidance (evaluate BEFORE the Decision Gate):** @@ -170,7 +170,7 @@ npx @databricks/appkit docs ./docs/plugins/analytics.md # example: specific doc Optionally use `--version ` to target a specific AppKit version. - **Required**: `--name`, `--profile`. Name: ≤26 chars, lowercase letters/numbers/hyphens only. Use `--features` only for **optional** plugins the user wants (plugins with `requiredByTemplate: false` or absent); mandatory plugins must not be listed in `--features`. - **Resources**: Pass `--set` for every required resource (each field in `resources.required`) for (1) all plugins with `requiredByTemplate: true`, and (2) any optional plugins you added to `--features`. Add `--set` for `resources.optional` only when the user requests them. - - **Discovery**: Use the parent `databricks-core` skill to resolve IDs (e.g. warehouse: `databricks warehouses list --profile ` or `databricks experimental aitools tools get-default-warehouse --profile `). + - **Discovery**: Use the companion `databricks-core` skill to resolve IDs (e.g. warehouse: `databricks warehouses list --profile ` or `databricks experimental aitools tools get-default-warehouse --profile `). **DO NOT guess** plugin names, resource keys, or property names — always derive them from `databricks apps manifest` output. Example: if the manifest shows plugin `analytics` with a required resource `resourceKey: "sql-warehouse"` and `fields: { "id": ... }`, include `--set analytics.sql-warehouse.id=`. diff --git a/skills/databricks-core/SKILL.md b/skills/databricks-core/SKILL.md index 60bdf7d..d6da46c 100644 --- a/skills/databricks-core/SKILL.md +++ b/skills/databricks-core/SKILL.md @@ -1,7 +1,6 @@ --- name: "databricks-core" description: "Databricks CLI operations: auth, profiles, data exploration, and bundles. Contains up-to-date guidelines for Databricks-related CLI tasks." -compatibility: Requires databricks CLI (>= v0.292.0) metadata: version: "0.1.0" --- diff --git a/skills/databricks-jobs/SKILL.md b/skills/databricks-jobs/SKILL.md index 59ef60a..7f2ebfb 100644 --- a/skills/databricks-jobs/SKILL.md +++ b/skills/databricks-jobs/SKILL.md @@ -1,15 +1,15 @@ --- name: databricks-jobs description: Develop and deploy Lakeflow Jobs on Databricks via DABs, Python SDK, or the CLI. Use when creating data engineering jobs with notebooks, Python wheels, SQL, dbt, or pipelines. Invoke BEFORE starting implementation. -compatibility: Requires databricks CLI (>= v1.0.0) metadata: version: "0.2.0" -parent: databricks-core --- # Lakeflow Jobs Development -**FIRST**: Use the parent `databricks-core` skill for CLI basics, authentication, profile selection, and data exploration commands. +Requires Databricks CLI v1.0.0 or newer. + +**FIRST**: Also load/read the `databricks-core` skill for CLI basics, authentication, profile selection, and data exploration commands. Do not rely on skill frontmatter to load it automatically. Lakeflow Jobs orchestrate data workflows with multi-task DAGs, flexible triggers, and comprehensive monitoring. Jobs support diverse task types and can be managed via Asset Bundles (DABs), Python SDK, or CLI. diff --git a/skills/databricks-lakebase/SKILL.md b/skills/databricks-lakebase/SKILL.md index db539e7..7bb67e3 100644 --- a/skills/databricks-lakebase/SKILL.md +++ b/skills/databricks-lakebase/SKILL.md @@ -1,15 +1,15 @@ --- name: databricks-lakebase description: "Databricks Lakebase Postgres: projects, scaling, connectivity, Lakebase synced tables, and Data API. Use when asked about Lakebase databases, OLTP storage, or connecting apps to Postgres on Databricks." -compatibility: Requires databricks CLI (>= v0.294.0) metadata: version: "0.1.0" -parent: databricks-core --- # Lakebase Postgres Autoscaling -**FIRST**: Use the parent `databricks-core` skill for CLI basics, authentication, and profile selection. +Requires Databricks CLI v0.294.0 or newer. + +**FIRST**: Also load/read the `databricks-core` skill for CLI basics, authentication, and profile selection. Do not rely on skill frontmatter to load it automatically. Lakebase is Databricks' serverless Postgres-compatible database, available on both AWS and Azure (GA). It provides fully managed OLTP storage with autoscaling, branching, and scale-to-zero. diff --git a/skills/databricks-model-serving/SKILL.md b/skills/databricks-model-serving/SKILL.md index febe743..540dd9e 100644 --- a/skills/databricks-model-serving/SKILL.md +++ b/skills/databricks-model-serving/SKILL.md @@ -1,15 +1,15 @@ --- name: databricks-model-serving description: "Databricks Model Serving (ops) plus MLflow model development (dev): manage serving endpoints, train and register models to Unity Catalog with @prod aliases, batch-score via spark_udf, build custom PyFunc / ResponsesAgent models, and discover Foundation Model API endpoints." -compatibility: Requires databricks CLI (>= v0.294.0) metadata: version: "0.3.0" -parent: databricks-core --- # Model Serving Endpoints -**FIRST**: Use the parent `databricks-core` skill for CLI basics, authentication, and profile selection. +Requires Databricks CLI v0.294.0 or newer. + +**FIRST**: Also load/read the `databricks-core` skill for CLI basics, authentication, and profile selection. Do not rely on skill frontmatter to load it automatically. Model Serving provides managed endpoints for serving LLMs, custom ML models, and external models as scalable REST APIs. Endpoints are identified by **name** (unique per workspace). diff --git a/skills/databricks-pipelines/SKILL.md b/skills/databricks-pipelines/SKILL.md index 35d7c15..7c6eaf6 100644 --- a/skills/databricks-pipelines/SKILL.md +++ b/skills/databricks-pipelines/SKILL.md @@ -1,15 +1,15 @@ --- name: databricks-pipelines description: Develop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks. Use when building batch or streaming data pipelines with Python or SQL. Invoke BEFORE starting implementation. -compatibility: Requires databricks CLI (>= v1.0.0) metadata: version: "0.3.0" -parent: databricks-core --- # Lakeflow Spark Declarative Pipelines Development -**FIRST**: Use the parent `databricks-core` skill for CLI basics, authentication, profile selection, and data discovery commands. +Requires Databricks CLI v1.0.0 or newer. + +**FIRST**: Also load/read the `databricks-core` skill for CLI basics, authentication, profile selection, and data discovery commands. Do not rely on skill frontmatter to load it automatically. ## Decision Tree diff --git a/skills/databricks-serverless-migration/SKILL.md b/skills/databricks-serverless-migration/SKILL.md index 5d000d3..89e4e3f 100644 --- a/skills/databricks-serverless-migration/SKILL.md +++ b/skills/databricks-serverless-migration/SKILL.md @@ -1,15 +1,15 @@ --- name: databricks-serverless-migration description: "Migrate Databricks workloads from classic compute to serverless compute. Use when migrating notebooks, jobs, pipelines, or Scala JARs (`spark_jar_task`) from classic clusters to serverless, checking if existing code is serverless-compatible, or writing new serverless-compatible code. Provides concrete fixes for the serverless Spark Connect architecture and guides the full migration. Not for classic DBR version upgrades or cluster configuration changes within classic compute." -compatibility: Requires databricks CLI (>= v0.292.0) metadata: version: "0.1.0" -parent: databricks-core --- # Serverless Compute Migration -**FIRST**: Use the parent `databricks-core` skill for CLI basics, authentication, and profile selection. +Requires Databricks CLI v0.292.0 or newer. + +**FIRST**: Also load/read the `databricks-core` skill for CLI basics, authentication, and profile selection. Do not rely on skill frontmatter to load it automatically. Analyze existing Databricks code for serverless compute compatibility and guide migration from classic clusters. The skill follows a 4-step migration lifecycle: **Ingest** the workload → **Analyze** for compatibility → **Test** via A/B comparison → **Validate** and iterate. diff --git a/skills/databricks-vector-search/SKILL.md b/skills/databricks-vector-search/SKILL.md index ad8eebc..92e2064 100644 --- a/skills/databricks-vector-search/SKILL.md +++ b/skills/databricks-vector-search/SKILL.md @@ -3,12 +3,11 @@ name: databricks-vector-search description: "Databricks Vector Search endpoints and indexes for RAG and semantic search; covers index types, search modes, end-to-end RAG patterns" metadata: version: "0.1.0" -parent: databricks-core --- # Databricks Vector Search -**FIRST**: Use the parent `databricks-core` skill for CLI basics, authentication, and profile selection. +**FIRST**: Also load/read the `databricks-core` skill for CLI basics, authentication, and profile selection. Do not rely on skill frontmatter to load it automatically. Patterns for creating, managing, and querying vector search indexes for RAG and semantic search applications.