Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/cli-readiness.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Confidence key: ✅ full CRUD verified · 🟡 partial / readonly verified ·
| --- | --- | --- |
| Auth (API key) | ✅ | None for personal keys. Service-account keys: untested end-to-end — we created an SA and then deleted it before creating a key scoped to it. |
| Chats | ✅ | `CreateChat.paradigm` — only `universal` observed; unknown if other paradigms exist (SQL-only? notebook?). Tool-selection flags (`sqlEnabled`, `pythonEnabled`, `webSearchEnabled`) defaults undocumented — must inspect `CreateChat` sample. `UpdateChat` only verified to touch `summary`; other mutable fields unknown. |
| Connectors | 🟡 | Three dialects fully probed: Postgres (password), Snowflake (password / key-pair / oauth-sso / oauth-individual — all four shipped), Databricks (access-token / client-credentials / oauth-sso / oauth-individual — probed, not yet shipped). See connector grid below. Remaining dialects (BigQuery, Redshift, MySQL, SQLServer, Supabase, MotherDuck, Tableau, PowerBI) still unverified. OAuth leaves POST the full create payload in a pending-OAuth state; user completes the browser handshake at `app.textql.com` to activate. |
| Connectors | 🟡 | Three dialects fully shipped: Postgres (password), Snowflake (password / key-pair / oauth-sso / oauth-individual — all four shipped), Databricks (access-token / client-credentials / oauth-sso / oauth-individual — all four shipped). See connector grid below. Remaining dialects (BigQuery, Redshift, MySQL, SQLServer, Supabase, MotherDuck, Tableau, PowerBI) still unverified. OAuth leaves POST the full create payload in a pending-OAuth state; user completes the browser handshake at `app.textql.com` to activate. |
| Service accounts | 🟡 | Created + deleted. NOT verified: creating an API key *on* a service account (the kebab menu has "Create API Key" — would need to confirm whether that uses `CreateApiKey` with a `memberId` override or a different RPC). |
| Dashboards | 🟡 | List/get/spawn/health covered. Create/update/delete NOT probed. |
| Playbooks | 🟡 | Get/list/reports/lineage covered. Create/update/delete/run-now NOT probed. |
Expand Down Expand Up @@ -65,7 +65,7 @@ Grading: ✅ Live-tested (probe + at least one real connector created) · 🟢 I
| --- | --- | --- | --- | --- | --- | --- |
| Postgres | 🟢 | — | — | — | — | — |
| Snowflake | 🟢 | 🟢 | — | — | 🟢* | 🟢* |
| Databricks | — | — | 🟦 | 🟦 | 🟦* | 🟦* |
| Databricks | — | — | 🟢 | 🟢 | 🟢* | 🟢* |
| BigQuery | ⚪ | ⚪ | ⚪ | ⚪ | ⚪ | ⚪ |
| Redshift | ⚪ | — | — | — | — | — |
| MySQL | ⚪ | — | — | — | — | — |
Expand All @@ -75,7 +75,7 @@ Grading: ✅ Live-tested (probe + at least one real connector created) · 🟢 I
| Tableau | ⚪ | — | — | — | — | — |
| PowerBI | ⚪ | — | — | — | — | — |

*OAuth leaves POST the full `CreateConnector` payload (including `clientId` / `clientSecret`) through the normal wire shape; the server accepts the row in a pending-OAuth state. The user then completes the browser handshake at `app.textql.com/auth/<dialect>/callback` to activate the connector — the CLI prints a note to that effect on success. Databricks OAuth leaves (not yet shipped) will follow the same pattern once probed.
*OAuth leaves POST the full `CreateConnector` payload (including `clientId` / `clientSecret`) through the normal wire shape; the server accepts the row in a pending-OAuth state. The user then completes the browser handshake at `app.textql.com/auth/<dialect>/callback` to activate the connector — the CLI prints a note to that effect on success.

Cells that show `—` are auth modes the dialect doesn't expose in the webapp's UI.

Expand Down
2 changes: 2 additions & 0 deletions e2e/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,9 @@ Live-smoke tests that drive real `app.textql.com` RPCs through the same verb pac
| `auth_test.go` | Keys + service-accounts CLI-driven create/rotate/revoke/delete (helper-backed legacy tests still live here for coverage); `--json` shape checks + error-path smokes for usage guards. |
| `chat_test.go` | Chat CRUD + streaming `send`, `--json` envelopes on list/show, CLI-path `chat new`, `history`/`bookmark`/`unbookmark`, `show <missing>` error-path. |
| `connector_test.go` | Connector CRUD + `--json` envelopes, CLI postgres create matrix (password-stdin × ssl on/off), `update --password-stdin`, `tables`/`examples`/`test` leaves, `get <missing>` error-path. |
| `connector_create_leaves_test.go` | Dialect-neutral `connectorCreateLeaf` helper + shared `extractConnectorID`; every Snowflake/Databricks create-leaf smoke runs its create/dry-run/id/cleanup/connectorType/get round-trip through this helper so the pattern can't drift. |
| `connector_snowflake_test.go` | Snowflake create leaves (password/keypair/oauth-sso/oauth-individual), per-mode env-gated. |
| `connector_databricks_test.go` | Databricks create leaves (access-token/client-credentials/oauth-sso/oauth-individual), per-mode env-gated on `ANA_E2E_DBX_*`. |
| `dashboard_test.go` | Dashboard list/get/folders read leaves (default + `--json`); `health`/`spawn` env-gated on `ANA_E2E_DASHBOARD_ID`. |
| `playbook_test.go` | Playbook list/get/reports/lineage read leaves (default + `--json`); id discovered via `list --json`. |
| `ontology_test.go` | Ontology list/get read leaves (default + `--json`); id is integer on the wire. |
Expand Down
37 changes: 37 additions & 0 deletions e2e/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,43 @@ differ in wire `authStrategy`:
| `ANA_E2E_SF_OAUTH_CLIENT_ID` | Snowflake OAuth client id |
| `ANA_E2E_SF_OAUTH_CLIENT_SECRET` | Client secret; piped via `--oauth-client-secret-stdin` |

### Databricks connector env

Databricks tests (`e2e/connector_databricks_test.go`) skip per-test when
their required vars are absent — same treatment as Snowflake. Four vars are
shared across every mode; set all four or every Databricks test skips:

| Variable | Meaning |
|-------------------------|----------------------------------------------------------------|
| `ANA_E2E_DBX_HOST` | Workspace hostname without scheme (e.g. `dbc-xxxx.cloud.databricks.com`) |
| `ANA_E2E_DBX_HTTP_PATH` | SQL warehouse path (`/sql/1.0/warehouses/<id>`) |
| `ANA_E2E_DBX_CATALOG` | Unity Catalog name |
| `ANA_E2E_DBX_SCHEMA` | Default schema |
| `ANA_E2E_DBX_PORT` | Optional port override (defaults to 443 when unset) |

Access Token mode (`TestConnectorCreateDatabricksAccessToken`):

| Variable | Meaning |
|-----------------------|----------------------------------------------------------------|
| `ANA_E2E_DBX_TOKEN` | Personal Access Token; piped via `--token-stdin` |

Client Credentials mode (`TestConnectorCreateDatabricksClientCredentials`):

| Variable | Meaning |
|-----------------------------|----------------------------------------------------------------|
| `ANA_E2E_DBX_CLIENT_ID` | Service Principal OAuth applicationId (UUID) |
| `ANA_E2E_DBX_CLIENT_SECRET` | Service Principal OAuth secret; piped via `--client-secret-stdin` |

OAuth SSO + OAuth individual (`TestConnectorCreateDatabricksOAuthSSO`,
`TestConnectorCreateDatabricksOAuthIndividual`) share the same vars — the
Databricks OAuth app credentials, distinct from the Service Principal
credentials used by Client Credentials mode:

| Variable | Meaning |
|-----------------------------------|---------------------------------------------------------|
| `ANA_E2E_DBX_OAUTH_CLIENT_ID` | Databricks OAuth app client id |
| `ANA_E2E_DBX_OAUTH_CLIENT_SECRET` | OAuth app secret; piped via `--client-secret-stdin` |

Invocations:

```sh
Expand Down
86 changes: 86 additions & 0 deletions e2e/connector_create_leaves_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
package e2e

import (
"fmt"
"regexp"
"strconv"
"strings"
"testing"

"github.com/highperformance-tech/ana-cli/e2e/harness"
)

// connectorIDRE extracts `connectorId: <int>` from the first line of non-JSON
// stdout emitted by every `connector create <dialect> <auth-mode>` leaf.
var connectorIDRE = regexp.MustCompile(`(?m)^connectorId:\s+(\d+)\s*$`)

// extractConnectorID pulls the integer id out of `connectorId: <int>` stdout.
// Fails the test if no match — every create leaf's contract is to emit this
// line on success, so a miss means the output shape drifted.
func extractConnectorID(t *testing.T, stdout string) int {
t.Helper()
m := connectorIDRE.FindStringSubmatch(stdout)
if len(m) != 2 {
t.Fatalf("could not find connectorId in stdout:\n%s", stdout)
}
id, err := strconv.Atoi(m[1])
if err != nil {
t.Fatalf("connectorId %q is not an int: %v", m[1], err)
}
return id
}

// connectorCreateLeaf bundles the invariants every connector-create smoke
// shares: run the command, skip post-create assertions in dry-run, extract +
// register the id, assert `connectorType: <DIALECT>`, run any leaf-specific
// stdout checks, then read the row back via `connector get` to confirm the
// server persisted the new connector.
//
// The helper exists so a parity slip (e.g., a new leaf forgetting the `get`
// round-trip) can only happen if a test intentionally bypasses this wrapper.
type connectorCreateLeaf struct {
// Name is the leaf identifier used in fatal error messages — typically
// "databricks access-token" or "snowflake oauth-sso".
Name string
// Args is the full argv passed to `h.RunStdin`, starting with
// "connector", "create", <dialect>, <auth-mode>, ...
Args []string
// Stdin is the stdin payload for secret flags (token, password, etc.).
// Empty when no --*-stdin flag is used.
Stdin string
// ConnectorType is the dialect tag asserted in stdout, e.g. "DATABRICKS"
// or "SNOWFLAKE". Matched against the literal `connectorType: <tag>` line.
ConnectorType string
// Extra runs after the common assertions and before the `connector get`
// round-trip. Use it for leaf-unique stdout fragments (OAuth endpoint
// note, per-member-lazy note, etc.). May be nil.
Extra func(stdout string)
}

// Run executes the leaf smoke. On non-dry-run success, the created connector
// id is registered for cleanup and read back via `connector get`. Returns the
// created id so callers can chain additional assertions if needed; in dry-run
// mode the returned id is 0.
func (l connectorCreateLeaf) Run(t *testing.T, h *harness.H) int {
t.Helper()
stdout, stderr, err := h.RunStdin(l.Stdin, l.Args...)
if err != nil {
t.Fatalf("connector create %s: %v\nstderr: %s", l.Name, err, stderr)
}
if h.DryRun() {
return 0
}
id := extractConnectorID(t, stdout)
h.RegisterConnectorCleanup(id)
typeLine := "connectorType: " + l.ConnectorType
if !strings.Contains(stdout, typeLine) {
t.Errorf("stdout missing %s:\n%s", typeLine, stdout)
}
if l.Extra != nil {
l.Extra(stdout)
}
if _, estderr, gerr := h.Run("connector", "get", fmt.Sprint(id)); gerr != nil {
t.Fatalf("connector get %d: %v\nstderr: %s", id, gerr, estderr)
}
return id
}
161 changes: 161 additions & 0 deletions e2e/connector_databricks_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
package e2e

import (
"os"
"strings"
"testing"

"github.com/highperformance-tech/ana-cli/e2e/harness"
)

// dbxCommonEnv holds the Databricks workspace fields every auth mode shares.
// `port` is optional here because the CLI's --port already defaults to 443;
// only override when the env sets a non-default value.
type dbxCommonEnv struct {
host string
httpPath string
catalog string
schema string
port string
}

// databricksCommonEnvOrSkip reads the mode-agnostic ANA_E2E_DBX_* env vars
// and skips the calling test if any required field (HOST, HTTP_PATH, CATALOG,
// SCHEMA) is empty. Mirrors snowflakeCommonEnvOrSkip — server does up-front
// validation so submitting a made-up spec would drown the suite in noise.
func databricksCommonEnvOrSkip(t *testing.T) dbxCommonEnv {
t.Helper()
env := dbxCommonEnv{
host: os.Getenv("ANA_E2E_DBX_HOST"),
httpPath: os.Getenv("ANA_E2E_DBX_HTTP_PATH"),
catalog: os.Getenv("ANA_E2E_DBX_CATALOG"),
schema: os.Getenv("ANA_E2E_DBX_SCHEMA"),
port: os.Getenv("ANA_E2E_DBX_PORT"),
}
if env.host == "" || env.httpPath == "" || env.catalog == "" || env.schema == "" {
t.Skip("e2e: ANA_E2E_DBX_HOST, ANA_E2E_DBX_HTTP_PATH, ANA_E2E_DBX_CATALOG, and ANA_E2E_DBX_SCHEMA must be set for Databricks tests")
}
return env
}

// databricksCommonArgs returns the --name/--host/--http-path/--catalog/--schema
// (+ optional --port override) flags shared by every Databricks auth-mode leaf.
func databricksCommonArgs(h *harness.H, suffix string, env dbxCommonEnv) []string {
args := []string{
"--name", h.ResourceName(suffix),
"--host", env.host,
"--http-path", env.httpPath,
"--catalog", env.catalog,
"--schema", env.schema,
}
if env.port != "" {
args = append(args, "--port", env.port)
}
return args
}

// databricksLeafArgs builds the full argv for `connector create databricks
// <auth-mode>` using the shared workspace flags. `suffix` seeds the name-based
// cleanup safety-net; `extra` carries the auth-mode-specific flags.
func databricksLeafArgs(h *harness.H, authMode, suffix string, env dbxCommonEnv, extra ...string) []string {
args := append([]string{"connector", "create", "databricks", authMode},
databricksCommonArgs(h, suffix, env)...)
return append(args, extra...)
}

// TestConnectorCreateDatabricksAccessToken smokes
// `connector create databricks access-token --token-stdin`. Requires
// ANA_E2E_DBX_TOKEN in addition to the common workspace env.
func TestConnectorCreateDatabricksAccessToken(t *testing.T) {
common := databricksCommonEnvOrSkip(t)
token := os.Getenv("ANA_E2E_DBX_TOKEN")
if token == "" {
t.Skip("e2e: ANA_E2E_DBX_TOKEN required for Databricks access-token mode")
}

h := harness.Begin(t)
h.RegisterConnectorCleanupByName(h.ResourceName("dbx-access-token"))
connectorCreateLeaf{
Name: "databricks access-token",
Args: databricksLeafArgs(h, "access-token", "dbx-access-token", common, "--token-stdin"),
Stdin: token + "\n",
ConnectorType: "DATABRICKS",
}.Run(t, h)
}

// TestConnectorCreateDatabricksClientCredentials smokes the M2M leaf.
// Requires ANA_E2E_DBX_CLIENT_ID + ANA_E2E_DBX_CLIENT_SECRET (Service
// Principal applicationId + OAuth secret) alongside the workspace env.
func TestConnectorCreateDatabricksClientCredentials(t *testing.T) {
common := databricksCommonEnvOrSkip(t)
clientID := os.Getenv("ANA_E2E_DBX_CLIENT_ID")
clientSecret := os.Getenv("ANA_E2E_DBX_CLIENT_SECRET")
if clientID == "" || clientSecret == "" {
t.Skip("e2e: ANA_E2E_DBX_CLIENT_ID and ANA_E2E_DBX_CLIENT_SECRET required for Databricks client-credentials mode")
}

h := harness.Begin(t)
h.RegisterConnectorCleanupByName(h.ResourceName("dbx-client-credentials"))
connectorCreateLeaf{
Name: "databricks client-credentials",
Args: databricksLeafArgs(h, "client-credentials", "dbx-client-credentials", common, "--client-id", clientID, "--client-secret-stdin"),
Stdin: clientSecret + "\n",
ConnectorType: "DATABRICKS",
}.Run(t, h)
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.

// TestConnectorCreateDatabricksOAuthSSO smokes the oauth-sso leaf. Asserts
// the success note references the configured endpoint (matches the Snowflake
// pattern). Requires ANA_E2E_DBX_OAUTH_CLIENT_ID +
// ANA_E2E_DBX_OAUTH_CLIENT_SECRET (Databricks OAuth app credentials, distinct
// from Service Principal credentials used by client-credentials).
func TestConnectorCreateDatabricksOAuthSSO(t *testing.T) {
common := databricksCommonEnvOrSkip(t)
clientID := os.Getenv("ANA_E2E_DBX_OAUTH_CLIENT_ID")
clientSecret := os.Getenv("ANA_E2E_DBX_OAUTH_CLIENT_SECRET")
if clientID == "" || clientSecret == "" {
t.Skip("e2e: ANA_E2E_DBX_OAUTH_CLIENT_ID and ANA_E2E_DBX_OAUTH_CLIENT_SECRET required for Databricks oauth-sso mode")
}

h := harness.Begin(t)
h.RegisterConnectorCleanupByName(h.ResourceName("dbx-oauth-sso"))
endpoint := h.Endpoint()
connectorCreateLeaf{
Name: "databricks oauth-sso",
Args: databricksLeafArgs(h, "oauth-sso", "dbx-oauth-sso", common, "--client-id", clientID, "--client-secret-stdin"),
Stdin: clientSecret + "\n",
ConnectorType: "DATABRICKS",
Extra: func(stdout string) {
if !strings.Contains(stdout, "complete OAuth at "+endpoint) {
t.Errorf("oauth-sso note should reference harness endpoint %q:\n%s", endpoint, stdout)
}
},
}.Run(t, h)
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.

// TestConnectorCreateDatabricksOAuthIndividual smokes the oauth-individual
// leaf. Asserts the per-member-lazy note since that's the only leaf-unique
// piece of stdout. Reuses the same ANA_E2E_DBX_OAUTH_CLIENT_* env pair —
// oauth-sso and oauth-individual share the same Databricks OAuth app.
func TestConnectorCreateDatabricksOAuthIndividual(t *testing.T) {
common := databricksCommonEnvOrSkip(t)
clientID := os.Getenv("ANA_E2E_DBX_OAUTH_CLIENT_ID")
clientSecret := os.Getenv("ANA_E2E_DBX_OAUTH_CLIENT_SECRET")
if clientID == "" || clientSecret == "" {
t.Skip("e2e: ANA_E2E_DBX_OAUTH_CLIENT_ID and ANA_E2E_DBX_OAUTH_CLIENT_SECRET required for Databricks oauth-individual mode")
}

h := harness.Begin(t)
h.RegisterConnectorCleanupByName(h.ResourceName("dbx-oauth-individual"))
connectorCreateLeaf{
Name: "databricks oauth-individual",
Args: databricksLeafArgs(h, "oauth-individual", "dbx-oauth-individual", common, "--client-id", clientID, "--client-secret-stdin"),
Stdin: clientSecret + "\n",
ConnectorType: "DATABRICKS",
Extra: func(stdout string) {
if !strings.Contains(stdout, "lazily at first query") {
t.Errorf("oauth-individual note should mention lazy per-member auth:\n%s", stdout)
}
},
}.Run(t, h)
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Loading
Loading