Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,22 @@
| [`shoploop-shopify`](../shoploop-shopify) | Shopify app · theme extensions · 3D configurator |
| [`shoploop-growth`](../shoploop-growth) | Google · Meta · Postiz · lifecycle · funnel analytics |

## Connectors

First-class, MCP-shaped, Temporal-driven connectors live under [`connectors/`](./connectors).
Each is the same shape — `primitives/` · `mcp/` (HTTP/SSE + bearer auth) · `temporal/` over a
single internal core. Not thin REST wrappers.

| Connector | Role in the loop | MCP port | Status |
|---|---|---|---|
| [`@shoploop/connector-shopify`](./connectors/shopify) | Catalog → `product_twin.usda` → scene composer (left half of the loop) | `:8787` | live |
| [`@shoploop/connector-google-ads`](./connectors/google-ads) | Performance Ingest #1 — ROAS/CTR/CAC → `PerformanceObservation` → Thompson Sampling → `brand_rules.json` | `:8788` | live |

```
Shopify catalog → product_twin.usda → render pack → Distribution
→ Google Ads Performance Ingest → Thompson Sampling → brand_rules.json (loop closes)
```

## Conversions

- **[Request Demo →](https://shoploop.ai/demo)**
Expand Down
13 changes: 13 additions & 0 deletions connectors/google-ads/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
GOOGLE_ADS_DEVELOPER_TOKEN=
GOOGLE_ADS_CLIENT_ID=
GOOGLE_ADS_CLIENT_SECRET=
GOOGLE_ADS_REFRESH_TOKEN=
GOOGLE_ADS_LOGIN_CUSTOMER_ID=
GOOGLE_ADS_API_VERSION=v17
SHOPLOOP_GADS_WRITE_ENABLED=false
BANDIT_QUEUE_PATH=./var/bandit/queue.jsonl
SHOPLOOP_MCP_TOKEN=
SHOPLOOP_GADS_MCP_PORT=8788
TEMPORAL_ADDRESS=localhost:7233
TEMPORAL_NAMESPACE=shoploop
PERF_OUT_DIR=./var/shoploop/perf
6 changes: 6 additions & 0 deletions connectors/google-ads/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
node_modules
dist
*.tsbuildinfo
.env
.env.local
var/
71 changes: 71 additions & 0 deletions connectors/google-ads/DESIGN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# DESIGN — connectors/google-ads

Companion to the root [`design.md`](../../design.md) and [`AGENTS.md`](../../AGENTS.md), and a
sibling of [`connectors/shopify/DESIGN.md`](../shopify/DESIGN.md). This doc covers only what is
specific to the Google Ads → performance-ingest connector.

## Position in the architecture loop

```
Distribution (Meta / TikTok / Klaviyo)
→ [THIS] Performance Ingest (ROAS, CTR, CAC) ← Google Ads is source #1
→ Thompson Sampling
→ brand_rules.json (loop closes)
```

The Shopify connector owns the *left* half of the loop (catalog → USD scene graph → render
pack). This connector owns the *right* half's **entry point**: turning channel performance into a
typed reward signal **before any bandit math exists**.

## The bandit loop (one paragraph)

Performance signal in, `brand_rules.json` arm update out, render pack regenerates. Each
`PerformanceObservation` carries a stable arm key (`campaign_id`, joinable to a product →
scene preset via the `shoploop.gads_campaign_id` Shopify metafield) and a reward
(`metrics.roas = conversions_value / cost`). The Thompson Sampling consumer treats each arm as a
Beta(α,β) posterior, updates it from observed ROAS vs spend, and rewrites the winning-scene bias
in `brand_rules.json` so the next render pack regenerates toward the arm with the highest sampled
ROAS. **The bandit is the moat; this connector is its input layer** — so the producer side is
locked to the contract and the consumer is stubbed (`recordPerformanceForBandit` just enqueues to
`BANDIT_QUEUE_PATH`). A real implementation drops in on the consumer side without touching this
connector.

## Single internal core

`client.ts` (`google-ads-api`, GAQL, API v17) is the only place Ads logic lives. The CLI, the MCP
tools, and the Temporal activities are thin adapters over the `GoogleAdsClient` interface — which
is trivially mocked by `FixtureClient` in tests, so a manual `ingest` and a scheduled workflow run
identical code.

## Why MCP-shaped, not a REST wrapper

A REST wrapper is commodity. The value is the first-class surface: campaigns, performance, search
terms, and change history are addressable by an agent (Claude/Cursor, Open Employee, Agentframe)
over the same HTTP/SSE + bearer pattern as the Shopify connector — on `:8788` so the two MCP
servers co-locate on one box. Write tools (`pause_campaign`, `resume_campaign`) exist but are
gated behind `SHOPLOOP_GADS_WRITE_ENABLED`, default off.

## Idempotency

Hourly schedule. Workflow id = `gads:<customerId>:<workflow>:<hour_bucket>` with
`WorkflowIdReusePolicy=REJECT_DUPLICATE`, so duplicate/retried fires within an hour collapse to one
execution. The ingest itself queries a 25h window (overlap) and dedupes by
`(date, campaign_id, ad_group_id, ad_id)` — belt and suspenders against gaps and double-counting.

## Contract (referenced, not vendored long-term)

`performance_observation.v1.json` lives in [`contracts/`](./contracts) for now; its eventual home
is `open-imagine/contracts/performance_observation.v1.json`. It is producer-agnostic
(`source ∈ {google-ads, meta, tiktok, klaviyo}`) so Meta/TikTok/Klaviyo ingest sources emit the
same shape into the same bandit.

## Cost discipline

CPU-only, no GPU. Varitea dogfood (1 customer, ~3 campaigns): API free (Basic Access), JSONL
storage <$1/mo, co-located on the Shopify `t4g.small` → ~$0/mo incremental.

## Boundaries

- No creative authoring / asset uploads — outbound push is a later PR.
- No render/image code → `open-imagine`.
- No bandit math here — producer side only.
148 changes: 148 additions & 0 deletions connectors/google-ads/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
# @shoploop/connector-google-ads

First-class **Google Ads → `PerformanceObservation`** primitives. Not a thin REST wrapper —
MCP-shaped, agent-callable, Temporal-driven. This is **Performance Ingest source #1**: the input
layer for the Thompson Sampling bandit that updates `brand_rules.json`.

```
Distribution (Meta / TikTok / Klaviyo) → [THIS] Performance Ingest (ROAS, CTR, CAC)
→ Thompson Sampling → brand_rules.json → render pack regenerates
```

Google Ads is the first source wired in — **Drink Varitea** already runs Google Ads. Mirrors
[`connectors/shopify`](../shopify) exactly: same `primitives/`, `mcp/`, `temporal/` layout, same
HTTP/SSE transport, same bearer auth, same Temporal idempotency convention.

> Contract referenced (vendored here until promoted):
> `open-imagine/contracts/performance_observation.v1.json` — see [`contracts/`](./contracts).
>
> **Ingest + signal only.** No creative authoring / asset uploads — outbound push is a later PR.

## The four primitives

| # | Primitive | Entry |
|---|-----------|-------|
| 1 | **Performance ingest** → one JSONL/day of `PerformanceObservation` | `ingestPerformance()` / `shoploop-google-ads ingest` |
| 2 | **MCP** read/write (7 tools, HTTP/SSE on `:8788`) | `shoploop-google-ads mcp` → `:8788/sse` |
| 3 | **Brand-rules feedback hook** (enqueue reward signal) | `recordPerformanceForBandit()` |
| 4 | **Scheduled ingest workflow** (hourly, idempotent) | `shoploop-google-ads worker` |

All four call the **same internal core** (`client.ts`) — importable from
`@shoploop/connector-google-ads`, callable from MCP, triggerable from Temporal. No duplication.

## Quickstart — Varitea dogfood path

```bash
cd connectors/google-ads
npm install
cp .env.example .env # fill in the GOOGLE_ADS_* OAuth2 creds
```

```bash
# .env essentials (OAuth2 refresh-token flow; Basic Access tier covers Varitea)
GOOGLE_ADS_DEVELOPER_TOKEN=...
GOOGLE_ADS_CLIENT_ID=...
GOOGLE_ADS_CLIENT_SECRET=...
GOOGLE_ADS_REFRESH_TOKEN=...
GOOGLE_ADS_LOGIN_CUSTOMER_ID=5128873230 # Varitea manager/customer id
GOOGLE_ADS_API_VERSION=v17
SHOPLOOP_GADS_WRITE_ENABLED=false # write tools OFF by default
BANDIT_QUEUE_PATH=./var/bandit/queue.jsonl
PERF_OUT_DIR=./var/shoploop/perf
```

### 1. Ingest performance

```bash
shoploop-google-ads ingest --customer 5128873230 --since 2026-06-12 --out ./var/shoploop/perf
# --until defaults to today; --since defaults to 7 days ago
```

**Expected output:** one JSONL per UTC day under `./var/shoploop/perf/<customerId>/YYYY-MM-DD.jsonl`.
Each line is a `PerformanceObservation` at campaign / ad_group / ad grain. For the Varitea Search
campaign (1 campaign, 2 ad groups, 5 ads) you get **8 observations/day** (1 + 2 + 5):

```json
{"schema":"performance_observation.v1","source":"google-ads","customer_id":"5128873230","date":"2026-06-12","level":"campaign","campaign_id":"20451234567","campaign_name":"Varitea — Search — Brand + Loose Leaf","ad_group_id":null,"ad_group_name":null,"ad_id":null,"metrics":{"impressions":1529,"clicks":75,"conversions":8,"cost_micros":58264164,"cost":58.264164,"conversions_value":304,"ctr":0.0490,"average_cpc":0.7769,"roas":5.22},"shopify_product_gid":null,"observed_at":"2026-06-19T00:00:00.000Z"}
```

`roas = conversions_value / cost` is the reward signal. Rows dedupe by
`(date, campaign_id, ad_group_id, ad_id)`, so overlapping windows are safe to re-run.

### 2. Join a campaign to its Shopify product

The join lives on the Shopify side as the product metafield **`shoploop.gads_campaign_id`**. We do
**not** duplicate Shopify client code — inject the Shopify connector's `update_metafield` tool:

```ts
import { linkCampaignToShopifyProduct } from "@shoploop/connector-google-ads";
import { createShopifyClient } from "@shoploop/connector-shopify";

const shopify = createShopifyClient({ /* ... */ });
await linkCampaignToShopifyProduct({
productGid: "gid://shopify/Product/7700112233445",
campaignId: "20451234567",
updateMetafield: shopify.updateMetafield, // the canonical Shopify write path
});
```

Read those metafields back into a `campaignId → productGid` map (`buildJoinMap`) and pass it as
`ingestPerformance({ shopifyJoin })` — every observation then carries `shopify_product_gid`, which
the bandit uses to find the right `brand_rules.json` arm.

### 3. Boot the MCP server

```bash
shoploop-google-ads mcp # Express + SSE on :8788 (/sse, /messages)
```

Bearer auth via `SHOPLOOP_MCP_TOKEN` (unset ⇒ open, dev only). CORS allowlist for Open Employee +
Agentframe origins. Tools:

| Tool | Kind |
|------|------|
| `list_campaigns(customerId, status?)` | read |
| `get_campaign(customerId, campaignId)` | read |
| `get_performance(customerId, range, granularity)` — daily/weekly rollups | read |
| `get_search_terms(customerId, campaignId, range)` — negative-keyword harvesting | read |
| `get_change_history(customerId, range)` — what changed and when | read |
| `pause_campaign(customerId, campaignId)` | **write** |
| `resume_campaign(customerId, campaignId)` | **write** |

Write tools are **refused** unless `SHOPLOOP_GADS_WRITE_ENABLED=true` (default off).

### 4. Run the Temporal worker

```bash
shoploop-google-ads worker # task queue: shoploop-google-ads
shoploop-google-ads worker --once --customer 5128873230 # dev: run hourly ingest once, on-demand
```

| Workflow | Trigger | Idempotency id |
|----------|---------|----------------|
| `IngestGoogleAdsPerformanceWorkflow(customerId)` | hourly; queries last 25h (overlap) | `gads:<customerId>:IngestGoogleAdsPerformanceWorkflow:<hour_bucket>` |
| `PauseUnderperformingCampaignWorkflow(customerId, campaignId, reason)` | bandit (later); no-op behind write flag | `gads:<customerId>:PauseUnderperformingCampaignWorkflow:<campaignId>:<hour_bucket>` |

The ingest workflow writes JSONL then flushes every observation onto the bandit queue
(`BANDIT_QUEUE_PATH`).

## Develop

```bash
npm run typecheck # tsc --noEmit (src + test)
npm test # vitest (primitives, mcp, workflow idempotency)
npm run build # emit dist/ (CLI bin → dist/cli.js)
```

## Cost (Varitea dogfood — 1 customer, ~3 campaigns)

- API quota: **free** (Basic Access tier).
- Storage (JSONL ~5 MB/mo): **<$1**.
- Co-locates on the same `t4g.small` as the Shopify MCP → **$0 incremental compute**.
- **Total: ~$0/mo incremental.**

## Boundaries

- No creative authoring / asset uploads → later PR.
- No render/image code → `open-imagine`.
- The real Thompson Sampling update is **stubbed** — this connector is the producer side only.
2 changes: 2 additions & 0 deletions connectors/google-ads/bin/cli.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/usr/bin/env node
import "../dist/cli.js";
80 changes: 80 additions & 0 deletions connectors/google-ads/contracts/performance_observation.v1.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "performance_observation.v1",
"title": "PerformanceObservation",
"description": "One performance datapoint from a distribution channel for one entity (campaign/ad_group/ad) on one day. Producer-agnostic reward signal for the Thompson Sampling bandit that updates brand_rules.json arms. EVENTUAL HOME: open-imagine/contracts/performance_observation.v1.json — vendored here until promoted.",
"type": "object",
"additionalProperties": false,
"required": [
"schema",
"source",
"customer_id",
"date",
"level",
"campaign_id",
"metrics",
"observed_at"
],
"properties": {
"schema": { "const": "performance_observation.v1" },
"source": {
"type": "string",
"description": "Channel that produced this observation.",
"enum": ["google-ads", "meta", "tiktok", "klaviyo"]
},
"customer_id": {
"type": "string",
"description": "Channel account id (Google Ads customer id, digits only, no dashes)."
},
"date": {
"type": "string",
"pattern": "^\\d{4}-\\d{2}-\\d{2}$",
"description": "UTC calendar day this datapoint aggregates."
},
"level": {
"type": "string",
"enum": ["campaign", "ad_group", "ad"],
"description": "Aggregation grain. Dedup key is (date, campaign_id, ad_group_id, ad_id)."
},
"campaign_id": { "type": "string" },
"campaign_name": { "type": "string" },
"ad_group_id": { "type": ["string", "null"] },
"ad_group_name": { "type": ["string", "null"] },
"ad_id": { "type": ["string", "null"] },
"metrics": {
"type": "object",
"additionalProperties": false,
"required": [
"impressions",
"clicks",
"conversions",
"cost_micros",
"cost",
"conversions_value",
"ctr",
"average_cpc",
"roas"
],
"properties": {
"impressions": { "type": "integer", "minimum": 0 },
"clicks": { "type": "integer", "minimum": 0 },
"conversions": { "type": "number", "minimum": 0 },
"cost_micros": { "type": "integer", "minimum": 0 },
"cost": { "type": "number", "minimum": 0, "description": "cost_micros / 1e6, channel currency." },
"conversions_value": { "type": "number", "minimum": 0 },
"ctr": { "type": "number", "minimum": 0, "description": "clicks / impressions." },
"average_cpc": { "type": "number", "minimum": 0, "description": "cost / clicks." },
"roas": { "type": "number", "minimum": 0, "description": "conversions_value / cost. THE reward signal for the bandit." }
}
},
"shopify_product_gid": {
"type": ["string", "null"],
"description": "Joined Shopify product gid via the shoploop.gads_campaign_id metafield. Null until the join is wired."
},
"observed_at": {
"type": "string",
"format": "date-time",
"description": "When this connector ingested the datapoint (ISO 8601)."
}
}
}
Loading
Loading