docs(llma): migrate onboarding docs to OpenTelemetry auto-instrumentation by richardsolomou · Pull Request #53668 · PostHog/posthog

richardsolomou · 2026-04-08T09:10:28Z

Problem

The in-app LLM analytics onboarding docs currently show PostHog SDK wrappers (posthog.ai.openai, @posthog/ai) as the primary integration method. We're moving toward standard OpenTelemetry auto-instrumentation as the recommended approach, keeping the wrappers available as a last resort for users who need them.

Changes

Migrates 28 provider onboarding guides from PostHog SDK wrapper approach to OpenTelemetry auto-instrumentation. Each migrated doc follows a consistent 3-step pattern:

Install dependencies — OTel SDK + provider-specific instrumentation + provider SDK
Set up OpenTelemetry tracing — TracerProvider with OTLPSpanExporter (Python) or PostHogTraceExporter (Node)
Call the provider — Native SDK usage, no posthog_ parameters

User identification uses the posthog.distinct_id resource attribute; custom properties like foo and conversation_id demonstrate the passthrough behavior.

Providers migrated (21 original + 7 more):

OpenAI-compatible: openai, groq, deepseek, mistral, cohere, fireworks-ai, xai, hugging-face, openrouter, together-ai, ollama, cerebras, perplexity
AI gateways: portkey, helicone, vercel-ai-gateway
Azure OpenAI, Anthropic
Frameworks: langchain, langgraph, llamaindex
Vercel AI (SimpleSpanProcessor update)
Newly migrated: autogen, instructor, mirascope, semantic-kernel, smolagents, pydantic-ai (already OTel-based in examples — docs just hadn't caught up)
Mastra (migrated to @mastra/posthog exporter — Mastra's native integration since there's no mature OTel path yet)

Kept as-is: google (no Node.js OTel instrumentation for @google/genai), aws-bedrock (already OTel), crewai, litellm, dspy, openai-agents, manual

Migration callout: Every migrated doc has a callout at the top linking to the full Node.js and Python examples on GitHub (current main/master), plus legacy wrapper examples pinned to the last commit before the migration. This helps both new users (who see the recommended OTel path) and existing users (who can find the old wrapper examples they're already using).

Manual capture doc: Split into multiple steps so it can have a useful table of contents (Capture LLM events manually → Event properties → Verify traces and generations).

Removed: "No proxy" callouts from all migrated docs — they made sense when wrapping SDK clients but don't apply to standard OTel tracing.

How did you test this code?

Agent-authored PR. Manually verified:

TypeScript compiles (pnpm --filter=@posthog/frontend format runs cleanly)
Each migrated doc follows the same pattern as other OTel docs
Example code in the docs matches what's in the real example repos (feat(examples): migrate AI examples to OpenTelemetry instrumentation posthog-js#3349 and feat(examples): migrate AI examples to OpenTelemetry instrumentation posthog-python#482)

No runtime tests since these are in-app onboarding content (static TSX returning step definitions).

Publish to changelog?

No

Docs update

This IS the docs update. Companion PR: PostHog/posthog.com#16236 (updates TOC frontmatter in posthog.com).

🤖 LLM context

Co-authored with Claude Code. Related PRs:

feat(examples): migrate AI examples to OpenTelemetry instrumentation posthog-js#3349 (Node.js OTel examples)
feat(examples): migrate AI examples to OpenTelemetry instrumentation posthog-python#482 (Python OTel examples)
docs(llma): update TOC for OTel onboarding migration posthog.com#16236 (docs.posthog.com TOC updates)

…tion Replace PostHog SDK wrapper approach with standard OpenTelemetry auto-instrumentation across 22 provider onboarding guides. Each doc now follows a 3-step pattern: install OTel deps, set up tracing, call the provider with the native SDK.

- Remove user.id from Vercel AI resource (posthog_distinct_id in experimental_telemetry is the established mechanism for that SDK) - Normalize service.name to 'my-app' in Vercel AI doc - Add anonymous events Blockquote to langchain, langgraph, llamaindex - Add Node tab to OpenAI embeddings step

greptile-apps · 2026-04-08T09:44:57Z

Vulnerabilities

No security concerns identified. The changes are documentation-only, migrating onboarding code snippets from PostHog SDK wrappers to OpenTelemetry auto-instrumentation. API keys in the snippets are clearly marked as placeholders.

Comments Outside Diff (1)

docs/onboarding/llm-analytics/portkey.tsx, line 24-43 (link)

Missing portkey-ai package in install instructions

The "Install dependencies" step only lists the OpenAI SDK and OTel packages, but the "Call Portkey" step imports portkey_ai (Python) and portkey-ai (Node) which are not included. Users copying the install command will hit an ImportError / Module not found error at runtime.

Python step (line 136–141) uses:

from portkey_ai import PORTKEY_GATEWAY_URL

Node step (line 158) uses:

import { PORTKEY_GATEWAY_URL } from 'portkey-ai'

The install blocks should include portkey-ai:

Prompt To Fix With AI

This is a comment left during a code review.
Path: docs/onboarding/llm-analytics/portkey.tsx
Line: 24-43

Comment:
**Missing `portkey-ai` package in install instructions**

The "Install dependencies" step only lists the OpenAI SDK and OTel packages, but the "Call Portkey" step imports `portkey_ai` (Python) and `portkey-ai` (Node) which are not included. Users copying the install command will hit an `ImportError` / `Module not found` error at runtime.

Python step (line 136–141) uses:
```python
from portkey_ai import PORTKEY_GATEWAY_URL
```

Node step (line 158) uses:
```typescript
import { PORTKEY_GATEWAY_URL } from 'portkey-ai'
```

The install blocks should include `portkey-ai`:



How can I resolve this? If you propose a fix, please make it concise.

Prompt To Fix All With AI

This is a comment left during a code review.
Path: docs/onboarding/llm-analytics/portkey.tsx
Line: 24-43

Comment:
**Missing `portkey-ai` package in install instructions**

The "Install dependencies" step only lists the OpenAI SDK and OTel packages, but the "Call Portkey" step imports `portkey_ai` (Python) and `portkey-ai` (Node) which are not included. Users copying the install command will hit an `ImportError` / `Module not found` error at runtime.

Python step (line 136–141) uses:
```python
from portkey_ai import PORTKEY_GATEWAY_URL
```

Node step (line 158) uses:
```typescript
import { PORTKEY_GATEWAY_URL } from 'portkey-ai'
```

The install blocks should include `portkey-ai`:

```suggestion
                    <Markdown>Install the OpenTelemetry SDK, the OpenAI instrumentation, the OpenAI SDK, and the Portkey SDK.</Markdown>

                    <CodeBlock
                        blocks={[
                            {
                                language: 'bash',
                                file: 'Python',
                                code: dedent`
                                    pip install openai portkey-ai opentelemetry-sdk opentelemetry-exporter-otlp-proto-http opentelemetry-instrumentation-openai-v2
                                `,
                            },
                            {
                                language: 'bash',
                                file: 'Node',
                                code: dedent`
                                    npm install openai portkey-ai @posthog/ai @opentelemetry/sdk-node @opentelemetry/resources @opentelemetry/instrumentation-openai
                                `,
                            },
                        ]}
                    />
```

How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (1): Last reviewed commit: "fix(llma): address review feedback on OT..." | Re-trigger Greptile}

…o docs

…mples

…smolagents, pydantic-ai docs to OTel

Match the posthog-js #3358 API: drop the manual SimpleSpanProcessor + PostHogTraceExporter wiring in favor of PostHogSpanProcessor. Convex keeps PostHogTraceExporter since BatchSpanProcessor doesn't work in V8.

Match posthog-python #494: drop the manual SimpleSpanProcessor + OTLPSpanExporter wiring in favor of PostHogSpanProcessor, and swap opentelemetry-exporter-otlp-proto-http for posthog[otel] in the pip install commands.

github-actions · 2026-04-14T08:52:27Z

Docs from this PR will be published at posthog.com

Project	Deployment	Preview	Updated (UTC)
posthog.com	🤷 Unknown	Preview	Apr 14, 2026, 8:52 AM

Preview will be ready in ~10 minutes. Click Preview link above to access docs at /handbook/engineering/

richardsolomou · 2026-04-15T10:42:38Z

This stack of pull requests is managed by Graphite. Learn more about stacking.

## Changes Updates `tableOfContents` frontmatter in LLM analytics installation docs to match the new step titles from the OTel migration in PostHog/posthog#53668. Each migrated doc now uses a 3-step TOC: 1. Install dependencies 2. Set up OpenTelemetry tracing 3. Call/Run {provider} 4. Verify traces and generations **Docs updated:** - OpenAI-compatible: openai, groq, deepseek, mistral, cohere, fireworks-ai, xai, hugging-face, openrouter, together-ai, ollama, cerebras, perplexity - Gateways: portkey, helicone - Azure OpenAI, Anthropic - Frameworks: langchain, langgraph, llamaindex - autogen, instructor, mirascope, semantic-kernel, smolagents, pydantic-ai - Mastra (now using `@mastra/posthog` exporter — 3-step TOC) - Manual capture (added a TOC for the first time — Capture LLM events manually → Event properties → Verify traces and generations) **Also includes** a branch-pinning change in `gatsby-config.js` and `gatsby/onPreBootstrap.ts` to point at `docs/otel-onboarding-docs` in the posthog repo, so the Vercel preview build shows the new docs before PostHog/posthog#53668 lands. **Must be reverted to `master` before merging.** **Note:** This PR should be merged after PostHog/posthog#53668 lands, since the TOC entries reference the new step titles from that PR. ## Checklist - [x] I've read the [docs](https://posthog.com/handbook/docs-and-wizard/docs-style-guide) and/or [content](https://posthog.com/handbook/content/posthog-style-guide) style guides. - [x] Words are spelled using American English - [x] Use relative URLs for internal links - [x] I've checked the pages added or changed in the Vercel preview build - [x] If I moved a page, I added a redirect in `vercel.json`

…482) ## Problem Our AI examples use PostHog's direct SDK wrappers (`posthog.ai.openai`, `posthog.ai.anthropic`, etc.) for tracking LLM calls. We want to silently deprecate these in favor of standard OpenTelemetry auto-instrumentation, which is more portable and follows industry conventions. ## Changes Migrates Python AI examples from PostHog wrappers to OpenTelemetry auto-instrumentation: - **OpenAI-compatible providers** (Groq, DeepSeek, Mistral, xAI, Together AI, Ollama, Cohere, Hugging Face, Perplexity, Cerebras, Fireworks AI, OpenRouter, Helicone, Vercel AI Gateway, Portkey) → `opentelemetry-instrumentation-openai-v2` - **OpenAI** (all files), **Azure OpenAI**, **Instructor**, **Autogen**, **Mirascope**, **Semantic Kernel**, **smolagents** → `opentelemetry-instrumentation-openai-v2` - **Anthropic** (chat, streaming, extended thinking) → `opentelemetry-instrumentation-anthropic` - **LangChain**, **LangGraph** → `opentelemetry-instrumentation-langchain` - **LlamaIndex** → `opentelemetry-instrumentation-llamaindex` - **Gemini** → `opentelemetry-instrumentation-google-generativeai` All OTel-based examples set resource attributes to demonstrate the full feature set: ```python resource = Resource( attributes={ SERVICE_NAME: "example-groq-app", "posthog.distinct_id": "example-user", "foo": "bar", "conversation_id": "abc-123", } ) ``` These map to `distinct_id` and custom event properties via PostHog's OTLP ingestion endpoint. **Kept as-is:** CrewAI (uses LiteLLM callbacks, internally manages its own TracerProvider), LiteLLM/DSPy (use LiteLLM's built-in PostHog callback), OpenAI Agents (uses dedicated `posthog.ai.openai_agents.instrument()`), Pydantic AI (already OTel via `Agent.instrument_all()`), AWS Bedrock (already OTel via `opentelemetry-instrumentation-botocore`). Key implementation details: - Uses `SimpleSpanProcessor` instead of `BatchSpanProcessor` so spans export immediately without needing `provider.shutdown()` - `# noqa: E402` on intentional late imports after `Instrumentor().instrument()` calls - Azure OpenAI example uses the generic `gpt-4o` model name instead of a deployment-specific one ## How did you test this code? Manually ran each example against real provider API keys via `llm-analytics-apps/run-examples.sh` to verify: 1. Each script runs successfully end-to-end 2. Traces arrive at PostHog as `$ai_generation` events 3. Resource attributes (`posthog.distinct_id`, `foo`, `conversation_id`) flow through as event properties 4. `distinct_id` is correctly set on each event All examples passing `ruff format` and `ruff check`. This is an agent-authored PR — I haven't manually tested each provider end-to-end beyond spot checks, though all examples follow the same pattern and the migration was verified on several providers. ## Publish to changelog? No ## Docs update The onboarding docs will be updated separately in PostHog/posthog#53668 and PostHog/posthog.com#16236. ## 🤖 LLM context Co-authored with Claude Code. Related PRs: - PostHog/posthog-js#3349 (Node.js examples) - PostHog/posthog#53668 (in-app onboarding docs) - PostHog/posthog.com#16236 (docs.posthog.com TOC updates)

…tion (#53668) ## Problem The in-app LLM analytics onboarding docs currently show PostHog SDK wrappers (`posthog.ai.openai`, `@posthog/ai`) as the primary integration method. We're moving toward standard OpenTelemetry auto-instrumentation as the recommended approach, keeping the wrappers available as a last resort for users who need them. ## Changes Migrates 28 provider onboarding guides from PostHog SDK wrapper approach to OpenTelemetry auto-instrumentation. Each migrated doc follows a consistent 3-step pattern: 1. **Install dependencies** — OTel SDK + provider-specific instrumentation + provider SDK 2. **Set up OpenTelemetry tracing** — `TracerProvider` with `OTLPSpanExporter` (Python) or `PostHogTraceExporter` (Node) 3. **Call the provider** — Native SDK usage, no `posthog_` parameters User identification uses the `posthog.distinct_id` resource attribute; custom properties like `foo` and `conversation_id` demonstrate the passthrough behavior. **Providers migrated (21 original + 7 more):** - OpenAI-compatible: openai, groq, deepseek, mistral, cohere, fireworks-ai, xai, hugging-face, openrouter, together-ai, ollama, cerebras, perplexity - AI gateways: portkey, helicone, vercel-ai-gateway - Azure OpenAI, Anthropic - Frameworks: langchain, langgraph, llamaindex - Vercel AI (SimpleSpanProcessor update) - **Newly migrated:** autogen, instructor, mirascope, semantic-kernel, smolagents, pydantic-ai (already OTel-based in examples — docs just hadn't caught up) - **Mastra** (migrated to `@mastra/posthog` exporter — Mastra's native integration since there's no mature OTel path yet) **Kept as-is:** google (no Node.js OTel instrumentation for `@google/genai`), aws-bedrock (already OTel), crewai, litellm, dspy, openai-agents, manual **Migration callout:** Every migrated doc has a callout at the top linking to the full Node.js and Python examples on GitHub (current `main`/`master`), plus legacy wrapper examples pinned to the last commit before the migration. This helps both new users (who see the recommended OTel path) and existing users (who can find the old wrapper examples they're already using). **Manual capture doc:** Split into multiple steps so it can have a useful table of contents (Capture LLM events manually → Event properties → Verify traces and generations). **Removed:** "No proxy" callouts from all migrated docs — they made sense when wrapping SDK clients but don't apply to standard OTel tracing. ## How did you test this code? Agent-authored PR. Manually verified: - TypeScript compiles (`pnpm --filter=@posthog/frontend format` runs cleanly) - Each migrated doc follows the same pattern as other OTel docs - Example code in the docs matches what's in the real example repos (PostHog/posthog-js#3349 and PostHog/posthog-python#482) No runtime tests since these are in-app onboarding content (static TSX returning step definitions). ## Publish to changelog? No ## Docs update This IS the docs update. Companion PR: PostHog/posthog.com#16236 (updates TOC frontmatter in posthog.com). ## 🤖 LLM context Co-authored with Claude Code. Related PRs: - PostHog/posthog-js#3349 (Node.js OTel examples) - PostHog/posthog-python#482 (Python OTel examples) - PostHog/posthog.com#16236 (docs.posthog.com TOC updates)

deployment-status-posthog · 2026-04-15T12:28:50Z

Deploy status

Environment	Status	Deployed At	Workflow
dev	✅ Deployed	2026-04-15 12:28 UTC	Run
prod-us	✅ Deployed	2026-04-15 12:45 UTC	Run
prod-eu	✅ Deployed	2026-04-15 12:46 UTC	Run

richardsolomou added 3 commits April 8, 2026 12:09

chore(llma): remove proxy callout from OTel onboarding docs

e95a64f

richardsolomou marked this pull request as ready for review April 8, 2026 09:42

richardsolomou mentioned this pull request Apr 8, 2026

docs(llma): update TOC for OTel onboarding migration PostHog/posthog.com#16236

Merged

5 tasks

richardsolomou requested a review from a team April 8, 2026 09:42

richardsolomou added 6 commits April 8, 2026 17:54

fix(llma): use posthog.distinct_id and add custom property examples t…

d0c19b0

…o docs

feat(llma): add migration callout linking to previous SDK wrapper exa…

ccb95d6

…mples

fix(llma): update callout wording to focus on current examples

96bdf3d

feat(llma): migrate autogen, instructor, mirascope, semantic-kernel, …

b483978

…smolagents, pydantic-ai docs to OTel

feat(llma): migrate mastra doc to @mastra/posthog exporter

0109746

feat(llma): split manual capture doc into steps for TOC

8d93882

richardsolomou mentioned this pull request Apr 9, 2026

feat(examples): migrate AI examples to OpenTelemetry instrumentation PostHog/posthog-python#482

Merged

carlos-marchal-ph approved these changes Apr 14, 2026

View reviewed changes

Comment thread docs/onboarding/llm-analytics/portkey.tsx Outdated

richardsolomou added 3 commits April 14, 2026 11:21

fix(llma): add missing deps to langchain, langgraph, and portkey docs

18a65eb

refactor(llma): use PostHogSpanProcessor in OTEL doc snippets

4a043d2

Match the posthog-js #3358 API: drop the manual SimpleSpanProcessor + PostHogTraceExporter wiring in favor of PostHogSpanProcessor. Convex keeps PostHogTraceExporter since BatchSpanProcessor doesn't work in V8.

refactor(llma): use PostHogSpanProcessor in Python OTEL doc snippets

cbd9a96

Match posthog-python #494: drop the manual SimpleSpanProcessor + OTLPSpanExporter wiring in favor of PostHogSpanProcessor, and swap opentelemetry-exporter-otlp-proto-http for posthog[otel] in the pip install commands.

richardsolomou mentioned this pull request Apr 15, 2026

docs(llma): add Dedalus Labs and Cloudflare AI Gateway onboarding docs #54663

Merged

richardsolomou merged commit 2ef6c6f into master Apr 15, 2026
134 checks passed

richardsolomou deleted the docs/otel-onboarding-docs branch April 15, 2026 12:00

a-lider mentioned this pull request Apr 15, 2026

fix(onboarding): add missing info type to CalloutBox component #54711

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs(llma): migrate onboarding docs to OpenTelemetry auto-instrumentation#53668

docs(llma): migrate onboarding docs to OpenTelemetry auto-instrumentation#53668
richardsolomou merged 12 commits into
masterfrom
docs/otel-onboarding-docs

richardsolomou commented Apr 8, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Apr 8, 2026 •

edited

Loading

Vulnerabilities

Comments Outside Diff (1)

Uh oh!

Uh oh!

github-actions Bot commented Apr 14, 2026

Uh oh!

richardsolomou commented Apr 15, 2026 •

edited

Loading

Uh oh!

Uh oh!

deployment-status-posthog Bot commented Apr 15, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

richardsolomou commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Changes

How did you test this code?

Publish to changelog?

Docs update

🤖 LLM context

Uh oh!

greptile-apps Bot commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Vulnerabilities

Comments Outside Diff (1)

Uh oh!

Uh oh!

github-actions Bot commented Apr 14, 2026

Uh oh!

richardsolomou commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

deployment-status-posthog Bot commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploy status

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

richardsolomou commented Apr 8, 2026 •

edited

Loading

greptile-apps Bot commented Apr 8, 2026 •

edited

Loading

richardsolomou commented Apr 15, 2026 •

edited

Loading

deployment-status-posthog Bot commented Apr 15, 2026 •

edited

Loading