Skip to content

docs(llma): migrate onboarding docs to OpenTelemetry auto-instrumentation #53668

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
richardsolomou merged 12 commits into from
Apr 15, 2026
Merged

docs(llma): migrate onboarding docs to OpenTelemetry auto-instrumentation #53668

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
55b9741
docs(llma): migrate onboarding docs to OpenTelemetry auto-instrumenta…
richardsolomou Apr 8, 2026
e95a64f
chore(llma): remove proxy callout from OTel onboarding docs
richardsolomou Apr 8, 2026
97177f3
fix(llma): address review feedback on OTel onboarding docs
richardsolomou Apr 8, 2026
d0c19b0
fix(llma): use posthog.distinct_id and add custom property examples t…
richardsolomou Apr 8, 2026
ccb95d6
feat(llma): add migration callout linking to previous SDK wrapper exa…
richardsolomou Apr 8, 2026
96bdf3d
fix(llma): update callout wording to focus on current examples
richardsolomou Apr 8, 2026
b483978
feat(llma): migrate autogen, instructor, mirascope, semantic-kernel, …
richardsolomou Apr 9, 2026
0109746
feat(llma): migrate mastra doc to @mastra/posthog exporter
richardsolomou Apr 9, 2026
8d93882
feat(llma): split manual capture doc into steps for TOC
richardsolomou Apr 9, 2026
18a65eb
fix(llma): add missing deps to langchain, langgraph, and portkey docs
richardsolomou Apr 14, 2026
4a043d2
refactor(llma): use PostHogSpanProcessor in OTEL doc snippets
richardsolomou Apr 14, 2026
cbd9a96
refactor(llma): use PostHogSpanProcessor in Python OTEL doc snippets
richardsolomou Apr 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
192 changes: 84 additions & 108 deletions docs/onboarding/llm-analytics/anthropic.tsx
View file
Open in desktop

Large diffs are not rendered by default.

108 changes: 52 additions & 56 deletions docs/onboarding/llm-analytics/autogen.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,110 +3,98 @@ import { OnboardingComponentsContext, createInstallation } from 'scenes/onboardi
import { StepDefinition } from '../steps'

export const getAutoGenSteps = (ctx: OnboardingComponentsContext): StepDefinition[] => {
const { CodeBlock, CalloutBox, Markdown, dedent, snippets } = ctx
const { CodeBlock, CalloutBox, Markdown, Blockquote, dedent, snippets } = ctx

const NotableGenerationProperties = snippets?.NotableGenerationProperties

return [
{
title: 'Install the PostHog SDK',
title: 'Install dependencies',
badge: 'required',
content: (
<>
<Markdown>
Setting up analytics starts with installing the PostHog SDK. The AutoGen integration uses
PostHog's OpenAI wrapper since AutoGen uses OpenAI under the hood.
</Markdown>
<CalloutBox type="info" icon="IconInfo" title="Full working examples">
<Markdown>
See the complete [Python
example](https://github.com/PostHog/posthog-python/tree/master/examples/example-ai-autogen)
on GitHub. If you're using the PostHog SDK wrapper instead of OpenTelemetry, see the [Python
wrapper
example](https://github.com/PostHog/posthog-python/tree/7223c52/examples/example-ai-autogen).
</Markdown>
</CalloutBox>

<Markdown>Install the OpenTelemetry SDK, the OpenAI instrumentation, and AutoGen.</Markdown>

<CodeBlock
language="bash"
code={dedent`
pip install posthog
pip install autogen-agentchat "autogen-ext[openai]" openai opentelemetry-sdk posthog[otel] opentelemetry-instrumentation-openai-v2
`}
/>
</>
),
},
{
title: 'Install AutoGen',
title: 'Set up OpenTelemetry tracing',
badge: 'required',
content: (
<>
<Markdown>
Install AutoGen with the OpenAI extension. PostHog instruments your LLM calls by wrapping the
OpenAI client that AutoGen uses internally.
Configure OpenTelemetry to auto-instrument OpenAI SDK calls and export traces to PostHog.
PostHog converts `gen_ai.*` spans into `$ai_generation` events automatically.
</Markdown>

<CodeBlock
language="bash"
language="python"
code={dedent`
pip install "autogen-agentchat" "autogen-ext[openai]"
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.resources import Resource, SERVICE_NAME
from posthog.ai.otel import PostHogSpanProcessor
from opentelemetry.instrumentation.openai_v2 import OpenAIInstrumentor

resource = Resource(attributes={
SERVICE_NAME: "my-app",
"posthog.distinct_id": "user_123", # optional: identifies the user in PostHog
"foo": "bar", # custom properties are passed through
})

provider = TracerProvider(resource=resource)
provider.add_span_processor(
PostHogSpanProcessor(
api_key="<ph_project_token>",
host="<ph_client_api_host>",
)
)
trace.set_tracer_provider(provider)

OpenAIInstrumentor().instrument()
`}
/>
</>
),
},
{
title: 'Initialize PostHog and AutoGen',
title: 'Run your agents',
badge: 'required',
content: (
<>
<Markdown>
Initialize PostHog with your project token and host from [your project
settings](https://app.posthog.com/settings/project), then create a PostHog OpenAI wrapper and
pass it to AutoGen's `OpenAIChatCompletionClient`.
Use AutoGen as normal. PostHog automatically captures an `$ai_generation` event for each LLM
call made through the OpenAI SDK that AutoGen uses internally.
</Markdown>

<CodeBlock
language="python"
code={dedent`
import asyncio
from posthog.ai.openai import OpenAI
from posthog import Posthog
from autogen_agentchat.agents import AssistantAgent
from autogen_ext.models.openai import OpenAIChatCompletionClient

posthog = Posthog(
"<ph_project_token>",
host="<ph_client_api_host>"
)

openai_client = OpenAI(
api_key="your_openai_api_key",
posthog_client=posthog,
)

model_client = OpenAIChatCompletionClient(
model="gpt-4o",
openai_client=openai_client,
api_key="your_openai_api_key",
)
`}
/>

<CalloutBox type="fyi" icon="IconInfo" title="How this works">
<Markdown>
AutoGen's `OpenAIChatCompletionClient` accepts a custom OpenAI client via the
`openai_client` parameter. PostHog's `OpenAI` wrapper is a proper subclass of
`openai.OpenAI`, so it works directly. PostHog captures `$ai_generation` events
automatically without proxying your calls.
</Markdown>
</CalloutBox>
</>
),
},
{
title: 'Run your agents',
badge: 'required',
content: (
<>
<Markdown>
Use AutoGen as normal. PostHog automatically captures an `$ai_generation` event for each LLM
call made through the wrapped OpenAI client.
</Markdown>

<CodeBlock
language="python"
code={dedent`
agent = AssistantAgent("assistant", model_client=model_client)

async def main():
Expand All @@ -118,6 +106,14 @@ export const getAutoGenSteps = (ctx: OnboardingComponentsContext): StepDefinitio
`}
/>

<Blockquote>
<Markdown>
**Note:** If you want to capture LLM events anonymously, omit the `posthog.distinct_id`
resource attribute. See our docs on [anonymous vs identified
events](https://posthog.com/docs/data/anonymous-vs-identified-events) to learn more.
</Markdown>
</Blockquote>

<Markdown>
{dedent`
You can expect captured \`$ai_generation\` events to have the following properties:
Expand Down
29 changes: 15 additions & 14 deletions docs/onboarding/llm-analytics/aws-bedrock.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ export const getAWSBedrockSteps = (ctx: OnboardingComponentsContext): StepDefini
language: 'bash',
file: 'Python',
code: dedent`
pip install boto3 opentelemetry-instrumentation-botocore opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
pip install boto3 opentelemetry-instrumentation-botocore opentelemetry-sdk posthog[otel]
`,
},
{
Expand Down Expand Up @@ -56,22 +56,21 @@ export const getAWSBedrockSteps = (ctx: OnboardingComponentsContext): StepDefini
code: dedent`
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.sdk.resources import Resource, SERVICE_NAME
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from posthog.ai.otel import PostHogSpanProcessor
from opentelemetry.instrumentation.botocore import BotocoreInstrumentor

resource = Resource(attributes={
SERVICE_NAME: "my-ai-app",
})

exporter = OTLPSpanExporter(
endpoint="<ph_client_api_host>/i/v0/ai/otel",
headers={"Authorization": "Bearer <ph_project_token>"},
)

provider = TracerProvider(resource=resource)
provider.add_span_processor(BatchSpanProcessor(exporter))
provider.add_span_processor(
PostHogSpanProcessor(
api_key="<ph_project_token>",
host="<ph_client_api_host>",
)
)
trace.set_tracer_provider(provider)

BotocoreInstrumentor().instrument()
Expand All @@ -83,17 +82,19 @@ export const getAWSBedrockSteps = (ctx: OnboardingComponentsContext): StepDefini
code: dedent`
import { NodeSDK } from '@opentelemetry/sdk-node'
import { resourceFromAttributes } from '@opentelemetry/resources'
import { PostHogTraceExporter } from '@posthog/ai/otel'
import { PostHogSpanProcessor } from '@posthog/ai/otel'
import { AwsInstrumentation } from '@opentelemetry/instrumentation-aws-sdk'

const sdk = new NodeSDK({
resource: resourceFromAttributes({
'service.name': 'my-ai-app',
}),
traceExporter: new PostHogTraceExporter({
apiKey: '<ph_project_token>',
host: '<ph_client_api_host>',
}),
spanProcessors: [
new PostHogSpanProcessor({
apiKey: '<ph_project_token>',
host: '<ph_client_api_host>',
}),
],
instrumentations: [new AwsInstrumentation()],
})
sdk.start()
Expand Down
Loading
Loading