Add OpenSearch observability integration#1370
Conversation
Add a new observability sample that demonstrates how to send OpenTelemetry traces and metrics from a Bedrock AgentCore agent to OpenSearch via Data Prepper. Follows the same minimal pattern as the existing Dynatrace and OpenLit integrations with a travel agent example using Strands Agents. Includes Docker Compose setup instructions for local testing and guidance for production deployment with Amazon OpenSearch Service. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds a new OpenSearch observability integration sample showing how to export OpenTelemetry traces/metrics from a Bedrock AgentCore (Strands) agent to OpenSearch via Data Prepper, and lists it in the integrations index.
Changes:
- Added a new
03-integrations/observability/opensearch/sample with OTEL SDK initialization, entrypoint, and Strands travel agent. - Added setup/deployment documentation for local Docker + production OpenSearch/AOSI usage.
- Updated
03-integrations/README.mdto include the new OpenSearch observability integration.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| 03-integrations/observability/opensearch/travel_agent.py | Adds a Strands agent + AgentCore entrypoint used by the sample. |
| 03-integrations/observability/opensearch/opensearch.py | Initializes OpenTelemetry tracing/metrics exporters targeting Data Prepper OTLP endpoints. |
| 03-integrations/observability/opensearch/main.py | Entrypoint that initializes OTEL before importing/running the agent app. |
| 03-integrations/observability/opensearch/pyproject.toml | Declares sample dependencies (AgentCore, Strands, OTEL exporters). |
| 03-integrations/observability/opensearch/README.md | Documents local Docker/Data Prepper setup and production deployment guidance. |
| 03-integrations/README.md | Adds OpenSearch to the Observability integrations list. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| ports: | ||
| - "4318:4318" # OTLP HTTP receiver | ||
| - "21890:21890" # Data Prepper OTLP traces | ||
| - "21891:21891" # Data Prepper OTLP metrics | ||
| depends_on: |
There was a problem hiding this comment.
The Docker Compose + Data Prepper pipeline example is internally inconsistent: the compose file advertises an OTLP HTTP receiver on port 4318, but the provided pipelines.yaml configures otel_trace_source on 21890 and otel_metrics_source on 21891. With the current opensearch.py exporter config (single OTEL_ENDPOINT), following these instructions will likely result in telemetry being sent to an endpoint that isn’t listening. Please align the README + example configs (e.g., either configure Data Prepper to receive on the same OTLP HTTP endpoint you document, or document separate trace/metric endpoints/ports and update the code accordingly).
| By default, Data Prepper listens on: | ||
| - Port 21890 for traces (OTLP/HTTP) | ||
| - Port 21891 for metrics (OTLP/HTTP) | ||
|
|
||
| Configure OTEL_ENDPOINT to point to your Data Prepper instance. | ||
| """ | ||
| auth = read_secret("opensearch_auth") | ||
| headers = {} | ||
| if auth: | ||
| headers["Authorization"] = f"Basic {auth}" | ||
|
|
||
| OTEL_ENDPOINT = os.environ.get( | ||
| "OTEL_ENDPOINT", | ||
| "http://localhost:4318", # Data Prepper OTLP HTTP endpoint | ||
| ) |
There was a problem hiding this comment.
init() documents Data Prepper listening on separate ports for traces (21890) and metrics (21891), but the implementation uses a single OTEL_ENDPOINT (defaulting to http://localhost:4318) for both /v1/traces and /v1/metrics. This doesn’t match the README’s sample pipelines.yaml (separate ports) and will make the sample confusing or non-functional. Consider supporting separate env vars for trace vs metric endpoints (with sensible defaults) or update the docstring/README to match the single-endpoint approach.
| [project] | ||
| name = "agent-core-opensearch" | ||
| version = "0.1.0" | ||
| description = "Amazon Bedrock AgentCore integration with OpenSearch observability" | ||
| readme = "README.md" | ||
| requires-python = ">=3.11" | ||
| dependencies = [ | ||
| "boto3", | ||
| "bedrock-agentcore", | ||
| "bedrock-agentcore-starter-toolkit", | ||
| "uv", | ||
| "strands-agents-tools", | ||
| "strands-agents", | ||
| "opentelemetry-sdk", | ||
| "opentelemetry-exporter-otlp-proto-http", | ||
| ] |
There was a problem hiding this comment.
This integration doesn’t include a uv.lock, while the other observability integrations do (e.g., 03-integrations/observability/dynatrace/uv.lock and 03-integrations/observability/openlit/uv.lock). Adding a lockfile would make the sample reproducible and consistent with the established pattern in this repo.
Address Copilot review feedback: - Use separate OTEL_TRACES_ENDPOINT (port 21890) and OTEL_METRICS_ENDPOINT (port 21891) to match Data Prepper's separate pipeline architecture - Remove incorrect port 4318 reference from Docker Compose - Align README, code, and pipelines.yaml configuration Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add docker-compose.yml, pipelines.yaml, and data-prepper-config.yaml as standalone files instead of inline README examples - Add uv.lock for reproducibility (matches dynatrace/openlit pattern) - Update README to reference the actual config files - Fix remaining OTEL_ENDPOINT reference in Amazon OpenSearch Service section Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Functional testing revealed two issues: - Data Prepper's otel_trace_source/otel_metrics_source use gRPC, not HTTP. Switched from opentelemetry-exporter-otlp-proto-http to opentelemetry-exporter-otlp-proto-grpc. - Data Prepper requires ssl: false in pipeline config for local dev. Also adds ready-to-use Docker Compose files (docker-compose.yml, pipelines.yaml, data-prepper-config.yaml) for quick local testing. Verified end-to-end: agent responds to prompts, traces (24 spans) and metrics (28 docs) successfully land in OpenSearch indices. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Summary
03-integrations/observability/opensearch/that demonstrates how to export OpenTelemetry traces and metrics from a Bedrock AgentCore agent to OpenSearch via Data Prepperopensearch.py(OTEL SDK config),main.py(entrypoint),travel_agent.py(Strands agent),pyproject.toml, andREADME.md03-integrations/README.mdto list the new OpenSearch integrationFiles Added
opensearch.pymain.pytravel_agent.pypyproject.tomlopentelemetry-exporter-otlp-proto-httpREADME.mdTest plan
uv run main.pystarts the HTTP server on port 8080🤖 Generated with Claude Code