Skip to content

test(inst/langchain4j): add mock ChatModel CI test and Ollama manual demo#11531

Draft
jbachorik wants to merge 7 commits into
jb/llm-ddt-3-lc4jfrom
jb/llm-ddt-4-demo
Draft

test(inst/langchain4j): add mock ChatModel CI test and Ollama manual demo#11531
jbachorik wants to merge 7 commits into
jb/llm-ddt-3-lc4jfrom
jb/llm-ddt-4-demo

Conversation

@jbachorik
Copy link
Copy Markdown
Contributor

@jbachorik jbachorik commented Jun 2, 2026

What Does This Do

Adds two demo/test classes to the langchain4j-1.0 instrumentation module:

  • MockLlmPipelineTest — JUnit test using a two-turn stub ChatModel (first turn returns a tool call request, second returns the final answer). Exercises CONTEXT_BUILD → AWAITING_INFERENCE → TOOL_EXECUTION → AWAITING_INFERENCE in a single CI-safe run. No external services required.

  • OllamaLlmPipelineDemo — standalone main class for generating real JFR recordings against a local Ollama server. Run with the dd-java-agent attached and dd.profiling.context.attributes.llm.phase.enabled=true to produce a recording with llm.agent.phase tags for backend ingestion.

Motivation

The mock test provides a deterministic CI validation that all three instrumentation hooks fire without error. The Ollama demo provides realistic inference latency data for the profiling backend.

Part of PROF-14858. Stacked on #11530 (LC4j module). Final PR in this stack.

Additional Notes

Run command for Ollama demo:

java -javaagent:dd-java-agent.jar \
  -Ddd.profiling.enabled=true \
  -Ddd.profiling.context.attributes.llm.phase.enabled=true \
  -cp <classpath> \
  datadog.trace.instrumentation.langchain4j.demo.OllamaLlmPipelineDemo

Prerequisites: ollama serve && ollama pull llama3

Contributor Checklist

  • Format the title according to the contribution guidelines
  • Assign the type: and (comp: or inst:) labels in addition to any other useful labels
  • Avoid using close, fix, or any linking keywords when referencing an issue
    Use solves instead, and assign the PR milestone to the issue
  • Update the CODEOWNERS file on source file addition, migration, or deletion
  • Update public documentation with any new configuration flags or behaviors
  • Add your completed PR to the merge queue by commenting /merge. You can also:
    • Customize the commit message associated with the merge with /merge --commit-message "..."
    • Remove your PR from the merge queue with /merge -c
    • Skip all merge queue checks with /merge -f --reason "reason"; please use this judiciously, as some checks do not run at the PR-level
    • Get more information in this doc

Jira ticket: PROF-14864

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jbachorik jbachorik added the AI label Jun 2, 2026
@datadog-datadog-prod-us1-2
Copy link
Copy Markdown
Contributor

datadog-datadog-prod-us1-2 Bot commented Jun 2, 2026

Pipelines

Fix all issues with BitsAI

⚠️ Warnings

🚦 15 Pipeline jobs failed

DataDog/apm-reliability/dd-trace-java | muzzle-dep-report   View in Datadog   GitLab

🔧 Fix in code (Fix with Cursor). Build failed: Cannot access last() element from an empty List in build.gradle:29

DataDog/apm-reliability/dd-trace-java | muzzle: [4/8]   View in Datadog   GitLab

🔧 Fix in code (Fix with Cursor). Muzzle validation failed during instrumentation tasks for multiple versions of langchain4j.

DataDog/apm-reliability/dd-trace-java | spotless   View in Datadog   GitLab

🔧 Fix in code (Fix with Cursor). Format violations detected in src/main/java/com/datadog/profiling/ddprof/DatadogProfiler.java. Run './gradlew spotlessApply' to fix.

View all 15 failed jobs.

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 64a9b3e | Docs | Datadog PR Page | Give us feedback!

@PerfectSlayer PerfectSlayer added tag: ai generated Largely based on code generated by an AI or LLM and removed AI labels Jun 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tag: ai generated Largely based on code generated by an AI or LLM

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants