Skip to content

Test MCP implementation with different LLMs and improve accordingly #26

@esgn

Description

@esgn

Context

This is a task we will repeat at different stages

  • Some models will not have access to MCP resources
  • Some small models will struggle with complex behavior
  • Some environments implement only part of the MCP standard or with specifics
  • MCP tools description must be tested with different models and different environments

Therefore we must test and improve accordingly.

Actions

Currently in ignfab/geocontext-test :

  • Allow the creation of a langgraph agent for test purpose
  • Create the first tests relying on the agent
  • Integrate either in geocontext (facing issues with corporate proxy) or demo-geocontext (refactoring required)

Models

  • anthropic:claude-haiku-4-5
  • anthropic:claude-sonnet-4-6
  • google_genai:gemini-2.5-flash
  • google_genai:gemini-3.1-flash-lite-preview
  • ollama:qwen3.5:9b ?
  • ... TO BE COMPLETED

Linked issues

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions