Skip to content
Merged
35 changes: 35 additions & 0 deletions .github/workflows/verify-litert-mcp.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
name: Verify LiteRT-LM MCP

on:
push:
paths:
- 'mcp-servers/litert-mcp/**'
- '.github/workflows/verify-litert-mcp.yml'
pull_request:
paths:
- 'mcp-servers/litert-mcp/**'
- '.github/workflows/verify-litert-mcp.yml'

jobs:
verify:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'

- name: Verify Server Startup (Initialize)
run: |
echo '{"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {}}' | python3 mcp-servers/litert-mcp/server.py

- name: Verify Tools List
run: |
OUTPUT=$(echo '{"jsonrpc": "2.0", "id": 2, "method": "tools/list", "params": {}}' | python3 mcp-servers/litert-mcp/server.py)
echo "$OUTPUT"
if [[ "$OUTPUT" != *"run_inference"* ]]; then
echo "Error: run_inference tool not found in output"
exit 1
fi
23 changes: 23 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# AGENTS.md

## Mission
To enable autonomous AI agents to efficiently reason, plan, and execute tasks within this repository, specifically focusing on the integration and orchestration of Model Context Protocol (MCP) servers and Video Intelligence pipelines.

## Scope of "Actionable Room"
Agents are granted permission and encouraged to:
1. **Modify and Expand `mcp-servers/`**: Create new MCP servers, update existing ones, and refactor code to improve modularity and performance.
2. **Improve Automation**: Create and edit GitHub Actions workflows (`.github/workflows/`) to add robust testing and verification for new features.
3. **Refactor for Clarity**: Improve documentation (READMEs) and code structure to facilitate better "Machine Readability" and "Human Understandability".

## Protocols
1. **Verify Before Submit**:
- Always run relevant verification scripts or tests before submitting changes.
- If no test exists for a new feature, **create one**.
2. **CI/CD Alignment**:
- Ensure all changes pass existing CI checks.
- When adding a new component (like an MCP server), add a corresponding CI workflow to ensure it remains functional.
3. **Cross-Platform Compatibility**:
- Write code that is compatible with Linux and Windows environments whenever possible (e.g., handling `asyncio` loops correctly).
4. **Documentation**:
- Update `README.md` files when interface changes occur.
- Document limitations (e.g., "Text-only CLI wrapper") clearly.
61 changes: 61 additions & 0 deletions mcp-servers/litert-mcp/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# LiteRT-LM MCP Server

This MCP server provides an interface to Google's **LiteRT-LM**, a high-performance runtime for Large Language Models (LLMs) on edge devices (Android, iOS, Linux, MacOS, Windows).

It allows you to run inference on local models (like Gemma, Phi, Qwen) directly from your MCP ecosystem.

**Note:** This server currently wraps the `lit` CLI. Multimodal inputs (image/audio) are enabled in the interface but require the C++ API or Python bindings. The CLI wrapper currently supports **Text-only inference** until CLI flags for multimodal are verified.

## Prerequisites

1. **LiteRT-LM**: You must have LiteRT-LM installed or built.
* Official Repository: [google-ai-edge/LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM)
* Follow the "Build and Run" instructions in the official repo to build the `lit` CLI or `litert_lm_main` binary.
* Alternatively, download prebuilt binaries if available for your platform.

2. **Models**: Download a supported `.litertlm` model.
* Models are available on Hugging Face: [LiteRT Community](https://huggingface.co/litert-community)

## Configuration

Set the following environment variables:

* `LIT_BINARY_PATH`: Path to the `lit` CLI executable or `litert_lm_main` binary. Defaults to `lit` (assuming it's in your PATH).
* `LIT_MODEL_PATH`: Default path to your `.litertlm` model file. (Optional, can be passed per request).

## Usage

### Tools

#### `run_inference`

Runs inference using the configured LiteRT-LM model.

* **Arguments**:
* `prompt` (string, required): The input text prompt.
* `model_path` (string, optional): Path to the `.litertlm` model file. Overrides `LIT_MODEL_PATH` env var.
* `image_path` (string, optional): Path to an image file for multimodal inference.
* `audio_path` (string, optional): Path to an audio file for multimodal inference.
Comment on lines +37 to +38
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The README.md lists image_path and audio_path as optional arguments. However, the server.py explicitly returns an error if these are provided, stating that multimodal input is not yet supported via the CLI wrapper. To avoid confusion, please update the README to clearly state this limitation or remove these arguments from the documentation until they are supported.

* `backend` (string, optional): Backend to use (`cpu`, `gpu`, `npu`). Defaults to `cpu`.

### Example

```json
{
"name": "run_inference",
"arguments": {
"prompt": "What is the capital of France?",
"backend": "cpu"
}
}
```
Comment on lines 43 to 51
Copy link

Copilot AI Jan 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The example demonstrates multimodal inference with an image_path, but the documentation and implementation both state that multimodal inputs are not currently supported. This creates confusion. Either remove the image_path from the example or provide a text-only example that matches the current capabilities.

Copilot uses AI. Check for mistakes.

## Setup for Development

This server uses a manual JSON-RPC implementation to avoid external dependencies in the base environment. Just run:

```bash
python3 server.py
```

Ensure `LIT_BINARY_PATH` is set correctly.
Loading
Loading