Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .github/workflows/skill-review.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Tessl Skill Review β€” runs on PRs that change any SKILL.md; posts scores as one PR comment.
# Docs: https://github.com/tesslio/skill-review
name: Tessl Skill Review

on:
pull_request:
branches: [main]
paths:
- "**/SKILL.md"

jobs:
review:
runs-on: ubuntu-latest
permissions:
pull-requests: write
contents: read
steps:
- uses: actions/checkout@v4
- uses: tesslio/skill-review@main
# Optional quality gate (off by default):
# with:
# fail-threshold: 70
145 changes: 66 additions & 79 deletions skills-submission/flutter-skill/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,22 @@
---
name: flutter-skill
description: Control and automate Flutter applications - inspect UI, perform gestures, validate state, take screenshots, and debug. Connects AI agents to running Flutter apps via Dart VM Service Protocol.
description: "Automate and test Flutter applications β€” launch apps, inspect widgets, tap elements, enter text, scroll, swipe, take screenshots, validate state, and debug via Dart VM Service Protocol. Use when the user wants to run Flutter app tests, automate Flutter UI interactions, inspect widget trees, debug a running Flutter app, or perform gesture-based testing."
---

# Flutter Skill

Give your AI Agent eyes and hands inside your Flutter app. This skill enables comprehensive control of Flutter applications for testing, debugging, and automation.
Control running Flutter applications for testing, debugging, and automation. Connects AI agents to Flutter apps via the Dart VM Service Protocol, exposing tools for UI inspection, gestures, state validation, screenshots, and log access.

## Installation

### Option 1: npx (Recommended)
```json
{
"flutter-skill": {
"command": "npx",
"args": ["flutter-skill"]
"mcpServers": {
"flutter-skill": {
"command": "npx",
"args": ["flutter-skill"]
}
}
}
```
Expand All @@ -24,91 +26,81 @@ Give your AI Agent eyes and hands inside your Flutter app. This skill enables co
dart pub global activate flutter_skill
```

Then configure:
```json
{
"flutter-skill": {
"command": "flutter_skill",
"args": ["server"]
"mcpServers": {
"flutter-skill": {
"command": "flutter_skill",
"args": ["server"]
}
}
}
```

## Available Tools

### Connection
- `connect_app` - Connect to a running Flutter app via WebSocket URI
- `launch_app` - Launch a Flutter app with auto-setup (adds dependencies, patches main.dart)

### UI Inspection
- `inspect` - Get interactive elements (buttons, text fields, etc.)
- `get_widget_tree` - Full widget tree structure with configurable depth
- `get_widget_properties` - Widget details (size, position, visibility)
- `get_text_content` - Extract all visible text from screen
- `find_by_type` - Find all widgets of a specific type

### Interactions
- `tap` - Tap element by key or text
- `double_tap` - Double tap gesture
- `long_press` - Long press gesture
- `swipe` - Swipe up/down/left/right
- `drag` - Drag from one element to another
- `scroll_to` - Scroll element into view
- `enter_text` - Input text into text field

### State Validation
- `get_text_value` - Get text field value
- `get_checkbox_state` - Get checkbox checked state
- `get_slider_value` - Get slider current value
- `wait_for_element` - Wait for element to appear (with timeout)
- `wait_for_gone` - Wait for element to disappear

### Screenshots
- `screenshot` - Capture full app screenshot (base64 PNG)
- `screenshot_element` - Capture specific element screenshot

### Navigation
- `get_current_route` - Get current route name
- `go_back` - Navigate back
- `get_navigation_stack` - Get navigation history

### Debug & Logs
- `get_logs` - Application logs
- `get_errors` - Error messages
- `get_performance` - Performance metrics
- `clear_logs` - Clear log buffer
- `hot_reload` - Trigger hot reload

## Usage Examples

### Test a Counter App
## Key Tools

| Category | Tools | Purpose |
|----------|-------|---------|
| **Connection** | `launch_app`, `connect_app` | Start or attach to a Flutter app |
| **Inspection** | `inspect`, `get_widget_tree`, `find_by_type` | Discover UI elements and widget structure |
| **Interaction** | `tap`, `enter_text`, `swipe`, `scroll_to`, `long_press`, `drag` | Perform gestures and input |
| **Validation** | `wait_for_element`, `wait_for_gone`, `get_text_value`, `get_checkbox_state` | Assert UI state |
| **Screenshots** | `screenshot`, `screenshot_element` | Capture visual state |
| **Navigation** | `go_back`, `get_current_route`, `get_navigation_stack` | Control and inspect navigation |
| **Debug** | `get_logs`, `get_errors`, `hot_reload`, `get_performance` | Diagnose issues |

## Workflow

### Core Testing Loop

```
1. Launch the app: launch_app with project_path="/path/to/app"
2. Inspect UI: inspect
3. Tap increment: tap with key="increment_button"
4. Verify: get_text_content to see updated counter
launch_app(project_path: "/path/to/app")
β†’ screenshot()
β†’ inspect()
β†’ tap(key: "element_key") / enter_text(key: "field_key", text: "value")
β†’ screenshot()
β†’ verify with wait_for_element / get_text_value
```

### Test a Login Flow
### Example: Login Flow

```
1. Enter email: enter_text with key="email_field", text="user@example.com"
2. Enter password: enter_text with key="password_field", text="password123"
3. Tap login: tap with key="login_button"
4. Wait for home: wait_for_element with key="home_screen", timeout=5000
launch_app(project_path: "/path/to/app")
screenshot()
inspect()
enter_text(key: "email_field", text: "user@example.com")
enter_text(key: "password_field", text: "password123")
tap(key: "login_button")
wait_for_element(key: "home_screen", timeout: 5000)
screenshot()
```

### Debug an Issue
**If `wait_for_element` times out:** Call `screenshot()` to see the current state, then `get_errors()` to check for crashes or failed network requests.

### Example: Debug a Running App

```
1. Connect: connect_app with uri="ws://127.0.0.1:xxxxx/ws"
2. Check errors: get_errors
3. View logs: get_logs
4. Take screenshot: screenshot
connect_app(uri: "ws://127.0.0.1:50000/ws")
get_errors()
get_logs()
screenshot()
inspect()
```

## Best Practices
## Validation Checkpoints

- **After `launch_app()`**: Verify a VM Service URI was returned. If not, check that Flutter is installed and the app compiles.
- **After `inspect()`**: Confirm interactive elements are returned. If empty, the app may still be loading β€” call `screenshot()` and retry.
- **After gestures** (`tap`, `enter_text`, `swipe`): Call `screenshot()` to confirm the UI updated as expected.
- **After navigation**: Use `wait_for_element(key: "target")` with a timeout. On timeout, call `get_errors()` to diagnose.

## Element Targeting Priority

### Use Widget Keys
For reliable element identification, target apps should use `ValueKey`:
1. **`key:`** (most reliable) β€” widget key set by the developer via `ValueKey`
2. **`text:`** β€” visible text content (breaks if text changes)
3. **`type:`** β€” widget type via `find_by_type` (may match multiple elements)

For reliable targeting, apps should use `ValueKey` on interactive elements:
```dart
ElevatedButton(
key: const ValueKey('submit_button'),
Expand All @@ -117,11 +109,6 @@ ElevatedButton(
)
```

### Element Finding Priority
1. **Key** (most reliable): `tap with key="submit_button"`
2. **Text content**: `tap with text="Submit"`
3. **Widget type**: `find_by_type with type="ElevatedButton"`

## Links

- [GitHub Repository](https://github.com/ai-dashboad/flutter-skill)
Expand Down
92 changes: 29 additions & 63 deletions skills/e2e-testing/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,11 @@
---
name: e2e-testing
description: AI-powered E2E testing for any app β€” Flutter, React Native, iOS, Android, Electron, Tauri, KMP, .NET MAUI. Test 8 platforms with natural language through MCP. No test code needed. Just describe what to test and the agent sees screenshots, taps elements, enters text, scrolls, and verifies UI state automatically.
version: 0.9.36
description: "AI-powered E2E testing for any app β€” Flutter, React Native, iOS, Android, Electron, Tauri, KMP, .NET MAUI. Connects via MCP to running apps so the agent can take screenshots, tap elements, enter text, scroll, inspect UI trees, and verify state with natural language. Use when the user wants to test an app's UI end-to-end, automate cross-platform testing, run smoke tests, validate form flows, or verify navigation without writing test code."
---

# AI E2E Testing β€” 8 Platforms, Zero Test Code

> Give your AI agent eyes and hands inside any running app.

flutter-skill is an MCP server that connects AI agents to running apps. The agent can see screenshots, tap elements, enter text, scroll, navigate, inspect UI trees, and verify state β€” all through natural language.
flutter-skill is an MCP server that connects AI agents to running apps across 8 platforms. The agent takes screenshots, taps elements, enters text, scrolls, navigates, inspects UI trees, and verifies state β€” all through natural language.

## Supported Platforms

Expand All @@ -23,8 +20,6 @@ flutter-skill is an MCP server that connects AI agents to running apps. The agen
| KMP Desktop | Gradle dependency |
| .NET MAUI | NuGet package |

**Test scorecard: 562/567 (99.1%) across all 8 platforms.**

## Install

```bash
Expand Down Expand Up @@ -52,48 +47,38 @@ Add to your AI agent's MCP config (Claude Desktop, Cursor, Windsurf, OpenClaw, e
}
```

### OpenClaw

If using OpenClaw, add to your gateway config under `mcp.servers`:

```yaml
mcp:
servers:
flutter-skill:
command: flutter-skill
args: ["server"]
```

## Quick Start

### 1. Initialize your app (one-time)

```bash
cd /path/to/your/app
flutter-skill init
```

Auto-detects project type and patches your app with the testing bridge.

**Verify:** Output should confirm the project type was detected and main entry point was patched. If it fails, check that you are in the project root and the framework is supported.

### 2. Launch and connect

```bash
flutter-skill launch .
```

**Verify:** A VM Service URI appears in the output (e.g. `ws://127.0.0.1:50000/ws`). If no URI appears, check that Flutter/the target framework is installed and the app compiles.

### 3. Test with natural language

Tell the agent what to test:
The agent follows this core loop:

> "Test the login flow β€” enter admin@test.com and password123, tap Login, verify Dashboard appears"
1. `screenshot()` β€” see the current screen
2. `inspect_interactive()` β€” discover all tappable/typeable elements with semantic refs
3. `tap(ref: "button:Login")` β€” tap using stable semantic reference
4. `enter_text(ref: "input:Email", text: "admin@test.com")` β€” type into field
5. `wait_for_element(key: "Dashboard")` β€” verify navigation succeeded
6. `screenshot()` β€” confirm final state

The agent will automatically:
1. `screenshot()` β†’ see the current screen
2. `inspect_interactive()` β†’ discover all tappable/typeable elements with semantic refs
3. `tap(ref: "button:Login")` β†’ tap using stable semantic reference
4. `enter_text(ref: "input:Email", text: "admin@test.com")` β†’ type into field
5. `wait_for_element(key: "Dashboard")` β†’ verify navigation
6. `screenshot()` β†’ confirm final state
**If `inspect_interactive()` returns no elements:** Take a screenshot to confirm the screen loaded, then check `get_logs()` for errors. The app may still be loading β€” retry after a short wait.

## Available MCP Tools

Expand All @@ -110,10 +95,10 @@ The agent will automatically:
| `go_back` | Navigate back |
| `press_key` | Send keyboard key events |

### Inspection (v0.8.0+)
### Inspection
| Tool | Description |
|------|-------------|
| `inspect_interactive` | **NEW** β€” Get all interactive elements with semantic ref IDs |
| `inspect_interactive` | Get all interactive elements with semantic ref IDs |
| `get_elements` | List all elements on screen |
| `find_element` | Find element by key or text |
| `wait_for_element` | Wait for element to appear (with timeout) |
Expand All @@ -132,7 +117,7 @@ The agent will automatically:
| `get_logs` | Read app logs |
| `clear_logs` | Clear log buffer |

## Semantic Refs (v0.8.0)
## Semantic Refs

`inspect_interactive` returns elements with stable semantic reference IDs:

Expand All @@ -155,50 +140,31 @@ enter_text(ref: "input:Email", text: "test@example.com")

## Testing Workflow

### Basic Flow
### Core Loop
```
screenshot() β†’ inspect_interactive() β†’ tap/enter_text β†’ screenshot() β†’ verify
```

### Comprehensive Testing
> "Explore every screen of this app. Test all buttons, forms, navigation, and edge cases. Report any bugs you find."

The agent will systematically:
- Navigate every screen via tab bars, menus, links
- Interact with every interactive element
- Test form validation (empty, invalid, valid inputs)
- Test edge cases (long text, special characters, emoji)
- Verify navigation flows (forward, back, deep links)
- Take screenshots at each step for verification

### Example Prompts

**Quick smoke test:**
> "Tap every tab and screenshot each page"

**Form testing:**
> "Fill the registration form with edge case data β€” emoji name, very long email, short password β€” and verify error messages"
Always call `screenshot()` before and after actions. Use `wait_for_element()` after navigation β€” apps need time to transition.

**Navigation:**
> "Test the complete user journey: sign up β†’ create post β†’ like β†’ comment β†’ delete β†’ sign out"
### Validation Checkpoints

**Accessibility:**
> "Check every screen for missing labels, small tap targets, and contrast issues"
- **After `screenshot()`**: Confirm the expected screen is visible before acting.
- **After `tap()` or `enter_text()`**: Call `screenshot()` to verify the UI responded.
- **After navigation**: Use `wait_for_element(key: "target_screen")` with a timeout. If it times out, call `screenshot()` and `get_logs()` to diagnose.
- **On unexpected state**: Call `get_logs()` and `inspect_interactive()` to understand what elements are present.

## Tips
### Element Targeting Priority

1. **Always start with `screenshot()`** β€” see before you act
2. **Use `inspect_interactive()` to discover elements** β€” don't guess at selectors
3. **Prefer `ref:` selectors** β€” more stable than text or coordinates
4. **`wait_for_element()` after navigation** β€” apps need time to transition
5. **Screenshot after every action** β€” verify the expected effect
6. **Use `press_key` for keyboard shortcuts** β€” test keyboard navigation
1. **`ref:`** (most reliable) β€” semantic ref from `inspect_interactive()`
2. **`key:`** β€” widget key set by the developer
3. **`text:`** β€” visible text content (fragile if text changes)
4. **Coordinates** β€” last resort, breaks on different screen sizes

## Links

- [GitHub](https://github.com/ai-dashboad/flutter-skill)
- [npm](https://www.npmjs.com/package/flutter-skill)
- [Documentation](https://github.com/ai-dashboad/flutter-skill/blob/main/docs/USAGE_GUIDE.md)
- [Demo Video](https://github.com/user-attachments/assets/d4617c73-043f-424c-9a9a-1a61d4c2d3c6)
- [pub.dev](https://pub.dev/packages/flutter_skill)
- [VSCode Extension](https://marketplace.visualstudio.com/items?itemName=AIDashboard.flutter-skill)