Skip to content

Add dynamic task suggestions to Gradio GUI#2

Open
dgaida wants to merge 5 commits intomasterfrom
add-dynamic-task-suggestions-16134512810370968264
Open

Add dynamic task suggestions to Gradio GUI#2
dgaida wants to merge 5 commits intomasterfrom
add-dynamic-task-suggestions-16134512810370968264

Conversation

@dgaida
Copy link
Copy Markdown
Owner

@dgaida dgaida commented Feb 9, 2026

This change adds a "Suggest Tasks" button to the Robot Control GUI. When clicked, the system:

  1. Queries the MCP server for currently detected objects.
  2. Retrieves all available robot skills (tools).
  3. Sends this context to an LLM to generate four realistic example tasks.
  4. Dynamically updates the example tasks in the Gradio interface using gr.render.

The implementation includes robust error handling, fallbacks to default examples, and uses asyncio.to_thread to keep the UI responsive during LLM generation.


PR created automatically by Jules for task 16134512810370968264 started by @dgaida

- Added 'Suggest Tasks' button to mcp_app.py
- Implemented suggest_tasks() in RobotMCPGUI class
- Uses get_detected_objects tool and available MCP skills to generate suggestions via LLM
- Uses gr.render for dynamic gr.Examples updates
- Improved responsiveness by running synchronous LLM calls in a separate thread

Co-authored-by: dgaida <23057824+dgaida@users.noreply.github.com>
@google-labs-jules
Copy link
Copy Markdown
Contributor

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

google-labs-jules Bot and others added 4 commits February 9, 2026 07:26
- Added 'Suggest Tasks' button to Gradio GUI in mcp_app.py
- Implemented suggest_tasks() in RobotMCPGUI class using LLMClient and MCP tools
- Uses gr.render for dynamic example updates
- Improved responsiveness and Python 3.8+ compatibility with run_in_executor
- Fixed CI collection failure by mocking broken 'text2speech' dependency in tests/conftest.py
- Added unit tests for suggest_tasks in tests/test_gui.py

Co-authored-by: dgaida <23057824+dgaida@users.noreply.github.com>
- Moved 'import pytest' to the top of the file to comply with PEP 8 and avoid E402 error.
- Kept the manual mocking of 'text2speech' as it is necessary for test collection.

Co-authored-by: dgaida <23057824+dgaida@users.noreply.github.com>
- Moved 'default_examples' to class level in RobotMCPGUI to support MagicMock(spec=...)
- Corrected import order in tests/conftest.py to fix E402 linting error while maintaining dependency mocks.

Co-authored-by: dgaida <23057824+dgaida@users.noreply.github.com>
- Fixed AttributeError in TestGradioInterface by explicitly setting the default_examples attribute on the mocked GUI object.
- This ensures the test is robust against differences between class and instance attributes during mocking.

Co-authored-by: dgaida <23057824+dgaida@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant