feat(excel): add support for Excel file processing in VisionAgent#121
Merged
adi-wan-askui merged 12 commits intomainfrom Aug 20, 2025
Merged
Conversation
- Introduced `ExcelSource` class to handle Excel files as input sources. - Updated `AgentBase` and related APIs to accept `ExcelSource` alongside existing image and PDF sources. - Implemented error handling for unsupported Excel processing in specific models. - Added tests for Excel file handling and processing. - Created utility functions for converting Excel content to markdown format. - Added dummy Excel file for testing purposes.
- Changed the type hint for the `root` attribute from `bytes` to `bytes | Path` to accurately reflect that it can be either the underlying Excel bytes or a file path.
…e related references - Renamed the `ExcelSource` class to `OfficeDocumentSource` to better reflect its functionality for handling various office document types. - Updated all references to `ExcelSource` across the codebase to `OfficeDocumentSource`. - Adjusted error messages to specify "Office Document" processing instead of just "Excel". - Enhanced the `Source` type to include `OfficeDocumentSource` for broader compatibility.
- Replaced occurrences of `Img` with `InputSource` in the codebase to standardize the type used for image, file and data uri inputs across various functions and classes. - Updated the `screenshot` parameter in methods to use `InputSource | None` instead of `Img | None` for better clarity and consistency. - Removed the `Excel` and `Pdf` type definitions from their respective modules, consolidating input types under `InputSource`. - Enhanced the `__all__` exports to include the new `InputSource` type for better accessibility.
- Updated docstrings for `source` and `screenshot` parameters in `AgentBase` class to clarify the types of input sources accepted, including image, PDF, and office document files. - Improved error messages in `AnthropicMessagesApi`, `AskUiInferenceApi`, and `OpenRouterModel` classes to provide clearer context regarding unsupported PDF and office document processing. - Reformatted docstring for `InputSource` type to enhance readability.
- Eliminated the file size validation for `OfficeDocumentSource` in the `read()` method, as it was deemed unnecessary for the current implementation.
…t.py` - Modified the test case `test_get_with_xlsx_with_default_model_with_chart_data` to change the query from "What does the chart show?" to "What is the salary of John?". - Updated the assertion to check for "10000" in the response instead of "count of names" to reflect the new query context.
- Introduced new pytest fixtures `path_fixtures_docs` and `path_fixtures_dummy_doc` to provide paths for the docs directory and a dummy document, respectively. - Added a test case `test_get_with_docs_with_default_model` to verify the response from the `VisionAgent` when querying with a dummy document. - Included a dummy document `dummy.docx` for testing purposes.
- Updated the test case `test_get_with_xlsx_with_gemini_model` to change the query to "What is the salary of Doe?" for clarity. - Introduced a new test case `test_get_with_xlsx_with_gemini_model_with_response_schema` to validate the response structure using the `SalaryResponse` schema. - Added `Salary` and `SalaryResponse` classes to define the expected response format. - Included assertions to verify the correctness of the salary data returned for multiple individuals.
- Introduced a new section detailing the use of the `markitdown` library for extracting data from documents like Docs and Excel files. - Highlighted key features of `markitdown`, including LLM-friendly output, inclusion of sheet names, enhanced image descriptions, no local inference requirements, optional dependencies, and Microsoft maintenance.
- Clarified the description of the `InputSource` type to specify that it includes both images and files for `askui.VisionAgent.get()` and images for `askui.VisionAgent.locate()`.
adi-wan-askui
deleted the
CL-1675-release-2025-09-python-extract-data-from-excel
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ExcelSourceclass to handle Excel files as input sources.AgentBaseand related APIs to acceptExcelSourcealongside existing image and PDF sources.