Feat: Add starrocks catalog support#8856
Feat: Add starrocks catalog support#8856chris-celerdata wants to merge 7 commits intomarimo-team:mainfrom
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
All contributors have signed the CLA ✍️ ✅ |
|
I have read the CLA Document and I hereby sign the CLA |
There was a problem hiding this comment.
Pull request overview
Adds first-class StarRocks support to marimo’s SQL datasource discovery by introducing a StarRocksEngine that can enumerate catalogs and introspect external-catalog objects via SHOW .../DESC SQL when SQLAlchemy’s inspector can’t.
Changes:
- Implement
StarRocksEngine(SQLAlchemyEngine)with multi-catalog discovery and external-catalog fallbacks. - Extend SQL identifier quoting to support the
starrocksdialect (backtick style) and add related tests. - Register the new engine in engine detection and add StarRocks optional test dependency / dependency manager entry.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
marimo/_sql/engines/starrocks.py |
New engine implementing StarRocks catalog/database/table discovery with SQL fallbacks for external catalogs. |
marimo/_sql/get_engines.py |
Adds StarRocksEngine to SUPPORTED_ENGINES so it’s selected before generic SQLAlchemyEngine. |
marimo/_sql/sql_quoting.py |
Adds starrocks to backtick-quoting dialects. |
marimo/_sql/sql.py |
Updates unsupported-engine error message to mention StarRocks. |
marimo/_dependencies/dependencies.py |
Adds DependencyManager.starrocks. |
pyproject.toml |
Adds starrocks to test-optional and bans module-level starrocks imports. |
tests/_sql/test_starrocks.py |
New unit tests for StarRocks engine behavior using mocks. |
tests/_sql/test_sql_quoting.py |
Adds StarRocks quoting/roundtrip cases and qualified-name quoting case. |
tests/_sql/test_get_engines.py |
Adds StarRocks engine selection test and datasource connection mapping assertion. |
Bundle ReportBundle size has no change ✅ |
|
hi @chris-celerdata, this sounds similar to an issue with other multi-db engines like Snowflake. There is a start of the fix here #8824 Would you mind having a look and seeing if that direction would fix this problem universally, rather than custom support for Starrocks? If not, we can also have a look (wish there was Starrocks cloud haha) |
|
There is a StarRocks cloud. It can also be deployed on Docker. If you want to work with external catalogs, the documentation can be found here. However, the I took a look at the other PR and it does not fix the problem; Note: I considered doing something similar to dialect_queries = {
"postgresql": "SELECT current_database()",
"mssql": "SELECT DB_NAME()",
"timeplus": "SELECT current_database()",
}This approach could be duplicated into |
|
I think that's the eventual direction. That PR decouples get_schema, so eventually we can call get_databases and get all databases connected. I'm okay to have this too, but there'll be some code drift once we merge that. In the meantime, would you want to create the frontend UI changes for adding a starrocks connection? |
|
I'd love to add StarRocks to the UI. I was referencing #8169, which talked about keeping the UI uncluttered. Has that changed? |
|
@chris-celerdata , yeah. But I'm checking with the team on if we're okay to add to the UI. The PR referenced #8824 has been merged, does this change this PR? Sorry for the back and forth on this |

📝 Summary
Implements #8765.
🔍 Description of Changes
This PR introduces a new engine which extends
SQLAlchemyEngine. When possible (e.g. working withdefault_catalog), the inherited methods are used; otherwise special "external" methods are used.Primary overridden functions:
get_databasesget_tables_in_schemaget_table_detailsBefore

After

Notes
Integration testing against a real StarRocks instance is not present. There is no "in-memory" equivalent like what
DuckDBandSQLitehave.Arrays of variable types default greedily to their inner types. There is currently no support for showing complex types with different icons from a standard string (i.e.
ARRAY<INT>as not simply an integer). I'm not sure if this is relevant, but I thought I would mention it anyway.📋 Checklist