feat: implement PLEXOS solution reader and setup master files for 9-12 versions

.gitignore

@@ -13,7 +13,6 @@
 *.sdf
 *.tax2010
 *.vcf
-*.xml
 
 ### Database ###
 *.accdb
@@ -315,6 +314,7 @@ Sessionx.vim
 [._]*.un~
 
 ### VisualStudioCode ###
+.vscode
 .vscode/*
 !.vscode/settings.json
 !.vscode/tasks.json

.pre-commit-config.yaml

@@ -29,6 +29,7 @@ repos:
         language: node
         additional_dependencies: ["prettier@3"]
         types: [markdown]
+        exclude: ^CHANGELOG\.md$
 
   - repo: builtin
     hooks:

README.md

@@ -56,11 +56,56 @@ db.add_membership(
 db.to_xml("modified_model.xml")
 ```
 
+## Versioned Schema Initialization
+
+When creating a brand-new database (not importing XML), you can preload the
+matching PLEXOS master template by version:
+
+```python
+from plexosdb import PlexosDB
+
+db = PlexosDB()
+db.create_schema(version=10)
+```
+
+Supported versions: 9, 10, 11, 12.
+
+Accepted inputs include integers, strings, and tuples. For example:
+
+```python
+db.create_schema(version="v11.0R4")
+db.create_schema(version=(12, 0, 3))
+```
+
+## Read PLEXOS Solution ZIP Tables with pandas
+
+```python
+from plexosdb import PlexosSolution
+import pandas as pd
+
+PLEXOS_SOLUTION = "/path/to/solution.zip"
+
+sol = PlexosSolution.from_zip(PLEXOS_SOLUTION)
+sol.to_sqlite("output.sqlite", if_exists="replace")
+
+table = "ST__Interval__Regions__Fixed_Load"
+sol.materialize_table(table, schema="report")
+df_table = pd.read_sql_query(f'SELECT * FROM report."{table}"', sol.connection)
+```
+
+This pattern is useful when you only need a few report/data tables from a large
+solution ZIP, because it materializes tables on demand.
+
 ## Documentation
 
 Full documentation is available at
 [natlabrockies.github.io/plexosdb](https://natlabrockies.github.io/plexosdb/).
 
+## Related Work
+
+For related previous/current work on querying PLEXOS outputs with DuckDB, see
+[plexos2duckdb](https://github.com/NatLabRockies/plexos2duckdb).
+
 ## Developer Setup
 
 plexosdb uses [uv](https://docs.astral.sh/uv/) for dependency management.

docs/source/howtos/create_db.md

@@ -12,6 +12,67 @@ from plexosdb import PlexosDB
 db = PlexosDB.from_xml("/path/to/model.xml")
 ```
 
+When using `from_xml(...)`, the schema is created automatically from the XML
+content and metadata in that file.
+
+## Create a New Empty Database with a Versioned Master Template
+
+When starting from an empty database (instead of an existing XML), you can now
+preload the versioned master template during schema creation.
+
+```python
+from plexosdb import PlexosDB
+
+db_base = PlexosDB()
+
+# Load default SQL schema + master template for PLEXOS v10
+ok = db_base.create_schema(version=10)
+assert ok
+```
+
+`create_schema(...)` returns a boolean status. It does not return a new
+`PlexosDB` instance.
+
+You can also pass custom SQL with `schema=...`:
+
+```python
+custom_schema = """
+CREATE TABLE IF NOT EXISTS my_table (
+	id INTEGER PRIMARY KEY,
+	name TEXT
+);
+"""
+
+db_custom = PlexosDB()
+db_custom.create_schema(schema=custom_schema)
+```
+
+Supported template versions are:
+
+- 9
+- 10
+- 11
+- 12
+
+Accepted version input formats include:
+
+- `10`
+- `"10.0"`
+- `"v10.0R2"`
+- `(10, 0, 2)`
+
+```python
+# Equivalent examples
+db_base.create_schema(version="v11.0R4")
+db_base.create_schema(version=(12, 0, 3))
+```
+
+If `version` is omitted, only the SQL schema is created (no master template is
+imported).
+
+Call `create_schema(...)` once per database instance. If the schema already
+exists, re-running schema SQL is skipped.
+
 ## Create empty schema (tables only)
 
 `create_schema()` by itself creates the table structure and config rows, but it
@@ -30,18 +91,6 @@ db = PlexosDB()
 db.create_schema()
 ```
 
-## Set schema version explicitly
-
-Use `version="..."` when your schema includes a `t_config` row for `Version` and
-you want to update that value during schema creation.
-
-```python
-from plexosdb import PlexosDB
-
-db = PlexosDB()
-db.create_schema(version="11.0")
-```
-
 ## Create empty schema and seed defaults
 
 Use `seed_defaults=True` when you want a fresh DB that can immediately add
@@ -102,7 +151,4 @@ VALUES (1, 'System', 1, 1, '00000000-0000-0000-0000-000000000001', 'Default syst
 
 db = PlexosDB()
 db.create_schema(schema=schema)
-
-# If your custom schema does not provide Version, you can still set it here:
-# db.create_schema(schema=schema, version="10.0")
 ```

docs/source/howtos/import_export.md

@@ -18,6 +18,36 @@ version = db.version
 print(f"Imported PLEXOS model version: {version}")
 ```
 
+## Importing a PLEXOS solution ZIP and reading report tables
+
+Use `PlexosSolution` when you want to analyze PLEXOS solution ZIP outputs. With
+the default `materialize="none"`, you can materialize only the table you need.
+
+```python
+from plexosdb import PlexosSolution
+import pandas as pd
+
+PLEXOS_SOLUTION = "/path/to/solution.zip"
+
+sol = PlexosSolution.from_zip(PLEXOS_SOLUTION)
+sol.to_sqlite("output.sqlite", if_exists="replace")
+
+table = "ST__Interval__Regions__Fixed_Load"
+sol.materialize_table(table, schema="report")
+df_table = pd.read_sql_query(f'SELECT * FROM report."{table}"', sol.connection)
+```
+
+How to use this flow:
+
+1. Create a `PlexosSolution` via `PlexosSolution.from_zip(zip_path)`.
+2. Call `to_sqlite(path, if_exists="replace")` to import the ZIP into SQLite.
+   Use `materialize="none"` (default) to avoid materializing every derived table
+   up-front.
+3. Call `materialize_table(table, schema="report")` for the specific table you
+   want.
+4. Read that table with pandas using `pd.read_sql_query(...)` and
+   `sol.connection`.
+
 ## Exporting to XML
 
 Export your database to a PLEXOS-compatible XML file:

docs/source/howtos/index.md

@@ -22,4 +22,5 @@ manage_relationships
 copy_objects
 bulk_operations
 remove_non_ascii
+inspect_solution
 ```

docs/source/howtos/inspect_solution.md

@@ -0,0 +1,87 @@
+# Inspecting a PLEXOS Solution
+
+This guide shows how to import a PLEXOS solution ZIP file into SQLite and
+inspect its table catalog using the `show_db_tables` helper.
+
+## Converting a solution
+
+Use `PlexosSolution` to import the ZIP into SQLite. Pass
+`decode_bin_values=False` to skip writing BIN payload data — the table catalog
+only needs the XML metadata tables and is then fast even for large solutions:
+
+```python
+from plexosdb import PlexosSolution, show_db_tables
+
+sol = PlexosSolution.from_zip("my_solution.zip")
+sol.to_sqlite("output.sqlite", if_exists="replace", decode_bin_values=False)
+```
+
+## Printing the table catalog
+
+Call `show_db_tables` directly on the `PlexosSolution` instance after
+`to_sqlite()` has been called:
+
+```python
+show_db_tables(sol)
+```
+
+### Example output
+
+```text
+┌───────────────────────────┬──────────────┬──────────────────────────────────────────────────────────┬────────────┬──────────────────────────────┬──────────────────────┬───────────────────────────┬──────────────────────────┬────────────────────────┬────────────────────┬──────────┬───────────────┬───────────────┐
+│       table_catalog       │ table_schema │                        table_name                        │ table_type │ self_referencing_column_name │ reference_generation │ user_defined_type_catalog │ user_defined_type_schema │ user_defined_type_name │ is_insertable_into │ is_typed │ commit_action │ TABLE_COMMENT │
+│          varchar          │   varchar    │                         varchar                          │  varchar   │           varchar            │       varchar        │          varchar          │         varchar          │        varchar         │      varchar       │ varchar  │    varchar    │    varchar    │
+├───────────────────────────┼──────────────┼──────────────────────────────────────────────────────────┼────────────┼──────────────────────────────┼──────────────────────┼───────────────────────────┼──────────────────────────┼────────────────────────┼────────────────────┼──────────┼───────────────┼───────────────┤
+│ Model model_2012 Solution │ data         │ ST__Interval__Batteries__Generation                      │ BASE TABLE │ NULL                         │ NULL                 │ NULL                      │ NULL                     │ NULL                   │ YES                │ NO       │ NULL          │ NULL          │
+│ Model model_2012 Solution │ data         │ ST__Interval__Batteries__Generation_Capacity             │ BASE TABLE │ NULL                         │ NULL                 │ NULL                      │ NULL                     │ NULL                   │ YES                │ NO       │ NULL          │ NULL          │
+│ Model model_2012 Solution │ data         │ ST__Interval__Batteries__Installed_Capacity              │ BASE TABLE │ NULL                         │ NULL                 │ NULL                      │ NULL                     │ NULL                   │ YES                │ NO       │ NULL          │ NULL          │
+│ Model model_2012 Solution │ data         │ ST__Interval__Batteries__Load                            │ BASE TABLE │ NULL                         │ NULL                 │ NULL                      │ NULL                     │ NULL                   │ YES                │ NO       │ NULL          │ NULL          │
+│ Model model_2012 Solution │ data         │ ST__Interval__Batteries__SoC                             │ BASE TABLE │ NULL                         │ NULL                 │ NULL                      │ NULL                     │ NULL                   │ YES                │ NO       │ NULL          │ NULL          │
+│ Model model_2012 Solution │ data         │ ST__Interval__Generators__Available_Capacity             │ BASE TABLE │ NULL                         │ NULL                 │ NULL                      │ NULL                     │ NULL                   │ YES                │ NO       │ NULL          │ NULL          │
+│ Model model_2012 Solution │ data         │ ST__Interval__Generators__Average_Heat_Rate              │ BASE TABLE │ NULL                         │ NULL                 │ NULL                      │ NULL                     │ NULL                   │ YES                │ NO       │ NULL          │ NULL          │
+│ Model model_2012 Solution │ data         │ ST__Interval__Generators__Capacity_Curtailed             │ BASE TABLE │ NULL                         │ NULL                 │ NULL                      │ NULL                     │ NULL                   │ YES                │ NO       │ NULL          │ NULL          │
+│ Model model_2012 Solution │ data         │ ST__Interval__Generators__Capacity_Factor                │ BASE TABLE │ NULL                         │ NULL                 │ NULL                      │ NULL                     │ NULL                   │ YES                │ NO       │ NULL          │ NULL          │
+│ Model model_2012 Solution │ data         │ ST__Interval__Generators__Emissions_Cost                 │ BASE TABLE │ NULL                         │ NULL                 │ NULL                      │ NULL                     │ NULL                   │ YES                │ NO       │ NULL          │ NULL          │
+│             ·             │      ·       │                            ·                             │     ·      │              ·               │          ·           │             ·             │            ·             │           ·            │         ·          │    ·     │       ·       │       ·       │
+│             ·             │      ·       │                            ·                             │     ·      │              ·               │          ·           │             ·             │            ·             │           ·            │         ·          │    ·     │       ·       │       ·       │
+│             ·             │      ·       │                            ·                             │     ·      │              ·               │          ·           │             ·             │            ·             │           ·            │         ·          │    ·     │       ·       │       ·       │
+│ Model model_2012 Solution │ report       │ ST__Year__Reserves__Price                                │ VIEW       │ NULL                         │ NULL                 │ NULL                      │ NULL                     │ NULL                   │ NO                 │ NO       │ NULL          │ NULL          │
+│ Model model_2012 Solution │ report       │ ST__Year__Reserves__Provision                            │ VIEW       │ NULL                         │ NULL                 │ NULL                      │ NULL                     │ NULL                   │ NO                 │ NO       │ NULL          │ NULL          │
+│ Model model_2012 Solution │ report       │ ST__Year__Reserves__Shortage                             │ VIEW       │ NULL                         │ NULL                 │ NULL                      │ NULL                     │ NULL                   │ NO                 │ NO       │ NULL          │ NULL          │
+│ Model model_2012 Solution │ report       │ ST__Year__Storages__Generation                           │ VIEW       │ NULL                         │ NULL                 │ NULL                      │ NULL                     │ NULL                   │ NO                 │ NO       │ NULL          │ NULL          │
+│ Model model_2012 Solution │ report       │ ST__Year__Storages__Inflow                               │ VIEW       │ NULL                         │ NULL                 │ NULL                      │ NULL                     │ NULL                   │ NO                 │ NO       │ NULL          │ NULL          │
+│ Model model_2012 Solution │ report       │ ST__Year__Storages__Initial_Volume                       │ VIEW       │ NULL                         │ NULL                 │ NULL                      │ NULL                     │ NULL                   │ NO                 │ NO       │ NULL          │ NULL          │
+│ Model model_2012 Solution │ report       │ ST__Year__Storages__Max_Volume                           │ VIEW       │ NULL                         │ NULL                 │ NULL                      │ NULL                     │ NULL                   │ NO                 │ NO       │ NULL          │ NULL          │
+│ Model model_2012 Solution │ report       │ ST__Year__Storages__Pump_Load                            │ VIEW       │ NULL                         │ NULL                 │ NULL                      │ NULL                     │ NULL                   │ NO                 │ NO       │ NULL          │ NULL          │
+└───────────────────────────┴──────────────┴──────────────────────────────────────────────────────────┴────────────┴──────────────────────────────┴────────────┴───────────────────────────┴──────────────────────────┴────────────────────────┴────────────────────┴──────────┴───────────────┴───────────────┘
+  267 rows (20 shown)  13 columns
+```
+
+Rows that do not fit within `max_rows` are replaced by three `·` rows. The
+default limit is 20; pass a different value to show more:
+
+```python
+with client as db:
+    show_db_tables(db, max_rows=50)
+```
+
+## Columns
+
+| Column               | Description                                                                 |
+| -------------------- | --------------------------------------------------------------------------- |
+| `table_catalog`      | Stem of the source ZIP file (used as the catalog name).                     |
+| `table_schema`       | SQLite schema name: `main`, `data`, `report`, or any other attached schema. |
+| `table_name`         | Name of the table or view.                                                  |
+| `table_type`         | `BASE TABLE` for tables; `VIEW` for views.                                  |
+| `is_insertable_into` | `YES` for writable tables, `NO` for views.                                  |
+| `is_typed`           | Always `NO`.                                                                |
+| remaining columns    | `NULL` — present for compatibility with `information_schema.tables`.        |
+
+## Schema meanings
+
+- **`main`** — raw tables imported directly from the PLEXOS solution XML (e.g.
+  `t_class`, `t_object`, `t_membership`).
+- **`data`** — derived result tables that can be materialized from BIN files
+  (e.g. `ST__Interval__Generators__Generation`).
+- **`report`** — same derived tables exposed as views with enriched metadata
+  joins.