Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
133 commits
Select commit Hold shift + click to select a range
093a257
Change in escape/wrapper.py to put a warning if M_vols too low
Apr 8, 2025
1da9cb2
Merge branch 'main' into ep/bug_escape
EmmaPostolec Apr 18, 2025
13d9272
Minor doc update
EmmaPostolec Apr 18, 2025
ff6bae5
Add toi561b toml file in input/planets
EmmaPostolec Apr 18, 2025
3ce1066
toi561b toml file
EmmaPostolec Apr 18, 2025
7112343
update
EmmaPostolec Apr 18, 2025
f426496
Add the post_processing_grid.py script and create a new folder to tes…
EmmaPostolec Apr 18, 2025
22e7fd9
Add histo plot
EmmaPostolec Apr 18, 2025
cc4cac9
Move post_processing_grid folder to tools
EmmaPostolec Apr 18, 2025
e6c59ed
try to do plot function : fail
EmmaPostolec Apr 18, 2025
8607942
Try to save extracted data in a csv file : still working on it
EmmaPostolec Apr 21, 2025
c944fcb
Save into csv ok now need to clean script
EmmaPostolec Apr 21, 2025
c453f46
post processing grid : generate csv file and then single plots. still…
EmmaPostolec Apr 21, 2025
2bacac7
try for grid plot. need to fix log single plots
EmmaPostolec Apr 22, 2025
06a68e8
Merge branch 'main' into ep/post_processing_grid
EmmaPostolec Apr 22, 2025
36858ca
Clean and clear comment in post_processing_grid.py
EmmaPostolec Apr 23, 2025
f3dc067
clean plot_grid.py
EmmaPostolec Apr 23, 2025
62057ea
Minor update to make it work fro another grid
EmmaPostolec Apr 23, 2025
abec185
Merge branch 'main' into ep/post_processing_grid
EmmaPostolec Apr 29, 2025
bc38fe1
Update toi561b.toml with johanna updated value in green in the table
EmmaPostolec May 1, 2025
493e68a
maybe i fixed the log scale problem youhou
EmmaPostolec May 1, 2025
63b3022
Almost there with log plots still needs to fix pb
EmmaPostolec May 2, 2025
d768f6b
Merge branch 'main' into ep/post_processing_grid
EmmaPostolec May 2, 2025
a04154d
Merge branch 'main' into ep/post_processing_grid
EmmaPostolec May 6, 2025
f35b076
test vs code commit
EmmaPostolec May 6, 2025
ed4b02e
minor changes
EmmaPostolec May 7, 2025
a361a7a
Merge branch 'main' into ep/post_processing_grid
EmmaPostolec May 7, 2025
0d44975
Merge branch 'main' into ep/post_processing_grid
EmmaPostolec May 7, 2025
c000b1a
small test to have consistent plot
EmmaPostolec May 8, 2025
2abe169
try to fix error log : now using ecdfplot for a single plot and its w…
EmmaPostolec May 11, 2025
d116058
clean a bit plot_grid.py
EmmaPostolec May 11, 2025
35779ea
plot for solidification time ok save before looping on all output
EmmaPostolec May 11, 2025
551d0cf
Single plots for all input param and output OK
EmmaPostolec May 11, 2025
7c93ab5
plot grid ok !!!
EmmaPostolec May 11, 2025
bbd119f
latest change before update main
EmmaPostolec May 23, 2025
2791d9b
Merge branch 'main' into ep/post_processing_grid
EmmaPostolec May 23, 2025
e8e2a2b
re-organize the code for easier use
EmmaPostolec Jun 2, 2025
02058b3
Merge branch 'main' into ep/post_processing_grid
EmmaPostolec Jun 2, 2025
bf4c5af
remove all files related to this branch from tools/
EmmaPostolec Jun 2, 2025
2077467
re-organize output + try to fix grid plot
EmmaPostolec Jun 2, 2025
dea5298
grid plot ok :)
EmmaPostolec Jun 2, 2025
38f65f4
extract more outpout from grid and test plots for different outpouts
EmmaPostolec Jun 3, 2025
598e6da
creating the proteus grid-analyze command and it worked yeah
EmmaPostolec Jun 3, 2025
d6a07ab
update doc with the proteus grid-analyze command
EmmaPostolec Jun 3, 2025
4cd918f
add the csv_error_cases but still need to work on this
EmmaPostolec Jun 5, 2025
f389ccb
commit by copilot ai : Update src/proteus/cli.py base
EmmaPostolec Jun 6, 2025
c2d8bcb
commit by copilot ai : Update input/planets/toi561b.toml
EmmaPostolec Jun 6, 2025
1573e04
commit copilot ai : Update docs/usage.md
EmmaPostolec Jun 6, 2025
408a6ef
commit copilot AI : Update docs/usage.md
EmmaPostolec Jun 6, 2025
0f07412
move the big run_grid_analyze funciton to the run_analysis_grid.py sc…
EmmaPostolec Jun 6, 2025
f8eb629
update_docs
EmmaPostolec Jun 6, 2025
ed410f1
try to fix the test
EmmaPostolec Jun 6, 2025
2f076d4
add pkl to gitignore
EmmaPostolec Jun 6, 2025
1c698f9
add seaborn as a dependency in pyproject.toml
EmmaPostolec Jun 6, 2025
fb2a907
Fix pre-commit hook issues: trailing whitespace, end-of-file, ruff
EmmaPostolec Jun 6, 2025
c0db852
try to fix cide style to pass the tests
EmmaPostolec Jun 6, 2025
2eca57e
fix import cm
EmmaPostolec Jun 11, 2025
2280407
input file to run toi561b grids + addition of tested parameters for t…
EmmaPostolec Aug 20, 2025
14f244c
resolve merge conflict before merging main
EmmaPostolec Aug 21, 2025
cb1b177
update input and grid submission files with latest main updates
EmmaPostolec Aug 26, 2025
9d8d21d
update files for more efficient grid runs
EmmaPostolec Aug 27, 2025
8b2be29
minor change toml files for grid runs
EmmaPostolec Aug 27, 2025
8a25af0
changes for toi561b grid with correct stellar spectrum
EmmaPostolec Aug 28, 2025
fcf8c67
need to run the grid paper 1 and toi561b
EmmaPostolec Oct 9, 2025
581e554
update to fix stellar spectrum error with janus
EmmaPostolec Oct 20, 2025
9d1e4a8
Merge branch 'main' into ep/post_processing_grid
EmmaPostolec Oct 20, 2025
1bc0839
Merge branch 'main' into ep/post_processing_grid
EmmaPostolec Oct 21, 2025
dec87d2
Merge branch 'main' into ep/post_processing_grid
EmmaPostolec Oct 21, 2025
06a43f9
Merge branch 'ep/janus_fix_escape' into ep/post_processing_grid
EmmaPostolec Oct 21, 2025
372c24d
Merge branch 'ep/janus_fix_escape' into ep/post_processing_grid
EmmaPostolec Oct 21, 2025
26bd0e3
Merge branch 'ep/janus_fix_escape' into ep/post_processing_grid
EmmaPostolec Oct 22, 2025
5f0f6e3
Merge branch 'main' into ep/post_processing_grid
EmmaPostolec Oct 22, 2025
5b4fa57
create a new csv file with only completed cases and add number in the…
EmmaPostolec Oct 26, 2025
ee39aa6
changes for plots paper 1 add ecdf species grid plot
EmmaPostolec Oct 28, 2025
c9ca7e4
update ecdf grid function to plot only values > 0 for species plot
EmmaPostolec Oct 28, 2025
4abed76
Merge branch 'main' into ep/post_processing_grid
EmmaPostolec Oct 28, 2025
3f3ab41
Merge branch 'main' into ep/post_processing_grid
EmmaPostolec Oct 31, 2025
37e1b9a
Merge branch 'main' into ep/post_processing_grid
EmmaPostolec Nov 11, 2025
bb957c0
mini updates before mergin main
EmmaPostolec Dec 10, 2025
f12f309
Fix usage for merging
EmmaPostolec Dec 10, 2025
2ecbd7c
test to update the post-processing script to new proteus version. nee…
EmmaPostolec Dec 11, 2025
84930fc
update toi561b file and new try to update the post processing steps
EmmaPostolec Dec 15, 2025
32183d4
generate csv file following harrison guidelines
EmmaPostolec Dec 15, 2025
145ce1c
plot status grid ok
EmmaPostolec Dec 15, 2025
6d12a53
almost there for ecdf
EmmaPostolec Dec 15, 2025
3f3d641
create toml grid analyze and use it for ecdf plot, clean and update f…
EmmaPostolec Dec 17, 2025
4be722a
update the command and doc
EmmaPostolec Dec 17, 2025
a8238f9
update minor typos
EmmaPostolec Dec 17, 2025
ebe169e
update doc
EmmaPostolec Dec 17, 2025
48f7eb7
Merge branch 'main' into ep/post_processing_grid
EmmaPostolec Dec 17, 2025
64febce
Remove folder_list file and simulations configuration files for my grid
EmmaPostolec Dec 17, 2025
0c61448
Update src/proteus/grid/post_processing.py
EmmaPostolec Dec 22, 2025
a75fbc5
Update src/proteus/grid/post_processing.py
EmmaPostolec Dec 22, 2025
fa577db
update toi561b time step
EmmaPostolec Feb 6, 2026
2ea72bb
Merge branch 'main' into ep/post_processing_grid
EmmaPostolec Mar 10, 2026
48267f0
Merge branch 'main' into ep/post_processing_grid
EmmaPostolec Mar 10, 2026
663658c
update
EmmaPostolec Mar 19, 2026
d54ed2a
Merge branch 'main' into ep/post_processing_grid
EmmaPostolec Mar 19, 2026
99a3c83
update the generate_csv funciton to avoid redundancy as suggested by …
EmmaPostolec Apr 7, 2026
08bac68
update doc of group_output_by_parameter funvtion as suggested by copilot
EmmaPostolec Apr 7, 2026
bf1b8fc
update doc of ecdf_grid_plot function
EmmaPostolec Apr 7, 2026
91ce43a
context manager in ecdf_grid_plot to close toml file for copilot issue
EmmaPostolec Apr 7, 2026
03ea9d3
update doc
EmmaPostolec Apr 7, 2026
df59de7
clean extra docstring in post_processing.py
EmmaPostolec Apr 7, 2026
d97a5bf
update docstring latex function
EmmaPostolec Apr 7, 2026
aefbc03
remove grid_analyse_toml_file = None by default
EmmaPostolec Apr 7, 2026
5877426
typo in example.grid_analyse.toml
EmmaPostolec Apr 7, 2026
9e879a9
update legend fO2 plot and update cli command to be consitent with PR…
EmmaPostolec Apr 7, 2026
21b3113
update doc with latest cli changes
EmmaPostolec Apr 7, 2026
25b30be
allow the user to choose the format of plot png or pdf
EmmaPostolec Apr 7, 2026
134833c
remove printing statement
EmmaPostolec Apr 7, 2026
acf29fc
Merge branch 'main' into ep/post_processing_grid
EmmaPostolec Apr 7, 2026
92348f6
update grid path from same config file and add preset_labels
EmmaPostolec Apr 7, 2026
36dcf4b
update the label for plotting
EmmaPostolec Apr 7, 2026
2651831
trying to get input param from grid config file
EmmaPostolec Apr 7, 2026
3bfaa36
update so input param are not duplicate for ecdf grid plot
EmmaPostolec Apr 8, 2026
875dee9
update scale log scale preset and update post_processing so it works …
EmmaPostolec Apr 8, 2026
a8196a1
clean test example.toml files, update docs and create succesfull pytest
EmmaPostolec Apr 8, 2026
e9f2f32
ruff changes to pass tests
EmmaPostolec Apr 8, 2026
04c654a
fix colormap problem
EmmaPostolec Apr 8, 2026
1c51c2a
fix: read colormap from top-level cfg key and fix ruff lint in platon.py
Copilot Apr 8, 2026
a398c59
fix removing 0 when it shouldnt with clean_series
EmmaPostolec Apr 8, 2026
10b92cd
fix mistake unit rho_obs in plot.py
EmmaPostolec Apr 8, 2026
4d8fc82
update the grammar in _escape.py
EmmaPostolec Apr 8, 2026
c635379
update doc mistake
EmmaPostolec Apr 8, 2026
77356bf
doc typo
EmmaPostolec Apr 8, 2026
ace2dac
comment for copilot on no argument in main
EmmaPostolec Apr 8, 2026
3d84a2b
to avoid copilot warning about not passing argument to main
EmmaPostolec Apr 8, 2026
5b6ce81
Merge branch 'ep/post_processing_grid' of github.com:FormingWorlds/PR…
EmmaPostolec Apr 8, 2026
5738d61
typo _escape.py
EmmaPostolec Apr 8, 2026
6e6ead5
ruff format on src/tests/ to pas test
EmmaPostolec Apr 8, 2026
1d8978a
test: add unit tests for post_processing helper functions
Copilot Apr 8, 2026
e65c564
test: address code review feedback on unit tests
Copilot Apr 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -274,3 +274,6 @@ tests/validation/
CLAUDE.md
.claude/
.mcp.json

# pickle files
*.pkl
33 changes: 33 additions & 0 deletions docs/How-to/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,39 @@ Use the CLI to package the results of a grid into a zip file; e.g. for sharing o
```console
proteus grid-pack -o output/grid_demo/
```
## Postprocessing of grid results

Results from a PROTEUS grid can be post-processed using the `proteus grid-analyse` command. Running the analysis generates summary CSV files containing all tested input parameters and the final time-step outputs for each simulation, along with a status overview plot showing the distribution of simulation outcomes (e.g., completed, running, error) and empirical cumulative distribution function (ECDF) plots that summarize selected output variables across all completed simulations. (For more details on ECDF plots, see the [Seaborn `ecdfplot` documentation](https://seaborn.pydata.org/generated/seaborn.ecdfplot.html).)

Before running the command, configure the post-processing options in your grid configuration file:`input/ensembles/example.grid.toml`. The following options can be set:

- `update_csv` — Generate or update the CSV summary files
- `plot_format` — Output format for plots (e.g., `png`, `pdf`)
- `plot_status` — Enable/disable the status summary plot
- `plot_ecdf` — Enable/disable ECDF plot generation
- `output_variables` — List of variables to include in ECDF plots

> **Note:**
> The variables specified in `output_variables` must match column names in `runtime_helpfile.csv`.
> The `solidification_time` is the only variable computed during post-processing and is not directly stored in the simulation output.

---

To post-process a grid and generate ECDF plots for further analysis, run the following command:

```
proteus grid-analyse --config input/ensembles/example.grid.toml
```
Comment thread
EmmaPostolec marked this conversation as resolved.


After execution, a `post_processing/` directory is created inside the grid folder with the following structure:

- `extracted_data/` This directory contains three CSV files:
- `{grid_name}_final_extracted_data_all.csv` which includes every run in the grid
- `{grid_name}_final_extracted_data_completed.csv` which contains only successful runs (used for ECDF plots)
- `{grid_name}_final_extracted_data_running_error.csv` for only failed simulations with status `Running` or `Error`.

- `grid_plots/` This directory contains a status summary plot and a ECDF plot
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grammar: “a ECDF plot” should be “an ECDF plot”.

Copilot uses AI. Check for mistakes.

## Retrieval scheme (Bayesian optimisation)

Expand Down
12 changes: 11 additions & 1 deletion input/ensembles/example.grid.toml
Original file line number Diff line number Diff line change
@@ -1,11 +1,21 @@
# Config file for running a grid of forward models

# Path to output folder where grid will be saved (relative to PROTEUS output folder)
output = "grid_demo/"
output = "scratch/grid_demo/"

# Make `output` a symbolic link to this absolute location. To disable: set to empty string.
symlink = ""

# Post-processing options
update_csv = true # Whether to update the summary CSV files before plotting
plot_format = "png" # Format for saving plots ("png" or "pdf")
plot_status = true # Generate status summary plot of the grid
plot_ecdf = true # Generate ECDF grid plot for input parameters tested in the grid and output_variables defined below
colormap = "berlin" # Colormap for ECDF plot (e.g. "viridis", "managua")
output_variables = ["solidification_time", "Phi_global",
"T_surf", "P_surf", "atm_kg_per_mol",
"esc_rate_total", "rho_obs"] # List of output variables to include in ECDF plot (name must match variable name in runtime_helpfile.csv)

# Path to base (reference) config file relative to PROTEUS root folder
ref_config = "input/demos/dummy.toml"

Expand Down
465 changes: 465 additions & 0 deletions input/planets/toi561b.toml

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ dependencies = [
"ruff",
"sympy",
"astropy",
"seaborn",
"torch",
"botorch",
"gpytorch",
Expand Down
18 changes: 18 additions & 0 deletions src/proteus/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -554,6 +554,24 @@ def observe(config_path: Path):
cli.add_command(offchem)
cli.add_command(observe)

# ----------------
# 'grid_analyse' postprocessing commands
# ----------------


@click.command()
@config_option
def grid_analyse(config_path: Path):
"""Generate grid analysis plots and CSV summary files from a grid
config_path : Path to the toml file containing grid analysis configuration
"""
from proteus.grid.post_processing import main

main(config_path)


cli.add_command(grid_analyse)
Comment thread
nichollsh marked this conversation as resolved.

# ----------------
# GridPROTEUS and BO inference scheme, runners
# ----------------
Expand Down
2 changes: 1 addition & 1 deletion src/proteus/config/_escape.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ class Zephyrus:
Attributes
----------
Pxuv: float
Pressure at which XUV radiation become opaque in the planetary atmosphere [bar]
Pressure at which XUV radiation becomes opaque in the planetary atmosphere (should be above Pxuv > 0 bar) [bar]
efficiency: float
Comment thread
EmmaPostolec marked this conversation as resolved.
Escape efficiency factor
tidal: bool
Expand Down
Loading
Loading