Cleanup and Restructure of Exported Results and Stored Networks

[KathiEsterl]

At the moment, several intermediate results are exported multiple times, sometimes with misleading naming/descriptions, creating a risk of confusion with final results.
The handling of networks within the etrago object also needs cleanup and clearer semantics.

Proposed restructuring of the results export:

results/
├── README.md
├── args.yml (or settings.yml?)
├── spatial_aggregation.json (busmap)
├── original_grid_topology/
├── premarket_optimization/
├── market_optimization/
├── grid_optimization/
├── temporal_disaggregation/
├── non_linear_powerflow/
└── disaggregation/

• README.md: gives an overview of all exported files and folders, explains which step produced which outputs.
• args.yml: stores the calculation settings corresponding to the run, alternative naming could be discussed if more suitable.
• spatial_aggregation.json: Explicitly documents the spatial clustering mapping.
• original_grid_topology/: contains the original grid topology export with one snapshot (former: clustering/)
• premarket_optimization/
• market_optimization/
• grid_optimization/
• temporal_disaggregation/
• non_linear_powerflow/
• disaggregation/

Folders should only contain PyPSA network CSV exports.
Naming and semantics should clearly distinguish intermediate from final results.

Additional TODOs:

• etrago.network should always reference the latest/current network state.
• etrago.market_model and etrago.grid_opti_model should remain available for final result comparison.
• The import function should automatically determine and load the “most final” available network based on the calculation workflow/order above.
• Evaluate whether NetCDF (.nc) files should be used instead of CSV table exports.

[ClaraBuettner]

Two thoughts on this:

• With the current implementation, a the overall result folder is > 80GB. So it would be good to find some ways to reduce it.
  I tried it with one disaggregated_network: When using csv files, the size is about 10GB, when using netcdf, it is 6GB. The export is also quicker. So I guess it makes sense to switch to netcdf.
• We should take into account that eGo currently uses etrago.disaggregated_network. If we change where the disaggregated network in etrago is stored, we need to change it in eGo accordingly.