Clessig/develop/fix io forcing#2445
Open
clessig wants to merge 17 commits into
Open
Conversation
Co-authored-by: Tharmeekan <tsivalingam@ethz.ch>
…2414) * Handle scalar datetimes in time encoders * Keep single target times one-dimensional --------- Co-authored-by: Christian Lessig <christian.lessig@ecmwf.int>
* ensure samples_per_mini_epoch >= world_size*batch_size * updated warning log
* Fix logging when NaNs occur * fix nan variables being included in terminal logged loss, replace nan by 0 * Removing incorrect reset --------- Co-authored-by: Sebastian Hickman <seb.hickman@gmail.com>
…fied projection logic (#2401) * added arctic region for plotting in stereographic projection and modified projection logic * new projection logic in plotter * restored attention and positional_encoding * added icon, arome, uwc-west regions * removing accidental file --------- Co-authored-by: Matteo Broccoli <mbroccol@login01.leonardo.local> Co-authored-by: iluise <72020169+iluise@users.noreply.github.com>
* Align usage of stream configs in model with dict semantics. * parse stream configs into dictconfig instead of list * Align data sampling to dict semantics, wrap self.streams into self.streams_datasets * change list to dictconfig for streams * fixes * ruffed * fix backward compatibility. * address comments * linter * Adjust incoming fresh code. * fix unit tests, rephrase comment
* multiple ranks in evaluation * shorter version * shorter version * Minor correction to ranks PR (#2420) * Fix minor bug * Minor correction * Linting --------- Co-authored-by: Savvas Melidonis <79579567+SavvasMel@users.noreply.github.com>
* remove dead code from train_logger * fix plot_loss_per_run * fix filter logic * address comments --------- Co-authored-by: Christian Lessig <christian.lessig@ecmwf.int>
* Changes to allow for score animations. Addition possibility for log-scale colorbar * Linting * Edit config files * Change to score map function name, move score function to new utils file, generalize single_animation function * Minor edits * Linting * Minor corrections to clarify the sa variable --------- Co-authored-by: iluise <72020169+iluise@users.noreply.github.com>
Co-authored-by: iluise <72020169+iluise@users.noreply.github.com>
* Delete unnecessary code * Linting
…text in legend entries (#2433) * Made legend font size and columns configurable and removed redundant string from plot legend entries and moved this to title * Reverting unnecessary changes * Put losses per run behind command line arg (typically not used currently)
…data (#2422) * Custom data reader for operational analysis that accounts for actual availability * Reduce mem footprint during training; add DataReaderAnemoiOperan * Config files * Fixed corner case: sample 0 that has no precedessor * Added additional check * Move OperAn data reader to extras readers * Cleaning up * Remove incorrect general setting for base permuation * Remove incorrect time_window_step settings * Linting * Improved encapsulation
github-actions
Bot
added
data
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fix IO with forcing datasets
Issue Number
Checklist before asking for review
./scripts/actions.sh lint./scripts/actions.sh unit-test./scripts/actions.sh integration-testlaunch-slurm.py --time 60