high performance computing

Here a brief introduction is provided for how to run job on high performance computing clusters (supercomputers), with JURECA used as an example.

Information on JURECA

JURECA utilises SLURM as workload manager. A manual on how to use SLURM with JURECA can be found here.

JURECA has 24 cores per node, thus jobs should use multiple of 12, or 24, cores. As long as 24, or less, jobs are given to a node, only physical cores are used, otherwise hyperthreading will be invoked.

Information on how to connect to JURECA via ssh can be obtained here

Quick reminder: Files can be copied directly from Linux command line, for instance, by utilising scp as follows (see):

# copy from local machine to remote machine
# single file
scp localfile user@host:/path/to/where/you/want/the/file

# directory
scp -r directory user@host:/path/to/where/you/want/the/directory

or

# copy from remote machine to local machine
# single file
scp user@host:/path/to/remote/file localfile

# directory
scp -r user@host:/path/to/remote/directory localdirectory

Running PROPTI on JURECA

Setting up SPOTPY

PROPTI itself does not contain optimisation algorithms, therefore the SPOTPY library needs to be available. You can install SPOTPY locally, i.e. in your home directory (~/.local/lib) with the command below or it could be made available as a module.

pip install --user spotpy

In the further example, the assumption is SPOTPY's availability as a module (see below).

Setting up PROPTI

PROPTI needs to be available as well, by cloning the git repository.

git clone https://github.com/FireDynamics/propti.git

The JURECA system supports environment modules, which allow to choose various versions of software, like compilers, MPI and other libraries.

Once you are logged in, you can list the available modules (just for information) with

module avail

These modules can be loaded manually. There is a script containing all needed modules (jureca.modules) to load all at once. You should check if all modules load correctly before proceeding, for example by sourcing the script:

source path/to/propti/resources/jureca/jureca.modules

For convenience, the modules have been included into the job script.

Setting up FDS

For parallel execution of PROPTI a serial version of FDS would be needed, because SPOTPY and FDS both use MPI. This leads to conflicts. Therefore, the execution of FDS needs to be encapsulated in a wrapper script in order to decouple the execution environments. Instead of calling the FDS executable directly, the PROPTI input file should point to said wrapper script.