1. ESMValTool

The Earth System Model eValuation Tool (ESMValTool) is a community-development that aims at improving diagnosing and understanding of the causes and effects of model biases and inter-model spread. The ESMValTool can process multiple model outputs, including NorESM, but these model output need to be post-processed to conform with the CMIP data standard (i.e., CMOR). ESMValTool support all the published model data output on the ESGF.

The esmvaltool-on-nird discussion group can be a useful resource if you plan to run ESMValTool on Nird.

1.1. Run ESMValTool on NIRD service node

The ESMValTool is currently installed under the dedicated IPCC service node (ipcc.nird.sigma2.no) for data post-processing. The following NIRD project areas are mounted under /projects of the IPCC node: NS2345K, NS2980K, NS9015K, NS9034K, NS9039K, NS9252K, NS9560K, NS9588K.

1.1.1. Activate and run ESMValTool

An example of steps to run the ESMValTool on ipcc.nird.sigma2.no

  1. login in the ipcc node:

    ssh -l username ipcc.nird.sigma2.no

  2. load ESMValTool on nird:

    conda activate /conda/esmvaltool/2.2.0/

(note, the esmvaltool may be upgraded in the future. Therefore, use ls /conda/esmvaltool/ to check which version is actually installed.)

  1. configure esmvaltool (first time use):

    • esmvaltool requires a configuration file with paths to model and observational data files, and some run instructions. By default, esmvaltool will look for this file in $HOME/.esmvaltool/config-user.yml.

    • esmvaltool can auto-generate a config file, but not the correct path settings, by running esmvaltool config get_config_user.

    • alternatively, create an empty directory in $HOME/.esmvaltool to store your own config files.

  2. copy the following config file to ~/.esmvaltool/:

    cp /projects/NS9252K/share/esmvaltool/config/config-ipcc.yml ~/.esmvaltool/

    • You may copy and modify the config-ipcc.yml file to store some intermediate data files and final plots to your own directory.

    • Rename config-ipcc.yml to config-user.yml if you don’t want to specify the config file for each run.

  3. copy a recipe to ~/your-recipes/:

    mkdir ~/your-recipes
cp /projects/NS9252K/share/esmvaltool/tested_recipes/recipe_validation_CMIP6.yml ~/your-recipes/

  4. run esmvaltool:

    esmvaltool run --config_file=~/.esmvaltool/config-ipcc.yml ./your-recipes/recipe_validation_CMIP6.yml

  5. Check for results. The output directory setting in config-ipcc.yml is directed to a web directory under the NS2345K project:

    /projects/NS2345K/www/diagnostics/esmvaltool/$USER/tmp

Sample results are under:

https://ns2345k.web.sigma2.no/diagnostics/esmvaltool/

1.1.2. Find sample recipes, model data and observational data

Shared resources for esmvaltool on the IPCC node is available from:

/projects/NS9252K/share/esmvaltool/

The config/config-ipcc.yml file should provide paths to relevant data on Nird. These settings are probably adequate for most users, but can be altered to include additional model output or observational datasets from other sources. Model data are stored in a directory structure following the DKRZ convention. A list of tested recipes is available under the tested_recipes folder.

auxiliary_data/

Auxiliary data needed to run some esmvaltool recipes, e.g. shapefiles for map plotting or data extraction.

config/

User and developer config files.

tested_recipes/

Some recipes that have been tested with esmvaltool for the IPCC node installation with NorESM1/2 supported.

1.1.3. Download data automatically with Synda

Synda is a tool to download and manage model data form the ESGF, it can be called by the esmvaltool as a command line option --synda_download=TRUE so that it can automatically download necessary model data as specified in the receipe of ESMValTool. For example,

esmvaltool run --config_file=~/.esmvaltool/config-ipcc.yml ./recipe_seaice.yml --synda_download=TRUE

Note, it is OPTIONAL to use synda to download data automatically during the runtime of ESMValTool. One can download the data by some other tools or scripts before running the ESMValTool.

If you do want to use synda, there are two major steps you need to do:

Activate the tool

It is installed with conda, so you can add synda exectable to your search path, e.g.,

export PATH:$PATH:/projects/NS9252K/conda/synda/bin/synda

Then the synda command should be available in your searching path.

Set up your credentials

  1. register (one of) the ESGF node and Globus transfer (not toally sure if account on globus is mandatory if this option will not be used, but seem yes according to my experience, you can firstly try without it)

  2. paste your username and password to in ~/.synda/conf/credentials.conf

  3. configure synda parameters in ~/.synda/conf/sdt.conf, use my setting as template.

  4. maybe you need to get globus token. Run the following command:

    synda token -p globus renew

it will give out something like:

Native App Authorization URL:
https://auth.globus.org/v2/oauth2/authorize?code_challenge=BrmiBhFVVuHVNyGDj6hn5N8M1-EKJNnNgptobIsbTqI&state=_default&redirect_uri=https%3A%2F%2Fauth.globus.org%2Fv2%2Fweb%2Fauth-code&response_type=code&client_id=83ec00c1-e67a-4356-9f1f-f7e31177e31a&scope=openid+email+profile+urn%3Aglobus%3Aauth%3Ascope%3Atransfer.api.globus.org%3Aall&code_challenge_method=S256&access_type=offline
Enter the auth code:

Paste the above https address to browser, and you will find a authen code, and past back to the command line.

1.2. Run ESMValTool on NIRD toolkit service

The ESMValTool is also available from NIRD Toolkit applications (Jupyter and JupyterHub), provided by Docker images.

NIRD Toolkit is a cloud infrastructure that gives access to compute nodes on Nird. The service is managed by Kubernetes which launches applications from Docker containers.

Access to the NIRD Toolkit service is provided according to a 3 tier ranking

  1. Owner : a project leader/executive officer of a NSxxxxK project

  2. Admin : a person authorized by an owner to manage NIRD Toolkit applications

  3. Member : a person authorized by the owner or admin to access a NIRD Toolkit application

In order to use NIRD Toolkit, an owner of of a NSxxxxK project should create a group in dataporten, and invite members to join, who may be given admin status by the owner or existing admin people. See Sigma2 information page about deploing service for details about the procedure.

1.2.1. Install and run an application

Applications in NIRD Toolkit are available from:

https://apps.sigma2.no/nird

An owner/admin can launch a new application by installing it, and request resources to be made available to the application via the Kybernetes management system. A member can launch an existing application that has previously been set up by an owner/admin. The application setup allows access to storage areas under NSxxxxK storage volumes (read-only by default) and a user storage area under a specific NSxxxxK/subfolder with write access, but only applications pre-defined in the docker image provided to the Kybernetes system can be used.

1.2.2. Run a pre-installed jupyterhub application (e.g. EOSC jupyterhub on nird)

ESMValTool is not included in the default docker images provided by Sigma2, but has been installed in modified docker images. These are created by building ESMValTool on top of an official Sigma2 docker image, and package in a new docker container. Such modified docker images are available for jupyter and jupyterhub applications.

A jupyterhub application with pre-installed esmvaltool is available at

  1. Launch a terminal, esmvaltool is designed to run from the command line.

  2. Activate the esmvaltool environment:

    conda activate esmvaltool

  3. Check that esmvaltool is working, and produce output:

    esmvaltool version

To run the pre-installed jupyterhub aplication, a user first needs to become a member of the user group. To get access to the hub please contact annefou[at]geo.uio.no <mailto:annefou@geo.uio.no> for the invitation URL link, or the noresm-core group.

To join the group, and get access to the jupyterhub, follow instuctions in email.

1.2.3. Install an ESMValTool docker image from source

  1. jupyterhub: nordicesmhub/jupyterhub-nird-toolkit

  2. jupyterhub: tomastorsvik/nird_jupyterhub-singleuser_esmvaltool

  3. jupyter: tomastorsvik/nird_jupyter-spark_esmvaltool

To install the docker image you need access to the Nird Toolkit service through a project, and install a new instance of e.g. “jupyter”. On the installation page you select whatever standard settings you like (application name, projectspace, persistent storage, machine type), and then select “Show advances configuration…” and replace the standard dockerimage with:

tomastorsvik/nird_jupyter-spark_esmvaltool:latest

One can activate the “JupyterLab” option in the configuration (but it is may not necessary to do so to run ESMValTool). To test if the image is installed correctly (provided the jupyter notebook environment builds without errors) you can open a terminal in the environment and type esmvaltool -h, which should give you the help page for the tool in the terminal window.

As the development of ESMValTool and the updating of the NIRD Toolkit base system are not syncronous, there may sometimes be conflits in the package dependencies of what ESMValTool wants and what NIRD Toolkit provides. Hence, it is sometimes necessary to build esmvaltool in a separate conda environment (not “base”). If esmvaltool -h fails to give the expected output, check if there exist any alternative environments by doing

conda env --list

If you find an esmvaltool environment, this can be activated by

source activate esmvaltool

At the time of writing (10 June 2020), the preferred activation method conda activate esmvaltool is not recognized inside a NIRD Toolkit application.

(Update: 8 March 2021, conda activate esmvaltool work on the https://eosc-nordic.uiogeo-apps.sigma2.no.)