2. Technical FAQ

2.1. How do I check the timing and cost of my experiments?

You will find a detailed overview of the model cost, throughput and run time for each model component in a subfolder named timing in the case folder (e.g. in ~/NorESM/cases/<casename>/timing/) The filename is on the form cesm_timing.<casename>.JOBID.yymmdd-hhmmss

2.2. My model experiment/simulation crashes and I don’t know why!

If you experience crashes while setting up, building, submitting or archiving the experiment, you may find useful information in the log file in the casefolder (e.g. in ~/NorESM/cases/<casename>/). The filename is the same as the 15 first letters of the casename with no file extension. If you don’t find a solution to your problem please post your question here: https://github.com/NorESMhub/NorESM/discussions

2.3. How do I know how many cpu hours I have left?

The terminal command cost will provide a detailed overview of available and used cpu hours for each project you are assigned as a user.

2.4. How do I know how much storage space I am using?

On NIRD and Betzy you can use the command dusage in the terminal. You can also get detailed information about specific projects by using dusage -p <project_name>

dusage -p nnxxxk

or

dusage -p nsxxxk

Another useful command to get information about the files is the find command. Typing

find $dirs  -type f -printf “%u %s hlink=%n %h/%f\n” > filelist.txt

in the terminal, the filelist.txt will contain the size and file number usage of every member in the project. The find command may take 30 minutes or more to run in large project spaces.

To find the total number of files for directory including subdirectories

find . -type f | wc -l

To find the total number of files for all subdirectories separately

find . -maxdepth 1 -type d | while read -r dir; do printf "%s:\t" "$dir"; find "$dir" -type f | wc -l; done

For getting info an all your projects (e.g. ns9252k ns9560k ns2345k), you may write a short bash script

#!/bin/bash
declare -a grps=(ns9252k ns9560k ns2345k)
count=${#grps[@]}
counter=0
while [ $counter -lt ${count} ]; do
    dusage -p ${grps[$counter]} | awk '{print substr($0, 1, 10) substr($0, 40, 32) }' | grep -v '\-\-'
    counter=$(( counter+1 ))
done

2.5. Manage externals

I get a Dictionary keys changed error when checking out externals in my cloned repository.

Error:

./manage_externals/checkout_externals
Processing externals description file : Externals.cfg
Checking status of externals: cam, dictionary keys changed during iteration

Solution:

Change to an older python version <= 3.7. If you have activated a conda environtment, you can deactivate conda (i.e. type deactivate conda in the terminal) when creating, building and submitting a case. The default python versions on BETZY and FRAM is 2.7.5 and will not create such errors.

2.6. Creating a case

I get a SyntaxError: invalid syntax error when creating a case in my cloned repository.

Error:

./create_newcase --case ../../../cases/$CASENAME --mach fram --res f19_tn14 --compset NHIST
Traceback (most recent call last):
  File "./create_newcase", line 9, in <module>
    from CIME.case          import Case
  File "/cluster/projects/nn2345k/$user/<noresm-base>/cime/scripts/Tools/../../scripts/lib/CIME/case/__init__.py", line 1, in <module>
    from CIME.case.case import Case
  File "/cluster/projects/nn2345k/$user/<noresm-base>/cime/scripts/Tools/../../scripts/lib/CIME/case/case.py", line 41, in <module>
    class Case(object):
  File "/cluster/projects/nn2345k/$user/<noresm-base>/cime/scripts/Tools/../../scripts/lib/CIME/case/case.py", line 72, in Case
    from CIME.case.case_submit import check_DA_settings, check_case, submit
  File "/cluster/projects/nn2345k/$user/<noresm-base>/cime/scripts/Tools/../../scripts/lib/CIME/case/case_submit.py", line 33
    print "limit0",resource.getrlimit(resource.RLIMIT_STACK)
          ^
SyntaxError: invalid syntax

Solution:

Change to an older python version. If you are working in an active conda environtment, you can deactivate conda (i.e. type deactivate conda in the terminal) when creating, building and submitting a case. The default python versions on BETZY and FRAM is 2.7.5 and will not create such errors.