Set up our CMIP6 python environment

This notebook sets up an environment using conda for our CMIP6 analysis.

This allows you to install and use specific versions of python packages, which has three advantages:

  1. You can use newer version of the packages than those available by default on the JASMIN servers.
  2. You can use consistent versions of packages across all your analysis on JASMIN, whether you are using the JASMIN Notebook Service or the JASMIN Scientific Analysis (sci) servers.
  3. You can allow others (including your future self) to configure the same version of packages to ensure reproducibility of your work.

Overview – two options

We can:

  1. Use a pre-installed environment created for the hackathon, which contains up-to-date versions of the most common packages
  2. Create our own environment with specific packages and versions

After running through either of these options, we then need to register our environment of choice with Jupyter, so that we can use it in Jupyter Notebooks.

Option 1: Use a pre-installed environment

The pre-installed environment contains Python 3.8, numpy, scipy, pandas, xrray, iris, matplotlib and cartopy, along with some other common packages.

If you'd like a package added, please let usknow in the #help channel on Slack.

You can find an overview of the packages we've installed (and their exact versions) in the following two files:

/gws/pw/j05/cop26_hackathons/bristol/env/environment.yml
/gws/pw/j05/cop26_hackathons/bristol/env/environment_frozen.yml

Register environment as a kernel

To register this environment with Jupyter (we do so by installing something called a kernel), we just need to run the following command once:

You can either run this command right here in the Jupyter Notebook, or at the command line via SSH (to do so, just take off the ! at the start).

If the command is successful, you'll see a message like Installed cmip6 Jupyter kernel for the hackathon.

You'll now need need to refresh the Jupyter Notebook page in your web browser before you'll see the cmip6 kernel.

Use kernel in our notebook

We now need to tell Jupyter to use the correct kernel when executing code. We need to do this once per notebook that we will be using (not this notebook – you could try, for instance, one of the other example notebooks).

In Jupyter this is called the 'kernel', and can be changed from either the top-right of the screen, or the "Kernel" menu:

Screenshot of kernel change mechanism

and then choosing the name of your kernel from the dropdown list:

Screenshot of kernel change dropdown

Use environment at the command line

If you're working at the command line, then we've made a handy script you can run to activate our hackathon environment.

You can use it by running this command:

source /gws/pw/j05/cop26_hackathons/bristol/activate-env

Option 2: Create our own environment

To create our own environment, we first need to define the packages we want, install them, and then register a kernel as above.

The instructions for this option are covered in a supplementary notebook.

Acknowledgements

By: James Thomas and William Seviour

Last updated: 28th May 2021