Summary and Setup

This lesson is about Interoperability in Climate Sciences.

Target audience

This lesson is intended for researchers in the climate and atmospheric sciences who handle multidimensional NetCDF datasets and wish to make their data and software more reusable by others.

Learning objectives

By the end of this lesson, learners will be able to:

  • Recognize the importance of interoperability for data reuse

  • Identify the key elements of interoperable data format

  • Explain when a NetCDF dataset is interoperable

  • Evaluate adherence of a NetCDF files to CF convention.

  • Access and analyse NetCDF files using OpenDAP and Python

  • Detect interoperability “holes” in a specific data format / data package

Prerequisite

To follow this lesson, learners should already be able to have :

  • Working knowledge in Python (write and execute short scripts in Python)
  • Awareness of NetCDF format

FIXME: Setup instructions live in this document. Please specify the tools and the data sets the Learner needs to have installed.

Data Sets

  • Make sure we the instructors create a toy dataset from existing ones in 4TU.researchdata also hosted in OpenDAP server and publish it again in 4TU.researchdata and OpenDAP server.

  • We are considering :

    SH 

     - IDRA datasets - https://data.4tu.nl/articles/_/12696887/1 CCO license

 - https://ruisdael-observatory.nl/data/ NOT available yet in 4TU.ResearchData

We want to modify one real dataset , such as it gets less complex and “more incomplete” for the part of reading and manipulating the dataset.

Why: The IDRA dataset has a license to allows us to reuse and also it has bigger size that we want to showcase how we can retrieve data stored in the web via a protocol that do not require the download of the data.

Implications:

  • We need to decide on which steps to take to modify the real good dataset.

  • We need to create the toy data and stored it back to 4tu to be able to showcase the web retrieval.

Software Setup

Python 3 Distribution

To download the latest Python 3 distribution for your operating system, please head to Python.org.

If you are on Linux, it is likely that the system Python 3 already installed will satisfy the requirements of this course (the material has been tested using the standard Python distribution version 3.11 but any supported version should work).

The course uses venv for virtual environment management and pip for package management. The material has not been extensively tested with other Python distributions and package managers, but most sections are expected to work with some modifications. For example, package installation and virtual environments would need to be managed differently, but Python script invocations should remain the same regardless of the Python distribution used.

You can test your Python installation from the command line with:

BASH 

$ python3 --version # on Mac/Linux
$ python --version # on Windows — Windows installation comes with a python.exe file rather than a python3.exe file

If you are using Windows and invoking python command causes your Git Bash terminal to hang with no error message or output, you may need to create an alias for the python executable python.exe, as explained in the troubleshooting section.

If all is well with your installation, you should see something like:

OUTPUT 

Python 3.11.4

To make sure you are using the standard Python distribution and not some other distribution you may have on your system, type the following in your shell:

BASH 

$ python3 # python on Windows

This should enter you into a Python console and you should see something like:

BASH 

Python 3.11.4 (main, Jun 20 2023, 17:23:00) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information. 
>>>

Press CONTROL-D or type exit() to exit the Python console.

venv and pip

If you are using a Python 3 distribution from Python.org, venv and pip will be automatically installed for you. If not, please make sure you have these two tools (that correspond to your Python distribution) installed on your machine.

IDE

VS Code

Alternatively, you can use Microsoft’s Visual Studio Code (VS Code).