Instructor Notes

Example of a note

Introduction

Instructor Note

It might be a good time to survey the participants to see how many of them have:

  • heard of NetCDF format before (n.b., it’s a prerequisite of the workshop)

  • have experience working with NetCDF format.


Instructor Note

This exercise is for discussion in Plenum and it serves as a good link to the next section.


Instructor Note

Examples of barriers to reuse datasets might include:

  • Missing metadata

  • Non-standard units or unclear variable names

  • File formats you could not easily open

  • Access restrictions or unstable URLs

  • Large data volumes and inefficient download workflows

  • Difficulty aligning datasets from multiple institutions

  • Lack of documentation on coordinate systems or time conventions

  • Inconsistent versions or unclear provenance

This discussion sets up the motivation for the rest of the workshop: practical, hands-on methods to make interoperable data using real tools such as NetCDF, CF conventions, and OPeNDAP.


Structural interoperability

Semantic interoperability

Technical interoperability: Data access protocols

Instructor Note

In most cases, a warning is shown. This warning is normal when using pydap with a THREDDS OPeNDAP server. It is not an error and your dataset should still load correctly. The warning simply means that PyDAP could not detect whether the server supports DAP2 or DAP4, so it defaults to DAP2, which is the older protocol.

The OPeNDAP protocol has two main versions:

DAP2 – legacy but widely supported (many THREDDS servers still use it)

DAP4 – newer, more efficient protocol

PyDAP tries to infer the protocol automatically. If it cannot, it falls back to DAP2, which triggers the warning. The server (opendap.4tu.nl) is a THREDDS server, and these typically expose DAP2 endpoints, so this behavior is expected.

  • Suppress the warning by changing the URL to start with dap2://

PYTHON 


url_dap2 = url.replace("https://", "dap2://").replace("http://", "dap2://")

ds_dap2 = xr.open_dataset(url_dap2, engine="pydap")

Instructor Note

You can go back to the exercise of the Episode of structural interoperability : Identify the structural elements in a NetCDF file


Technical interoperability: API

Instructor Note

This section could be shown as a live demo or a step-by-step walkthrough, depending on the audience and format of the lesson. The key is to demonstrate how to interact with the API using command-line tools like curl, and to explain the underlying concepts of RESTful APIs as you go through the examples.


Instructor Note

Open this link : https://data.4tu.nl/v2/articles/03c249d6-674c-47cf-918f-1ef9bdafe749/files in the browser to check the uuid of a file to download (the readme, the last file) for the following step.


Cloud-Native Layouts

Instructor Note

For an introductory lesson, it is sufficient to describe Zarr in terms of independently readable chunks. More advanced Zarr layouts can also use sharding, where several chunks are grouped into larger storage objects.


Instructor Note

This activity works well as guided live coding.

The example below uses a NetCDF3 file, so NetCDF3ToZarr is used. NetCDF4/HDF5 datasets require the corresponding HDF5 translator.

Use the direct /fileServer/ endpoint because Kerchunk needs access to the bytes of the original file rather than the /dodsC/ OPeNDAP service.


Interoperable Infrastructure in the AI Era