Instructor Notes

Example of a note

Introduction

Instructor Note

It might be a good time to survey the participants to see how many of them have:

  • heard of NetCDF format before (n.b., it’s a prerequisite of the workshop)

  • have experience working with NetCDF format.


Instructor Note

This exercise is for discussion in Plenum nad it can serves as a good link to the next section


Instructor Note

Examples of barriers to reuse datasets might include:

  • Missing metadata

  • Non-standard units or unclear variable names

  • File formats you could not easily open

  • Access restrictions or unstable URLs

  • Large data volumes and inefficient download workflows

  • Difficulty aligning datasets from multiple institutions

  • Lack of documentation on coordinate systems or time conventions

  • Inconsistent versions or unclear provenance

This discussion sets up the motivation for the rest of the workshop: practical, hands-on methods to make interoperable data using real tools such as NetCDF, CF conventions, and OPeNDAP.


Structural interoperability

Important teaching distinction

Avoid saying that CSV is “not interoperable.” CSV is one of the most widely exchangeable formats available. Its limitation is that it is weakly typed and weakly self-describing. A well-designed CSV accompanied by a machine-readable schema can be more reusable than a poorly structured file in a richer binary format.


Semantic interoperability

Boundary question

Ask participants to separate the following questions:

  1. Where and how is a unit recorded? — mainly structural.
  2. Is the unit valid and compatible with the declared physical quantity? — semantic and conformance-related.
  3. Do two variables with convertible units represent the same measurement context and statistical treatment? — semantic and scientific comparability.

When the web checker is unavailable

The web service may occasionally be unavailable or unable to retrieve a remote endpoint. Prepare one downloaded report in advance or run the command-line Compliance Checker before the lesson.

Do not frame the activity as “the tool decides whether the file is good.” Frame it as:

The checker tests selected machine-readable rules. Researchers must still interpret the scientific consequences of each finding.


Technical interoperability: Data access protocols

Instructor Note

Most of the cases , a warning is prompted. This warning is normal when using pydap with a THREDDS OPeNDAP server. It is not an error and your dataset should still load correctly. The warning simply means that PyDAP could not detect whether the server supports DAP2 or DAP4, so it defaults to DAP2, which is the older protocol.

The OPeNDAP protocol has two main versions:

DAP2 – legacy but widely supported (many THREDDS servers still use it) DAP4 – newer, more efficient protocol

PyDAP tries to infer the protocol automatically. If it cannot, it falls back to DAP2, which triggers the warning.The server (opendap.4tu.nl) is a THREDDS server, and these typically expose DAP2 endpoints, so this behavior is expected.

  • suppress the warning by replacing the url to start with dap2://

PYTHON 


ds_dap2 = ds.replace("https://", "dap2://").replace("http://", "dap2://")

Instructor Note

You can go back to the exercise of the Episode of structural interoperability : Identify the structural elements in a NetCDF file


Instructor Note

You can use this part for discussion in the group.

Exercise: What would Ash need to check before trusting the comparison?

In small groups, inspect the two datasets and answer the following questions.

  1. Do both files contain the same processed radar variables?
  2. Do the variables use the same dimensions?
  3. Are the units the same across the two files?
  4. Are the time coordinates encoded in the same way?
  5. Is the range coordinate comparable across both years?
  6. Which metadata attributes help Ash understand the origin of the data?
  7. Which metadata attributes are still missing or unclear?
  8. Would you trust a direct comparison of equivalent_reflectivity_factor across these two files? Why or why not?
  9. What would you document before sharing the combined subset with another researcher?
  10. What would be the benefit of storing the harmonised subset as Zarr?

Key message

Ash can access both IDRA files through OPeNDAP, inspect their NetCDF structure, select the same radar variable, and create a small combined subset. But meaningful comparison still depends on metadata, units, dimensions, coordinates, provenance, and clear documentation of the processing steps.

This is the practical meaning of interoperability: different datasets become useful together only when software can access them, humans can understand them, and workflows can reuse them reliably.


Technical interoperability: API

Instructor Note

This section could be shown as a live demo or a step-by-step walkthrough, depending on the audience and format of the lesson. The key is to demonstrate how to interact with the API using command-line tools like curl, and to explain the underlying concepts of RESTful APIs as you go through the examples.


Instructor Note

Open this link : https://data.4tu.nl/v2/articles/03c249d6-674c-47cf-918f-1ef9bdafe749/files in the browser to check the uuid of a file to download (the readme, the last file) for the following step.


Cloud-Native Layouts

Instructor Note

This session can be delivered as a live-coding demonstration.
Walk through each step, explain the concepts, and let learners follow along.


Instructor Note

Common issues when working with Kerchunk:

  • Use the /fileServer/ endpoint (not /dodsC/)
  • NetCDF3 files require NetCDF3ToZarr
  • If you see async errors, set "asynchronous": True
  • Prefer engine="kerchunk" over "reference://" for simplicity

Interoperable Infrastructure in the AI Era