Instructor Notes

Example of a note

Introduction

Instructor Note

It might be a good time to survey the participants to see how many of them have:

  • heard of NetCDF format before (n.b., it’s a prerequisite of the workshop)

  • have experience working with NetCDF format.


Instructor Note

This exercise is for discussion in Plenum and it serves as a good link to the next section.


Instructor Note

Examples of barriers to reuse datasets might include:

  • Missing metadata

  • Non-standard units or unclear variable names

  • File formats you could not easily open

  • Access restrictions or unstable URLs

  • Large data volumes and inefficient download workflows

  • Difficulty aligning datasets from multiple institutions

  • Lack of documentation on coordinate systems or time conventions

  • Inconsistent versions or unclear provenance

This discussion sets up the motivation for the rest of the workshop: practical, hands-on methods to make interoperable data using real tools such as NetCDF, CF conventions, and OPeNDAP.


Structural interoperability

Important teaching distinction

Avoid saying that CSV is “not interoperable.” CSV is one of the most widely exchangeable formats available. Its limitation is that it is weakly typed and weakly self-describing. A well-designed CSV accompanied by a machine-readable schema can be more reusable than a poorly structured file in a richer binary format.


Semantic interoperability

Boundary question

Ask participants to separate the following questions:

  1. Where and how is a unit recorded? — mainly structural.
  2. Is the unit valid and compatible with the declared physical quantity? — semantic and conformance-related.
  3. Do two variables with convertible units represent the same measurement context and statistical treatment? — semantic and scientific comparability.

When the web checker is unavailable

The web service may occasionally be unavailable or unable to retrieve a remote endpoint. Prepare one downloaded report in advance or run the command-line Compliance Checker before the lesson.

Do not frame the activity as “the tool decides whether the file is good.” Frame it as:

The checker tests selected machine-readable rules. Researchers must still interpret the scientific consequences of each finding.


Technical interoperability: Data access protocols

Instructor Note

Most of the cases , a warning is prompted. This warning is normal when using pydap with a THREDDS OPeNDAP server. It is not an error and your dataset should still load correctly. The warning simply means that PyDAP could not detect whether the server supports DAP2 or DAP4, so it defaults to DAP2, which is the older protocol.

The OPeNDAP protocol has two main versions:

DAP2 – legacy but widely supported (many THREDDS servers still use it) DAP4 – newer, more efficient protocol

PyDAP tries to infer the protocol automatically. If it cannot, it falls back to DAP2, which triggers the warning.The server (opendap.4tu.nl) is a THREDDS server, and these typically expose DAP2 endpoints, so this behavior is expected.

  • suppress the warning by replacing the url to start with dap2:// instead of http:// or https://:

PYTHON 


ds_dap2 = ds.replace("https://", "dap2://").replace("http://", "dap2://")

Instructor Note

You can go back to the exercise of the Episode of structural interoperability : Identify the structural elements in a NetCDF file


Instructor Note

This is to guide the instructor what to expect from this demo, which can be optional depending of the time.

Questions

  • Can we open both datasets remotely without downloading the full files?
  • Do both datasets contain the same variables?
  • Do the key radar variables have the same dimensions and units?
  • Can we extract the same variable from both years?
  • Can we combine a selected variable into one analysis-ready object?
  • Why might we want to save this combined subset as Zarr?

The core narrative is:

Ash finds two apparently comparable IDRA radar files from the same radar system, but before combining them she must check whether they are compatible at several levels:

  • Technical access
    • Can both NetCDF files be opened remotely through OPeNDAP?
  • Structural interoperability
    • Do both datasets have comparable dimensions, variables, coordinates, and array shapes?
  • Semantic interoperability
    • Do key radar variables mean the same thing in both files, and do they use the same units and metadata?
  • Practical workflow interoperability
    • Can Ash extract a common variable, make a small subset, combine both years, preserve metadata, and save the result as Zarr for reuse?

Technical interoperability: API

Instructor Note

This section could be shown as a live demo or a step-by-step walkthrough, depending on the audience and format of the lesson. The key is to demonstrate how to interact with the API using command-line tools like curl, and to explain the underlying concepts of RESTful APIs as you go through the examples.


Instructor Note

Open this link : https://data.4tu.nl/v2/articles/03c249d6-674c-47cf-918f-1ef9bdafe749/files in the browser to check the uuid of a file to download (the readme, the last file) for the following step.


Cloud-Native Layouts

Instructor Note

This session can be delivered as a live-coding demonstration.
Walk through each step, explain the concepts, and let learners follow along.


Instructor Note

Common issues when working with Kerchunk:

  • Use the /fileServer/ endpoint (not /dodsC/)
  • NetCDF3 files require NetCDF3ToZarr
  • If you see async errors, set "asynchronous": True
  • Prefer engine="kerchunk" over "reference://" for simplicity

Interoperable Infrastructure in the AI Era