Hi, I'm Eirini 👋

👩‍🎓 PhD Candidate @Max Planck Institute for Psycholinguistics

• worked with lots of tabular data with observations from experiments with human participants
• got interested in open science and reproducibility

💼 Trainer on Research Data Management @TU Delft and 4TU.ResearchData

• train researchers and data supporters on good research data management practice

Session outline

• Reproducibility & the FAIR principles
  • What is reproducibility and why is it important?
  • What are the FAIR principles?
• Data Carpentry: Data organisation for tabular data*
  • Good data entry practices
  • Formatting data tables in spreadsheets
  • Importing and exporting data

The content presented in this part comes from the Data Carpentry lesson Data Organization in Spreadsheets for Ecologists and is used under a CC-BY licence.

What is reproducibility?

Image from The Turing Way: A Handbook for Reproducible Data Science (The Turing Way Community, 2021) used under a CC-BY licence.

Why is reproducibility important?

From an altruistic point of view, reproducibility is beneficial for the research community, and arguably society. In many cases, it is a minimum requirement for verifying research; when research can be verified, and by extension corrected, science is built on solid ground. Being able to reuse and build on available data and code saves time and resources, with the potential to speed up innovation.

But reproducibility can also be beneficial for individual researchers!

Reproducibility: benefits for individual researchers

There aren't only altruistic reasons to work reproducibly; it can be good for your career too!

Markowetz (2015) notes five selfish reasons to work reproducibly.

Reason 1: reproducibility helps to avoid disaster

• 1 in 2 examined papers had inconsistencies in the reported p-values

• 1 in 8 examined papers had inconsistencies large enough that they would affect the conclusions

Reason 1: reproducibility helps to avoid disaster (cont'd)

• 1 in 5 papers in genetics is thought to have an error because of the default settings in Microsoft Excel reading in gene names as dates

• To be continued...

Reason 2: reproducibility makes it easier to write papers

• with literate programming it's easy to combine text and code

  • meaning you can call your results into your document directly. No more copy-pasting = no more-copy pasting errors ✨

  • if you collect more data or change your data cleaning, you just need to rerun the analyses and all results will update

• working reproducibly often involves automation and/or good documentation, making it more likely you'll actually know what you did when it's time to write up 💃🏼

Reason 3: reproducibility helps reviewers see it your way

Sharing the underlying code and data makes it possible for the reviewers to:

• understand exactly how you cleaned and analysed the data

• try out different analyses themselves

• spot mistakes in your code👀

Reason 4: reproducibility enables continuity of your work

Good documentation of the data and code means:

• you'll be able to go to a project after a break (e.g., holidays, peer review)

• it's easier for others to pick up a project (important in long-term projects)

Reason 5: reproducibility helps to build your reputation

Sharing and documenting your data and code well:

• helps you build a reputation as an honest and careful researcher. This could be contributing to a citation advantage of about 25% for publications with linked data (Colavizza et al., 2020).

• if there is ever a problem with one of your papers you can show you did everything in good faith

Interim summary 1

Working reproducibly doesn't mean that there will definitely not be any mistakes in your work. But it does make it easier to figure out what went wrong and, by extension, to fix it.

Working reproducibly can also make the paper writing and reviewing process easier and more productive.

And it can help others build on your work and build your reputation.

FAIR and open data and code

Looking at this matrix again, it's clear that for reproducibility to be possible, it is necessary to be able to get a copy of the data and code.

Additionally, code and data need to be Findable, Accessible, Interoperable, and Reusable, or in short FAIR (Wilkinson et al., 2016).

Findable 🔍

First people need to be able to find the data!

For data to be findable, they need to be described with rich metadata, or information about data. These metadata can be generic (e.g. title, author name, keywords) or discipline specific.

Data also need to be assigned a unique and persistent identifier. A commonly used identifier is the Digital Object Identifier (DOI). Such identifiers make it easy to find data, but also to link them with other relevant information (e.g. a publication).

Accessible 🔓

After people have found the data they need to be able to access them!

This could mean that the data are publicly available in a data repository, like 4TU.ResearchData.

⚠️ Personal data and other sensitive information

If you're working with personal data, classified data, or other sensitive information, you may need to keep the data restricted and require authentication or authorisation before others can access the data.

In those cases, metadata should still be accessible.

Interoperable ⚙️

Data often need to be integrated with other data.

This integration is easier when data make reference to other relevant datasets.

It's also important to make data available in open file formats, that anyone can open.

Using controlled vocabularies is also highly recommended, if these exist in your field.

In how many different ways can you say "female"?*

Adapted from Silvester et al., 2015.

Dates

To avoid ambiguity, use the RFC3339 standard: YYYYMMDD (or YYYY-MM-DD).

This image was created by cmglee, Canuckguy and many others for Wikimedia Commons and is used under a CC-BY-SA licence

Reusable ♻️

Let's say people were able to find, access, and open your data. Now for the last part!

To be able to reuse your work, people need to be able to understand it. This means you need to provide good documentation:

• at a minimum, you should provide a README file where you describe what the project is about, how the files are organised and how to reproduce the project

Example of a README file

Reusable ♻️

Let's say people were able to find, access, and open your data. Now for the last part!

To be able to reuse your work, people need to be able to understand it. This means you need to provide good documentation:

• at a minimum, you should provide a README file where you describe what the project is about, how the files are organised and how to reproduce the project
• when working with tabular data, you should also provide a data dictionary where you explain what the different variables mean, the measurement units, how missingness is encoded etc.

Some data

Example of a data dictionary

• Date collected: the date the data were collected in YYYY-MM-DD format
• Species: a code for the species of the animal detected. See below for a table of what the codes stand for
• Sex: the sex of the animal detected, M for male and F for female
• Weight: the weight of the animal detected measured in grams

Missing data are coded as NA.

Reusable (cont'd) ♻️

Another crucial component of reusability is usage licences. Just because something is shared online, doesn't mean anyone can use it. That's because the creator* of the work holds the copyright to it.

So, you need to tell people what they are allowed to do with your data and code. You do this by providing a usage licence.

Note that usage licences are different for data and for code:

• Commonly used licences for data, presentations, papers etc. are the Creative Commons licences
• For code, you can check a list of OSI-approved licences

Don't panic!

This image was created by Scriberia for The Turing Way community and is used under a CC-BY licence. DOI: 10.5281/zenodo.3332807

Interim summary 2

Making your data and code Findable, Accessible, Interoperable and Reusable can take you a long way towards making your work reproducible.

There are many things you need to do to make your work FAIR, but:

1. these things will benefit you
2. you don't have to start doing everything immediately
3. we're here to help 🤝 ✨ In this workshop you'll learn good practices for working with tabular data, how to generate and add metadata to your data, how to validate your data, and how to publish your data in a data repository.

So let's get started 🎉

Tabular data

Tidy data

A good framework for working with tabular data is that of tidy data. The concept of tidy data is borrowed from R and the tidyverse packages within it (Wickham, 2014).

llustrations from the Openscapes blog Tidy Data for reproducibility, efficiency, and collaboration by Julia Lowndes and Allison Horst.

Tidy data

llustrations from the Openscapes blog Tidy Data for reproducibility, efficiency, and collaboration by Julia Lowndes and Allison Horst.

Why tidy data?

Using a consistent way to organise your data, e.g. following the tidy data structure, helps you be more efficient when working with your data. That's because you can create (and reuse) tools that expect that kind of structure instead of starting from scratch every time.

Like R, NumPy and pandas can use vectorised processes that make working with large datasets more effcient and faster. Placing variables in columns makes using vectorised processes easier.

Example

Summary (long)

Summary (wide)

⚠️ Keep your data tidy from the start

How you collect your data will vary depending on your field and specific project:

• If you can control how the data are created, make sure you use the tidy data structure from the beginning.
• If you cannot control how the raw data are structured, wrangle them into the tidy format, ideally using a scripted programming language, like Python or R.

Spreadsheet programmes (e.g. Microsoft Excel)

Researchers often use spreadsheet programmes like Microsoft Excel to work with their tabular data.

Although such programmes are good for some things, they aren't always appropriate...

For example, when 16000 COVID cases went unreported in England when the old .xls file format ran out of rows.

Problems with Excel

⛔ Data wrangling, visualisation & analysis

• easy to make mistakes (accidental deletions, misapplying formulas)

• not reproducible (including mistakes)

Never edit your raw data in Excel!!!

⚠️ Data entry

• data importing and exporting problems

• non machine-readable formatting

Data importing and exporting problems

We've known since 2016 that many genetics papers (1 in 5) have mistakes because Microsoft Excel interprets gene names as dates.

Also, for any language scientists out there working with a language that's not English, always use UTF-8 text encoding.

Non machine-readable formatting

We often organise data the way we humans like to work, i.e. by relying on context. That doesn't work for software like Python or R that we use to analyse data. So we need to structure our data in a way computers can understand.

Non machine-readable formatting includes:

• notes in the margin

• spatial layout of data

• field formatting

• using special characters and spaces

How to code missing values*

Adapted slightly from White et al. (2013)

Summary: Tips for structuring tabular data

• Leave the raw data alone!
• Be consistent
• Put variables in columns
• Put observations in rows
• Put as little information as possible in a single cell
• Avoid spaces and special characters
• Use controlled vocabularies and standards (e.g. for dates)
• Create documentation (e.g. README and data dictionary)
• Think about how to code missing vales (and add that in the documentation!)
• Do not use formatting (colours, font, bolding)
• Use data validation to avoid errors (Frictionless Data framework)

Exercise 📝

• Download the messy data
• Open up the data in a spreadsheet programme
• Notice the two tabs: Two RAs conducted surveys, one in 2013 and one in 2014. They kept track of the data in their own way in the tabs 2013 and 2014 respectively. Now you're the person in charge of this project and you want to start analysing the data.
• You have 7 minutes to clean up as much of the dataset as you can.
• Once the 7 minutes are up:
  • take a screenshot of your cleaned dataset
  • add a comment with what confused or annoyed you the most
• Look at the posts of the other participants and give them a +1 if you like their solution/comment

Using open file formats

• Accessible: usable by multiple types of software that are freely available (e.g. note that Microsoft requires a paid licence)

• Life expectancy: open file formats are more suitable for long-terms preservation because they don't rely ona single software programme

• Publishing: data repositories, journals and funding agencies may have requirements for open file formats

• Universal format: interoperable format that produces the same results when it is imported by various software programmes, including plain text editors

Which open file format? 🤔

Exporting from Excel to .csv

1. From the top menu, select File and Save as
2. In the Format field select CSV UTF-8 (Comma delimited) (.csv) from the list
3. Double check the file name and the location where you want to save the data and hit Save

My data contains commas!

Enclose the data fields with double quotes and double check that the file you are exporting can be read in correctly.

Exporting from Excel to .tsv

1. From the top menu, select File and Save as
2. In the Format field select Text (Tab delimited) (.txt) from the list
3. Double check the file name and the location where you want to save the data and hit Save
4. Rename the file you just saved and change the extension from .txt to .tsv

Resources

Metadata:

• FAIRsharing.org
• Digital Curation Centre
• Research Data Alliance Metadata Directory

Licences:

• Creative Commons licences (for data)
• OSI-approved licences (for code)

References

References