Open Data

The use of open data facilitates new insights into science, industry and society. Did you know that there are 21 mountains called “Schwarzhorn” in Switzerland? Or that Bahnhofstrasse in Zurich was named in 1863 and is 1,173 metres long? What historic buildings and monuments are there right on your doorstep? Open data and open government data (OGD) make these and many more snippets of information possible. By combining various datasets, for instance, applications 1 that work with different visualizations can be developed. But how is data made open and what characterizes “open data”? Open Definition 2 words it as follows:

In principle, all data can be opened, i.e. rendered accessible, whether it be your own photos on image platforms such as Wikimedia Commons, cooking recipes on a forum or travelogues on your own blog. Typically, however, open data increasingly involves larger datasets. After all, a certain amount of data is crucial to develop corresponding applications and thus generate added value.

Meanwhile, many libraries, archives or museums run online platforms where “data” is available openly and freely. This might be digitized images and/or the metadata, i.e. the describing information, on this data. Based on metadata on a historical collection, an application could be developed that shows the places where the objects’ copyright holders come from on a map. Or a user can pinpoint the era from which most works stem. It then becomes fascinating to combine the two pieces of information in a dynamic visualization: do the places of origin change over time?

In academia, it is usually research data, i.e. data collected from observations or experiments, which is rendered publicly accessible. This promotes international exchanges and drives research forward.

Open government data is also increasingly available to the population at large. For instance, a wide range of data can be searched for and subsequently downloaded in the desired format in the City of Zurich’s open data catalogue 4 . This might be information on catering businesses by year and district, locations of fountains, playgrounds, churches or day-nurseries in the City of Zurich or even greenhouse gas emissions over several decades, to name but a few examples.

For open data to be compatible with various programmes and for it to be downloaded and processed further on computers all over the world, a universal definition of “openness” is required. 5 Tim Berners-Lee, the inventor of the World Wide Web, proposes a five-star open data model. 6 On the first, bottommost level, data – regardless of the format – is provided under an open licence. 7 The next level requires the data to be restructured – as an Excel table, for instance. On the next level, open data is characterized by a non-proprietary, i.e. open, file format, which might be a CSV file, for example. The second-highest level also requires URIs (unique resource identifiers) to enable data to be located simply and persistently. On the fifth level, open data is linked to other data as linked open data to render the context visible.

Besides the positive aspects and opportunities to gain new insights, naturally open data also poses challenges – none more so than data protection: certain data, such as customer information, must not simply be rendered openly accessible. This concerns patient data in medicine or personal data, for example, where absolute anonymity must be guaranteed. Thus, there always needs to be a thorough legal explanation of which data is allowed to be rendered freely accessible in the first place.

Institutions have to use substantial resources to select data in advance and seek the necessary legal clarifications. A technical download option also needs to be provided for developers and programmers to reach the open data as uncomplicatedly as possible. Moreover, a certain level of expertise on the part of the users is required to programme a visualization or application from the raw data. Otherwise, the finest data streams on the highest open level are useless.

Needless to say, institutions and authorities rendering open data accessible is only one aspect. Processing data further into visualizations or applications is just as important to obtain added value from the open data. So-called hackathons are a prime example of this. Data providers, hackers and anyone else who is interested get together and endeavour to develop new applications or visualizations based on open data in the space of two to three days. The Swiss Open Cultural Data Hackathon was also launched against this backdrop in 2015 and took place for the third time in September 2017. It yielded many new projects, all based on open data.

ETH Library was present as a data provider too and supplied metadata on Carl Gustav Jung’s letters, which are kept in the holdings of the ETH Zurich University Archives. In a project team formed on location called Jung-Rilke Correspondence Networks, an intensive discussion on the letter holdings of C.G. Jung and Rainer Maria Rilke (from the Swiss Literary Archives) took place. The goal was to gradually edit and enrich the data with a view to subsequently visualizing it in a different way. For instance, the geocoding of the sender locations available enabled the correspondence networks to be depicted on maps. 8

The publication of results is a basic principle of modern academia. It enables scientists to identify errors and support, reject or build on theories. Openness is essential for modern science to be able to self-correct and keep improving. 9 One new concept is the idea that not only evaluations and interpretations of data should be published, but also already pure measurement series and raw data from experiments. This means so-called open research data.

There are good reasons as to why research data should also be published as open data. These include citing academic studies more frequently if the underlying data is publicly accessible. Such studies have greater credibility. Moreover, high-quality research data can be published additionally in specialist journals if necessary and ultimately the research project can be re-used for new projects with reproducible results. 10

In the internet age, open research data is more current than ever. Public-funded data and research results should especially be freely accessible to everyone according to the guidelines of the funding institutions – not least to advance research. Consequently, the Swiss National Science Foundation (SNSF) has demanded data management plans (DMPs) as a component of project applications since October 2017. A DMP outlines how the research data is generated, collected, documented, published or rendered publicly accessible and ultimately archived digitally within the scope of a project and thus helps researchers plan the life cycle of their data. 11

The Swiss National Science Foundation (SNSF) guidelines 12 for researchers provide key criteria for compiling a data management plan. They are based on the FAIR principles 13 : research data should be Findable, Accessible, Interoperable, Re-Usable. Four areas need to be covered with a data management plan:

• Data collection and documentation
• Ethical, legal and security-related issues
• Data storage and retention
• Exchange and re-use of data

The Digital Curation Office at ETH Library offers comprehensive advice and support on all aspects of data management. 14 Researchers from ETH Zurich also have the possibility of publishing their data on the Research Collection, ETH Zurich’s repository for publications and research data, from where it is exported automatically into the ETH Data Archive for long-term archiving. 15

As an information service provider, ETH Library uses the possibility of linked open data by enriching the entries in its own catalogue with dynamic links to Wikipedia, the German Digital Library and other sources.

As a supplier of open bibliographical metadata and digital copies, ETH Library would like to contribute towards the open data movement. Whenever possible, its own data is provided for further use without legal restrictions, i.e. under a public domain mark 16 or what is known as a CC0 licence 17 . If the prerequisites for this are not fulfilled, as open a licence as possible is selected from the Creative Commons “kit”. Specifically, bibliographical metadata as metadata sets of different publication types are available for download as a packet or via a direct interface (Z 39.50). Moreover, ETH Library’s Image Archive enables links to the holdings connected to GND 18 -referenced people via what is referred to as a BEACON file. Thousands of digitized documents – both texts and images – can be viewed and downloaded from various platforms:

• E-Pics Image Archive Online: ETH Library’s Image Archive with images of the academic and technical history of Switzerland, landscapes and townscapes, the Swissair Photo Archive and the Archive of the photographic agency Comet Photo AG. Freely accessible image series are regularly presented on the Crowdsourcing blog. Selected digital copies and „born digitals” are also provided for re-use with an open licence on Wikimedia Commons, which simplifies their integration in Wikipedia articles.
• e-rara.ch: digitized books from Swiss libraries from the 15th to the 20th century.
• e-manuscripta.ch: digitized manuscript materials from Swiss libraries and archives.

The fact that the principle of open data is spreading ever further and governments all over the world are increasingly tending to open their data is clearly evident on the Open Data Barometer. 19 Since 2013 the portal has recorded how authorities have been providing open data with regard to accountability, innovation and social impact. Although Switzerland has managed to improve every year, with a score of 43/100 it is only in 22nd place out of 77 nations evaluated. Only if authorities and organisations continue to act with open data in mind and the public at large learns just how much potential open data harbours can the trend towards an open information society be expedited. ETH Library is dedicated to advancing this development and continuing to promote and raise awareness of the notion of open data.