Compliance with the PSI guideline
The European Union's Public Sector Information (PSI) Directive was revised on September 26, 2019. In particular, this new version 2019/1024 regulates guidelines for handling data, such as those acquired at the Conrad Observatory. The following briefly summarizes how these guidelines affect the observatory and in what form they are applied at the observatory. Terms printed in italics correspond exactly in content and form to the wording of PSI Directive 2019/1024 Article 2 (definitions). At the Conrad Observatory, dynamic data, research data and high-quality data sets are acquired and generated. These data sets are provided with internationally recognized standard licenses, anonymized and published in open, machine-readable formats.
Dynamic data and data formats
At the Conrad Observatory, earth observation data is continuously recorded and stored in all geophysical areas. This dynamic data is acquired in real time and we differentiate between raw data, data that corresponds exactly to the sensor measurement, and adjusted data, data that is subject to an automated real-time testing process as well as simple automated corrections (offset, baseline) or conversions with calibration values. Filtered data products and analyzed data sets, which can already be assigned to the category of high-quality data sets, are also generated in real time. All data sets from the observatory, including all meta information, are archived indefinitely in the form of SQL databases and in Common Data Format (CDF) and backed up multiple times. The CDF format corresponds to the standard of machine-readable, open, accessible and reusable format required by the directive. Meta information corresponds to the international standard of the respective discipline, as required by the relevant international bodies. Since a detailed description would go beyond the scope of a short summary due to the number of disciplines, we refer you to an appendix that has yet to be created. In addition to these archiving formats, data formats are also generated as required by the international networks in each discipline and used in the corresponding community. These data formats are also all open, documented and generally machine-readable. According to PSI Directive 2019/1024 Article 5, in addition to the data formats, the accessibility of the data must also be taken into account. Dynamic data and high-quality data can be accessed via API (application programming interface) of the Conrad Observatory's open data portal, web service, in an automated, real-time and machine-readable manner. The API is fully documented on the web service page and is based on international standards.
Data licensing, reuse and mass downloading
All observation data from the Conrad Observatory are automatically provided with a standard license in accordance with the PSI guidelines. Currently (as of June 2023) all available data is subject to a Creative Common License CC-BY-4.0, which states that the data may be reproduced and redistributed in any format or medium and that it is permitted to remix the data , change and build on it for any purpose, including commercial. Appropriate copyright and rights information must be provided, a link to the license must be included and whether changes have been made. In accordance with Article 6 of PSI Guideline 2019/1024, we reserve the right not to use data in the future, even with restrictive licenses (e.g. CC-BY-NC-4.0) and possibly for commercial applications, due to the work required to provide, document, reproduce and anonymize them be made available free of charge. The license information is an integral part of the data metainformation and is included in every data set, whether web service or mass download. For research data, unless otherwise required by the respective research funding agency, CC-BY-4.0 is also used and the data sets are linked on the respective project page in the observatory's data portal (PSI Article 10). In addition to the dynamic data mentioned above, two other forms of data are generated at the observatory: quasi-definitive and definitive data. These records are checked manually. The data check is again carried out according to internationally established technical standards to ensure the comparability of observatories and their methods/data. The INTERMAGNET technical manual is an example. Definitive data sets are created annually and are no longer adjusted or changed after publication. They are available for mass download in the data archive of the data portal and are also offered via international networks.
Technical limitations and transparency
Due to the enormous amount of data acquired at the observatory, only the most internationally common and frequently used data are currently available for real-time download, as well as filtered data products online or via API. The observatory's IT infrastructure, network connection and staffing make real-time availability and provision of all data, including high-resolution data, impossible. A reference to this restriction, an overview of all data, and options for obtaining these data sets can also be found in the observatory's data portal. The observatory also acquires numerous data relevant to building and infrastructure safety that does not fall under the PSI guidelines.
Referencing and monitoring
Dynamic data sets are subject to change and evolving data review processes, both automated and manual. Only with definitive data can we guarantee that the information contained therein will not change. If Observatory data is reused, the standard license requires referencing. For definitive data, the relevant observatory yearbook should be cited with details of the analysis and data verification methodology. With dynamic data, it is currently not possible to clearly reference the data set and you should stick to the "DataTerms" suggestion contained in the meta information. Data usage via the data portal's APIs is logged.