This article is the first in a two-part series that provides an overview of the FAIR data principles and looks at how ArcGIS supports geospatial data providers in making your data FAIR.
Understanding FAIR
We all need and benefit from easy access to interoperable data for decision-making. The FAIR data principles were published in Nature Scientific Data1 in the spirit of improved data management among researchers who increasingly depend on data that are easily Findable, Accessible, Interoperable, and Reusable (FAIR). Beyond researchers, it turns out that these same principles are of universal value to a broad range of data managers and users.
The spirit of FAIR is to improve the discoverability of interoperable datasets. If a person or a machine/agent does not know about a dataset, how would we ever find, connect to, and reuse it?
It is important to note that FAIR data is not the same as open data; these are complementary concepts. As explained by Maastricht University,2 open data is data that can be freely used, shared, and built on by anyone. The FAIR principles provide a set of best practices for sharing data in ways that can respect any ethical, legal, or contractual restrictions. They do not state that data must be free of charge or unencumbered by license conditions. Instead, the FAIR principles focus on the future interoperability of data and infrastructure, within which data sources are properly documented and accessible. Between the lines, FAIR helps providers understand that shared data need a bit of care to be truly useful.
The principles refer to three types of entities: data (or any digital object), metadata (information describing the data), and infrastructure (tools and systems). The FAIR principles are further broken down into sub-prnciples elaborated in the appendix below. For instance, principle F4 defines that both metadata and data are registered or indexed in a searchable resource: the infrastructure component.
The GO FAIR initiative3 recently provided a handy annotated version to understand the FAIR Principles more thoroughly. According to GO FAIR, the vision is to foster the coherent development of the global Internet of FAIR Data & Services (IFDS), focusing on early outcomes in the European Open Science Cloud (EOSC).4 See also the European Cloud Initiative – Building a competitive data and knowledge economy in Europe.5 But of course, the FAIR principles are of general interest worldwide!
The four FAIR principles are as follows:
Findable data are the first step to reusable data. Machine-readable data and metadata, combined with search engine optimization (SEO) techniques, help make your data findable by web search engines like Google and explorable by humans.
Accessible data lets users know how to access the data they find, possibly including authentication and authorization. Accessible can apply to open public data and private and secure data available only to authorized users.
Interoperable data can be integrated with other data and systems, leveraging open standards and specifications.
Reusable data are richly described by metadata, released with a transparent data usage license, and adhere to domain-relevant community standards. Reusability is a primary goal of the FAIR data principles.
Industry adoption of FAIR
The geospatial community has embraced the FAIR data principles and has long appreciated the need for accessible and interoperable data. In 2021, the Open Geospatial Consortium (OGC) standards development organization updated its mission statement, embracing these principles: “The organization represents over 500 businesses, government agencies, research organizations, and universities united with a desire to make location information FAIR – Findable, Accessible, Interoperable, and Reusable.”
As a principal member of the OGC, Esri is united in supporting this vital mission. So, what does it mean in concrete terms for Esri to support the FAIR data principles? Ultimately, it is about helping you, our users who produce, publish, collaborate, share, and (re)use the world’s location-based data. Our goal is to ensure that the ArcGIS system enables FAIR data with appropriate geospatial infrastructure.
Esri has a history of supporting the spirit of the FAIR principles:
- 2014 – When Open Data Isn’t “Useful” Data
- 2017 – Beyond Open Data
- 2017 – ArcGIS Hub: A Catalyst for Creating Smart Communities
- 2020 – Ties between the FAIR Guiding Principles for scientific data management, stewardship, publishing, and GIS
- 2021 – Esri’s ArcGIS enables thousands of datasets, maps, and apps for location
ArcGIS: Helping you implement the FAIR principles
These are grand principles, so how are they put into practice?
It’s important to understand that data do not become FAIR automagically and that data FAIRness is not a binary but rather a range or spectrum. Put into practice, FAIR results from an organization’s efforts and the tools that support them. For example, organizations must publish data as web services, ensure their content has proper metadata, and carry appropriate licensing. Tools must support these efforts through user interfaces, automation, workflows, infrastructure, and compliance that help make FAIR easy to implement.
As you produce, publish, collaborate, share, use and reuse data, you work across different software and SaaS products. ArcGIS helps you to execute FAIR data principles throughout your workflows. Let’s look at an overview of these FAIR principles in practice:
As you produce and manage data and metadata, domain-relevant community and international standards are put into practice using ArcGIS Pro.
Using ArcGIS Enterprise and ArcGIS Online, you publish human- and machine-readable web services and their closely coupled metadata in well-known formats (such as JSON and XML) with international open standards and open specifications. API keys allow requisite authentication for quick access to popular locations services such as basemaps, routing, and geocoding. ArcGIS identities provide the necessary authorization mechanism for controlled access. This enables collaboration among individuals and organizations so you can work with your partners more easily.
Content is shared with appropriate audiences as open or private data, based on the permission granted by the data owner, and described with clear license terms of use. The FAIR principles come to life as you share content through ArcGIS Hub. Shared data and web services are easily found, accessed, and federated using the Hub catalog. In addition, Hub helps you engage with your communities along the way.
GIS professionals, analysts, and developers use and reuse interoperable web services using the ArcGIS REST API and various mapping tools from ArcGIS, open source, and third-party APIs. Finally, ArcGIS low-code and no-code app builders assure that web services are easily reused and accessible to non-specialists through information products that answer questions and convey understanding.
Conclusion
Hopefully, this post motivates you to investigate and then apply the FAIR data principles.
Stay tuned for Part II, where we’ll enumerate the FAIR principles, their attributes, and how ArcGIS helps you fulfill the basic requirements. In Part II, we’ll follow the path of a dataset through this end-to-end workflow to show how ArcGIS supports each of the sub-principles (F1, F2, A1, A2…) detailed in the appendix below.
Join the conversation in the Esri Community.
References
1 Wilkinson, M. D. et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci. Data 3:160018 doi: 10.1038/sdata.2016.18 (2016). https://www.nature.com/articles/sdata201618
2 Maastricht University, 2020. The FAIR principles explained (YouTube video) https://youtu.be/5OeCrQE3HhE?t=61 accessed 4-April, 2021
3 https://www.go-fair.org/go-fair-initiative
4 https://www.go-fair.org/go-fair-initiative/vision-and-strategy/
5 European Commission, 2016. European Cloud Initiative – Building a competitive data and knowledge economy in Europe https://eur-lex.europa.eu/legal-content/en/TXT/?uri=CELEX:52016DC0178
Appendix: ArcGIS supports the FAIR data principles
The following table details how ArcGIS supports each of the FAIR principles from a technical product perspective, following the framework provided by the GO FAIR initiative:
| F: FINDABLE | ArcGIS support |
|---|---|
| F1. (Meta)data are assigned a globally unique and persistent identifier | ArcGIS items are assigned a globally unique and persistent identifier referred to as the Item ID |
| F2. Data are described with rich metadata (defined by R1 below) | ArcGIS items and their associated metadata are closely coupled and can be richly described by the item owner. In ArcGIS, metadata is saved with the item it describes. The ArcGIS Metadata model is a superset that supports a wide variety of international metadata standards and profiles, including ISO, FGDC, and INSPIRE (called metadata “styles” in ArcGIS). Esri Geoportal Server provides additional support for Dublin Core and custom profiles. ArcGIS’s default metadata style, Item Description, lets you see and edit a concise description of an item. Item Description metadata style is roughly equivalent to the Dublin Core Metadata Element Set for describing Items; some element values are populated automatically. ArcGIS Online provides users an item information completeness score to help improve metadata. |
| F3. Metadata clearly and explicitly include the identifier of the data they describe | ArcGIS metadata clearly and explicitly includes the file identifier of the data they describe. In ArcGIS, an item’s metadata is stored as part of the item; metadata is stored differently for each item based on that item’s requirements and flow through the system as the items are published and shared. |
| F4. (Meta)data are registered or indexed in a searchable resource | ArcGIS SW and SaaS support a wide variety of item types that are indexed by metadata elements that support browse and search in a focused way. Items in ArcGIS are available for searching within the ecosystem, including search queries on ArcGIS Living Atlas of the World, ArcGIS Enterprise, and ArcGIS Online. Search queries on arcgis.com/home/search will return content shared with Everyone in ArcGIS Online. Items shared to your ArcGIS Hub Content Library are indexed so content is easily discoverable. Search queries on hub.arcgis.com, will return all related content shared by you, teams across your organization, and curated content belonging to other hubs around the world.
Hub item pages are search engine optimized (SEO) to make your content easily findable through popular search engines. |
| A: ACCESSIBLE | ArcGIS support |
| A1. (Meta)data are retrievable by their identifier using a standardized communications protocol | ArcGIS items are retrievable by a unique identifier using a standardized communications protocol via the ArcGIS API and well-known URL. Data are available in common JSON and Image formats. Users can append /info/metadata to the end of a service’s REST URL to get an XML file containing all the service’s metadata. Developers can parse this metadata and present it to the end user of the service in the desired format. |
| A1.1 The protocol is open, free, and universally implementable | ArcGIS default web services use the GeoServices REST open specification. Originally developed by Esri, in 2010 the specification was transferred to the Open Web Foundation. Esri is committed to an open and secure design architecture. A wide variety of international open, free, and universally implementable web service standards are supported, such as OGC feature and image services and APIs, as well as content standards from organizations such as NENA, IHO, ISO and INSPIRE. The UN Standards Guide Edition 3 (2022) notes that open does not necessarily imply free of cost. Depending on a Standards Development Organization’s business model, costs of developing standards are recovered from membership fees or sales. As previously noted, ArcGIS support a variety of international open metadata standards. |
| A1.2 The protocol allows for an authentication and authorization procedure, where necessary | The ArcGIS Platform supports secure access to location services and private data. It ensures that only valid, authorized users and services access protected information. API keys allow requisite authentication for quick access to popular locations services such as basemaps, routing, and geocoding; and ArcGIS Identities provide the necessary authorization mechanism for access to private user and organization content for richer experiences. |
| A2. Metadata are accessible, even when the data are no longer available | ArcGIS data and metadata are tightly coupled. Once created, metadata is copied, moved, and deleted with the item as it’s managed by ArcGIS. Content deprecation (without deletion) is supported. Esri’s open source Geoportal Server also supports metadata cataloging and access decoupled from the data, for example for historical archived content. |
| I: INTEROPERABLE | ArcGIS support |
| I1. (Meta)data use a formal, accessible, shared, and broadly applicable language for knowledge representation. | ArcGIS web services provide a simple open Web interface where all resources exposed by the REST API are accessible through URL endpoints. GeoServices REST is a proven and easy to understand method for a broad range of clients and applications. GeoServices use broadly accessible JSON language for metadata and feature encoding. The ArcGIS APIs and SDKs include many helper classes, functions, and utilities for working with the ArcGIS REST APIs. Metadata are expressed using broadly applicable human- and machine-readable HTML and XML representations. Metadata are automatically transformed by ArcGIS Hub into DCAT / DCAT-AP using the Resource Description Framework (RDF) vocabulary designed to facilitate interoperability between data catalogs published on the Web, expressed using JSON-LD. |
| I2. (Meta)data use vocabularies that follow FAIR principles | As described by the University of Texas Libraries, controlled vocabularies are standardized and organized arrangements of words and phrases and provide a consistent way to describe data, such as thesauri and established lists. Within an item, ArcGIS supports user-defined attribute domains to constrain values such as name, description, and field types. Metadata creators can assign terms from controlled vocabularies that follow FAIR principles, such as ISO, INSPIRE, and FGDC categories, to improve content quality and information retrieval. |
| I3. (Meta)data include qualified references to other (meta)data | ArcGIS items can include qualified references generated automatically by the system, as well as reference and provided by the user in the metadata. For example, ArcGIS Online item details automatically provide “created from / published as” overview information. |
| R: REUSABLE | ArcGIS support |
| R1. (Meta)data are richly described with a plurality of accurate and relevant attributes | ArcGIS item properties, such as spatial reference and attribute fields, can be added automatically to its metadata by ArcGIS Pro. Metadata flow with data as web services are published and shared. Tools enable users to richly describe data attributes using field attributes and descriptions that are accessible within web service layers, maps, and apps. For example, field aliases provide an important way to provide a more user-friendly description of the content. Esri regularly provides good practice guidance documents and tools needed to improve the display, sharing, and reuse of content, such as when sharing open data with ArcGIS Hub. |
| R1.1. (Meta)data are released with a clear and accessible data usage license | ArcGIS provides the tools for content owners and publishers to easily provide terms of use and licensing describing how an item can or cannot be used. ArcGIS dynamically displays license icon and links for Creative Commons license (CC0, CC BY, CC BY-SA) and Open Data Commons license (PDDL, ODbL, ODC-By) on Hub item pages. Note that FAIR principles can apply to open (public) as well as private/secure data. Terms of use may include special restrictions, disclaimers, terms and conditions, or limitations on using the content. |
| R1.2. (Meta)data are associated with detailed provenance | Without knowledge of data accuracy, provenance, and age, you cannot have a high level of confidence in decisions based on that data. Provenance can be as simple as a metadata citation attributing credit to an item’s source recorded by the user, or if required, ArcGIS can automatically maintain an item’s geoprocessing history within its metadata, for example, to support legal questions where the provenance and processing history of the data are required to provide a response. |
| R1.3. (Meta)data meet domain-relevant community standards | As described herein, ArcGIS supports a wide array of domain-relevant standards and specifications for data, web services, and metadata. For example, DCAT and DCAT-AP are domain-relevant global standards enabling your content to be seamlessly federated with Open Data sites, such as data.gov and data.europa.eu. |
banner Photo by Niklas Ohlrogge on Unsplash
Commenting is not enabled for this article.