2021 ESIP Summer Meeting

Over the past several decades, considerable progress has been made in the development of metadata standards and formats for publishing Earth and environmental data. Repositories for scientific data products have also proliferated. In the ecological research community, much of this development has centered on publishing tabular data. As data products become more complex, voluminous, and distributed, there is a clear need to develop new methods to handle the creation, publication, and discovery of datasets that don’t fit within the standards and workflows established for the most common cases. In this session we address several special cases that have become increasingly common in Earth and environmental datasets, such as drone data, sequencing and genomic data, image or document collections, and datasets distributed across multiple repositories. Presentations will focus on emerging best-practices, metadata standards, specialized repositories, provenance metadata, and methods for linking datasets with disparate structures, metadata, or published locations. A breakout group session will foster collaboration and coordination among the numerous groups that are leading efforts in this area.

Session goals and preparation:

To prepare for the session, please think about what types of Earth, environmental, or ecological datasets you work with in your research or data management activities, and consider where there are gaps in your knowledge of how to publish, access, or use these datasets. In general, are session goals are:   

1. Introduce participants to recent developments around publishing, discovering, and re-using modern ecological and environmental datasets.
2. Answer the question: Should this community be doing more to develop best practices for publishing modern datasets as they become more complex and varied? (And what? how? where?)

During the session participants will enter notes and questions in a Collaborative Doc.

Session Agenda:

• 1:30-1:35p ET: Arrival and orientation - Greg Maurer, Corinna Gries, and Renée F. Brown
• 1:35-1:45p ET: Data package design for special cases - A best practices guide developed by the LTER Network and the Environmental Data Initiative - Renée F. Brown
• 1:45-1:55p ET: Advancing metadata formats for modern ecological and environmental data - Carl Boettiger
• 1:55-2:05p ET: Exemplars and moving pieces in drone data practice, conventions, and tooling - Jane Wyngaard
• 2:05-2:15p ET: Open science expectations for simulation based research - Doug Schuster
• 2:15-2:25p ET: Engaging a global scientific community for publishing code: CoMSES.net and the Open Modeling Foundation initiative - Michael Barton
• 2:30-2:45p ET: Collaborative breakout sessions
• 2:45-2:55p ET: Report outs from breakout sessions
• 2:55-3:00p ET: Wrap up and closing

View Notes

The ESIP cluster “Sustainable Data Management” promotes mechanisms for repositories to collaborate to preserve their holdings (https://wiki.esipfed.org/Sustainable_Data_Management). Their current project is to produce recommendations for member repositories on implementing guidance principles within frameworks like FAIR (https://doi.org/10.1038/sdata.2016.18) and TRUST (https://doi.org/10.1038/s41597-020-0486-7). We are also including a third framework: the CARE principles for indigenous data governance (http://doi.org/10.5334/dsj-2020-043), where CARE stands for Collective Benefit, Authority to Control, Responsibility, and Ethics. These principles extend data management concerns to be more people- and purpose-oriented, and to respect indigenous sovereignty. As stated in the Data Science Journal paper, “The ‘CARE Principles for Indigenous Data Governance’ empower Indigenous Peoples by shifting the focus from regulated consultation to value-based relationships that position data approaches within Indigenous cultures and knowledge systems to the benefit of Indigenous Peoples”. This session will present the cluster’s recent examination of the CARE principles, how these are related to repository activities, and extend FAIR and TRUST. Introductory material on CARE and the cluster’s work will be presented, followed by discussion.

View Notes

Cloud systems have raised the visibility of online data and users are becoming (more) comfortable using data that they do not first download as files. OPeNDAP has been building web services to enable access to remote data for over two decades. In this session we will describe how data providers can make their data holdings accessible to tools like Python/Jupyter Notebook, Matlab, and ArcGIS Pro with little or no reformatting. At the same time, we will also show how those same data can be moved to the cloud and accessed both directly, using the zarr API, and using an OPeNDAP server.

This session will also cover different formats that can be served using OPeNDAP. We will show how the Hyrax OPeNDAP server is installed and configured, with a focus on the most important aspects of the service, including best practices for using OPeNDAP servers as a component in a larger data system design. We will also cover demonstrations of various clients accessing the server and discuss built-in interfaces in different clients, so that providers can point users to tutorials they are likely to find useful.

View Notes

In this session we will openly discuss and strive to achieve consensus on how best to govern SWEET as a longstanding, domain level, semantic web resource. As such, there are several outstanding questions which either need to be addressed or past decisions confirmed and then documented (likely on the SWEET wiki). Examples, though not comprehensive, include:
1. How are SWEET issues or proposals raised?
2. What are the criteria used to evaluate SWEET issues or proposals?
3. Who, or whom, evaluates the issues or proposals?
4. Is there a ‘statute of limitations’ for any such issues or proposals?
5. How does the community arrive at a decision?
6. How is that decision recorded and/or documented for the community?
7. How or what is put in place to help ensure every member is abiding by those decisions?
8. Based on discussion of previous items, is a SWEET manager required or can the community self manage under a more specific set of guidelines?

View Notes

Informatics efforts emerged largely in isolation across a number of disciplines. This new discipline, generally cast as the science and engineering of information systems originated in the middle of the last century and has undergone many adaptations and in the last two decades flourishing into discipline-specific fields like geoinformatics, bioinformatics, astroinformatics and more. Recently, certain core elements in informatics have been recognized as applicable across disciplines. Hence, efforts at systematizing the common (or core, i.e. discipline neutral) aspects of informatics have been successful: use cases, human-centered design, iterative approaches, information models and more are some of the key elements. Dr. Peter Fox has been instrumental in convening the Earth Science Informatics community, defining Informatics and Data Science in Earth Sciences, for his vision of “X-informatics” and the evolution of these fields as interdisciplinary research becomes widely accepted, and new challenges arise from the increased attention to a data-intensive approach in general. This includes creating or adapting informatics to address data that are high-dimensional, heterogeneous, sparse or with uncertain quality. We would like to dedicate this session to Dr. Peter Fox, a visionary, champion and an avid explorer of boundaries when it comes to Informatics and its benefits in scientific research. This session will showcase the field of Informatics, its history, current research, visions for the future and the role Dr. Peter Fox has in shaping these ideas and approaches.

Featured presentations:

• Mineral Informatics: Analytics, Visualization, and the Legacy of Peter Fox (Robert Hazen)
• X-informatics: making data science down to earth in the real world (Xiaogang (Marshall) Ma)

Since the ESIP Winter meeting, the Air Quality Cluster has been hard at work developing several use cases for air quality data and tools. We are now ready to try developing some applications, and will be kicking off a competitive application development effort at our Summer Meeting session. Esri, an ESIP member, and the company behind ArcGIS, has generously offered us the use of the ArcGIS Online (AGOL) framework for this hackathon competition, providing accounts and credits to hackathon participants to support development activities for a full month. The objective is to develop AQ data visualization applications that will aid both citizens and local governments in decision making related to Air Quality issues.

Using the AQ Cluster's use case as a guide, we will form three core project teams that will compete to develop the best AQ visualization application. We will conduct preparatory work during the May and June AQC meetings, to help participants pre-register, set up AGOL accounts, and get briefed on using AGOL. At the Summer Meeting session, done remotely, we’ll break into teams. Each team will develop an application concept together an implementation plan. We'll come together at the end of the session, and each team will provide a brief report. At the August 26 AQ Cluster meeting, teams will present their application to a set of ESIP-selected judges with prizes awarded for most effective and most creative. A more complete description of the concept can be found at https://docs.google.com/document/d/19ajXwjepWzaXv0AKIXZqKnEW8nkJ7KmQC3rd1bwVuPk/edit?usp=sharing

View Notes

Open science is the ability to share and reproduce analysis without sharing a computer. We recognize users have limited resources, such as network bandwidth and memory, and often this prevents them from thinking outside the box when it comes to scaling and sharing science. Open science presents a clear need to standardize on and deliver more cloud-friendly data formats and services. During this session, we highlight advances in cloud-friendly data and services and strive to answer some ongoing research in how these formats and services will support new scales of science and do so openly.

Cloud-friendly data formats and services are central to delivering new innovation in Earth science. With cloud-optimized data formats and services, Earth scientists can achieve new scales of analyses and deliver reproducible research output and information products.
The conversation about data formats is not one that will be “closed” with a decision on “one format to rule them all”. We propose a session centered around discussions which surface new advances in data formats and standards which specifically support sharing and scaling science on the cloud. Many call these formats “cloud-friendly” and “cloud-optimized” formats, respectively.

Putting data on the cloud in cloud-friendly formats is a starting point. Necessary to the utility of this data is the metadata, tools and services which support users accessing these datasets. There have been new advances in cloud-friendly services as well, however there is a lot of room for improvement. During this session, we focus not just on the data formats themselves, but on the usability of those formats made possible by the support system around using them.

Agenda (150 minutes):

Part 1: Lightning Talks - Provide a "lay of the land" and fodder for discussion:

• Aimee Barciauskas, Welcome to this session: What do we mean by cloud-optimized and why does it matter?
• 60 minutes of 7-10 minute lightning talks
  • Trevor Skaggs, Element84, will speak on Entwine Point Tile store generated for ATL06
  • Joe Roberts, NASA JPL, will speak about the Metadata Raster Format (MRF) and how it supports the features of NASA GIBS
  • Charles Stern will talk about motivations and progress made on pangeo-forge
  • Stavros Papadopoulos, creator of TileDB, will present "Time to depart from file formats and focus on engines and APIs"
  • Steve Olson  and Shane Mill will talk about NOAA's EDR API and how it enables programmatic access to both conventional and cloud-optimized data formats
  • Aaron Friesz will talk about the platform NASA LPDAAC has built to leverage cloud-optimized data formats.
• 15 minutes: organize into 3-4 sub groups for continuing the conversation on a specific topic or presentation.

15 minutes: Break 

Part 2: Small Group Discussions - Attendees and speakers will use this time to dive into discussions, questions and expertise on a sub-topic or specific question.

• 30 minutes: Small groups meet in a virtual sub-space. A session organizer or ESIP coordinator will meet with each small group to facilitate conversation and take notes.
• 25 minutes: Small groups present back to the larger group what was discussed
• 5 minutes: Wrap-up

View Notes

Since the ESIP Winter meeting, the Air Quality Cluster has been hard at work developing several use cases for air quality data and tools. We are now ready to try developing some applications, and will be kicking off a competitive application development effort at our Summer Meeting session. Esri, an ESIP member, and the company behind ArcGIS, has generously offered us the use of the ArcGIS Online (AGOL) framework for this hackathon competition, providing accounts and credits to hackathon participants to support development activities for a full month. The objective is to develop AQ data visualization applications that will aid both citizens and local governments in decision making related to Air Quality issues.

Using the AQ Cluster's use case as a guide, we will form three core project teams that will compete to develop the best AQ visualization application. We will conduct preparatory work during the May and June AQC meetings, to help participants pre-register, set up AGOL accounts, and get briefed on using AGOL. At the Summer Meeting session, done remotely, we’ll break into teams. Each team will develop an application concept together an implementation plan. We'll come together at the end of the session, and each team will provide a brief report. At the August 26 AQ Cluster meeting, teams will present their application to a set of ESIP-selected judges with prizes awarded for most effective and most creative. A more complete description of the concept can be found at https://docs.google.com/document/d/19ajXwjepWzaXv0AKIXZqKnEW8nkJ7KmQC3rd1bwVuPk/edit?usp=sharing

View Notes

In this working session, we will work together to curate some obscure-but-public data about Flint water quality for preparation for deposit in a suitable repository. This data is public but obscure, and currently stored in a series of spreadsheets. Our curation efforts will involve creating metadata, cleaning the datasets, and making them easier to understand. Our hope is to deposit this data in Open Data Flint for easier access by journalists and the general public, but other repositories are an option. https://www.icpsr.umich.edu/web/pages/odf/index.html

View Notes

This session is a working session of the Physical Samples Curation Cluster. During this session, we will report on related activities in other ESIP Clusters, updates from the relevant external Sample Communities (iSamples, SESAR, ESS DIVE, RDA, IMLGS, etc.) and continue efforts from the Cluster monthly meetings.

To continue efforts from the monthly meetings, we will have focused discussions on three topics. First, identified at the 2021 ESIP Winter Meeting, the development of recommendations for samples for journals and publishers. Second, we will review the landscape of ongoing efforts related to sample metadata (including top level categorizations, controlled vocabularies, and registration workflows). Finally, we will provide space for discussing leading practices and community questions (e.g. recommendations for identifiers and barcodes to streamline sample analysis and publications workflows).

The Physical Samples Curation Cluster is a forum for the community supporting physical samples in the earth, space, and environmental sciences which includes but is not limited to geological and biological samples. The cluster’s goal is to enhance discoverability, access, and use of sample collections.

Meeting notes - https://bit.ly/36VARj7
Session slides - https://bit.ly/3zbz4Cx


Agenda

• Introduction (5 minutes)
• Presentations on related activities (30 minutes)
  • iSamples and Sampling Nature 
  • RDA  Physical Samples and Collections in the Research Data Ecosystem IG
  • Research Artifact Citation Cluster update
  • COPDESS Cluster update
  • Other
• Breakout activity (45 minutes)
  • Recap/introduction to breakout activity (5 minutes)
  • Breakout group discussions (30 minutes)
  • Group reports (10 minutes)
• Closing/Wrap Up (10 minutes)

Session Agenda:

    Invited Presentations (1 hour) (see abstracts below)
        1. “Making Data Decision Ready” by David Green, Program Manager of the NASA Disaster Applications program
        2. “Creating Trust in Earth Observation Data” by Jasmine Muir, FrontiersSI, Australia
        3. “Evolving Operational Readiness Levels within ESIP’s Disaster Lifecycle Cluster” by Dave Jones/Karen Moe, ESIP Disasters Lifecycle Cluster
        4. “ESIP Information Quality Cluster Overview and Recent Efforts” by Yaxing Wei et al., ESIP Information Quality Cluster

    Panel Discussion (30 mins)


Session Description:

The ESIP Information Quality Cluster (IQC) has been collaborating across ESIP clusters and beyond ESIP with national and international domain experts on a number of fronts toward establishing a baseline of standards and best practices for Earth science data quality. These efforts include 1) exposition of the state-of-the-art practices and establishing recommendations to further promote the quantification, characterization, communication, and use of uncertainty information for broad classes of Earth science data, including on-orbit, airborne, field, and assimilated/modeled data; 2) developing community guidelines for consistently curating and representing dataset-level quality information; and 3) identifying challenges and potential approaches for improving citizen science data quality. The development of uncertainty recommendations and quality information guidelines is driven by community needs and the expected outcomes would have a high impact on the community. For example, the IQC is partnering with the ESIP Disasters Lifecycles Cluster (DLC) to mature a framework for determining the Operational Readiness Levels (ORLs) for data products driving disaster management decision-making. In this session, we will share updates with the ESIP community on the current status of those efforts and further strengthen the collaboration between the IQC and other clusters of ESIP to demonstrate the implications of recommendations and guidelines being developed by the IQC.

View Notes

Presentations Abstract:

1. Making Data Decision Ready (David Green)
It is true that data drives decisions, but not just any data. The NASA Disasters program embraces moving toward improvements in capturing, representing, and enabling data quality for risk reduction. Relevant and diverse data types must be discoverable sooner, more useful in complex scenarios, and used by a wider range of actors. These aims have motivated improvements to lower latency for faster dissemination of warnings and forecasts as well as higher resolution to increase local awareness but have not addressed analysis and decision readiness. From a Disasters program perspective, accurate, precise, and fit-for-use hazard data is still not sufficient since disaster “risk” is the consequence of vulnerability, exposure, and coping capacity. The NASA Disasters program utilizes an earth systems perspective and a user-centric approach, which advocates for data interoperability and open geospatial standards to facilitate integration and analysis readiness. Similarly, there have been tremendous improvements in geospatial information systems and frontier technologies, including collaborative tools for sharing, cloud processing, artificial intelligence, visualization, and natural language. We are moving closer to having a portfolio of data capabilities and attributes that make data decision ready, but the issues of representing quality must also mirror the shift we have experienced in supporting the Sendai Framework for Disaster Risk Reduction, specifically data quality for improved understanding of systemic risk and risk management. Metrics of data quality must inform the choices people make throughout the disaster management cycle, the situational awareness support needed during response, and the confidence in guidance as actions evolve. Certainly, within an environmental and socio-economic context the Disasters Program is starting to see the feasibility of quality in decision ready data. Knowing the data quality can also incentivize and support collaborative decision making with the increasing variety and velocity of data needed to meet measureable thresholds for guiding smarter and more resilient actions. 

2.  Creating Trust in Earth Observation Data (Jasmine Muir)
The presentation will provide an introduction to the Australian and New Zealand Data Quality Interest Group and their work on community standards for FAIR and Quality data. The standards will be demonstrated through a use case on creating trust in satellite Earth observation data and derived products.

Help us push the frontiers of democratic participation in environmental governance by joining this design workshop on a new data portal that enables members of environmental advocacy groups to ask geography-based questions about environmental enforcement!

Background: Vital data about federal enforcement actions against facilities that pollute the soil, air, and water is currently available but largely inaccessible in the U.S. Environmental Protection Agency’s (EPA) Enforcement and Compliance History Online (ECHO) database. We have been working for 1.5 years with data analysts, nonprofits, and community groups—and now with ESIP Lab funding—to develop well-documented and open source cloud-based Jupyter Notebooks that make ECHO data readily accessible and reportable by zip code, hydrologic unit code (to assess watersheds), state, and congressional district. However, we now have so many tools and reports that they can be hard to navigate and access!

What we’re making: We are now building a web portal to share our tools and reports. Our vision is an intuitive map-centric interface for three types of public interaction:

1. Accessing already-generated reports
2. Accessing our Jupyter Notebooks to generate custom reports (e.g. Clean Water Act violations in the Niagara River watershed)
3. Sharing these custom reports and some context about why the findings are important or how they are surprising.

Where you come in: Are there best practices we should know about for displaying these kinds of reports and tools? What are similar projects we should look at during the design process? For example, EPA’s How’s My Waterway tool, justicemap.org, and DataONE (possible integration potential?)

This workshop will take place in two parts:

• Part 1 is an introduction to the reports and tools. We will familiarize participants with the project through both a presentation and hands-on use of a Notebook.
• Part 2 is a design workshop exploring ideas for the web portal: a structured, facilitated discussion focused on developing user scenarios to inform web development.

View Notes