This event has ended. Create your own event on Sched.
For over 20 years, ESIP meetings have brought together the most innovative thinkers and leaders around Earth observation data, thus forming a community dedicated to making Earth observations more discoverable, accessible and useful to researchers, practitioners, policymakers, and the public. The theme of this year’s meeting is Leading Innovation in Earth Science Data Frontiers.
Back To Schedule
Thursday, July 22 • 11:00am - 12:30pm
Best Practices for Reusability of Machine Learning Models: Guideline and Specification

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
Machine Learning (ML) is the frontier in revolutionizing how we conduct research across all Earth Science disciplines. ML techniques including deep learning are increasingly being applied to problems in Earth science for classification, regression, or clustering applications. Development of ML models involves choices of model architecture, training data, hyperparameters, training techniques and so on. These steps complicate reproducibility of ML model results unless the developer has shared a detailed description and code of their application. In addition, many ML model development efforts rely on existing modeling architectures and “pre-trained” models on benchmark data. Therefore, it is necessary to develop a set of guidelines for researchers and developers in the Earth science community to follow for publishing their models to ensure they are reproducible and reusable. This would also require a model metadata specification so enable cataloging and discoverability of models.

In this session speakers will present the latest developments for FAIR principles as it related to ML models, the activities of NASA ESDS WG on model reusability, and introduce Geospatial ML Model Catalog (GMLMC) to get the participants familiar with these efforts, and engage them to advance the guideline and specification. In the second part of the session, participants will engage in a set of breakout sessions and discuss questions related to FAIR principles and reusability of ML models. 

11:00 am - Welcome, Introduction & logistics
11:05 am - The road toward defining FAIR for Machine Learning, Fotis Psomopoulos 
11:15 am - ML Model Reusability ESDWG, Sanjay Purushotham, Hamed Alemohammad
11:25 am - Geospatial Machine Learning Model Catalog (GMLMC), Jon Duckworth
11:35 am - Breakout Sessions
Q&A for breakout sessions:
  1. How should FAIR be applied to ML? What changes/definitions are needed? Should this address only ML models, and/or also processes, and/or also platforms, etc.? 
  2. How much do you reuse your ML or other models? What are the good practices you follow for model reusability and/or reproducibility?
  3. What tools and specifications are needed to study if model reusability is feasible?  
  4. What metadata related to model metrics should be included in the GMLMC to communicate performance and possible bias?
  5. What properties should be considered as metadata to capture requirements of the training fragment in GMLMC? Should there be different requirements for reproducing the model results vs reusing the model?
12:10 pm - Break out report out and Q&A
12:30 pm - Closing remarks

View Notes

Organizers & Speakers
avatar for Hamed Alemohammad

Hamed Alemohammad

Executive Director and Chief Data Scientist, Radiant Earth Foundation
avatar for Jon Duckworth

Jon Duckworth

Tech Lead & Geospatial Software Engineer, Radiant Earth Foundation
Jon Duckworth is passionate about making geospatial data and tools accessible to people and organizations who are working to make the world a better place. He has extensive experience building scalable data pipelines to support machine learning on satellite imagery and bringing geospatial... Read More →
avatar for Sanjay Purushotham

Sanjay Purushotham

University of Maryland Baltimore County
avatar for Fotis Psomopoulos

Fotis Psomopoulos

Researcher, Institute of Applied Biosciences, Centre for Research and Technology Hellas

Thursday July 22, 2021 11:00am - 12:30pm EDT