Machine Learning (ML) is the frontier in revolutionizing how we conduct research across all Earth Science disciplines. ML techniques including deep learning are increasingly being applied to problems in Earth science for classification, regression, or clustering applications. Development of ML models involves choices of model architecture, training data, hyperparameters, training techniques and so on. These steps complicate reproducibility of ML model results unless the developer has shared a detailed description and code of their application. In addition, many ML model development efforts rely on existing modeling architectures and “pre-trained” models on benchmark data. Therefore, it is necessary to develop a set of guidelines for researchers and developers in the Earth science community to follow for publishing their models to ensure they are reproducible and reusable. This would also require a model metadata specification so enable cataloging and discoverability of models.
In this session speakers will present the latest developments for FAIR principles as it related to ML models, the activities of NASA ESDS WG on model reusability, and introduce Geospatial ML Model Catalog (GMLMC) to get the participants familiar with these efforts, and engage them to advance the guideline and specification. In the second part of the session, participants will engage in a set of breakout sessions and discuss questions related to FAIR principles and reusability of ML models.
Agenda:11:00 am - Welcome, Introduction & logistics
11:05 am - The road toward defining FAIR for Machine Learning, Fotis Psomopoulos
11:15 am - ML Model Reusability ESDWG, Sanjay Purushotham, Hamed Alemohammad
11:25 am - Geospatial Machine Learning Model Catalog (GMLMC), Jon Duckworth
11:35 am - Breakout Sessions
Q&A for breakout sessions:
- How should FAIR be applied to ML? What changes/definitions are needed? Should this address only ML models, and/or also processes, and/or also platforms, etc.?
- How much do you reuse your ML or other models? What are the good practices you follow for model reusability and/or reproducibility?
- What tools and specifications are needed to study if model reusability is feasible?
- What metadata related to model metrics should be included in the GMLMC to communicate performance and possible bias?
- What properties should be considered as metadata to capture requirements of the training fragment in GMLMC? Should there be different requirements for reproducing the model results vs reusing the model?
12:10 pm - Break out report out and Q&A
12:30 pm - Closing remarks
View Notes
