Mount Sinai Researchers Develop Software to Measure the Findability, Accessibility, Interoperability, and Reusability of Biomedical Digital Research Objects
The FAIRshake Toolbox Developed to Promote Information Exchange in Biomedicine
Mount Sinai investigators, together with an international consortium of colleagues, have developed a new toolbox to allow scientists to assess how easily the digital objects they generate as a result of their research projects can be used by other research teams. The findings, reported today in Cell Systems, apply to digital objects including datasets, software, source code, and other products that scientists develop that may be useful to the research community.
“The goal is to upgrade the way we are managing data in biomedical research—sharing and reusing data from these projects to maximally extract value,” said Avi Ma’ayan, PhD, Director of the Mount Sinai Center for Bioinformatics, and senior researcher on the study. “While funders, researchers, and other stakeholders recognized that we need to do a better job with data management in biomedical research, the path toward doing so wasn’t clear.” The team hopes that as biomedical researchers apply the toolbox to evaluate their work, they’ll become more aware of simple steps they can take to improve the usability of their digital objects.
In 2017 and 2018 the National Institutes of Health (NIH) funded the NIH Data Commons Pilot Phase Consortium, which worked to find ways to help improve the sharing of digital research objects through a cloud-based environment. To implement such a virtual environment, researchers needed to develop standards and a platform to help scientists adhere to the FAIR (findable, accessible, interoperable, and reusable) guidelines which were previously established by members of the consortium.
First, the team developed metrics and rubrics to assess FAIRness, but recognized that these metrics and rubrics may not apply equally to all research domains. So they created the toolbox, called FAIRshake, which allows those creating digital objects not only to assess their FAIRness but also to develop new standards to define and evaluate FAIRness. Thus, the researchers stated that “FAIRshake allows for the coexistence of multiple metrics and rubrics, enabling the community to develop standards more democratically.”
Dr. Ma’ayan emphasizes that research does not need to be open-access in order to be FAIR. “It just requires people to provide a clear license of how others should use their data,” he said.
The FAIRshake toolbox includes both manual and automated tools for assessing FAIRness, which ask questions such as, “Does the digital object have machine-readable metadata?” The toolbox has been tested over the past year on thousands of digital objects and the team has posted tutorials on Youtube about how to use FAIRshake.
While Dr. Ma’ayan says that there has been a “natural improvement of accessibility and adoption of those standards even without any push from us or from NIH,” he also explains that the team was surprised by some of the trends they noticed when performing assessments with the toolkit. He said that few researchers implemented application programming interfaces (APIs)—software that allows different computer programs to communicate—which are “critical for bringing data together.” Researchers also frequently failed to provide licenses to tell other scientists how to get permission to use the data.
As research products become more FAIR, scientists will be able to make connections and build things that were not possible before using tools like machine learning to “transform the way we are reusing data,” says Dr. Ma’ayan.
About the Mount Sinai Health System
The Mount Sinai Health System is New York City's largest academic medical system, encompassing eight hospitals, a leading medical school, and a vast network of ambulatory practices throughout the greater New York region. Mount Sinai is a national and international source of unrivaled education, translational research and discovery, and collaborative clinical leadership ensuring that we deliver the highest quality care—from prevention to treatment of the most serious and complex human diseases. The Health System includes more than 7,200 physicians and features a robust and continually expanding network of multispecialty services, including more than 400 ambulatory practice locations throughout the five boroughs of New York City, Westchester, and Long Island. The Mount Sinai Hospital is ranked No. 14 on U.S. News & World Report's "Honor Roll" of the Top 20 Best Hospitals in the country and the Icahn School of Medicine as one of the Top 20 Best Medical Schools in country. Mount Sinai Health System hospitals are consistently ranked regionally by specialty and our physicians in the top 1% of all physicians nationally by U.S. News & World Report.