Mount Sinai Researchers Develop Software to Measure the Findability, Accessibility, Interoperability, and Reusability of Biomedical Digital Research Objects
The FAIRshake Toolbox Developed to Promote Information Exchange in Biomedicine
Mount Sinai investigators, together with an international consortium of colleagues, have developed a new toolbox to allow scientists to assess how easily the digital objects they generate as a result of their research projects can be used by other research teams. The findings, reported today in Cell Systems, apply to digital objects including datasets, software, source code, and other products that scientists develop that may be useful to the research community.
“The goal is to upgrade the way we are managing data in biomedical research—sharing and reusing data from these projects to maximally extract value,” said Avi Ma’ayan, PhD, Director of the Mount Sinai Center for Bioinformatics, and senior researcher on the study. “While funders, researchers, and other stakeholders recognized that we need to do a better job with data management in biomedical research, the path toward doing so wasn’t clear.” The team hopes that as biomedical researchers apply the toolbox to evaluate their work, they’ll become more aware of simple steps they can take to improve the usability of their digital objects.
In 2017 and 2018 the National Institutes of Health (NIH) funded the NIH Data Commons Pilot Phase Consortium, which worked to find ways to help improve the sharing of digital research objects through a cloud-based environment. To implement such a virtual environment, researchers needed to develop standards and a platform to help scientists adhere to the FAIR (findable, accessible, interoperable, and reusable) guidelines which were previously established by members of the consortium.
First, the team developed metrics and rubrics to assess FAIRness, but recognized that these metrics and rubrics may not apply equally to all research domains. So they created the toolbox, called FAIRshake, which allows those creating digital objects not only to assess their FAIRness but also to develop new standards to define and evaluate FAIRness. Thus, the researchers stated that “FAIRshake allows for the coexistence of multiple metrics and rubrics, enabling the community to develop standards more democratically.”
Dr. Ma’ayan emphasizes that research does not need to be open-access in order to be FAIR. “It just requires people to provide a clear license of how others should use their data,” he said.
The FAIRshake toolbox includes both manual and automated tools for assessing FAIRness, which ask questions such as, “Does the digital object have machine-readable metadata?” The toolbox has been tested over the past year on thousands of digital objects and the team has posted tutorials on Youtube about how to use FAIRshake.
While Dr. Ma’ayan says that there has been a “natural improvement of accessibility and adoption of those standards even without any push from us or from NIH,” he also explains that the team was surprised by some of the trends they noticed when performing assessments with the toolkit. He said that few researchers implemented application programming interfaces (APIs)—software that allows different computer programs to communicate—which are “critical for bringing data together.” Researchers also frequently failed to provide licenses to tell other scientists how to get permission to use the data.
As research products become more FAIR, scientists will be able to make connections and build things that were not possible before using tools like machine learning to “transform the way we are reusing data,” says Dr. Ma’ayan.
About the Mount Sinai Health System
The Mount Sinai Health System is New York City's largest integrated delivery system, encompassing eight hospitals, a leading medical school, and a vast network of ambulatory practices throughout the greater New York region. Mount Sinai's vision is to produce the safest care, the highest quality, the highest satisfaction, the best access and the best value of any health system in the nation. The Health System includes approximately 7,480 primary and specialty care physicians; 11 joint-venture ambulatory surgery centers; more than 410 ambulatory practices throughout the five boroughs of New York City, Westchester, Long Island, and Florida; and 31 affiliated community health centers. The Icahn School of Medicine is one of three medical schools that have earned distinction by multiple indicators: ranked in the top 20 by U.S. News & World Report's "Best Medical Schools", aligned with a U.S. News & World Report's "Honor Roll" Hospital, No. 12 in the nation for National Institutes of Health funding, and among the top 10 most innovative research institutions as ranked by the journal Nature in its Nature Innovation Index. This reflects a special level of excellence in education, clinical practice, and research. The Mount Sinai Hospital is ranked No. 14 on U.S. News & World Report's "Honor Roll" of top U.S. hospitals; it is one of the nation's top 20 hospitals in Cardiology/Heart Surgery, Diabetes/Endocrinology, Gastroenterology/GI Surgery, Geriatrics, Gynecology, Nephrology, Neurology/Neurosurgery, and Orthopedics in the 2019-2020 "Best Hospitals" issue. Mount Sinai's Kravis Children's Hospital also is ranked nationally in five out of ten pediatric specialties by U.S. News & World Report. The New York Eye and Ear Infirmary of Mount Sinai is ranked 12th nationally for Ophthalmology, Mount Sinai St. Lukes and Mount Sinai West are ranked 23rd nationally for Nephrology and 25th for Diabetes/Endocrinology, and Mount Sinai South Nassau is ranked 35th nationally for Urology. Mount Sinai Beth Israel, Mount Sinai St. Luke's, Mount Sinai West, and Mount Sinai South Nassau are ranked regionally.