Crowdsourcing for Scientific Discovery: Mount Sinai Researchers Find Novel Ways to Analyze Data for Drug and Target Discovery
In a unique project, researchers at the Icahn School of Medicine at Mount Sinai have crowdsourced the annotation and analysis of a large number of gene expression profiles from the National Center for Biotechnology Information’s (NCBI) Gene Expression Omnibus (GEO). More than 70 volunteers from 25 countries helped Mount Sinai researchers analyze the data, enabling the identification of new associations between genes, diseases, and drugs – something that a smaller number of unaided researchers, or an automated computer program, would not be able to achieve. An article published today in the journal Nature Communications describes the crowdsourcing project.
Omics repositories, which are virtual storehouses for raw gene expression data, contain thousands of studies. Such an abundance of data opens opportunities for integrative analyses that can uncover new knowledge that was missed or was not possible in the initial publication of the data. For example, while a dataset from a given study may have been used for a particular published article, that same dataset may contain evidence whose value can only become realized when combined with data from another study. Then, it might become apparent that a drug can be repurposed to treat a different disease. Several computerized search engines have been designed to comb through this data. However, for these tools to be effective they require heavy, time-consuming human curation to ensure accuracy.
That is where crowdsourcing can be useful. For this project, the 70 volunteers were recruited through a massive open online course (MOOC), which was being taught on the Coursera MOOC platform by Avi Ma’ayan, PhD, Professor of Pharmacological Sciences and Director of the Mount Sinai Center of Bioinformatics at the Icahn School of Medicine at Mount Sinai. The student volunteers were asked first to identify relevant studies in the NCBI GEO database – in this case, studies in which single-gene or single-drug perturbations were applied to mammalian cells, or in which normal versus diseased tissues were compared. Once the studies were selected, the volunteers extracted metadata from the studies, and then computed differential expression using a custom-designed Chrome browser extension developed by the Mount Sinai researchers.
This process extracted information about gene signatures – observations of groups of genes whose combined expression is associated with a particular condition or drug action – which were stored in a new database. Then, Dr. Ma’ayan and his team used the database to visualize and analyze the signatures on a web portal known as Crowd Extracted Expression of Differential Signatures, or CREEDS, which was developed by the Ma’ayan Lab at Mount Sinai. Over the course of the project, over 3,100 single-gene perturbations from more than 2,300 studies were submitted, as well as 1,238 single-drug perturbations from nearly 450 studies.
“There is an incredible amount of data stored in these databases, but much of it has not been fully explored,” said Dr. Ma’ayan. “The profiling and extrication of gene expression signatures is time-consuming and labor-intensive, and cannot be completely automated. By utilizing volunteers, so called ‘citizen-scientists,’ we were able to bring a much greater scale of human curation and quality control than we could have performed alone. By combining that human touch with automated programs, we could process much more data than would have been otherwise possible.”
Ultimately, the manually extracted signatures were used as a training set to help a program that uses machine learning to process all the data currently available in GEO for adding more drug, gene, and disease signatures to the CREEDS database. While researchers generally find that the quality of automatically generated signatures is subpar compared to those created by humans, such crowdsourced efforts can be integrated with machine learning to help refine the data. Instances that the computer programs find more difficult can be presented to the crowdsourced human curators with suggestions; this allows for higher quality data, while reducing effort required of the volunteer.
“We are grateful to the volunteers who helped demonstrate that citizen-scientists, working with researchers towards a common goal, can achieve remarkable results that have a real impact,” said Dr. Ma’ayan. “Such collective efforts can help us discover new drugs, new causes of diseases, and new scientific knowledge.”
While many new relationships between genes, drugs, and diseases were identified, further hypotheses can be formed through additional analysis of the data, which Dr. Ma’ayan and his team have made available to the public on the CREEDS portal. To interact with the portal, visit http://amp.pharm.mssm.edu/creeds.
About the Mount Sinai Health System
The Mount Sinai Health System is New York City's largest academic medical system, encompassing eight hospitals, a leading medical school, and a vast network of ambulatory practices throughout the greater New York region. Mount Sinai is a national and international source of unrivaled education, translational research and discovery, and collaborative clinical leadership ensuring that we deliver the highest quality care—from prevention to treatment of the most serious and complex human diseases. The Health System includes more than 7,200 physicians and features a robust and continually expanding network of multispecialty services, including more than 400 ambulatory practice locations throughout the five boroughs of New York City, Westchester, and Long Island. The Mount Sinai Hospital is ranked No. 14 on U.S. News & World Report's "Honor Roll" of the Top 20 Best Hospitals in the country and the Icahn School of Medicine as one of the Top 20 Best Medical Schools in country. Mount Sinai Health System hospitals are consistently ranked regionally by specialty and our physicians in the top 1% of all physicians nationally by U.S. News & World Report.