Mount Sinai Researchers Use New Deep Learning Approach to Enable Analysis of Electrocardiograms as Language
HeartBEiT is much more precise at highlighting areas of interest, in this case for diagnosing heart attacks (myocardial infarction). Image Credit: Augmented Intelligence in Medicine and Science Laboratory at the Icahn School of Medicine at Mount Sinai
Mount Sinai researchers have developed an innovative artificial intelligence (AI) model for electrocardiogram (ECG) analysis that allows for the interpretation of ECGs as language. This approach can enhance the accuracy and effectiveness of ECG-related diagnoses, especially for cardiac conditions where limited data is available on which to train.In a study published in the June 6 online issue of npj Digital Medicine DOI: 10.1038/s41746-023-00840-9, the team reported that its new deep learning model, known as HeartBEiT, forms a foundation upon which specialized diagnostic models can be created. The team noted that in comparison tests, models created using HeartBEiT surpassed established methods for ECG analysis.
“Our model consistently outperformed convolutional neural networks [CNNs], which are commonly used machine learning algorithms for computer vision tasks. Such CNNs are often pretrained on publicly available images of real-world objects,” says first author Akhil Vaid, MD, Instructor of Data-Driven and Digital Medicine (D3M) at the Icahn School of Medicine at Mount Sinai. “Because HeartBEiT is specialized to ECGs, it can perform as well as, if not better than, these methods using a tenth of the data. This makes ECG-based diagnosis considerably more viable, especially for rare conditions which affect fewer patients and therefore have limited data available.”
Thanks to their low cost, non-invasiveness, and wide applicability to cardiac disease, more than 100 million electrocardiograms are performed each year in the United States alone. Nonetheless, the ECG’s usefulness is limited in scope since physicians cannot consistently identify, with the naked eye, patterns representative of disease, particularly for conditions which do not have established diagnostic criteria or where such patterns may be too subtle or chaotic for human interpretation. Artificial intelligence is now revolutionizing the science, however, with most of the work to date centered on CNNs.
Mount Sinai is taking the field in a bold new direction by building on the intense interest in so-called generative AI systems such as ChatGPT, which are built on transformers—deep learning models that are trained on massive datasets of text to generate human-like responses to prompts from users on almost any topic. Researchers are using a related image-generating model to create discrete representations of small parts of the ECG, enabling analysis of the ECG as language.
“These representations may be considered individual words, and the whole ECG a single document,” explains Dr. Vaid. “HeartBEiT understands the relationships between these representations and uses this understanding to perform downstream diagnostic tasks more effectively. The three tasks we tested the model on were learning if a patient is having a heart attack, if they have a genetic disorder called hypertrophic cardiomyopathy, and how effectively their heart is functioning. In each case, our model performed better than all other tested baselines.”
Researchers pretrained HeartBEiT on 8.5 million ECGs from 2.1 million patients collected over four decades from four hospitals within the Mount Sinai Health System. Then they tested its performance against standard CNN architectures in the three cardiac diagnostic areas. The study found that HeartBEiT had significantly higher performance at lower sample sizes, along with better “explainability.” Elaborates senior author Girish Nadkarni, MD, MPH, Irene and Dr. Arthur M. Fishberg Professor of Medicine at Icahn Mount Sinai, Director of The Charles Bronfman Institute of Personalized Medicine, and System Chief, Division of Data-Driven and Digital Medicine, Department of Medicine: “Neural networks are considered black boxes, but our model was much more specific in highlighting the region of the ECG responsible for a diagnosis, such as a heart attack, which helps clinicians to better understand the underlying pathology. By comparison, the CNN explanations were vague even when they correctly identified a diagnosis.”
Indeed, through its sophisticated new modeling architecture, the Mount Sinai team has greatly enhanced the manner and opportunities by which physicians can interact with the ECG. “We want to be clear that artificial intelligence is by no means replacing diagnosis by professionals from ECGs,” explained Dr. Nadkarni, “but rather augmenting the ability of that medium in an exciting and compelling new way to detect heart problems and monitor the heart’s health.”
The paper is titled “A foundational vision transformer improves diagnostic performance for electrocardiograms.”
This study was funded by the National Heart, Lung, and Blood Institute of the NIH, grant number R01HL155915, and by the National Center for Advancing Translational Sciences of the NIH, grant number UL1TR004419.
To view the full list of authors and competing interests, see DOI: 10.1038/s41746-023-00840-9.
About the Icahn School of Medicine at Mount Sinai
The Icahn School of Medicine at Mount Sinai is internationally renowned for its outstanding research, educational, and clinical care programs. It is the sole academic partner for the eight- member hospitals* of the Mount Sinai Health System, one of the largest academic health systems in the United States, providing care to a large and diverse patient population.
Ranked 14th nationwide in National Institutes of Health (NIH) funding and among the 99th percentile in research dollars per investigator according to the Association of American Medical Colleges, Icahn Mount Sinai has a talented, productive, and successful faculty. More than 3,000 full-time scientists, educators, and clinicians work within and across 44 academic departments and 36 multidisciplinary institutes, a structure that facilitates tremendous collaboration and synergy. Our emphasis on translational research and therapeutics is evident in such diverse areas as genomics/big data, virology, neuroscience, cardiology, geriatrics, as well as gastrointestinal and liver diseases.
Icahn Mount Sinai offers highly competitive MD, PhD, and Master’s degree programs, with current enrollment of approximately 1,300 students. It has the largest graduate medical education program in the country, with more than 2,000 clinical residents and fellows training throughout the Health System. In addition, more than 550 postdoctoral research fellows are in training within the Health System.
A culture of innovation and discovery permeates every Icahn Mount Sinai program. Mount Sinai’s technology transfer office, one of the largest in the country, partners with faculty and trainees to pursue optimal commercialization of intellectual property to ensure that Mount Sinai discoveries and innovations translate into healthcare products and services that benefit the public.
Icahn Mount Sinai’s commitment to breakthrough science and clinical care is enhanced by academic affiliations that supplement and complement the School’s programs.
Through the Mount Sinai Innovation Partners (MSIP), the Health System facilitates the real-world application and commercialization of medical breakthroughs made at Mount Sinai. Additionally, MSIP develops research partnerships with industry leaders such as Merck & Co., AstraZeneca, Novo Nordisk, and others.
The Icahn School of Medicine at Mount Sinai is located in New York City on the border between the Upper East Side and East Harlem, and classroom teaching takes place on a campus facing Central Park. Icahn Mount Sinai’s location offers many opportunities to interact with and care for diverse communities. Learning extends well beyond the borders of our physical campus, to the eight hospitals of the Mount Sinai Health System, our academic affiliates, and globally.
-------------------------------------------------------
* Mount Sinai Health System member hospitals: The Mount Sinai Hospital; Mount Sinai Beth Israel; Mount Sinai Brooklyn; Mount Sinai Morningside; Mount Sinai Queens; Mount Sinai South Nassau; Mount Sinai West; and New York Eye and Ear Infirmary of Mount Sinai.
About the Mount Sinai Health System
Mount Sinai Health System is one of the largest academic medical systems in the New York metro area, with more than 43,000 employees working across eight hospitals, over 400 outpatient practices, nearly 300 labs, a school of nursing, and a leading school of medicine and graduate education. Mount Sinai advances health for all people, everywhere, by taking on the most complex health care challenges of our time — discovering and applying new scientific learning and knowledge; developing safer, more effective treatments; educating the next generation of medical leaders and innovators; and supporting local communities by delivering high-quality care to all who need it.
Through the integration of its hospitals, labs, and schools, Mount Sinai offers comprehensive health care solutions from birth through geriatrics, leveraging innovative approaches such as artificial intelligence and informatics while keeping patients’ medical and emotional needs at the center of all treatment. The Health System includes approximately 7,300 primary and specialty care physicians; 13 joint-venture outpatient surgery centers throughout the five boroughs of New York City, Westchester, Long Island, and Florida; and more than 30 affiliated community health centers. We are consistently ranked by U.S. News & World Report's Best Hospitals, receiving high "Honor Roll" status, and are highly ranked: No. 1 in Geriatrics and top 20 in Cardiology/Heart Surgery, Diabetes/Endocrinology, Gastroenterology/GI Surgery, Neurology/Neurosurgery, Orthopedics, Pulmonology/Lung Surgery, Rehabilitation, and Urology. New York Eye and Ear Infirmary of Mount Sinai is ranked No. 12 in Ophthalmology. U.S. News & World Report’s “Best Children’s Hospitals” ranks Mount Sinai Kravis Children's Hospital among the country’s best in several pediatric specialties.
For more information, visit https://www.mountsinai.org or find Mount Sinai on Facebook, Twitter and YouTube.