A New Artificial Intelligence Tool for Cancer

Scientists at Harvard Medical School have designed a versatile, ChatGPT-like AI model capable of performing an array of diagnostic tasks across multiple forms of cancers.

The new AI system, described Sept. 4 in Nature, goes a step beyond many current AI approaches to cancer diagnosis, the researchers said.

Current AI systems are typically trained to perform specific tasks — such as detecting cancer presence or predicting a tumor’s genetic profile — and they tend to work only in a handful of cancer types. By contrast, the new model can perform a wide array of tasks and was tested on 19 cancer types, giving it a flexibility similar to that of large language models such as ChatGPT.

Get more HMS news here

While other foundation AI models for medical diagnosis based on pathology images have emerged recently, this is believed to be the first to predict patient outcomes and validate them across several international patient groups.

“Our ambition was to create a nimble, versatile ChatGPT-like AI platform that can perform a broad range of cancer evaluation tasks,” said study senior author Kun-Hsing Yu, assistant professor of biomedical informatics in the Blavatnik Institute at Harvard Medical School. “Our model turned out to be very useful across multiple tasks related to cancer detection, prognosis, and treatment response across multiple cancers.”

The AI model, which works by reading digital slides of tumor tissues, detects cancer cells and predicts a tumor’s molecular profile based on cellular features seen on the image with superior accuracy to most current AI systems. It can forecast patient survival across multiple cancer types and accurately pinpoint features in the tissue that surrounds a tumor — also known as the tumor microenvironment — that are related to a patient’s response to standard treatments, including surgery, chemotherapy, radiation, and immunotherapy. Finally, the team said, the tool appears capable of generating novel insights — it identified specific tumor characteristics previously not known to be linked to patient survival.

The findings, the research team said, add to growing evidence that AI-powered approaches can enhance clinicians’ ability to evaluate cancers efficiently and accurately, including the identification of patients who might not respond well to standard cancer therapies.

“If validated further and deployed widely, our approach, and approaches similar to ours, could identify early on cancer patients who may benefit from experimental treatments targeting certain molecular variations, a capability that is not uniformly available across the world,” Yu said.

Training and performance

The team’s latest work builds on Yu’s previous research in AI systems for the evaluation of colon cancer and brain tumors. These earlier studies demonstrated the feasibility of the approach within specific cancer types and specific tasks.

The new model, called CHIEF (Clinical Histopathology Imaging Evaluation Foundation), was trained on 15 million unlabeled images chunked into sections of interest. The tool was then trained further on 60,000 whole-slide images of tissues including lung, breast, prostate, colorectal, stomach, esophageal, kidney, brain, liver, thyroid, pancreatic, cervical, uterine, ovarian, testicular, skin, soft tissue, adrenal gland, and bladder. Training the model to look both at specific sections of an image and the whole image allowed it to relate specific changes in one region to the overall context. This approach, the researchers said, enabled CHIEF to interpret an image more holistically by considering a broader context, instead of just focusing on a particular region.

Following training, the team tested CHIEF’s performance on more than 19,400 whole-slide images from 32 independent datasets collected from 24 hospitals and patient cohorts across the globe.

Overall, CHIEF outperformed other state-of-the-art AI methods by up to 36 percent on the following tasks: cancer cell detection, tumor origin identification, predicting patient outcomes, and identifying the presence of genes and DNA patterns related to treatment response.

Because of its versatile training, CHIEF performed equally well no matter how the tumor cells were obtained — whether via biopsy or through surgical excision. And it was just as accurate, regardless of the technique used to digitize the cancer cell samples. This adaptability, the researchers said, renders CHIEF usable across different clinical settings and represents an important step beyond current models that tend to perform well only when reading tissues obtained through specific techniques.