When rural Minnesota music teacher Shelly Engfer-Triebenbach started getting winded while doing the Hokey Pokey with her students, she knew something was wrong. She went to a doctor who gave her antibiotics for what was initially diagnosed as pneumonia, but days later learned that the shortness of breath was due to stage 4 non-small cell lung cancer.
She was 40 years old. The average age of lung cancer patients at diagnosis is 70.
“That didn’t set right with me,” said Engfer-Triebenbach, now 46. “I sought four other opinions before I settled on a doctor who said, ‘You’re too young to have this disease. You fit the criteria of an ALK mutation.’”
Engfer-Triebenbach’s first oncologist had tested her for ALK and EGFR, two common mutations that drive non-small cell lung cancer (among others) and told her she was negative for both. Her second oncologist suggested she be retested at a different lab then started her on a chemotherapy.
“I didn’t have a lot of hope at the time,” she said. “If you don’t have those mutations, you’re lucky to live a year. My oncologist sent in my test, started me on a second line of chemo, which was really awful, then three weeks later said, ‘You’re not going to believe this. You’re ALK positive.’ It was like winning the lung cancer lottery. There are so many other things available. It was a total life-changer.”
Targeted oral therapies for these two common mutations — ALK, short for anaplastic lymphoma kinase, and EGFR, short for epidermal growth factor receptor — have been life-changers for many patients. So much so, that they’re now offered as a first-line treatment for patients diagnosed with stage 4 non-small cell lung cancer.
But not all lung cancer patients are being tested and, as in the case of Engfer-Treibenbach, some test results can be flawed.
That’s why researchers at Fred Hutchinson Cancer Research Center in Seattle are road-testing a big data workaround, using a type of artificial intelligence called natural language processing, or NLP, to delve into the pathology reports of patients with metastatic non-small cell lung cancer to find these two treatable mutations. The results of their first study were published today in the Journal of Clinical Cancer Informatics; additional studies are already underway.
“We’re trying to develop an AI method to capture EGFR and ALK mutations in patients’ records,” said lead author Dr. Bernardo Goulart, a researcher and lung cancer oncologist with Fred Hutch and its clinical care partner Seattle Cancer Care Alliance.
“It’s all about helping patients and clinicians move forward with advancing targeted therapies and personalized medicine,” he continued. “We know these patients should be treated up front with specific oral targeted therapies that are highly effective and less toxic than chemotherapy and recommended by all the guidelines. We wanted to know, in real practice, if this patient population was getting timely access to these oral drugs and what their outcomes were. We also wanted to know whether patients were getting tested.”
Goulart’s team, which included the Hutch’s Drs. Steve Schwartz, Scott Ramsey, Christina Baik and researchers from the University of Kentucky, worked with Hutch NLP engineer Emily Silgard to develop and train an algorithm to plumb the Washington state and Kentucky cancer registries for metastatic lung cancer patients diagnosed and tested for the two mutations between 2011 and 2013.
Cancer patients’ electronic health records, including tumor pathology reports, are customarily fed into state cancer registries that go on to form the National Cancer Institute’s Surveillance, Epidemiology and End Results Program, known as SEER. The SEER database, which serves as a proxy for the U.S. cancer patient population, provides valuable information to researchers on cancer incidence and mortality, as well as patient demographics, tumor characteristics and treatment.
NCI and SEER, which funded the Hutch study, hope to improve the registry by tracking and recording genomic biomarkers, such as EGFR and ALK, crucial information patients and doctors use to make treatment decisions (Hutch researchers are also working with NCI to fill in the recurrence data gaps). Manual abstraction of genetic data is time-consuming and labor intensive. Automating the process could make it more efficient and cost-effective and bump up the value of the national database.
“SEER and the NCI are trying to develop ways to report on important molecular markers in cancer efficiently. One of the questions is, ‘Can they use NLP for that?’” Goulart said. “The ultimate goal is to have a way to identify patients treated in real world scenarios and clinics whose lung cancers carry those mutations.”
After training the NLP algorithms, the research team sent it digging through the electronic pathology reports of Washington’s Cancer Surveillance System, or CSS, housed at Fred Hutch, and the Kentucky Cancer Registry, housed at the University of Kentucky.
If a test for the mutations was found, the NLP algorithm reported whether the patient tested positive or negative. It also reported on the type of test that was used to determine the mutation.
Then the researchers pitted man against machine. Two practicing oncologists dove in and read the records for each of the qualifying cancer patients: 1,634 from Washington and 565 from Kentucky.
“They read all the pathology reports from two registries and generated a gold standard ‘internal validation data set’ of what these EGFR and ALK results were when available,” said Goulart. “Then we had the NLP run the same reports and show their results and compared them with the manually extracted data.”
As it turned out, the NLP algorithm did quite well against actual oncologists in the CSS registry housed at Fred Hutch. On the EGFR mutation, the algorithm scored 97 percent on sensitivity and 98 percent on precision. On ALK, it scored 95 percent on sensitivity and 100 percent on precision. The NLP did not perform as well with the Kentucky registry, however, scoring 29 percent on sensitivity and 48 percent on precision for EGFR mutations and 100 percent on sensitivity and 2 percent on precision for ALK mutations.
The algorithm also identified the type of genetic test used with patients: Mutational analysis was the standard for EGFR testing and FISH, or fluorescence in situ hybridization, was the standard for determining an ALK mutation.
"The appropriate test technique is key for EGFR therapies to work," said Goulart. "The NLP accurately detects EGFR by mutational analysis and ALK by all techniques, particularly FISH, which is by far the most common ALK test used in practice."
“Overall, this study is important because we now have — at least here in the Cancer Surveillance System registry — a way to identify patients with EGFR and ALK mutations,” Goulart said. “We can now track their outcomes and their survival.”
The NLP algorithm, when coupled with other data analysis, will also allow Hutch public health researchers to determine who is and isn’t getting tested, who is and isn’t getting timely access to the new oral therapies and what the cost is to patients and insurance companies for both.
As for the NLP’s poor performance in the Kentucky registry, Goulart said it might be due to the out-of-state registry’s use of different formats or because the CSS was used to train the NLP.
“We developed the NLP algorithms here,” he said. “Machine learning tends to be valid for situations in which the group where you use them for testing is similar to the group where you train them.”
One major issue illuminated by the study: Nearly 50 percent of the patients in the two registries had no information about EGFR and ALK tests in their records.
“One possibility is that the oncologist never ordered the test,” Goulart said. “Or the test occurred but the results never made it into the registry."
Goulart said it's impossible to know what the issue is, but he shared the information with the NCI.
"My number one concern was the lack of completeness in reporting the tests," he said. "But that information can be added into the registry later. Moving forward, we need to make sure all labs report the results to the SEER registry for that region.”
Engfer-Triebenbach, whose cancer journey led her to establish the patient advocacy group ALK Fusion (other mutation-specific patient groups include ROS1nders and EGFR Resisters), applauded the Hutch study.
“It’s exciting that they’re using AI to look at this,” she said. “Some of these rural doctors, like at the clinic where I was first diagnosed, just don’t have this knowledge. They don’t seek it out or they’re in charge of so many cancers they can’t keep up on all of them.”
And knowledge, she said, is definitely power.
“I was told by three separate doctors that I had three to six months to live,” she said. “One of them told me to ‘Start getting your affairs in order.’ Then I had 12 rounds of really awful chemotherapy before being switched to the ALK inhibitor, which was great. There were no side effects or they were so limited I could still golf 18 holes with my kids; I could play tennis with them. I went back to the classroom and taught part-time again. I started coaching. It was a totally different lifestyle.”
Goulart said a precision approach for lung cancer is only the beginning. He believes the NLP algorithm could also dig into the electronic health records of other patient populations dealing with other cancers.
“This study could absolutely serve as a prototype to look at molecular mutations in other tumors,” he said. “We did it in lung cancer but there’s no reason that you can’t do this in other tumor settings where patients might benefit from oral drugs, like melanoma patients with the B-RAF mutations. And we can do even a better job designing a more robust AI that can be validated across all cancer registries, not just in ours.”
Diane Mapes is a staff writer at Fred Hutchinson Cancer Center. She has written extensively about health issues for NBC News, TODAY, CNN, MSN, Seattle Magazine and other publications. A breast cancer survivor, she blogs at doublewhammied.com and tweets @double_whammied. Email her at dmapes@fredhutch.org. Just diagnosed and need information and resources? Visit our Patient Care page.
Every dollar counts. Please support lifesaving research today.
For the Media