News reports in January that President Donald Trump passed a widely used test that screens for mild cognitive impairment, bringing the little-known clinical tool into the public eye. Google searches for the test – the Montreal Cognitive Assessment (MoCA) – peaked when dozens of media reports shared parts or all test and political commentators fired around it.
The president’s supporters proudly passed the test, bragging about Trump’s perfect 30-for-30 score and using it to laugh at those who questioned Trump’s mental state. Others chuckled at the seemingly simple components of the test, such as asking test subjects to correctly draw the times on a clock and identify animals.
But the laugh may be on all of us, according to a research letter published Monday, July 16 in JAMA Neurology.
In the brief announcement, physicians led by Hourmazd Haghbayan of the University of Toronto collected data on the extent of public exposure receiving the validated clinical device. Most importantly, they pointed out that several studies have suggested that such publicity may make the MoCA less reliable for detecting early stages of cognitive decline or dementia in test subjects. Essentially, seeing or taking the test allows candidates to learn the questions — and the answers — resulting in artificially improved scores. This could potentially obscure signs of early cognitive decline in overall clinical assessments that rely on the tool, they warn.
“The presence of a learning effect may lead to decreased sensitivity, affecting the main utility of the MoCA in identifying patients with mild cognitive impairment,” they write. “Future research is needed to validate test performance in populations with direct exposure to the questionnaire content or assessment scheme.”
In the meantime, Dr. Haghbayan and colleagues suggest that clinicians ask patients if they have been exposed to the MoCA before giving it to them. They also recommend that clinicians use alternative versions than the one most emphasized in news articles:i.e, version one as opposed to versions two and three.
Surpass the test
Before their warning and recommendations, the doctors were trying to assess the extent of the test’s exposure. Simple Google News and Google Trends data suggested broad coverage. The doctors found 190 unique news articles discussing MoCA as part of Trump’s tests. Of these, 53.7 percent (102 articles) included all or some of the questions of the test, while 44 percent (84 articles) included the test in its entirety. Seventeen percent invited readers to take it, and nearly nine percent published the answer key. Outside of news articles, Internet searches for the MoCA skyrocketed compared to searches for other cognitive tests, the doctors found.
This is a problem, as previous research has suggested that people can easily learn the test and score better after being exposed to it. For example, a 2015 study of 468 participants found that those who repeated the test within a year of taking it for the first time were able to improve their scores. Another 2015 study found similar evidence of a learning effect. Most importantly, the study indicated that those who initially scored low on the MoCA had a significantly higher score when they took it again 12 months later.
The authors concluded:
These results suggest that the MoCA may be sensitive to field effects, especially between the first and second administrations. These field effects should be taken into account when the MoCA is used repeatedly to screen for cognitive status in healthy older adults.
Controversial conjecture
Widespread coverage of the MoCA and the president’s test results followed intense media coverage of Trump’s potential mental state and health — and debate over whether such coverage was acceptable. According to the White House medical staff, Trump was the first president to undergo such cognitive tests, which were done at his own request amid all the media attention.
In the past, such media coverage was strictly off limits. Ethical standards, such as the Goldwater Rule passed by the American Psychiatric Association in 1973, prohibit mental health professionals from speculating publicly about the mental state of public figures who have not personally examined them. However, in the current political climate, some doctors and journalists have broken such rules. They claim that their speculation is for the greater good when they see a danger in the behavior of a public figure. Others have argued to uphold the ethical standard, pointing out that remote diagnoses are not reliable and that throwing labels around can increase the stigma of mental illness.
Journalists from FiveThirtyEight added the argument, saying that media speculation about the mental state of public figures is “beyond the scope of our profession.” They also argued that it is not helpful in Trump’s case. Many of the characteristics and behaviors that led to media speculation were observable in the president before his election.
The new letter in JAMA Neurology can extend the argument even further, suggesting that focusing on a public figure’s mental state can have additional widespread negative consequences, such as potentially weakening the usefulness of a common assessment tool.
JAMA Neurology2018. DOI: 10.1001/jamaneurol.2018.1777 (About DOIs).