On October 20, 2014, the Stanford Center on Longevity (the “Longevity Center”) released a statement presented as “A Consensus on the Brain Training Industry from the Scientific Community.” The statement is critical of the brain fitness industry, and given Posit Science’s leadership in that industry, I want to take the time to respond to it carefully.
I fundamentally agree with a basic thrust of the statement—no one should make science claims for brain training with little or no evidence, and no one should mislead the public about what the science shows by exaggeration or omission. And I also agree with the regrettable points raised by the statement that some companies in our nascent brain fitness industry engage in this type of marketing behavior.
Having said that, I strongly disagree with the claims in the statement that there is little or no evidence of efficacy of cognitive training. There are now more than 70 peer-reviewed papers about the benefits of the BrainHQ exercises from Posit Science. More than 50 of those papers focus specifically on the benefits of our cognitive exercises in aging. These studies have involved many thousands of research participants. Multiple scientific reviews and meta-analyses have confirmed these positive results. From this enormous body of scientific work, the Longevity Center statement mentions in passing only a single study, and the summary statement ignores the literature entirely. Regardless of whether this omission is intentional or an oversight, this is poor scientific practice, and it raises significant questions about the quality of their process.
The statement did make several good points about the scientific obligations of brain fitness providers. I’d like to highlight them, and note how Posit Science has lived up to these goals. At Posit Science, we strongly agree that:
- Consumers who want exercises that have been shown to work should expect companies to provide peer-reviewed, randomized, controlled trials: At Posit Science, we say our programs work because they have been shown to work in dozens of such trials, including the three largest randomized controlled cognitive training trials ever conducted: the ACTIVE, IMPACT, and IHAMS trials.
- No one should say or imply products have scientific evidence where there is no or little evidence. At Posit Science, every statement we make regarding the proven benefits of our cognitive training programs is rigorously analyzed by a team of scientists against a broad body of evidence, including more than 70 published peer-reviewed articles on the benefits of exercises we bring to market, to make sure that each statement is accurate.
- Such evidence is stronger if run independently. Posit Science directly conducted the first two published studies of our cognitive training exercises, because we believed it was incumbent on us to jump-start the science of this field. With these two small exceptions, all of the more than 70 peer-reviewed publications using our exercises have been from studies run by independent academic scientists, typically funded by the National Institutes of Health.
- Better studies employ active controls. Our studies are conducted with strong control or comparison groups. Control activities have included adult education (through computerized classes and quizzes), crossword puzzles, computer games, and, in the case of studies on the impact of exercises on driving abilities, drivers’ education classes.
- Studies have more weight when run at multiple sites or results are replicated in subsequent studies. A large number of the studies on our exercises have been multi-site studies or have results that have been replicated. The three largest studies—ACTIVE, IMPACT, and IHAMS—were all run at multiple sites.
The statement also recommended four tests that brain training programs need to pass to be fully credible. At Posit Science, our programs have passed these four tests:
- Are the positive changes noticed in real life indices of cognitive health? Yes. Our exercises are unique among what is available to the public, because there have been so many peer-reviewed demonstrations of generalization to real-life indices of cognitive health. These include better driving (e.g., at-fault crash risk reduced by 48%; 36% fewer dangerous on-road driving maneuvers; better ability to drive under a variety of challenges); better health outcomes (e.g., better scores on measures of health-related quality of life, including the SF-36 and SF-1, commonly used in VA and CMS studies, and 3.3% lower predicted healthcare costs, based on the use of the federal government’s model developed by AHRQ for DHS, and used by Medicare and Medicaid); better mood (e.g., 38% less risk of onset of depression; 30% less risk of deepening of depression; 68% greater feelings of control) better self-reported performance in a normed measure of everyday cognitive activities (e.g., remembering why you walked into a room; following a conversation in a noisy place); reduced fall and mobility risk; improved ability to hear in a noisy environment; and many, many others. Together, these results show that positive changes are not just on the training exercises themselves, but, crucially, are on real-life indices of cognitive health.
- Does the improvement encompass a broad array of tasks that constitute a particular ability, or does it just reflect the acquisition of specific skills? The outcome measures in IMPACT encompassed two distinct composite measures of cognitive function, each composed of multiple standardized neuropsychological tests; the outcome measures in ACTIVE and IHAMS were broad measures of health-related quality of life and mood; and the outcome measures in ACTIVE additionally included real-world automobile crash measures. Together, these measures show improvements on a broad array of tasks that constitute generalized abilities.
- Do the gains persist for a reasonable amount of time? Yes. IMPACT documented benefits persisting for three months after training, IHAMS showed persistence of at least one year, and ACTIVE has now shown that some benefits can last for ten years following training. In aggregate, it is now clear that cognitive training works like physical exercise—the benefits persist following training, and, absent further exercise, the effects diminish over time.
- What role do motivations and expectations play in bringing about improvements in cognition when observed? Motivations and expectations are important, and can be controlled for by the use of active controls. IMPACT used an adult education control, IHAMS used crossword puzzles, and ACTIVE employed three different kinds of cognitive training. In the dozens of other published studies on Posit Science’s exercises, the lion’s share also included relevant active controls.
We also agree with the statement that brain fitness providers should be criticized for exaggerated or misleading claims in marketing their products. At Posit Science, we are working to build a brain fitness industry based on solid science and clinical evidence. We have a rigorous process for marketing claims and are receptive to a dialogue with anyone who has concerns about our claims.
Given the criticism of marketing language made by the statement, critics should take care not to engage in the same kinds of mistakes. And unfortunately, by ignoring or disregarding this published scientific literature, the Longevity Center statement misleads the public regarding the true status of scientific evidence for the efficacy of brain training.
In the spirit of promoting better science and better understanding of the science that has already been done, I offer two points of guidance to the Longevity Center for their use in future reviews:
- Comprehensive Data Review: Any consensus statement that reviews the state of the science should include, at minimum, a comprehensive review and rating of the scientific publications that went into the analysis. Without this, it is not possible to distinguish facts from opinions, which is an essential goal of any review process. Future statements should follow best practices for scientific reviews, and include a list of publications reviewed, with a rigorous and transparent process (including a scoring scale describing how the evidence was evaluated and by whom, and a summary of the scores). For examples of how high-quality scientific reviews conduct and document their data gathering and analytic processes, Longevity Center coordinators should read this review from Dr. George Rebok and colleagues, or this review from Drs. Nicola Gates and Michael Valenzuela, or this review from Dr. Elizabeth Zelinski and Shoshana Hindin. Each carefully gathers all relevant studies, evaluates the strength of the study and claims from the data, and then analyzes the data as a group. Each also documents positive effects from cognitive training using these very standard scientific review and meta-analytic approaches. Scientists, as well as the public, should be swayed by evidence, not advertising or opinions.
- Gathering Consensus: Any consensus statement should seek input from all stakeholders, particularly, in this area, from neuroscientists who specialize in applied plasticity research. Without this, key knowledge and perspectives can easily be missed. Future Longevity Center reviews should involve representatives from industry, as well as academia. They should not claim to speak for the “scientific community” when there is clearly not a consensus across that community.
Why did the Longevity Center issue this flawed statement? Frankly, I don’t know. Despite our involvement with most of the published research showing efficacy in this area, we were not invited to participate. From my recent conversations with participants (some of whom signed the statement and some of whom refused to do so), I believe that the Longevity Center was primarily focused on the intensive television, radio, and internet-based advertising that has become common from some of our competitors. This wave of advertising has frustrated many scientists, perhaps leading to the Longevity Center statement. However, that frustration does not obviate the need to comprehensively review the evidence before drafting a statement, and does not excuse replacing scientific rigor with exaggerations, omissions, polemics, and press releases.
Companies in this space have to decide whether to spend their time, effort, and assets on studies and developing new science, or on marketing and advertising. From the start, our focus has been to make exercises that work, which requires advancing the science through good research. Other companies have focused on marketing and advertising. Regrettably, by ignoring the substantial research that has been done, the Longevity Center statement undermines companies in this nascent industry who genuinely care about and invest in science. It also likely reinforces a belief, amongst others, that the investment of time and effort in science will only yield a return of endless academic debate.
At Posit Science, we believe it is important to look at evidence when coming to a scientific consensus, with a particular focus on evidence from new science that can help advance the field to a new plateau of understanding. Earlier in his career, our co-founder, Dr. Michael Merzenich, was among the first to show evidence of plasticity in the adult brain. The “scientific consensus” at that time was that plasticity ended with adulthood. Dr. Merzenich was pilloried for his early publications, and it literally took decades and hundreds of articles from Dr. Merzenich and others to move the so-called “consensus.” Now, virtually every introductory neuroscience textbook treats “lifelong” plasticity as an accepted fact. Dr. Merzenich was also considered outside the “consensus” for his early belief that it was possible to cure certain types of deafness with a neural prosthetic. His research on plasticity led to his co-invention of the cochlear implant—technology that has now helped give or restore hearing to hundreds of thousands of people. We are now going through this same process around the evidence for cognitive training—and we’re going to end up at the same place.
Mike Merzenich taught me as a young scientist to care more about what the right answer is than about “being right.” Discovery comes from adapting to new facts. These are the same values he inculcated at Posit Science, where we work with hundreds of university-based scientists around the world to invent new training regimens, test them and validate them—so we can get them out of the lab to people they can help. Our mission at Posit Science is not to maximize shareholder value or to make personal fortunes or reputations. Our mission is simply stated as “ Science to the people!” We hope others who are like-minded will join us in achieving that goal.
Is there more research to be done on cognitive training? Sure. The more, the better—and there are still some open questions. That’s why we have many more studies underway right now. But I am pleased to confirm that the first wave of evidence is already in—properly designed cognitive training programs do work—and well-run, independent, randomized controlled trials have shown it.
Henry W Mahncke, PhD
CEO, Posit Science