Read this series from the beginning.
Read the previous installment of this series (Part 4).
We here at the TM-Free Blog have received a comment from David W. Orme-Johnson. He’s a former Maharishi University of Management psychology department chairman and a frequent defender and promoter of TM research studies. I’ve mentioned him and his website a number of times in previous parts of this series, “The dubious research claims of Transcendental Meditation.” He submitted his comment in response to the second part posted here at the TM-Free Blog on September 22, 2019, subtitled ‘"Peer review" doesn't always mean "quality" or "accuracy.”’ In 2015, he claimed to have no formal affiliation with the TM movement on his website, but a published comment that same year indicated that he was then associated with the “Research Desk, Maharishi Foundation USA.”
Read the previous installment of this series (Part 4).
 |
| David W. Orme-Johnson. From "Inaugurating the Dawn of the Age of Enlightenment," 1975 |
As a matter of editorial policy, the contributors to the TM-Free Blog don’t particularly care to reprint the TM movement’s marketing materials. There are thousands of TM movement operated or affiliated websites worldwide that serve that purpose. There is also, of course, Orme-Johnson’s own website, “Truth About TM,” that contains plenty of defenses of TM research and a few rebuttals of critics from his point of view, that’s evidently very similar to that of the TM organization. Parts of his submitted comment are of a promotional nature, therefore I won’t be running his comment in the comments, or as a post, verbatim.
 |
| The "evidence-based" claim, as it appears on the tm.org website at the time of this writing. |
The second paper the blog uses to criticize TM research is a preliminary study by the late Sarina Grosswald and co-authors on ADHD. In children with ADHD, the study showed statistically significant reductions in stress, anxiety, and improvements in ADHD symptoms and executive function. The authors were obviously not naïve about experimental design, as the blog portrays them, because they said in the title that this was “An exploratory study”.
This passage in Orme-Johnson’s comment leaves me chuckling to myself. The idea that calling their work “an exploratory study” excuses blatant problems and errors in their methods is, in my view, ridiculous. If the experimenters really wanted to draw serious attention to their work, which makes a rather novel if not controversial claim, why wouldn’t they take particular care to ensure that what they were doing wasn’t obviously problematic? Relying on self-reporting and self-controls, ensuring that the schools’ most prominent authority figure was an enthusiastic TM meditator, and coaching the student subjects to expect positive results - and recording that coaching on video! - are not excusable because the study was “exploratory.” If anything, these faults show that the claimed results were most likely the result of the kind of wishful thinking and confirmation bias common among TM meditators, and aren’t really worth considering for further study. But a paper like this, published in a third- or fourth-rate journal, provides the kind of story that’s commonly circulated among TM insiders, that reinforces, for them, that what they’re doing, spending time and money on, is a valid and worthwhile cause. It also provides fodder for press releases that are sent to reporters who aren’t in a position to approach these matters with the skepticism, resources or time to do anything other than to publish these claims verbatim without the most basic examination of how these conclusions were reached.
Orme-Johnson continues, and attempts to fault my reporting and analysis:
In contradistinction to good science, the blog fails to report a second study on ADHD by these authors. This study found that TM improved verbal fluency, and that a parents’ survey reported that the students significantly improved on five ADHD-symptoms over the six months of the study. Significantly, the study found that TM reduced objective brain measures associated with ADHD (decreases in theta/beta ratios) and TM increased brain measures associated with improved cognitive function (increased theta EEG coherence, a trend for increased alpha and beta1 EEG coherence). These objective neurophysiological measures corroborate the behavioral observations of improved ADHD, and are clearly not fakable or suggestible.
Without addressing the faulty methods identified by others in the initial study, he’s moved on to talk about another matter, another study which was outside the scope of my blog post. That there was another study done by the same people, making similar claims and using similar methods, but adding the TM movement’s research hobby-horse of “EEG coherence” to the mix, is marginally relevant to my point, which was that the same errors and methodological faults are common across many TM research studies performed by meditators formally affiliated with the TM organization, in this case, in two studies thirty years apart. But if he’s going to bring up this additional study, I’ll play.
First red flag on this one is that Orme-Johnson hasn’t provided a reference to the “second study on ADHD,” despite having footnoted more than a dozen other references in his comment. No problem, as it turns out this study is actually included on the list I’ve been working with from the tm.org website, which is billed as only containing research published in “peer reviewed journals.” Here’s the citation as they list it, in the “Brain Function” section, italicized/bolded exactly as they have:
Travis F, Grosswald S, Stixrud W. ADHD, brain functioning, and Transcendental Meditation practice. Mind & Brain, The Journal of Psychiatry 2011 2(1):73-81
So, let’s go find this journal. If you Google the full journal title, nothing resembling a journal’s website shows up in the results, and that’s odd since I’m used to easily locating the website for most journal titles pretty much instantly with that kind of search on the name. But on the first page of the results of a Google search on just the journal’s name, the eighth and the tenth are the text of a Maharishi University of Management press release touting this mystery “second study” to which Orme-Johnson refers. Hmm.
But the ninth result is a link to the article at ResearchGate, which is a kind of social network for researchers. I find it very handy at times, because researchers may upload their own papers, or as they say on the front page of the website, “Share your work from any stage of the research cycle to gain visibility.” Sometimes papers that were published in high-quality journals, that are only otherwise available behind a rather expensive journal’s paywall appear on ResearchGate. At the same time, ResearchGate is not peer-reviewed or edited, so almost anything might show up as an uploaded file there.
In this case, the copy of this paper on ResearchGate is the only one available online. It was evidently uploaded to ResearchGate by Sarina Grosswald herself, and it looks much like a journal article in any other scientific journal. But then you look at the bottom of every page and see a website address - www.slm-psychiatry.com - which is a dead link to a blank page.
Looking up that address in the Internet Archive’s Wayback Machine shows that the website went away sometime between mid-2012 and mid-2014, after three issues of the journal were produced. The ADHD paper in question was in the final issue. Sarina Grosswald is also listed as a member of the editorial board of the journal.
A bit of further digging through the Wayback Machine, turns up the name and address of the publisher, “San Lucas Medical” of Middlesex in the UK. This publisher was briefly producing journals under ten different titles.
As I mentioned in the previous part of this series, in the scientific publishing field the “predatory journal” has become a more frequent issue for researchers in recent years. As simply defined by this page at a University of Texas website, “Predatory Journals take advantage of authors by asking them to publish for a fee without providing peer-review or editing services.” Given the apparent fly-by-night nature of this “Mind and Brain” journal, was it in fact a predatory journal? Given the insistence by TM promoters, including Orme-Johnson, that editing and peer review actually count for something - has he, at the same time, in the course of defending the validity of research on TM performed by longtime TM meditators, called my attention to a paper that they paid to have placed in a publication that publishes most anything without reviewing it, based on the ability to pay for that service? And as a bonus, this same paper is represented on the tm.org website research list as if it were published in a legitimate, peer-reviewed, scientific journal.
The ‘San Lucas Medical” (SLM) journals, and the publisher, were in business so briefly that they don’t appear on lists of predatory journals that were beginning to appear around the time that these journals went out of business. But an anonymous comment at the Retraction Watch website points to UK government public records that reveal a link that certainly suggests that the SLM journals were, in fact, predatory, pay-for-play operations. This is because at least one of the directors of SLM, Asif Ali, was later the director of another predatory journal publisher, Premier Publishing, which appears on a list of known predatory publishers and that has also since gone out of business.
As for the contents of the paper itself? Orme-Johnson highlights an obvious, likely flaw: again, as with the first ADHD study, evaluation of the children’s symptoms by professionals is not the sole means of evaluating any changes in their behavior. He seems to think that the parents having been surveyed to determine if the children are improving or not is worth mentioning. I tend to think that the parent’s views would likely be the most unreliable and biased means one could ever choose to evaluate a research subject’s behavior; this would be particularly true when they’ve likely been told that their children are participating in a study of a method that is consistently oversold as always being positively life-changing and effective, as TM is always sold to prospective meditators.
Orme-Johnson seems quite insistent that the addition of EEG recordings in the second study, allegedly demonstrating changes in “coherence,” somehow further legitimizes the methods that were used in, and the conclusions of the first study. The problem is that “EEG coherence,” I think, should be considered part of a common article of faith in the TM movement, and not a clear scientific reality. While coherent brain activity is observed when people are active, particularly when performing creative tasks, the assertion that coherence observed in meditators is always beneficial is a position that is largely that of meditating researchers affiliated with TM institutions. These researchers provide a very tenuous basis for an implied claim that appears in TM marketing materials and websites, where “whole brain creativity,” EEG coherence and an observed increase in EEG coherence among meditators are strung together to give the incorrect impression that a concrete and generally accepted causative relationship among these three items exists, that supports the practice of TM because it supposedly improves creativity among meditators. Here’s one example that recently appeared on a number of TM websites:
Creativity, perhaps more than any other aspect of life, depends on the extent to which the brain is able to work coherently as one whole. In scientific circles this is also referred to as “whole brain creativity”. This can be measured through EEG coherence measurements, and transcending [through Transcendental Meditation], more than any other experience ever researched has been found to increase EEG coherence.
Another issue with the TM movement’s vague talk of “EEG coherence” in which coherence is always viewed as a good and beneficial thing, is that there is considerable ambiguity as to what TM researchers mean by that term, since the type and pattern of coherence is of diagnostic importance, not that it simply exists in greater or lesser amounts. For example, EEG coherence patterns are observed in epileptics and are associated with epileptic seizures. More relevant to these claims that attempt to show that TM practice and allegedly resulting EEG coherence reduces ADHD symptoms, is that in one recent study (note the red lines in figure 3), an increase of coherence in certain patterns has been associated with ADHD patients and not with subjects in a control group.
Another recent paper reported that in a study that compared brain coherence measurements between scalp and intracranial (inside the brain, implanted) electrodes in monkeys, there wasn’t a reliable relationship between the two kinds of measurements. The authors advised caution when interpreting EEG coherence effects:
Although we found inter-area coherence at both scales of measurement, we did not find that scalp-level coherence was reliably related to coherence between brain areas measured intracranially on a trial-to-trial basis, despite that scalp-level EEG was related to other important features of neural oscillations, such as trial-to-trial variability in overall amplitudes. This suggests that caution must be exercised when interpreting EEG coherence effects, and new theories devised about what aspects of neural activity long-range coherence in the EEG reflects.
To sum up, I tend to think that paying to get published in an evidently “predatory journal” would be “in contradistinction to good science,” which is how Orme-Johnson attempts to characterize my criticisms, in which he notes that I failed to cite exactly that kind of paper and journal that wasn’t relevant to my argument. Claims in published papers implicating EEG coherence as the means by which TM practice is beneficial, by TM affiliated researchers, which have most often appeared in the neuroscience journal whose faults I discussed in the previous part of this series, may be much less than they appear, and seem to exist primarily to provide some marketable set of claims of “scientific” validation (including, by way of Maharishi University of Management press releases) that don’t hold up very well to further scrutiny. These claims aren’t supported by any scientific consensus outside the bubble of TM-affiliated research, as is again true in this case, where the study was funded by the David Lynch Foundation and performed by people with well-known formal affiliations with TM organizations. But what do I know, I’m just an unpublished college dropout, which Orme-Johnson also attempts to address elsewhere in his submitted comment:
I looked the blog writer on Google Scholar and did not find he had single publications in a peer-reviewed journal. His blog would not make it through peer-review process because it does not tell the whole story about the two TM papers it takes issue with.
I make no secret, here and elsewhere, of being a University of Maryland dropout. Unlike the edifice, which I personally liken to a Potemkin village, that the TM organization has constructed over the past 50 years or so in Fairfield, Iowa, in which to house and feed its authorities, I’ve made no claim to any particular level of scholarly authority or institutional affiliation; what you see is what you get. I can at least proofread my work and rarely let slip a broken sentence like the first one Orme-Johnson wrote here. Despite the missing words, again he’s stating the obvious: no, my name doesn’t appear in a peer-reviewed journal, although I’ve co-written a few pieces in other fields that have actually been printed on paper, a series of articles in a specialty newsmagazine and twice in one of the longest running alternative newspapers in the United States. My professional achievements, and for that matter, failures, are a matter of public record.
This sort of comment, I think, is indicative of someone who’s used to discrediting critics in the presence of like-minded advocates of Transcendental Meditation; after all, he’s created and maintained a website for this purpose, for over a dozen years now. But science is not a black box tended only by credentialed, published scientists. Actually, I think it’s essential that non-scientists, particularly those in any position of responsibility, develop some minimum ability to evaluate scientific claims. It’s really not that hard, and there are plenty of resources online (here’s a popular one and here’s a handy poster) that can help people develop that sort of understanding of how science works. This is particularly true in the realm of public policy and politics, relevant to all matters involving Transcendental Meditation since the movement, and now the David Lynch Foundation, have long attempted, and continue to work toward, inserting TM into government, government-sponsored programs, prisons, and public schools, despite court decisions in this country standing in the way of such efforts.
What I see at Orme-Johnson’s website is a single-minded devotion to the advancement of TM as the usual religious-tradition-based, panacea and solution for everything that its founder described it to be, and not anything resembling an objective, dispassionate evaluation of both the possible benefits of meditation for some people, and the limits of scientific research, particularly when that research is performed by people who by definition have more than a casual interest in the outcome.
Maharishi University of Management (MUM) would not exist except for the fact that its primary purpose is to serve as a means of attempting to legitimize, in the Western world, a particular interpretation of a spiritual tradition by a religious sect, Maharishi being the most recognizable leader of that sect. By centering an unusual institution of higher learning on TM, or as they more generally call it, “Maharishi Vedic Science,” their attempted research in these fields has been elevated to a level of legitimacy, and lack of scrutiny, that it has never deserved.
It is in that context that research conducted by MUM, or anyone currently or ever formally associated with it, should be critically viewed. Every objection raised against research by such individuals, focused on TM and all the rest of the Maharishi-related products may well chip away at the legitimacy of the institution in which they work, that they support, and that in many cases provides them with employment. That relationship is very seldom explicitly disclosed in the published research, and in my view it constitutes an undisclosed conflict of interest, which customarily and technically need not be disclosed, if MUM were just another university. But MUM isn’t just another university, it’s completely dependent on the assumed legitimacy of the products and practices connected with the religious tradition (so-called “Vedic science”) on which it was founded.
No comments:
Post a Comment