Simple Steps for the detection of bad science in ME Research
, trust – 1
keywords – definition, advocacy, cdc, ccc, icc, Oxford
Oct 31, 2016
It's come to my attention that bad science and good science in the field of Myalgic Encephalomyelitis are often sloshed together. In a disease for which there is such low funding, and therefore fewer studies, this is especially dangerous.
So, I hope to do my part in helping with this simple guide.
It starts out with a few premises I think any rational, scientifically-minded, person would agree with:
1.) Any definition of a disease or illness should include the central defining feature of that condition.
2.) People with idiopathic fatigue vastly outnumber nearly every disease category (including Myalgic Encephalomyelitis).
3.) In the absence of other clues clinicians who studied outbreaks firsthand should be given the assumption that they know how to diagnose the illness in question, even if their study cites a definition that fails the first premise.
Now the fun stuff–How do we determine if a study in Myalgic Encephalomyelitis is "bad science"? And what cases fall in the gray area wherein they may constitute evidence but confirmatory work is required?
ITS DEFINITION LETS IN THE ENTIRE CITY ZOO.
The first and easiest step is to consider which studies which used definitions failing premise 1.
The Oxford Definition (1992) requires only "chronic fatigue lasting at least 6 months" and the Reeves Definition (otherwise called Updated CDC Definition, or CDC 2005 Definition or CDC Empirical Definition or the Definition with too many names) requires that patients possess at least 4 symptoms from the more-well known 1994 Fukuda Definition but with no hard requirement for reduced functioning and significant overlap with the significantly, more common, major depression.
The Oxford Definition and Reeves Definition fail premise 1 (they do not include the central feature of 24-72 hour delayed post-exertional collapse*) and premise 2 by being too porous to exclude idiopathic fatigue and severe fatigue from other causes.
So, for an example of this logic: the widely disseminated PACE trial and its more recent descendants turn up on the heap of bad science.
BUT, OTHER DEFINITIONS CLEARLY FAIL PREMISE ONE!
The widely used Fukuda 1994 definition fails premise one as well. While it requires at least four signs and symptoms associated with Myalgic Encephalomyelitis and "chronic fatigue" it doesn't require that the core symptom of the disease: "delayed onset Post-Exertional Collapse" is met.
Also, one must wonder about the people who receive this diagnosis before getting treatment for their true problem: for example, someone suffering undiagnosed chronic migraines could likely get a diagnosis of "CFS" according to the Fukuda Definition.
This leads to a dilemma.
In the United States the Fukuda Definition was used nearly exclusively in the years 94-late 00s. How is one to decide then if a study is good or bad? Should all research from this period be trashed?
In my opinion, No.
This leads us to premise 3. Is there a good reason to assume the authors knew what they were talking about? Doctors and Researchers who were present at the earliest outbreaks such as Royal Free (1955), Incline Village (1980s) and Lyndonville (mid 1980s) can be assumed to know how to diagnose a Myalgic Encephalomyelitis patient. Such people include Ramsay, Cheney, Peterson, Bell, among others.
...Isn't this a bit circular you ask? Maybe they were just always bad diagnosticians and that's why "outbreaks" occurred? The best answer is to refer to the history of these outbreaks: there is ample evidence each occurred, and additionally the patient and journalistic descriptions are congruent with the descriptions of the clinicians involved in studying them.
So, it seems reasonable enough to me, that these clinicians should be trusted to diagnose Myalgic Encephalomyelitis even if their studies methodologies cite a weak definition (such as Fukuda 1994 in the materials and methods).
A second tactic, is to see, if possible, if the article you are investigating listed how many patients had post-exertional collapse (malaise) as a symptom? If the percentage is high, then it likely the study has scientific value even if the Fukuda Definition was used.
Some examples of studies that fit the above description, and are therefore probably trustworthy: The blood volume studies by Bell and Streeten, the NK cell studies by Klimas, most/all of the studies by the charity ME research UK** in this time period.
ARE THERE DEFINITIONS WHICH INCLUDE "Delayed Post-Exertional Collapse" BUT ARE OTHERWISE POOR?
Perhaps you refer to the Canadian Consensus Definition or the International Consensus Definition? I've yet to see a strong argument as to why these two are "bad." If we confine ourselves to the earlier three premises we should accept any study using the ICC or CCC as good science: at least insofar as patient selection is concerned.
A recent definition by the IOM, for Systemic Exertional Intolerance Disorder includes "Post-Exertional Malaise" but is otherwise considered by some researchers and many patients to be bad. Why? An argument in a recent study is that these criteria lack exclusions for psychiatric disorder and don't require central, however non-unique, symptoms to ME such as sleep disorder, cognitive disorder etc.
POST-EXERTIONAL MALAISE aka POST-EXERTIONAL NEUROIMMUNE EXHAUSTION aka DELAYED POST-EXERTIONAL COLLAPSE: SUFFICIENT OR ONLY NECESSARY?
Though advances have been made in measuring it;(1,2) reporting delayed Post Exertional Collapse is currently subjective (at least in the definitions considered) and it could be reported in other diseases and conditions. Therefore many argue it is necessary but not sufficient to define Myalgic Encephalomyelitis. The argument is widely observed, albeit non-unique, signs and symptoms are needed: such as Cognitive Impairment(s), Unrestful Sleep, and Widespread Pain.
I tend to concur. Thanks for reading.
Reeves Definiton (2005): Bad!
Oxford Definition (1992): Bad!!!
Fukuda Definition (1994): Untrustworthy, evaluate studies using it with care.
* A marked, often several magnitudes, generalized worsening of symptoms including: pain, neurological, cognitive, sleep difficulty, increased difficulty maintaining upright posture, generalized muscle aches from the pre-exertion state of function.
** This refers to the Research Charity: ME Research UK, previously called MERGE. Led by Drs. Vance Spence, Neil Abbot (recently retired) and Sue Waddle.
(1) Snell CR, Stevens SR, Davenport TE, Van Ness JM. Discriminative validity of
metabolic and workload measurements for identifying people with chronic fatigue
syndrome. Phys Ther. 2013 Nov;93(11):1484-92. doi: 10.2522/ptj.20110368. PubMed PMID: 23813081.
(2) Keller BA, Pryor JL, Giloteaux L. Inability of myalgic
encephalomyelitis/chronic fatigue syndrome patients to reproduce VO₂peak indicates functional impairment. J Transl Med. 2014 Apr 23;12:104. doi: 10.1186/1479-5876-12-104. PubMed PMID: 24755065; PubMed Central PMCID: