About a year ago when I became aware of the libel suit brought by the British Chiropractic Association (BCA) against Dr Simon Singh, the first thing that struck me—as someone who is a working scientist—was the unscientific nature of the action.
Last week's ruling on meaning by the Court of Appeal has confirmed me in that view.
Many words have been expended on the Court of Appeal's judgement. I have particularly appreciated the clear legal exposition of my host, Jack of Kent, Heresy Corner's dissenting take and a perceptive examination of the BCA's mindset by Dave Gorman.
I would like to expend just a few more words to consider paragraph 26 of the judgement, which seems to me to go to the scientific heart of the matter. Let us look at exactly what it says:
26. What "evidence" signifies depends heavily on context. To a literalist, any primary fact – for example, that following chiropractic intervention a patient's condition improved – may be evidence of a secondary fact, here that chiropractic works. To anyone (and not only a scientist) concerned with the establishment of dependable generalisations about cause and effect, such primary information is as worthless as evidence of the secondary fact as its converse would be. The same may equally well be true of data considerably more complex than in the facile example we have given: whether it is or not is what scientific opinion is there to debate. If in the course of the debate the view is expressed that there is not a jot of evidence for one deduction or another, the natural meaning is that there is no worthwhile or reliable evidence for it. That is as much a value judgment as a contrary viewpoint would be.
There are several interesting points here.
First, it is pleasing to see that their Lordships have a sophisticated understanding of the nature and variability of what constitutes evidence, even in science. I can only speculate that they have read either or both of Ernst and Singh's Trick or Treatment and Ben Goldacre's Bad Science, which give detailed expositions on the factors that determine the quality of scientific experiments, with particular reference to clinical trials. Not all experiments are equal. Controls—the deceptive application of a mock treatment—are vital. Blinding of the scientists and the participants to whether any given participant is receiving the mock or the test treatment provides the most unbiased way of determining treatment efficacy. But not all investigations adhere to these standards.
Second, their Lordships draw an interesting distinction between "literalists" who are prepared to make facile causal links between observations and those "concerned with the establishment of dependable generalisations about cause and effect". This latter group includes but is not restricted to professional scientists. But, given the use of that word "dependable", it is difficult not to envisage that this group must include those who are willing to apply the scientific method rigorously. Few people, I imagine, would be prepared to submit to treatments developed by "literalists".
Third, by highlighting this distinction, their Lordships draw attention to an unhappy fact about the scientific profession: not all scientists are equal. Some of them are cleverer than others. Some of them are more critical in their interpretation of experimental data. And some of them produce work that is not very good.
As a result of this variability, we have the necessary cut and thrust of scientific debate—the free exchange of criticism that is so important to the method—in which scientists robustly evaluate this or that piece of evidence. (For anyone interested in the technical details, there is an interesting case study here.) Just because an experiment has been done does not mean that useful evidence has necessarily accrued. The good stuff is kept—for the time being—and used as the basis for further investigation. The bad stuff is discarded and left to lie, lifeless and unread, in the scientific literature.
Crucially, their Lordships contend that it is within the context of this scientific rough and tumble that assessments such as "not a jot of evidence" can readily be taken to mean "no... reliable evidence". These argumentative evaluations of evidence are part of the widely understood terms and conditions of scientific debate. It is a blessed relief that they have been thrown out of court and returned to their proper place: the public domain.
There is an interesting follow-on to this judicial dissection. In the very next paragraph of the ruling, the Court of Appeal mention a revealing example of the BCA's approach to scientific evidence:
27. ...Dr Singh's defence includes, in §8(25), a survey of controlled clinical trials on the efficacy of chiropractic in treating infantile colic, none of which, he contends, affords objective support for the BCA's claim. The BCA, in §9(23) of its reply, relies (among other studies) on a 1989 observational study of 316 children, of which it is said:
"This... measured the number of hours each child spent in crying. It showed a reduction in crying time from 5.2 hours each day to 0.65 hours each day at 14 days. This was a very substantial improvement. There was no control group. However, the study constitutes evidence."
Their Lordships are careful to point out that this is not the only study cited by the BCA in support of their claims. But it is telling that this paper [Klougart et al. (1989) J. Manipulative Physiol Ter, Vol. 12, p281-8] is cited at all*.
The Klougart study is considered by the BCA to constitute "evidence". But there is no control and therefore no way to gauge the efficacy of the chiropractic intervention.
This is not evidence that a scientist worth their mettle would consider to be of any real value. This is not evidence that I would tolerate in an argument from one of my undergraduates. This is not evidence that a child at primary school learning about science at key stage 1 of the national curriculum would consider to be derived from a "fair test".
The BCA is a professional organisation that represents the interests of chiropractors who claimed to be able to treat non-spinal childhood ailments by manipulating their spines. Those are bold claims given our modern understanding of human physiology. That the BCA is happy to cite the paper by Klougart and colleagues as worthwhile evidence supporting chiropractic treatments tells me they are far less critical, far less scientific, than the public has a right to expect them to be.
*This study is the second paper listed in the press release put out by the BCA in June 2009 to demonstrate that there is a "significant amount" of evidence to support the claims that chiropractic could treat childhood ailments such as colic.
The totality of this evidence was subjected to intense scrutiny in the blogoshpere and the pages of the British Medical Journal and found to consist of irrelevant or poor quality trials.
No purely anonymous comments will be published; always use a name for ease of reference by other commenters.