Further studies are needed to assess the competing therapies for neck pain☆

Was there selection bias? 

Ernst did not search the main chiropractic electronic databases, MANTIS and Chirolars, because of cost concerns and instead asked chiropractors to identify their research. Did this strategy miss any articles? A recently published systematic review on mobilization and manipulation20 identified 173 possibly relevant articles and reviewed 20 articles on mobilization and manipulation, and that search strategy was completed to 1997 only. We have recently performed an updated search to 2002 as part of a National Institutes of Health–Consortial Center for Chiropractic Research funded grant, and we found 33 trials related to the topic of mobilization and manipulation. Some of these articles might well have been excluded from the Ernst review, given the restrictive inclusion of chiropractic interventions only. However, the pool of articles related to manipulation and mobilization is more like 20 to 33 articles, not the 10 or so found by Ernst.

Prior research has demonstrated that several people need to make decisions about which articles to include.22 The article by Gross et al20 showed that agreement on article selection, performed by 2 people, was good but not perfect (estimated kappa, 0.68; range, 0.53 to 0.8). Haines et al.22 showed that there was good but not perfect agreement on which articles should be included (estimated kappas, 0.67 to 0.75) and that article selection was influenced by several factors, such as familiarity with author and journal as well as perceived influence of the author and the journal. Thus there are several reasons to believe that the Ernst review suffered from selection bias and that these factors artifactually reduced the apparent evidence from which Ernst draws conclusions.

Why limit the review to chiropractic spinal manipulation? 

Ernst suggests that the articles on spinal manipulation should be limited to chiropractic manipulation. The rationale provided is that chiropractors use different techniques. No supporting references are offered for this statement. He also states that chiropractors have a different philosophical approach to treatment. What he does not mention is that these techniques are practiced by many other practitioner types, including physical therapists29 and physicians,14 that the first randomized trial of spinal manipulation was performed by a medical doctor,45 and that many of the standard textbooks on manipulative techniques used by chiropractors were written by medical doctors, including Cyriax,13 Mennel,36 Maigne,33 and Haldeman.23 It is not clear what the relevance of philosophical differences between spinal manipulators is, and this problem is not unique to chiropractic. For instance, we have several theories but no clear answers on the mechanism of action of methotrexate49 and hydroxychloroquine43 for rheumatoid arthritis, yet they continue to have clinical relevance to patients. We do not know how acetaminophen works as an analgesic,41 although it is used to treat neck pain. A more convincing argument that technique differences are important might have been based on evidence of differences in efficacy or side effects related to technique differences. There are no randomized trials of “different” techniques in spinal manipulation, suggesting that one is preferred. The studies by Haldeman et al25 of serious complications of spinal manipulation (stroke and death) found that no technique was safest or riskiest. These complications are not unique to chiropractors, and complications from manipulation have occurred at the hands of physical therapists and physicians.21 Thus the selection of chiropractic spinal manipulation alone seems arbitrary and not based on evidence of differences in benefit or harm.

It is also not clear why he would have excluded studies of whiplash. For many trials the description of the neck pain population is insufficient, so it is not clear whether patients with whiplash are included or excluded in the primary studies. Another review also found no evidence of systematic differences in treatment approaches used for those with whiplash and no systematic differences in treatment outcomes for patients with whiplash as compared to those with other types of nonpathologic neck pain.20 This arbitrary decision would further serve to artificially reduce the pool of studies for evaluation.

Was the assessment of study quality adequate? 

Ernst reports that 2 studies were of poor quality and 2 were of moderate quality. This was based on the Jadad scoring system.28 What was left unstated was that the Jadad instrument was developed for studies of drugs, in which double blinding is readily accomplished with a matching placebo. In fact, the Jadad scoring system awards 2 points for randomization (1 point for stating it was random and 1 point for describing an acceptable method), 2 points for double-blinding (1 point for stating it was double-blinded and 1 point for describing an acceptable method), and 1 point for describing dropouts and follow-up. For almost all hands-on physical techniques, the person delivering the treatment cannot be blinded. Many of them are hard to blind to patients, if patients have experienced them in the past. This is true for spinal manipulation.

It is possible to have a blinded outcome assessor for some outcomes. In the case of pain assessment, the best outcome assessor is rightly the patient; therefore it is difficult to blind pain assessment. Thus we point out 2 errors with Ernst's quality assessment of quality. The first is that the Jadad quality scoring system, as he uses it, has a maximum score of 3 not 4 points, because there is no option in the Jadad scoring system, as Jadad described it, for scoring points if a trial uses a blinded outcome assessor (ie, single-blinding). The second is that the review excludes trials of outcome measures, such as range of motion, that can be assessed by a blinded outcome assessor. These issues would artificially reduce the apparent methodologic quality of the trials reported. The Cochrane Back Pain group, recognizing the deficiency of the Jadad system when applied to trials of physically based treatments, recommends the scoring system of van Tulder et al.47 In the van Tulder scoring system, which is based on 19 methodologic elements that good randomized trials should report, trials score methodologic points for using a blinded outcome assessor.

Was data extraction adequate? 

Mistakes can easily be made by using a single data extractor. L'Abbe et al,31 in one of the earliest reports on the quality of meta-analysis, suggested that having 2 data extractors was important. To quote the 1987 article, “data extraction can be tedious, subject to error and hence potential bias. We recommend that two persons familiar with the clinical topic do the data extraction independently and cross-check their reports to protect against this form of error.” This is also the view of the Cochrane Back Pain Group and of the Cochrane Collaboration in general, as suggested in their handbook,37 as well as reported in standard texts on systematic reviews and meta-analyses.8, 15

Is pain the only outcome of interest? 

Why limit the data extraction to pain only? Although pain might be the cardinal symptom of neck troubles, neck pain only becomes clinically important when people seek and consume health care (seek out health care providers and use their services), lose function (such as range of motion, strength), lose sleep, lose time from work, and have impairments in their health-related quality of life. It is possible to have high pain levels and little disability and, conversely, lower pain levels and high disability.12 It is also possible to have neck pain associated with low back pain and headache, and more than half of individuals with neck pain have associated back pain.10 Other contemporary systematic reviews in neck pain have extracted outcome measures, other than pain, including range of motion, and function,20 and the Cochrane Back Pain group recommends extracting several outcome measures.47

Chiropractic care in social context 

Meeker and Haldeman35 have recently published a thoughtful review of the history and future challenges of chiropractic care, entitled “Chiropractors: A Profession at the Crossroads.” In this report the authors noted the historical philosophy of chiropractic, as Ernst does, but also described how many chiropractors are moving away from these concepts. It would seem that the chiropractic profession has recognized the need to undertake the science supporting their practices and that the National Institutes of Health is supportive of these endeavors.

There have recently been several well-conducted, large, randomized trials performed by chiropractors with PhD level research training. One of these trials6 was discussed by Ernst, whereas a recent publication comparing chiropractic mobilization and manipulation was not.27 The National Institutes of Health, through their National Center for Complementary and Alternative Medicine, supported the trial by Hurwitz et al.27

Having discussed the changing face of efficacy data, what are the data for harm? We agree with Ernst that chiropractic care has been a politicized process. However, rather than report some of the recent data on harm, Ernst largely quotes his own studies and an opinion piece from the Medical Letter on Drugs and Therpeutics,34 which quotes one of his letters to the editor of the Canadian Medical Association Journal on chiropractic safety. The “Medical Letter” does not present data on the incidence, prevalence, or risk factors for harm related to spinal manipulation, and Ernst does not in this review. However, there have been studies and reviews on the harm associated with manipulation that are relevant to this discussion, and they will be briefly outlined. The comprehensive review by Assendelft et al3 reported that the estimates of vertebrobasilar artery complications ranges from 1 in 20,000 to 1 in 1,000,000, although they pointed out the poor quality of this literature. They further suggested that the apparent high proportion of complications related to chiropractic might not relate solely to the use of high-velocity thrust techniques. Alternate explanations offered included selection bias for complications inherent in the use of the English language literature, wherein chiropractors make up a larger proportion of those applying spinal manipulation, and misclassification of injuries.3 A population-based study of chiropractic manipulation and stroke in the Canadian province of Ontario identified age younger than 45 years and more than 3 visits for chiropractic care in the last week were risk factors for vertebrobasilar events.42 However, the authors noted that the rarity of these events made the association difficult to study, in spite of high volumes of chiropractic treatments during the 6-year period under review. On the basis of reports to the Canadian malpractice carrier for chiropractors, the suggested incidence of arterial dissections is estimated to be less than 1 in 1,000,000.24 A Danish retrospective cohort study also suggested that the occurrence of cerebrovascular accidents after manipulation to the neck was estimated to be 1 in 1,000,000 cervical manipulations.30 Although severe complications with spinal manipulation are uncommon, short-term side effects to spinal manipulation are not.32

In contrast to the numerous trials of manipulation for neck pain noted above, the situation for medicinal therapies is humbling. A 1996 review found no placebo-controlled trials on acetaminophen, nonsteroidal anti-inflammatory drugs (NSAIDs), or opioids for neck pain.2 A recent update of medications and injections for neck pain also failed to disclose any placebo-controlled trials of acetaminophen, NSAIDs, or opioids specifically with a neck pain population.40 Yet acetaminophen and NSAIDs continue to be recommended in standard textbooks4 and used in practice.5, 11 Acetaminophen is recommended for neck pain in the absence of evidence of benefit and in the face of harm to the gastrointestinal tract,20 liver,44 and kidney.17 Likewise, there are no randomized, placebo-controlled trials of NSAIDs in a neck pain population, although the gastrointestinal toxicity of these drugs is well known, with serious toxicity occurring in the order of - persons.19 NSAIDS can provoke heart failure in those at risk,18 and the newer cyclooxygenase specific inhibitors are equally likely to cause renal failure compared to the traditional NSAIDs.1

Finally, although it appears that spinal manipulation has no distinct advantages over other therapies from an efficacy point of view, Americans with low back pain are more satisfied with chiropractic care than with medical care,7, 26, 46 even though chiropractors cost more. This appears to be true for patients with neck pain as well, because a Canadian study showed patients were satisfied with chiropractic care, and their satisfaction correlated with pain relief and reduction in pain-related disability.48

In short, we agree that further studies of all therapies for neck pain are needed and that comparative studies of common treatments for neck pain are especially needed. We are beyond the point of benefiting from interdisciplinary finger pointing, and we suggest that interdisciplinary cooperation is needed to address the pressing scientific issues of assessing benefits and harms of competing therapies for patients with neck pain.