Advertisement

Experimental Comparison of Parametric Versus Nonparametric Analyses of Data From the Cold Pressor Test

      Highlights

      • The appropriateness of using parametric analysis on cold pressor data was studied.
      • In most published studies, normality of data distribution was not mentioned.
      • However, the vast majority of these studies used parametric analyses.
      • Cold pressor test measures collected at 8 independent studies were not normally distributed.
      • Monte Carlo simulations reveal that a nonparametric approach is more appropriate.

      Abstract

      Parametric statistical methods are common in human pain research. They require normally distributed data, but this assumption is rarely tested. The current study analyzes the appropriateness of parametric testing for outcomes from the cold pressor test (CPT), a common human experimental pain test. We systematically reviewed published CPT studies to quantify how often researchers test for normality and how often they use parametric versus nonparametric tests. We then measured the normality of CPT data from 7 independent small to medium cohorts and 1 study of >10,000 subjects. We then examined the ability of 2 common mathematical transformations to normalize our skewed data sets. Lastly, we performed Monte Carlo simulations on a representative data set to compare the statistical power of the parametric t-test versus the nonparametric Wilcoxon Mann-Whitney test. We found that only 39% of published CPT studies (47/122) mentioned checking data distribution, yet 72% (88/122) used parametric statistics. Furthermore, among our 8 data sets, CPT outcomes were virtually always nonnormally distributed, and mathematical transformations were largely ineffective in normalizing them. The simulations demonstrated that the nonparametric Wilcoxon Mann-Whitney test had greater statistical power than the parametric t-test for all scenarios tested: For small effect sizes, the Wilcoxon Mann-Whitney test had up to 300% more power.

      Perspective

      These results demonstrate that parametric analyses of CPT data are routine but incorrect and that they likely increase the chances of failing to detect significant between-group differences. They suggest that nonparametric analyses become standard for CPT studies and that assumptions of normality be routinely tested for other types of pain outcomes as well.

      Key words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to The Journal of Pain
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Bradley J.V.
        A common situation conducive to bizarre distribution shapes.
        Am Stat. 1977; 31: 147-150
        • Bradley J.V.
        Robustness?.
        Br J Math Stat Psychol. 1978; 31: 144-152
        • Breivik H.
        • Collett B.
        • Ventafridda V.
        • Cohen R.
        • Gallacher D.
        Survey of chronic pain in Europe: Prevalence, impact on daily life, and treatment.
        Eur J Pain. 2006; 4: 287-333
        • Bridge P.D.
        • Sawilowsky S.S.
        Increasing physicians' awareness of the impact of statistics on research outcomes: Comparative power of the t-test and Wilcoxon rank-sum test in small samples applied research.
        J Clin Epidemiol. 1999; 52: 229-235
        • Cohen J.
        Statistical Power Analysis for the Behavioral Sciences.
        2nd ed. Erlbaum, Hillsdale, NJ1988
        • Colliver J.A.
        • Manchikanti L.
        • Markwell S.J.
        Evaluation and comparison of the distributions of gastric pH and hydrogen ion concentration.
        Anesthesiology. 1987; 67: 391-394
        • Dworkin R.H.
        • Turk D.C.
        • Katz N.P.
        • Rowbotham M.C.
        • Peirce-Sandner S.
        • Cerny I.
        • Clingman C.S.
        • Eloff B.C.
        • Farrar J.T.
        • Kamp C.
        • McDermott M.P.
        • Rappaport B.A.
        • Sanhai W.R.
        Evidence-based clinical trial design for chronic pain pharmacotherapy: A blueprint for ACTION.
        Pain. 2011; 152: S107-S115
        • Harden N.
        • Cohen M.
        Unmet needs in the management of neuropathic pain.
        J Pain Symptom Manage. 2003; 25: S12-S17
        • Institute of Medicine (US) Committee on Advancing Pain Research, Care, and Education
        Relieving Pain in America: A Blueprint for Transforming Prevention, Care, Education, and Research.
        National Academies Press (US), Washington, DC2011
        • Jain K.K.
        Current challenges and future prospects in management of neuropathic pain.
        Expert Rev Neurother. 2008; 8: 1743-1756
        • Johansen A.
        • Schirmer H.
        • Stubhaug A.
        • Nielsen C.S.
        Persistent post-surgical pain and experimental pain sensitivity in the Tromso study: Comorbid pain matters.
        Pain. 2014; 155: 341-348
        • Koenig J.
        • Jarczok M.N.
        • Ellis R.J.
        • Bach C.
        • Thayer J.F.
        • Hillecke T.K.
        Two-week test-retest stability of the cold pressor task procedure at two different temperatures as a measure of pain threshold and tolerance.
        Pain Pract. 2014; 14: E126-E135
        • Manly B.F.J.
        Randomization, Bootstrap and Monte Carlo Methods in Biology.
        2nd ed. Chapman-Hall/CRC, Boca Raton, FL1997
        • Micceri T.
        The unicorn, the normal curve, and other improbable creatures.
        Psychol Bull. 1989; 105: 156-166
        • Modir J.G.
        • Wallace M.S.
        Human experimental pain models 2: The cold pressor model.
        Methods Mol Biol. 2010; 617: 165-168
        • Moran J.L.
        • Solomon P.
        Worrying about normality.
        Crit Care Resusc. 2002; 4: 316-319
        • Pud D.
        • Eisenberg E.
        • Sprecher E.
        • Rogowski Z.
        • Yarnitsky D.
        The tridimensional personality theory and pain: Harm avoidance and reward dependence traits correlate with pain perception in healthy volunteers.
        Eur J Pain. 2004; 8: 31-38
        • Pud D.
        • Golan Y.
        • Pesta R.
        Hand dominancy—A feature affecting sensitivity to pain.
        Neurosci Lett. 2009; 467: 237-240
        • Sawilowsky S.S.
        Nonparametric tests of interaction in experimental design.
        Rev Educ Res. 1990; 60: 91-126
        • Sawilowsky S.S.
        Comments on using alternatives to normal theory statistics in social and behavioral science.
        Can Psychol. 1993; 34: 398-406
        • Sawilowsky S.S.
        Comments on using robust statistics in social and behavioral science.
        Br J Math Stat Psychol. 1998; 51: 49-52
        • Sawilowsky S.S.
        Fermat, Schubert, Einstein, and Behrens-Fisher: The probable difference between two means when σ12σ22.
        J Mod Appl Stat Meth. 2002; 1: 461-472
        • Sawilowsky S.S.
        Misconceptions leading to choosing the t test over the Wilcoxon Mann-Whitney test for shift in location parameter.
        J Mod Appl Stat Meth. 2005; 4: 598-600
        • Sawilowsky S.S.
        New effect size rules of thumb.
        J Mod Appl Stat Meth. 2009; 8: 597-599
        • Sawilowsky S.S.
        • Blair R.C.
        A more realistic look at the robustness and type II error properties of the t test to departures from population normality.
        Psychol Bull. 1992; 111: 353-360
        • Stabell N.
        • Stubhaug A.
        • Flaegstad T.
        • Nielsen C.S.
        Increased pain sensitivity among adults reporting irritable bowel syndrome symptoms in a large population-based study.
        Pain. 2013; 154: 385-392
        • Torrance N.
        • Smith B.H.
        • Lee A.J.
        • Aucott L.
        • Cardy A.
        • Bennett M.I.
        Analysing the SF-36 in population-based research. A comparison of methods of statistical approaches using chronic pain as an example.
        J Eval Clin Pract. 2009; 15: 328-334
        • Treister R.
        • Pud D.
        • Ebstein R.P.
        • Eisenberg E.
        Dopamine transporter genotype dependent effects of apomorphine on cold pain tolerance in healthy volunteers.
        PLoS One. 2013; 8: e63808
        • Treister R.
        • Pud D.
        • Ebstein R.P.
        • Laiba E.
        • Gershon E.
        • Haddad M.
        • Eisenberg E.
        Associations between polymorphisms in dopamine neurotransmitter pathway genes and pain response in healthy humans.
        Pain. 2009; 147: 187-193
        • Tukey J.W.
        Exploratory Data Analysis.
        1st ed. Addison-Wesley, Massachusetts1977
        • Turk D.C.
        • Dworkin R.H.
        • Burke L.B.
        • Gershon R.
        • Rothman M.
        • Scott J.
        • Allen R.R.
        • Atkinson J.H.
        • Chandler J.
        • Cleeland C.
        • Cowan P.
        • Dimitrova R.
        • Dionne R.
        • Farrar J.T.
        • Haythornthwaite J.A.
        • Hertz S.
        • Jadad A.R.
        • Jensen M.P.
        • Kellstein D.
        • Kerns R.D.
        • Manning D.C.
        • Martin S.
        • Max M.B.
        • McDermott M.P.
        • McGrath P.
        • Moulin D.E.
        • Nurmikko T.
        • Quessy S.
        • Raja S.
        • Rappaport B.A.
        • Rauschkolb C.
        • Robinson J.P.
        • Royal M.A.
        • Simon L.
        • Stauffer J.W.
        • Stucki G.
        • Tollett J.
        • von Stein T.
        • Wallace M.S.
        • Wernicke J.
        • White R.E.
        • Williams A.C.
        • Witter J.
        • Wyrwich K.W.
        • Initiative on Methods, Measurement and Pain Assessment in Clinical Trials
        Developing patient-reported outcome measures for pain clinical trials: IMMPACT recommendations.
        Pain. 2006; 125: 208-215
        • Vadalouca A.
        • Siafaka I.
        • Argyra E.
        • Vrachnou E.
        • Moka E.
        Therapeutic management of chronic neuropathic pain: An examination of pharmacologic treatment.
        Ann N Y Acad Sci. 2006; 1088: 164-186
        • Vickers A.J.
        Parametric versus non-parametric statistics in the analysis of randomized trials with non-normally distributed data.
        BMC Med Res Methodol. 2005; 5: 35
        • von Baeyer C.L.
        • Piira T.
        • Chambers C.T.
        • Trapanotto M.
        • Zeltzer L.K.
        Guidelines for the cold pressor task as an experimental pain stimulus for use with children.
        J Pain. 2005; 6: 218-227

      Linked Article

      • Letter to the Editor: Experimental Comparison of Parametric Versus Nonparametric Analyses of Data From the Cold Pressor Test
        The Journal of PainVol. 17Issue 1
        • Preview
          We read with great interest the recent article by Treiser et al,7 published in the June 2015 issue of the Journal of Pain, entitled “Experimental comparison of parametric versus nonparametric analyses of data from the Cold Pressor Test.” On the basis of systematic reviews and intensive Monte Carlo simulations, Treiser et al noted frequent but inappropriate usage of the parametric t-test and recommended use of the nonparametric Wilcoxon-Mann-Whitney (WMW) test for cold pressor test data, as well as routinely checking assumptions of normality for other types of pain outcomes.
        • Full-Text
        • PDF