Using Multiple Daily Pain Ratings to Improve Reliability and Assay Sensitivity: How Many Is Enough?

Published:October 02, 2014DOI:


      • In persons with chronic pain, a single pain rating produced less than adequate reliability.
      • A composite of 5 daily ratings resulted in reliability above .90.
      • The 5-day composite score was interchangeable with the mean for all available ratings.
      • This supports IMMPACT recommendations for improving assay sensitivity with multiday assessment.
      • Higher measure reliability can improve assay sensitivity, power, and statistical precision.


      The Initiative for Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) has reported diminished assay sensitivity in pain treatment trials and recommended investigation of the causes. Specific recommendations included examination of outcome measure reliability and lengthening the baseline measurement period to allow more measurements to be collected. This secondary data analysis evaluated the minimum number of daily pain intensity ratings required to obtain a reliability of at least .90 and whether a composite of this smaller number of ratings was interchangeable with the composite of all ratings. Veterans Affairs medical center patients made 14 daily calls to an automated telephone system to report their average daily pain intensity rating. A single daily rating produced less than adequate reliability (intraclass correlation coefficient = .65), but a composite of the average of 5 ratings resulted in reliability above .90. A Bland-Altman analysis revealed that the differences between a 5-day composite and the composite of all ratings were small (mean .09 points, standard deviation = .45; 95% confidence interval = −.05 to .23) and below the threshold for a clinically meaningful difference, indicating that the 2 measurements are interchangeable. Our results support the IMMPACT recommendations for improving assay sensitivity by collecting a multiple-day baseline of pain intensity ratings.


      This study examined the minimum number of pain ratings required to achieve reliability of .90 and examined whether this smaller subset of ratings could be used interchangeably with a composite of all available ratings. Attention to measure reliability could enhance the assay sensitivity, power, and statistical precision of pain treatment trials.

      Key words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to The Journal of Pain
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Bland J.M.
        • Altman D.G.
        Statistical methods for assessing agreement between two methods of clinical measurement.
        Lancet. 1986; 327: 307-310
        • Bolton J.E.
        Accuracy of recall of usual pain intensity in back pain patients.
        Pain. 1999; 83: 533-539
        • Dworkin R.H.
        • Turk D.C.
        • Peirce-Sandner S.
        • Burke L.B.
        • Farrar J.T.
        • Gilron I.
        • Jensen M.P.
        • Katz N.P.
        • Raja S.N.
        • Rappaport B.A.
        • Rowbotham M.C.
        • Backonja M.M.
        • Baron R.
        • Bellamy N.
        • Bhagwagar Z.
        • Costello A.
        • Cowan P.
        • Fang W.C.
        • Hertz S.
        • Jay G.W.
        • Junor R.
        • Kerns R.D.
        • Kerwin R.
        • Kopecky E.A.
        • Lissin D.
        • Malamut R.
        • Markman J.D.
        • McDermott M.P.
        • Munera C.
        • Porter L.
        • Rauschkolb C.
        • Rice A.S.
        • Sampaio C.
        • Skljarevski V.
        • Sommerville K.
        • Stacey B.
        • Steigerwald I.
        • Tobias J.
        • Trentacosti A.M.
        • Wasan A.D.
        • Wells G.A.
        • Williams J.
        • Witter J.
        • Ziegler D.
        Considerations for improving assay sensitivity in chronic pain clinical trials: IMMPACT recommendations.
        Pain. 2012; 153: 1148-1158
        • Farrar J.T.
        • Young J.P.
        • LaMoreaux L.
        • Werth J.L.
        • Poole R.M.
        Clinical importance of changes in chronic pain intensity measured on an 11-point numerical pain rating scale.
        Pain. 2001; 94: 149-158
        • Heapy A.A.
        • Sellinger J.J.
        • Higgins D.M.
        • Chatkoff D.K.
        • Bennett T.C.
        • Kerns R.D.
        Using interactive voice response to measure pain and quality of life.
        Pain Med. 2007; 8: S145-154
      1. Higgins DH, Sellinger JJ, Chatkoff DK, Heapy AA, Shulman M, Bennett T, Bellmore W, Kerns RD: Feasibility of interactive voice response for monitoring pain treatment effects. Paper presented at the annual meeting of the American Psychological Association, New Orleans, LA, 2006.

      2. International Conference on Harmonisation. E10: Choice of control groups and related issues in clinical trials. Available at: Accessed December 26, 2013.

        • Jamison R.N.
        • Raymond S.A.
        • Levine J.G.
        • Slawsby E.A.
        • Nedeljkovic S.S.
        • Katz N.P.
        Electronic diaries for monitoring chronic pain: 1-year validation study.
        Pain. 2001; 91: 277-285
        • Jensen M.P.
        • Karoly P.
        Self-report scales and procedures for assessing pain in adults.
        in: Turk D.C. Melzack R. Handbook of Pain Assessment. 3rd ed. Guilford Press, New York2011: 19-44
        • Jensen M.P.
        • McFarland C.A.
        Increasing the reliability and validity of pain intensity measurement in chronic pain patients.
        Pain. 1993; 55: 195-203
        • Katz J.
        • Melzack R.
        Measurement of pain.
        Surg Clin North Am. 1999; 79: 231-252
        • Kerns R.D.
        • Finn P.
        • Haythornthwaite J.
        Self-monitored pain intensity: Psychometric properties and clinical utility.
        J Behav Med. 1988; 11: 71-82
        • Lachin J.M.
        The role of measurement reliability in clinical trials.
        Clin Trials. 2004; 1: 553-566
        • McDonald C.J.
        • Mazzuca S.A.
        • McCabe G.P.
        How much of the placebo “effect” is really statistical regression?.
        Stat Med. 1983; 2: 417-427
        • Nunnally J.C.
        • Bernstein I.H.
        Psychometric Theory.
        2nd ed. McGraw-Hill, New York1978
        • Perkins D.O.
        • Wyatt R.J.
        • Bartko J.J.
        Penny-wise and pound-foolish: The impact of measurement error on sample size requirements in clinical trials.
        Biol Psychiatry. 2000; 47: 762-766
        • Serlin R.C.
        • Mendoza T.R.
        • Nakamura Y.
        • Edwards K.R.
        • Cleeland C.S.
        When is cancer pain mild, moderate or severe? Grading pain severity by its interference with function.
        Pain. 1995; 61: 277-284
        • Shrout P.E.
        • Fleiss J.L.
        Intraclass correlations: Uses in assessing rater reliability.
        Psychol Bull. 1979; 86: 420-428
        • Spearman C.C.
        Correlation calculated from faulty data.
        Br J Psychol. 1910; 3: 271-295
        • Turner J.A.
        • Deyo R.A.
        • Loeser J.D.
        • Von Korff M.
        • Fordyce W.
        The importance of placebo effects in pain treatment and research.
        J Am Med Assoc. 1994; 271: 1609-1614