Test-Retest Reliability of Pain-Related Brain Activity in Healthy Controls Undergoing Experimental Thermal Pain


      • Test-retest reliability of pain-related activity has not been extensively explored.
      • This study found generally “good” reliability for pain-related brain activity.
      • Self-reported pain, however, showed “excellent” test-retest reliability.
      • Findings suggest sufficient reliability of fMRI for valuable use in research.
      • However, self-report was more reliable than brain activity in this study.


      Although functional magnetic resonance imaging (fMRI) has been proposed as a method to elucidate pain-related biomarkers, little information exists related to psychometric properties of fMRI findings. This knowledge is essential for potential translation of this technology to clinical settings. The purpose of this study was to assess the test-retest reliability of pain-related brain activity and how it compares to the reliability of self-report. Twenty-two healthy controls (mean age = 22.6 years, standard deviation = 2.9) underwent 3 runs of an fMRI paradigm that used thermal stimuli to elicit experimental pain. Functional MRI summary statistics related to brain activity during thermal stimulation periods were extracted from bilateral anterior cingulate cortices and anterior insula. Intraclass correlations (ICCs) were conducted on these summary statistics and generally showed “good” test-retest reliability in all regions of interest (ICC range = .32–.88; mean = .71); however, these results did not surpass ICC values from pain ratings, which fell within the “excellent” range (ICC range = .93–.96; mean = .94). Findings suggest that fMRI is a valuable tool for measuring pain mechanisms but did not show an adequate level of test-retest reliability for fMRI to potentially act as a surrogate for individuals' self-report of pain.


      This study is one of the first reports to demonstrate the test-retest reliability of fMRI findings related to pain processing and provides a comparison to the reliability of subjective reports of pain. This information is essential for determining whether fMRI technology should be potentially translated for clinical use.

      Key words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to The Journal of Pain
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Apkarian A.V.
        • Hashmi J.A.
        • Baliki M.N.
        Pain and the brain: Specificity and plasticity of the brain in clinical chronic pain.
        Pain. 2011; 152: S49-S64
        • Aron A.R.
        • Gluck M.A.
        • Poldrack R.A.
        Long-term test–retest reliability of functional MRI in a classification learning task.
        Neuroimage. 2006; 29: 1000-1006
        • Bennett C.M.
        • Miller M.B.
        How reliable are the results from functional magnetic resonance imaging?.
        Ann N Y Acad Sci. 2010; 1191: 133-155
        • Bijur P.E.
        • Silver W.
        • Gallagher E.J.
        Reliability of the visual analog scale for measurement of acute pain.
        Acad Emerg Med. 2001; 8: 1153-1157
        • Borsook D.
        • Becerra L.
        • Hargreaves R.
        Biomarkers for chronic pain and analgesia. Part 1: The need, reality, challenges, and solutions.
        Discov Med. 2011; 11: 197-207
        • Borsook D.
        • Becerra L.
        • Hargreaves R.
        Biomarkers for chronic pain and analgesia. Part 2: How, where, and what to look for using functional imaging.
        Discov Med. 2011; 11: 209-219
        • Brandt D.J.
        • Sommer J.
        • Krach S.
        • Bedenbender J.
        • Kircher T.
        • Paulus F.M.
        • Jansen A.
        Test-retest reliability of fMRI brain activity during memory encoding.
        Front Psychiatry. 2013; 4: 163
        • Brooks J.C.
        • Nurmikko T.J.
        • Bimson W.E.
        • Singh K.D.
        • Roberts N.
        fMRI of thermal pain: Effects of stimulus laterality and attention.
        Neuroimage. 2002; 15: 293-301
        • Caceres A.
        • Hall D.L.
        • Zelaya F.O.
        • Williams S.C.R.
        • Mehta M.A.
        Measuring fMRI reliability with the intra-class correlation coefficient.
        Neuroimage. 2009; 45: 758-768
        • Cicchetti D.V.
        • Sparrow S.A.
        Developing criteria for establishing interrater reliability of specific items: Applications to assessment of adaptive behavior.
        Am J Ment Defic. 1981; 86: 127-137
        • Coghill R.C.
        • McHaffie J.G.
        • Yen Y.F.
        Neural correlates of interindividual differences in the subjective experience of pain.
        Proc Natl Acad Sci U S A. 2003; 100: 8538-8542
        • Davis K.D.
        Neuroimaging of pain: What does it tell us?.
        Curr Opin Support Palliat Care. 2011; 5: 116-121
        • Duerden E.G.
        • Albanses M.C.
        Localization of pain-related brain activation: A meta-analysis of neuroimaging data.
        Hum Brain Mapp. 2013; 34: 109-149
        • Eaton K.P.
        • Szaflarski J.P.
        • Altaye M.
        • Ball A.L.
        • Kissela B.M.
        • Banks C.
        • Holland S.K.
        Reliability of fMRI for studies of language in post-stroke aphasia subjects.
        Neuroimage. 2008; 41: 311-322
        • Gallagher E.J.
        • Bijur P.E.
        • Latimer C.
        • Silver W.
        Reliability and validity of a visual analog scale for acute abdominal pain in the ED.
        Am J Emerg Med. 2002; 20: 287-290
        • Grafton K.V.
        • Foster N.E.
        • Wright C.C.
        Test-retest reliability of the Short-Form McGill Pain Questionnaire: Assessment of intraclass correlation coefficients and limits of agreement in patients with osteoarthritis.
        Clin J Pain. 2005; 21: 73-82
        • Jacob T.
        • Baras M.
        • Zeev A.
        • Epstein L.
        Low back pain: Reliability of a set of pain measurement tools.
        Arch Phys Med Rehabil. 2001; 82: 735-742
        • Jones D.H.
        • Kilgour R.D.
        • Comtois A.S.
        Test-retest reliability of pressure pain threshold measurements of the upper limb and torso in young healthy women.
        J Pain. 2007; 8: 650-656
        • Jensen M.P.
        The validity and reliability of pain measures in adults with cancer.
        J Pain. 2003; 4: 2-21
        • Mackey S.C.
        Central neuroimaging of pain.
        J Pain. 2013; 14: 328
        • Maldjian J.A.
        • Laurienti P.J.
        • Kraft R.A.
        • Burdette J.H.
        An automated method for neuroanatomic and cytoarchitectonic atlas-based interrogation of fMRI data sets.
        Neuroimage. 2003; 19: 1233-1239
        • Meindl T.
        • Teipel S.
        • Elmouden R.
        • Mueller S.
        • Koch W.
        • Dietrich O.
        • Coates U.
        • Reiser M.
        • Glaser C.
        Test–retest reproducibility of the default-mode network in healthy individuals.
        Hum Brain Mapp. 2011; 31: 237-246
        • McCaffrey M.
        • Beebe A.
        Giving narcotics for pain.
        Nursing. 1989; 19: 161-165
        • McGrath C.L.
        • Kelley M.E.
        • Holtzheimer P.E.
        • Dunlop B.W.
        • Craighead W.E.
        • Franco A.R.
        • Craddock R.C.
        • Mayberg H.S.
        Toward a neuroimaging treatment selection biomarker for major depressive disorder.
        JAMA Psychiatry. 2013; 12: 1-9
        • Plichta M.M.
        • Schwarz A.J.
        • Grimm O.
        • Morgen K.
        • Mier D.
        • Haddad L.
        • Gerdes A.B.M.
        • Sauer C.
        • Tost H.
        • Esslinger C.
        • Colman P.
        • Wilson F.
        • Kirsch P.
        • Meyer-Lindenberg A.
        Test–retest reliability of evoked BOLD signals from a cognitive–emotive fMRI test battery.
        Neuroimage. 2012; 60: 1746-1758
        • Price D.D.
        • McGrath P.A.
        • Rafii A.
        • Buckingham B.
        The validation of visual analogue scales as ratio scale measures for chronic and experimental pain.
        Pain. 1983; 17: 45-60
        • Price D.D.
        • Patel R.
        • Robinson M.E.
        • Staud R.
        Characteristics of electronic visual analogue and numerical scales for ratings of experimental pain in healthy subjects and fibromyalgia patients.
        Pain. 2008; 140: 158-166
        • Robinson M.E.
        • Staud R.
        • Price D.D.
        Pain measurement and brain activity: Will neuroimages replace pain ratings?.
        J Pain. 2013; 14: 323-327
        • Rombouts S.A.
        • Barkhof F.
        • Hoogenraad F.G.
        • Sprenger M.
        • Scheltens P.
        Within-subject reproducibility of visual activation patterns with functional magnetic resonance imaging using multislice echo planar imaging.
        Magn Reson Imaging. 1998; 16: 105-113
        • Staud R.
        Predictors of clinical pain intensity in patients with fibromyalgia syndrome.
        Curr Pain Headache Rep. 2005; 9: 316-321
        • Sullivan M.D.
        • Cahana A.
        • Derbyshire S.
        • Loeser J.D.
        What does it mean to call chronic pain a brain disease?.
        J Pain. 2013; 14: 317-322
        • Wager T.D.
        • Atlas L.Y.
        • Lindquist M.A.
        • Roy M.
        • Woo C.W.
        • Kross E.
        An fMRI-based neurologic signature of physical pain.
        N Engl J Med. 2013; 368: 1388-1397
        • Wartolowska K.
        How neuroimaging can help us to visualise and quantify pain?.
        Eur J Pain Suppl. 2011; 5: 323-327
        • Williamson A.
        • Hoggart B.
        Pain: A review of three commonly used pain rating scales.
        J Clin Nurs. 2005; 14: 798-804