Advertisement

Data Interpretation in Analgesic Clinical Trials With Statistically Nonsignificant Primary Analyses: An ACTTION Systematic Review

Published:October 22, 2014DOI:https://doi.org/10.1016/j.jpain.2014.10.003

      Highlights

      • We investigated spin in RCTs with nonsignificant primary analyses.
      • Authors and editors should prioritize transparent reporting of RCTs.
      • Transparent reporting includes unbiased interpretation and recognizing limitations.

      Abstract

      Peer-reviewed publications of randomized clinical trials (RCTs) are the primary means of disseminating research findings. “Spin” in RCT publications is misrepresentation of statistically nonsignificant research findings to suggest treatment benefit. Spin can influence the way readers interpret clinical trials and use the information to make decisions about treatments and medical policies. The objective of this study was to determine the frequency with which 4 types of spin were used in publications of analgesic RCTs with nonsignificant primary analyses in 6 major pain journals. In the 76 articles included in our sample, 28% of the abstracts and 29% of the main texts emphasized secondary analyses with P values <.05; 22% of abstracts and 29% of texts emphasized treatment benefit based on nonsignificant primary results; 14% of abstracts and 18% of texts emphasized within-group improvements over time, rather than primary between-group comparisons; and 13% of abstracts and 10% of texts interpreted a nonsignificant difference between groups in a superiority study as comparable effectiveness. When considering the article conclusion sections, 21% did not mention the nonsignificant primary result, 22% were presented with no uncertainty or qualification, 30% did not acknowledge that future research was required, and 8% recommended the intervention for clinical use.

      Perspective

      This article identifies relatively frequent “spin” in analgesic RCTs. These findings highlight a need for authors, reviewers, and editors to be more cognizant of how analgesic RCT results are presented and attempt to minimize spin in future clinical trial publications.

      Key words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to The Journal of Pain
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Al-Marzouki S.
        • Roberts I.
        • Evans S.
        • Marshall T.
        Selective reporting in clinical trials: Analysis of trial protocols accepted by The Lancet.
        Lancet. 2008; 372: 201
        • Altwairgi A.K.
        • Booth C.M.
        • Hopman W.M.
        • Baetz T.D.
        Discordance between conclusions stated in the abstract and conclusions in the article: Analysis of published randomized controlled trials of systemic therapy in lung cancer.
        J Clin Oncol. 2012; 30: 3552-3557
        • Boutron I.
        • Dutton S.
        • Ravaud P.
        • Altman D.G.
        Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes.
        J Am Med Assoc. 2010; 303: 2058-2064
        • Brenner R.J.
        On the more insidious manifestations of bias in scientific reporting.
        J Am Coll Radiol. 2010; 7: 490-494
        • Chan A.W.
        • Altman D.G.
        Identifying outcome reporting bias in randomised trials on PubMed: Review of publications and survey of authors.
        BMJ. 2005; 330: 753
        • Chan A.W.
        • Hrobjartsson A.
        • Haahr M.T.
        • Gotzsche P.C.
        • Altman D.G.
        Empirical evidence for selective reporting of outcomes in randomized trials: Comparison of protocols to published articles.
        JAMA. 2004; 291: 2457-2465
        • Davis C.E.
        Secondary endpoints can be validly analyzed, even if the primary endpoint does not provide clear statistical significance.
        Control Clin Trials. 1997; 18: 557-560
        • Dworkin R.H.
        • O'Connor A.B.
        • Kent J.
        • Mackey S.C.
        • Raja S.N.
        • Stacey B.R.
        • Levy R.M.
        • Backonja M.
        • Baron R.
        • Harke H.
        • Loeser J.D.
        • Treede R.D.
        • Turk D.C.
        • Wells C.D.
        Interventional management of neuropathic pain: NeuPSIG recommendations.
        Pain. 2013; 154: 2249-2261
        • Fleming T.R.
        Clinical trials: Discerning hype from substance.
        Ann Intern Med. 2010; 153: 400-406
        • Fletcher R.H.
        • Black B.
        “Spin” in scientific writing: Scientific mischief and legal jeopardy.
        Med Law. 2007; 26: 511-525
        • Gewandter J.S.
        • McDermott M.P.
        • McKeown A.
        • Smith S.M.
        • Williams M.R.
        • Hunsinger M.
        • Farrar J.
        • Turk D.C.
        • Dworkin R.H.
        Reporting of missing data and methods used to accommodate them in recent analgesic clinical trials: ACTTION systematic review and recommendations.
        Pain. 2014; 155: 1871-1877
        • Gewandter J.S.
        • Smith S.M.
        • Mckeown A.
        • Burke L.B.
        • Hertz S.H.
        • Hunsinger M.
        • Katz N.P.
        • Lin A.H.
        • McDermott M.P.
        • Rappaport B.A.
        • Williams M.R.
        • Turk D.C.
        • Dworkin R.H.
        Reporting of primary analyses and multiplicity adjustment in recent analgesic clinical trials: ACTTION systematic review and recommendations.
        Pain. 2014; 155: 461-466
        • Ioannidis J.P.
        Why most published research findings are false.
        PLoS Med. 2005; 2: e124
        • Latronico N.
        • Metelli M.
        • Turin M.
        • Piva S.
        • Rasulo F.A.
        • Minelli C.
        Quality of reporting of randomized controlled trials published in Intensive Care Medicine from 2001 to 2010.
        Intensive Care Med. 2013; 39: 1386-1395
        • Lundh A.
        • Sismondo S.
        • Lexchin J.
        • Busuioc O.A.
        • Bero L.
        Industry sponsorship and research outcome.
        Cochrane Database Syst Rev. 2012; 12: MR000033
        • Marco C.A.
        • Larkin G.L.
        Research ethics: Ethical issues of data reporting and the quest for authenticity.
        Acad Emerg Med. 2000; 7: 691-694
        • Mathieu S.
        • Boutron I.
        • Moher D.
        • Altman D.G.
        • Ravaud P.
        Comparison of registered and published primary outcomes in randomized controlled trials.
        JAMA. 2009; 302: 977-984
        • Moher D.
        • Hopewell S.
        • Schulz K.F.
        • Montori V.
        • Gøtzshce P.C.
        • Cevereaux P.J.
        • Elbourne D.
        • Egger M.
        • Altman D.G.
        • Consolidated Standards of Reporting Trials Group
        CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomized trials.
        J Clin Epidemiol. 2010; 63: e1-e37
        • Moher D.
        • Liberati A.
        • Tetzlaff J.
        • Altman D.G.
        • The PRISMA Group
        Preferred Reporting Items for Systematic reviews and Meta-Analyses: The PRISMA Statement.
        BMJ. 2009; 339: b2535
        • O’Neill R.T.
        Secondary endpoints cannot be validly analyzed if the primary endpoint does not demonstrate clear statistical significance.
        Cont Clin Trials. 1997; 18: 550-556
        • Smith S.M.
        • Wang A.T.
        • Pereira A.
        • Chang R.D.
        • Mckeown A.
        • Greene K.
        • Rowbotham M.C.
        • Burke L.B.
        • Coplan P.
        • Gilron I.
        • Hertz S.H.
        • Katz N.P.
        • Lin A.H.
        • McDermott M.P.
        • Papadopoulos E.J.
        • Rappaport B.A.
        • Sweeney M.
        • Turk D.C.
        • Dworkin R.H.
        Discrepancies between registered and published primary outcome specifications in analgesic trials: ACTTION systematic review and recommendations.
        Pain. 2013; 154: 2769-2774
        • Sun X.
        • Briel M.
        • Busse J.W.
        • You J.J.
        • Akl E.A.
        • Mejza F.
        • Bala M.M.
        • Bassler D.
        • Mertz D.
        • Diaz-Granados N.
        • Vandvik P.O.
        • Malaga G.
        • Srinathan S.K.
        • Dahm P.
        • Johnston B.C.
        • Alonso-Coello P.
        • Hassouneh B.
        • Truong J.
        • Dattani N.D.
        • Walter S.D.
        • Heels-Ansdell D.
        • Bhatnagar N.
        • Altman D.G.
        • Guyatt G.H.
        The influence of study characteristics on reporting of subgroup analyses in randomised controlled trials: Systematic review.
        BMJ. 2011; 342: d1569
        • Turner E.H.
        • Matthews A.M.
        • Lindardatos E.
        • Tell R.A.
        • Rosenthal R.
        Selective publishing of antidepressant trials and its influence on apparent efficacy.
        N Engl J Med. 2008; 358: 252-260
        • Vedula S.S.
        • Bero L.
        • Scherer R.W.
        • Dickersin K.
        Outcome reporting in industry-sponsored trials of gabapentin for off-label use.
        N Engl J Med. 2009; 361: 1963-1971
        • Vera-Badillo F.E.
        • Shapiro R.
        • Ocana A.
        • Amir E.
        • Tannock I.F.
        Bias in reporting of end points of efficacy and toxicity in randomized, clinical trials for women with breast cancer.
        Ann Oncol. 2013; 24: 1238-1244
        • You B.
        • Gan H.K.
        • Pond G.
        • Chen E.X.
        Consistency in the analysis and reporting of primary end points in oncology randomized controlled trials from registration to publication: A systematic review.
        J Clin Oncol. 2012; 30: 210-216