Advertisement

Systematic Review of Research Methods and Reporting Quality of Randomized Clinical Trials of Spinal Cord Stimulation for Pain

Open AccessPublished:June 20, 2020DOI:https://doi.org/10.1016/j.jpain.2020.05.001

      Highlights

      • This review assessed methods of trials of spinal cord stimulation for pain.
      • Included studies assessed not only pain, but also functioning and opioid use.
      • Methodological and reporting deficiencies were common.
      • Challenges unique to this field include difficulties with blinding.
      • Posting of study protocols with a priori criteria would enhance transparency.

      Abstract

      This systematic review assessed design characteristics and reporting quality of published randomized clinical trials of spinal cord stimulation (SCS) for treatment of pain in adults and adolescents. The study protocol was registered with PROSPERO (CRD42018090412). Relevant articles were identified by searching the following databases through December 31, 2018: MEDLINE, Embase, WikiStim, The Cochrane Database of Systematic Reviews, and The Cochrane Central Register of Controlled Trials. Forty-six studies were included. Eighty-seven percent of articles identified a pain-related primary outcome. Secondary outcomes included physical functioning, health-related quality of life, and reductions in opioid use. Nineteen of the 46 studies prespecified adverse events as an outcome, with 4 assessing them as a primary outcome. Eleven studies stated that they blinded participants. Of these, only 5 were assessed as being adequately blinded. The number of participants enrolled was generally low (median 38) and study durations were short (median 12 weeks), particularly in studies of angina. Fifteen studies employed an intention-to-treat analysis, of which only seven specified a method to accommodate missing data. Review of these studies identified deficiencies in both reporting and methodology. The review's findings suggest areas for improving the design of future studies and increasing transparency of reporting.

      Perspective

      This article presents a systematic review of research methods and reporting quality of randomized clinical trials of SCS for the treatment of various pain complaints. The review identifies deficiencies in both methodology and reporting, which may inform the design of future studies and improve reporting standards.

      Key words

      The use of spinal cord stimulation (SCS) for the treatment of chronic pain was first described in 1967.
      • Shealy CN
      • Mortimer JT
      • Reswick JB
      Electrical inhibition of pain by stimulation of the dorsal columns: Preliminary clinical report.
      Its development was based on the gate-control theory of pain, introduced a few years earlier by Melzack and Wall, who hypothesized that a “gate” in the dorsal horn of the spinal cord dictated transmission of nociception within the central nervous system.
      • Melzack R
      • Wall PD
      Pain mechanisms: A new theory.
      The hypothetical gate could be closed when stimulation of large diameter myelinated fibers associated with touch, pressure, or vibration predominated over stimulation of thinner, unmyelinated pain fibers, in turn attenuating or eliminating noxious signaling to the brain. SCS was thought to represent a means of electrically closing the gate by stimulation of such large afferent fibers.
      Small clinical trials assessing the efficacy of SCS in reducing pain in various chronic conditions, including low back pain, angina, peripheral vascular disease, peripheral neuropathy, complex regional pain syndrome, type I, and irritable bowel syndrome, were first published in the early 1970s. In 1984 the United States (US) Food and Drug Administration (FDA) approved SCS to treat chronic intractable pain in the trunk or legs, including pain associated with failed back surgery syndrome. The first randomized clinical trials (RCTs) of SCS were published in the mid-1990s,
      • North RB
      • Kidd DH
      • Lee MS
      • Piantodosi S
      A prospective, randomized study of spinal cord stimulation versus reoperation for failed back surgery syndrome: Initial results.
      ,
      • Suy R
      • Gybels J
      • Van Damme H
      • Martin D
      • Van Maele R
      • Delaporte C
      Spinal cord stimulation for ischemic rest pain. The Belgian randomized study.
      and the first systematic review of efficacy of this approach shortly thereafter.
      • Turner JA
      • Loeser JD
      • Bell KG
      Spinal cord stimulation for chronic low back pain. A systematic literature synthesis.
      Current applications of SCS can broadly be divided into conventional, high-frequency, and high-frequency burst stimulation.
      • Miller JP
      • Eldabe S
      • Buchser E
      • Johanek LM
      • Guan Y
      • Linderoth B
      Parameters of spinal cord stimulation and their role in electrical charge delivery: A review.
      In conventional SCS, the patient typically experiences paresthesias in the area of coverage. With high-frequency stimulation (HF-SCS), the stimulus is delivered at the dorsal columns at a higher frequency than conventional SCS. In burst stimulation, the stimulus is delivered at a low frequency with closely spaced high frequency pulses. As with HF-SCS, the patients are unlikely to experience paresthesias.
      While systematic reviews have supported the efficacy of the various SCS modalities in terms of pain reduction, there are limitations of existing evidence and challenges in conducting research.
      • Grider J
      • Manchikanti L
      • Carayannopoulos A
      • Sharma ML
      • Balog CC
      • Harned ME
      • Grami V
      • Justiz R
      • Nouri KH
      • Hayek SM
      • Vallejo R
      Effectiveness of spinal cord stimulation in chronic spinal pain: A systematic review.
      ,
      • Mailis-Gagnon A
      • Furlan MD PhD AD
      • Sandoval JA
      • Taylor RS
      Spinal cord stimulation for chronic pain.
      Limitations include the fact that many studies have small sample sizes, and have relatively short durations.
      • Mailis-Gagnon A
      • Furlan MD PhD AD
      • Sandoval JA
      • Taylor RS
      Spinal cord stimulation for chronic pain.
      Challenges in designing SCS trials include the requirement for highly specialized care, the invasiveness of the procedure, and the costs of surgery and the devices.
      • Geurts JW
      • Joosten EA
      • van Kleef M
      Current status and future perspectives of spinal cord stimulation in treatment of chronic pain.
      The requirement for paresthesia with conventional SCS makes complete blinding with placebo or sham interventions a more complex undertaking.

      North RB, Shipley J: Chapter 4 - Clinical study designs for neuromodulation, in Krames Elliot S, Peckham P Hunter, Rezai Ali R, (eds.): Neuromodulation, 2nd ed. London, United Kingdom, Academic Press, 2018, pp 41-51.

      Study duration may be an issue in that a device in the control arm may have been superseded by an updated device by the point at which a trial is completed, ie, the device field is currently evolving faster than the duration of many RCTs.
      • Neugebauer EA
      • Rath A
      • Antoine SL
      • Eikermann M
      • Seidel D
      • Koenen C
      • Jacobs E
      • Pieper D
      • Laville M
      • Pitel S
      • Martinho C
      Specific barriers to the conduct of randomised clinical trials on medical devices.
      There may also be issues with patient selection; eg, clinical presentation or pain characteristics may influence the likelihood of a successful outcome.
      • Mailis-Gagnon A
      • Furlan MD PhD AD
      • Sandoval JA
      • Taylor RS
      Spinal cord stimulation for chronic pain.
      ,
      • North R
      • Shipley J
      Practice parameters for the use of spinal cord stimulation in the treatment of chronic neuropathic pain.
      With these limitations and challenges in mind, the objectives of this review of RCTs of SCS for pain were to: 1) assess research methods employed, 2) document which outcomes were assessed and how they were reported, and 3) summarize patient characteristics. By documenting current design and reporting, and by highlighting possible deficiencies in both, we hope to provide the foundation for recommendations for standardization and improvement in future trials.

      Methods

      The methods of this systematic review conformed to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (prisma-statement.org/) (Appendix e-1). The study protocol was registered with PROSPERO (CRD42018090412).

      PROSPERO: International prospective register of systematic reviews. National Institute for Health Research. Registered May 2018. Updated November 2018. Available at: http://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42018090412. Accessed July 9, 2020

      This protocol was also used as a framework for smaller, but related systematic reviews of placebo/sham-controlled trials and of cost-effectiveness studies.
      • Duarte RV
      • McNicol E
      • Colloca L
      • Taylor RS
      • North RB
      • Eldabe S
      Randomised placebo/sham controlled trials of spinal cord stimulation: A systematic review and methodological appraisal.
      ,
      • Duarte RV
      • Nevitt S
      • McNicol E
      • Taylor RS
      • North RB
      • Eldabe S
      Are all placebo/sham controls the same? Lessons from a systematic review and meta-analysis of placebo/sham controlled cross-over randomised trials of spinal cord stimulation for neuropathic pain.

      Search

      MEDLINE, Embase, The Cochrane Database of Systematic Reviews, and The Cochrane Central Register of Controlled Trials (CENTRAL) were systematically searched to identify eligible studies up to February 7, 2018 (date of search) using appropriate search terminology (Appendix e-2). The same search terminology (restricted to publication year 2018) was repeated in January 2019 to identify eligible studies published as of December 31, 2018. WIKISTIM monthly newsletters were reviewed from February through November 2018 to identify new citations added to the topic of SCS.
      • North RB
      • Shipley J
      WIKISTIM.org: An on-line database of published neurostimulation studies.
      ,

      Wikistim.org. The Neuromodulation Foundation, Inc. Copyright 2013-2019. Available at: https://www.wikistim.org/. Accessed July 9, 2020

      The reference lists of all included studies were also reviewed for additional studies. The results of the search were used to inform 2 further analyses.
      • Duarte RV
      • McNicol E
      • Colloca L
      • Taylor RS
      • North RB
      • Eldabe S
      Randomised placebo/sham controlled trials of spinal cord stimulation: A systematic review and methodological appraisal.
      ,
      • Duarte RV
      • Nevitt S
      • McNicol E
      • Taylor RS
      • North RB
      • Eldabe S
      Are all placebo/sham controls the same? Lessons from a systematic review and meta-analysis of placebo/sham controlled cross-over randomised trials of spinal cord stimulation for neuropathic pain.

      Study Selection

      Inclusion criteria were RCTs (of any design including parallel, cross-over, or cluster) evaluating SCS for any acute or chronic pain condition. Nonrandomized studies, abstracts, conference proceedings, and studies without a pain-related outcome were excluded. Studies of dorsal root ganglion stimulation were excluded (unless they also assessed SCS) as the anatomic target of dorsal root ganglion stimulation is more akin to peripheral nerve stimulation, and it is not clear to what extent the relevant mechanisms of pain relief might be the same for both. There were no exclusions based on lack of patient or clinician blinding, study duration, language, or sample size. Each stage of study selection was performed in duplicate and checked for agreement between reviewers (any combination of EM, MF, KB, or ER). Eligibility was determined by reading the abstract of each study identified by the search. Studies that clearly did not satisfy the inclusion criteria were eliminated, and full texts of the remaining studies were obtained. Two review authors read each of these full texts independently and reached agreement by discussion. Where agreement could not be reached, a third review author adjudicated. No attempts were made to anonymize studies before assessment.

      Data Extraction

      Four investigators (authors EM, MF, KB, ER) performed data extraction. A combination of 2 of the 4 investigators independently extracted information from each study and adjudicated discrepancies. A data extraction form was piloted prior to use (Appendix e-3). A coding manual was developed to clarify entry and promote consistency in the evaluation of study methodology (Appendix e-4). Risk of bias for methods related to randomization and adequacy of blinding (if applicable) was assessed as ``high risk,” ``low risk,” or ``unclear risk” according to criteria outlined in the Cochrane Handbook for Systematic Reviews of Interventions.
      • Higgins JPT
      • Altman DG
      • Sterne JAC
      Chapter 8: Assessing risk of bias in included studies.
      Angina studies were assessed separately from other chronic pain syndromes because of assumed differences in inclusion criteria and other study design features.
      Information related to population, funding source(s), inclusion criteria, interventions and controls, use of a trial/screening phase, duration, primary and secondary outcomes related to pain, collection of adverse events (AEs), methods of data analysis (eg, sample size calculations, clinical significance, handling of missing data), and baseline demographics (eg, number of participants, sex, and age) were collected. Descriptive statistics were used to provide the central tendency (mean or median), variability (expressed as ranges or standard deviations [SDs]), or percent of studies reporting a certain method. Descriptions were grouped into available data about the 1) population, 2) intervention, 3) comparator, 4) outcomes, and 5) study characteristics, with special attention to design and methodology.

      Results

      Search Results

      Of the 1,227 de-duplicated records identified from the initial search of multiple databases, 1,108 were excluded, based on screening the title or abstract. An additional 68 were excluded after full text review: 15 were abstracts or protocols only, 16 were not randomized, 9 were studies of cost only, 13 were duplicate publications (ie, they reported data that had been reported in a previous manuscript), 13 had no pain outcome, and 2 were excluded for other reasons (Fig 1). The initial search was re-run through December 31st, 2018 and identified a further 2 studies, resulting in a total of 46 included studies,
      • Al-Kaisy A
      • Palmisani S
      • Pang D
      • Sanderson K
      • Wesley S
      • Tan Y
      • McCammon S
      • Trescott A
      Prospective, randomized, sham-control, double blind, crossover trial of subthreshold spinal cord stimulation at various Kilohertz frequencies in subjects suffering from failed back surgery syndrome (SCS Frequency Study).
      • De Andres J
      • Monsalve-Dolz V
      • Fabregat-Cid G
      • Villanueva-Perez V
      • Harutyunyan A
      • Asensio-Samper JM
      • Sanchis-Lopez N
      Prospective, randomized blind effect-on-outcome study of conventional vs high-frequency spinal cord stimulation in patients with pain and disability due to failed back surgery syndrome.
      • de Jongste MJ
      • Hautvast RW
      • Hillege HL
      • Lie KI
      Efficacy of spinal cord stimulation as adjuvant therapy for intractable angina pectoris: A prospective, randomized clinical study.
      • de Jongste MJL
      • Staal MJ
      Preliminary results of a randomized study on the clinical efficacy of spinal cord stimulation for refractory severe angina pectoris.
      • de Ridder Plazier M
      • Kamerling N
      • Menovsky T
      • Vanneste S
      Burst spinal cord stimulation for limb and back pain.
      • de Vos CC
      • Meier K
      • Zaalberg PB
      • Nijhuis HJ
      • Duyvendak W
      • Vesper J
      • Enggaard TP
      • Lenders MW
      Spinal cord stimulation in patients with painful diabetic neuropathy: A multicentre randomized clinical trial.
      • Deer T
      • Slavin KV
      • Amirdelfan K
      • North RB
      • Burton AW
      • Yearwood TL
      • Tavel E
      • Staats P
      • Falowski S
      • Pope J
      • Justiz R
      Success using neuromodulation with BURST (SUNBURST) study: Results from a prospective, randomized controlled trial using a novel burst waveform.
      • Deer TR
      • Levy RM
      • Kramer J
      • Poree L
      • Amirdelfan K
      • Grigsby E
      • Staats P
      • Burton AW
      • Burgher AH
      • Obray J
      • Scowcroft J
      Dorsal root ganglion stimulation yielded higher treatment success rate for complex regional pain syndrome and causalgia at 3 and 12 months: A randomized comparative trial.
      • Di Pede F
      • Zuin G
      • Giada F
      • Pinato G
      • Turiano G
      • Bevilacqua M
      • Cazzin R
      • Raviele A
      Long-term effects of spinal cord stimulation on myocardial ischemia and heart rate variability: Results of a 48-hour ambulatory electrocardiographic monitoring.
      ,
      • Eddicks S
      • Maier-Hauff K
      • Schenk M
      • Müller A
      • Baumann G
      • Theres H
      Thoracic spinal cord stimulation improves functional status and relieves symptoms in patients with refractory angina pectoris: The first placebo-controlled randomised study.
      ,
      • Eisenberg E
      • Burstein Y
      • Suzan E
      • Treister R
      • Aviram J
      Spinal cord stimulation attenuates temporal summation in patients with neuropathic pain.
      ,
      • Eldabe S
      • Thomson S
      • Duarte R
      • Brookes M
      • deBelder M
      • Raphael J
      • Davies E
      • Taylor R
      The effectiveness and cost-effectiveness of spinal cord stimulation for refractory angina (RASCAL study): A pilot randomized controlled trial.
      ,
      • Hautvast RW
      • DeJongste MJ
      • Staal MJ
      • van Gilst WH
      • Lie KI
      Spinal cord stimulation in chronic intractable angina pectoris: A randomized, controlled efficacy study.
      ,
      • Jessurun GA
      • DeJongste MJ
      • Hautvast RW
      • Tio RA
      • Brouwer J
      • van Lelieveld ST
      • Crijns HJ
      Clinical follow-up after cessation of chronic electrical neuromodulation in patients with severe coronary artery disease: A prospective randomized controlled study on putative involvement of sympathetic activity.
      • Jivegard LEH
      • Augustinsson LE
      • Holm J
      • Risberg B
      • Ortenwall P
      Effects of spinal cord stimulation (SCS) in patients with inoperable severe lower limb ischaemia: A prospective randomised controlled study.
      • Kapural L
      • Yu C
      • Doust MW
      • Gliner BE
      • Vallejo R
      • Sitzman BT
      • Amirdelfan K
      • Morgan DM
      • Brown LL
      • Yearwood TL
      • Bundschu R
      Novel 10-kHz high-frequency therapy (HF10 Therapy) is superior to traditional low-frequency spinal cord stimulation for the treatment of chronic back and leg pain.
      ,
      • Kemler MA
      • Barendse GA
      • Van Kleef M
      • de Vet HC
      • Rijks CP
      • Furnee CA
      • Van Den Wildenberg FA
      Spinal cord stimulation in patients with chronic reflex sympathetic dystrophy.
      ,
      • Klomp HM
      • Spincemaille GH
      • Steyerberg EW
      • Habbema JD
      • van Urk H
      • ESES Study Group
      Spinal-cord stimulation in critical limb ischaemia: A randomised trial.
      ,
      • Kriek N
      • Groeneweg JG
      • Stronks DL
      • Huygen FJPM
      Preferred frequencies and waveforms for spinal cord stimulation in patients with complex regional pain syndrome: A multicentre, double-blind, randomized and placebo-controlled crossover trial.
      ,
      • Kumar K
      • Taylor RS
      • Jacques L
      • Eldabe S
      • Meglio M
      • Molet J
      • Thomson S
      • O'Callaghan J
      • Eisenberg E
      • Milbouw G
      • Buchser E
      Spinal cord stimulation versus conventional medical management for neuropathic pain: A multicentre randomised controlled trial in patients with failed back surgery syndrome.
      • Kumar V
      • Prusik J
      • Lin Y
      • Hwang R
      • Feustel P
      • Pilitsis JG
      Efficacy of alternating conventional stimulation and high frequency stimulation in improving spinal cord stimulation outcomes: A pilot study.
      • Lanza GA
      • Grimaldi R
      • Greco S
      • Ghio S
      • Sarullo F
      • Zuin G
      • De Luca A
      • Allegri M
      • Di Pede F
      • Castagno D
      • Turco A
      Spinal cord stimulation for the treatment of refractory angina pectoris: A multicenter randomized single-blind study (the SCS-ITA trial).
      • Lanza GA
      • Sestito A
      • Sgueglia GA
      • Infusino F
      • Papacci F
      • Visocchi M
      • Ierardi C
      • Meglio M
      • Bellocci F
      • Crea F
      Effect of spinal cord stimulation on spontaneous and stress-induced angina and 'ischemia-like' ST-segment depression in patients with cardiac syndrome X.
      • Lind G
      • Winter J
      • Linderoth B
      • Hellström PM
      Therapeutic value of spinal cord stimulation in irritable bowel syndrome: A randomized crossover pilot study.
      ,
      • Mannheimer C
      • Eliasson T
      • Augustinsson LE
      • Blomstrand C
      • Emanuelsson H
      • Larsson S
      • Norrsell H
      • Hjalmarsson A
      Electrical stimulation versus coronary artery bypass surgery in severe angina pectoris: The ESBY study.
      ,
      • McNab D
      • Khan SN
      • Sharples LD
      • Ryan JY
      • Freeman C
      • Caine N
      • Tait S
      • Hardy I
      • Schofield PM
      An open label, single-centre, randomized trial of spinal cord stimulation vs. percutaneous myocardial laser revascularization in patients with refractory angina pectoris: The SPiRiT trial.
      ,
      • Meier K
      • Nikolajsen L
      • Sörensen JC
      • Jensen TS
      Effect of spinal cord stimulation on sensory characteristics: A randomized, blinded crossover study.
      ,
      • North JM
      • Hong KS
      • Cho PY
      Clinical outcomes of 1 kHz subperception spinal cord stimulation in implanted patients with failed paresthesia-based stimulation: Results of a prospective randomized controlled trial.
      ,
      • North RB
      • Brigham DD
      • Khalessi A
      • Calkins SK
      • Piantadosi S
      • Campbell DS
      • Daly MJ
      • Dey PB
      • Barolat G
      • Taylor R
      Spinal cord stimulator adjustment to maximize implanted battery longevity: A randomized, controlled trial using a computerized patient-interactive programmer.
      ,
      • North RB
      • Kidd DH
      • Farrokhi F
      • Piantadosi SA
      Spinal cord stimulation versus repeated lumbosacral spine surgery for chronic pain: A randomized, controlled trial.
      ,
      • North RB
      • Kidd DH
      • Petrucci L
      • Dorsi MJ
      Spinal cord stimulation electrode design: A prospective, randomized, controlled trial comparing percutaneous with laminectomy electrodes: Part II–clinical outcomes.
      ,
      • Perruchoud C
      • Eldabe S
      • Batterham AM
      • Madzinga G
      • Brookes M
      • Durrer A
      • Rosato M
      • Bovet N
      • West S
      • Bovy M
      • Rutschmann B
      Analgesic efficacy of high-frequency spinal cord stimulation: A randomized double-blind placebo-controlled study.
      ,
      • Schu S
      • Slotty PJ
      • Bara G
      • von Knop M
      • Edgar D
      • Vesper J
      A prospective, randomised, double-blind, placebo-controlled study to examine the effectiveness of burst spinal cord stimulation patterns for the treatment of failed back surgery syndrome.
      ,
      • Schultz DM
      • Webster L
      • Kosek P
      • Dar U
      • Tan Y
      • Sun M
      Sensor-driven position-adaptive spinal cord stimulation for chronic pain.
      ,
      • Slangen R
      • Schaper NC
      • Faber CG
      • Joosten EA
      • Dirksen CD
      • van Dongen RT
      • Kessels AG
      • van Kleef M
      Spinal cord stimulation and pain relief in painful diabetic peripheral neuropathy: A prospective two-center randomized controlled trial.
      ,
      • Spincemaille GH
      • Klomp HM
      • Steyerberg EW
      • Habbema JD
      Pain and quality of life in patients with critical limb ischaemia: Results of a randomized controlled multicentre study on the effect of spinal cord stimulation.
      • Spincemaille GH
      • Klomp HM
      • Steyerberg EW
      • Habbema JD
      Spinal cord stimulation in patients with critical limb ischemia: A preliminary evaluation of a multicentre trial.
      • Spincemaille GH
      • Klomp HM
      • Steyerberg EW
      • van Urk H
      • Habbema JD
      Technical data and complications of spinal cord stimulation: Data from a randomized trial on critical limb ischemia.
      • Suy R
      • Gybels J
      • Van Damme H
      • Martin D
      • Van Maele R
      • Delaporte C
      Spinal cord stimulation for ischemic rest pain. The Belgian randomized study.
      • Thomson SJ
      • Tavakkolizadeh M
      • Love-Jones S
      • Patel NK
      • Gu JW
      • Bains A
      • Doan Q
      • Moffitt M
      Effects of rate on analgesia in Kilohertz frequency spinal cord stimulation: Results of the PROCO randomized controlled trial.
      • Tjepkema-Cloostermans MC
      • de Vos
      • Wolters CC
      • Dijkstra-Scholten R
      • Lenders MWPM C
      Effect of burst stimulation evaluated in patients familiar with spinal cord stimulation.
      ,
      • van Bussel CM
      • Stronks DL
      • Huygen FJ
      Dorsal column stimulation vs. dorsal root ganglion stimulation for complex regional pain syndrome confined to the knee: Patients’ preference following the trial period.
      ,
      • Van Havenbergh T
      • Vancamp T
      • Van Looy P
      • Vanneste S
      • De Ridder D
      Spinal cord stimulation for the treatment of chronic back pain patients: 500-Hz vs. 1000-Hz burst stimulation.
      ,
      • Washburn S
      • Catlin R
      • Bethel K
      • Canlas B
      Patient-perceived differences between constant current and constant voltage spinal cord stimulation systems.
      ,
      • Wolter T
      • Kiemen A
      • Porzelius C
      • Kaube H
      Effects of sub-perception threshold spinal cord stimulation in neuropathic pain: A randomized controlled double-blind crossover study.
      • Youn Y
      • Smith H
      • Morris B
      • Argoff C
      • Pilitsis J
      The effect of high-frequency stimulation on sensory thresholds in chronic pain patients.
      • Zipes DP
      • Svorkdal N
      • Berman D
      • Boortz-Marx R
      • Henry T
      • Lerman A
      • Ross E
      • Turner M
      • Irwin C
      Spinal cord stimulation therapy for patients with refractory angina who are not candidates for revascularization.
      12 of which assessed outcomes in patients with angina.
      • de Jongste MJ
      • Hautvast RW
      • Hillege HL
      • Lie KI
      Efficacy of spinal cord stimulation as adjuvant therapy for intractable angina pectoris: A prospective, randomized clinical study.
      ,
      • de Jongste MJL
      • Staal MJ
      Preliminary results of a randomized study on the clinical efficacy of spinal cord stimulation for refractory severe angina pectoris.
      ,
      • Di Pede F
      • Zuin G
      • Giada F
      • Pinato G
      • Turiano G
      • Bevilacqua M
      • Cazzin R
      • Raviele A
      Long-term effects of spinal cord stimulation on myocardial ischemia and heart rate variability: Results of a 48-hour ambulatory electrocardiographic monitoring.
      ,
      • Eddicks S
      • Maier-Hauff K
      • Schenk M
      • Müller A
      • Baumann G
      • Theres H
      Thoracic spinal cord stimulation improves functional status and relieves symptoms in patients with refractory angina pectoris: The first placebo-controlled randomised study.
      ,
      • Eldabe S
      • Thomson S
      • Duarte R
      • Brookes M
      • deBelder M
      • Raphael J
      • Davies E
      • Taylor R
      The effectiveness and cost-effectiveness of spinal cord stimulation for refractory angina (RASCAL study): A pilot randomized controlled trial.
      ,
      • Hautvast RW
      • DeJongste MJ
      • Staal MJ
      • van Gilst WH
      • Lie KI
      Spinal cord stimulation in chronic intractable angina pectoris: A randomized, controlled efficacy study.
      ,
      • Jessurun GA
      • DeJongste MJ
      • Hautvast RW
      • Tio RA
      • Brouwer J
      • van Lelieveld ST
      • Crijns HJ
      Clinical follow-up after cessation of chronic electrical neuromodulation in patients with severe coronary artery disease: A prospective randomized controlled study on putative involvement of sympathetic activity.
      ,
      • Lanza GA
      • Grimaldi R
      • Greco S
      • Ghio S
      • Sarullo F
      • Zuin G
      • De Luca A
      • Allegri M
      • Di Pede F
      • Castagno D
      • Turco A
      Spinal cord stimulation for the treatment of refractory angina pectoris: A multicenter randomized single-blind study (the SCS-ITA trial).
      ,
      • Lanza GA
      • Sestito A
      • Sgueglia GA
      • Infusino F
      • Papacci F
      • Visocchi M
      • Ierardi C
      • Meglio M
      • Bellocci F
      • Crea F
      Effect of spinal cord stimulation on spontaneous and stress-induced angina and 'ischemia-like' ST-segment depression in patients with cardiac syndrome X.
      ,
      • Mannheimer C
      • Eliasson T
      • Augustinsson LE
      • Blomstrand C
      • Emanuelsson H
      • Larsson S
      • Norrsell H
      • Hjalmarsson A
      Electrical stimulation versus coronary artery bypass surgery in severe angina pectoris: The ESBY study.
      ,
      • McNab D
      • Khan SN
      • Sharples LD
      • Ryan JY
      • Freeman C
      • Caine N
      • Tait S
      • Hardy I
      • Schofield PM
      An open label, single-centre, randomized trial of spinal cord stimulation vs. percutaneous myocardial laser revascularization in patients with refractory angina pectoris: The SPiRiT trial.
      ,
      • Zipes DP
      • Svorkdal N
      • Berman D
      • Boortz-Marx R
      • Henry T
      • Lerman A
      • Ross E
      • Turner M
      • Irwin C
      Spinal cord stimulation therapy for patients with refractory angina who are not candidates for revascularization.
      In addition, seven extension studies were identified.
      • Kapural L
      • Yu C
      • Doust MW
      • Gliner BE
      • Vallejo R
      • Sitzman BT
      • Amirdelfan K
      • Morgan DM
      • Yearwood TL
      • Bundschu R
      • Yang T
      Comparison of 10-kHz high-frequency and traditional low-frequency spinal cord stimulation for the treatment of chronic back and leg pain: 24-month results from a multicenter, randomized, controlled pivotal trial.
      ,
      • Kemler MA
      • De Vet HC
      • Barendse GA
      • Van Den Wildenberg FA
      • Van Kleef M
      The effect of spinal cord stimulation in patients with chronic reflex sympathetic dystrophy: Two years' follow-up of the randomized controlled trial.
      • Kemler MA
      • De Vet HC
      • Barendse GA
      • Van Den Wildenberg FA
      • Van Kleef M
      Spinal cord stimulation for chronic reflex sympathetic dystrophy - Five-year follow-up.
      • Kemler MA
      • Reulen JP
      • Barendse GA
      • van Kleef M
      • de Vet HC
      • van den Wildenberg FA
      Impact of spinal cord stimulation on sensory characteristics in complex regional pain syndrome type IA randomized trial.
      ,
      • Kumar K
      • Taylor RS
      • Jacques L
      • Eldabe S
      • Meglio M
      • Molet J
      • Thomson S
      • O'Callaghan J
      • Eisenberg E
      • Milbouw G
      • Buchser E
      The effects of spinal cord stimulation in neuropathic pain are sustained: A 24-month follow-up of the prospective randomized controlled multicenter trial of the effectiveness of spinal cord stimulation.
      ,
      • van Beek M
      • Slangen R
      • Schaper NC
      • Faber CG
      • Joosten EA
      • Dirksen CD
      • van Dongen RT
      • Kessels AG
      • van Kleef M
      Sustained treatment effect of spinal cord stimulation in painful diabetic peripheral neuropathy: 24-month follow-up of a prospective two-center randomized controlled trial.
      ,
      • Van Eijs F
      • Smits H
      • Geurts JW
      • Kessels AG
      • Kemler MA
      • Van Kleef M
      • Joosten EA
      • Faber CG
      Brush-evoked allodynia predicts outcome of spinal cord stimulation in complex regional pain syndrome type 1.
      Figure 1
      Figure 1Preferred reporting items for systematic reviews and meta-analyses flow diagram. Flow diagram which reflects the search and screening of articles to be included in the review.
      No RCTs were found in acute pain or in cancer pain. The timeline of evidence reflects the evolution of SCS, from early studies comparing conventional (paresthesia-based) SCS to usual care, through studies where both groups received conventional SCS, but that compared different stimulation parameters, to the newest studies that compared burst- or HF-SCS with conventional SCS or other burst/HF settings.

      Coder Discrepancies

      In total, 4,386 items were coded in our main analysis and 1,548 in our angina analysis. In the main analysis 591(14%) reviewer discrepancies occurred. A median of 10 reviewer discrepancies per study were noted in the extraction of angina studies for a rate of 9%. Disagreements were resolved by discussion between extracting reviewers/authors with consultation from a third reviewer/author if disagreement persisted. Disagreements most frequently occurred in assessments of the role of the sponsor, the type of analysis performed, and reporting of clinical significance.

      Non-angina Studies

      Basic Characteristics

      Study publication dates ranged from 1994 to 2018. Of the 34 non-angina studies, 38% were registered on a clinical trial website (Table 1). A minority of studies (32%) included US sites. A funding source was specified in 82% of studies, the majority being industry-sponsored. Most studies did not explicitly specify the role of the sponsor in the research.
      Table 1Summary Characteristics of Included RCTs
      CharacteristicResult
      Year published
       Median (range)2014 (1994–2018)
      Registered (%)38.2
      Included U.S. Sites (%)32.4
      Single center (%)50
      Multicenter (%)50
      Funding source
      Adds to more than 100% as some studies had multiple funding sources.
      (%)
       Reported funding source82.4
        Industry64.7
        Government17.7
        Institutional8.8
        Professional society2.9
      Role of sponsor specified (%)
       Study design11.8
       Data collection8.9
       Data analysis14.7
       Manuscript preparation8.9
       Supplied devices11.8
       Role was NOT stated58.8
      Except where designated otherwise, denominator is 34 studies.
      low asterisk Adds to more than 100% as some studies had multiple funding sources.

      Population

      For the 34 trials, the median number of randomized participants was 38 (range 10–171; Table 2). Average age had less variability than the number of participants; mean average age in years was 54.7 (8.6). Sex was reported in all studies. The range of percentages of female participants across studies was 6 to 92%; the mean proportion of female participants was 50%.
      Table 2Summary of Participant Enrollment & Demographics
      CharacteristicResult
      Number of participants randomized after screen
       Median (range)38 (10–171)
      Number of participants completing (primary outcome)
       Median (range)33 (9–171)
       Unclear (%)17.6
      Age (median years; range)53.5 (38.3–73)
      Gender (% female)
       Mean % (SD)50.1 (18.8)
      INCLUSION CRITERIA
      One or more of the following required for inclusion (%)
       Minimum pain intensity score41.2
       Minimum pain duration55.9
       Minimum disability score5.9
       Failed other treatments47.1
       Willing to stop pain meds/keep stable dose23.5
      Pain Parameters for Study Entry
      For minimum pain intensity, mean score on 0–10 scale (SD)5.2 (0.42)
      For minimum pain duration, median (range) months6 (0.5-–24)
      Pain location identified for inclusion (%)91.2
      Pain location
      Adds to more than 100% because some studies specified more than one location.
      (%)
       Leg pain47.1
       Failed back surgery29.4
       Back pain23.5
       CRPS-117.7
       Peripheral vascular disease14.7
       Other unspecified neuropathic pain14.7
       Diabetic neuropathy5.9
       History of nerve-related injury5.9
       Irritable bowel syndrome2.9
       Heart failure0
       Other14.7
      Except where designated otherwise, denominator is 34 studies.
      low asterisk Adds to more than 100% because some studies specified more than one location.
      The number of participants reported in each study as having completed the primary outcome assessment, and the number of participants analyzed, was similar to the number enrolled across the 34 trials; however, there was more variability amongst the completers in each study compared to the groups of those enrolled. In 6 studies, the number of participants completing the study was unclear.
      • de Ridder Plazier M
      • Kamerling N
      • Menovsky T
      • Vanneste S
      Burst spinal cord stimulation for limb and back pain.
      ,
      • Jivegard LEH
      • Augustinsson LE
      • Holm J
      • Risberg B
      • Ortenwall P
      Effects of spinal cord stimulation (SCS) in patients with inoperable severe lower limb ischaemia: A prospective randomised controlled study.
      ,
      • North RB
      • Kidd DH
      • Petrucci L
      • Dorsi MJ
      Spinal cord stimulation electrode design: A prospective, randomized, controlled trial comparing percutaneous with laminectomy electrodes: Part II–clinical outcomes.
      ,
      • Spincemaille GH
      • Klomp HM
      • Steyerberg EW
      • Habbema JD
      Pain and quality of life in patients with critical limb ischaemia: Results of a randomized controlled multicentre study on the effect of spinal cord stimulation.
      • Spincemaille GH
      • Klomp HM
      • Steyerberg EW
      • Habbema JD
      Spinal cord stimulation in patients with critical limb ischemia: A preliminary evaluation of a multicentre trial.
      • Spincemaille GH
      • Klomp HM
      • Steyerberg EW
      • van Urk H
      • Habbema JD
      Technical data and complications of spinal cord stimulation: Data from a randomized trial on critical limb ischemia.
      ,
      • Van Havenbergh T
      • Vancamp T
      • Van Looy P
      • Vanneste S
      • De Ridder D
      Spinal cord stimulation for the treatment of chronic back pain patients: 500-Hz vs. 1000-Hz burst stimulation.
      Some studies specified that participants meet a minimum pain intensity, pain duration, or both, in order to be included. Of the 41% of studies that required participants meet a minimum pain intensity score, the mean score was 5.2 (on a 0–10 visual analog or numeric scale). Of the 56% of studies that required a minimum duration of pain, the median number of months was 6, and the requirements ranged from 2 weeks to 24 months to qualify. Only two studies required that participants meet a pre-determined disability score for entry into their trials.
      • Kapural L
      • Yu C
      • Doust MW
      • Gliner BE
      • Vallejo R
      • Sitzman BT
      • Amirdelfan K
      • Morgan DM
      • Brown LL
      • Yearwood TL
      • Bundschu R
      Novel 10-kHz high-frequency therapy (HF10 Therapy) is superior to traditional low-frequency spinal cord stimulation for the treatment of chronic back and leg pain.
      ,
      • Thomson SJ
      • Tavakkolizadeh M
      • Love-Jones S
      • Patel NK
      • Gu JW
      • Bains A
      • Doan Q
      • Moffitt M
      Effects of rate on analgesia in Kilohertz frequency spinal cord stimulation: Results of the PROCO randomized controlled trial.
      The proportion of studies requiring failure of other treatments was 47%, and 24% of studies required participant willingness to keep dosing stable or stop pain medication entirely.
      Thirty-one of the 34 studies clearly identified one or more types or locations of pain as an inclusion criterion. Leg pain, failed back surgery syndrome, and back pain were identified in 16, 10, and 8 of the 34 studies, respectively.

      Interventions and Comparators

      With advances in SCS technology, devices now allow for variations of settings such as frequency, placement, amplitude, and pulse width, to an extent that earlier versions did not. The most commonly used intervention was what has come to be called ``conventional” or ``paresthesia-based” SCS (ie, permanently implanted percutaneous cylindrical leads or paddle electrodes placed in the midline epidural space); it was employed in 65% of the studies (Table 3). Other interventions included stimulation at high frequencies, and at burst intervals. The range of comparator or control interventions is also listed in Table 3.
      Table 3Summaries of the Interventions and Comparators
      CharacteristicResult
      Intervention
      Type of SCS
      Adds to more than 100% because some studies permitted more than 1.
      (%)
       Conventional paresthesia-based64.7
       High-frequency23.5
       High-frequency burst17.7
       Other settings (eg, “shuffle” between conventional and high-frequency)20.6
      Control (comparator)
      Adds to more than 100% because some studies permitted more than 1.
      (%)
       Conventional paresthesia-based SCS38.2
       High-frequency or high-frequency burst8.8
       Dorsal root ganglion stimulation5.9
       Placebo
      SCS unit switched off or programmed to produce subthreshold stimulation.
      29.4
       Usual care (clinician decides)20.6
       Physical therapy5.9
       Surgery2.9
       Usual care (defined by protocol)2.9
      Co-administration of other noninvasive treatments allowed for all arms of the study (%)
       Yes64.7
       Not specified32.4
      Methods allowed for SCS adjustments (%)
       Yes58.8
        Intervention only23.5
        Control only2.9
        All arms allowed adjustments32.4
      SCS adjustments
      Adds to more than 100% because some studies permitted more than 1.
      (%)
       >1 adjustment allowed35.2
       Amplitude (voltage)52.9
       Electrode location/placement14.7
       Frequency/kHz14.7
       Pulse width (microsecond)14.7
       On/off8.8
       Not specified5.9
       Stimulation waveform0
      Except where designated otherwise, denominator is 34 studies.
      low asterisk Adds to more than 100% because some studies permitted more than 1.
      SCS unit switched off or programmed to produce subthreshold stimulation.
      Researchers reported permitting supplementation of the treatment intervention by co-administration of noninvasive therapy (such as physical therapy, medication) in 65% of studies. Similarly, study methods in 59% of publications noted that adjustments (eg, in programming of SCS) were allowed to the assigned intervention. The greatest proportion of studies (32%) allowed adjustments in both intervention and control arms; 24% were permitted in the intervention alone; only 1 study reported allowing adjustments to only the control group.
      • Kapural L
      • Yu C
      • Doust MW
      • Gliner BE
      • Vallejo R
      • Sitzman BT
      • Amirdelfan K
      • Morgan DM
      • Brown LL
      • Yearwood TL
      • Bundschu R
      Novel 10-kHz high-frequency therapy (HF10 Therapy) is superior to traditional low-frequency spinal cord stimulation for the treatment of chronic back and leg pain.
      In 2 of the 14 parallel group studies, participants who failed to achieve adequate pain management with an assigned intervention were allowed to switch to the alternative treatment.
      • Kumar K
      • Taylor RS
      • Jacques L
      • Eldabe S
      • Meglio M
      • Molet J
      • Thomson S
      • O'Callaghan J
      • Eisenberg E
      • Milbouw G
      • Buchser E
      Spinal cord stimulation versus conventional medical management for neuropathic pain: A multicentre randomised controlled trial in patients with failed back surgery syndrome.
      ,
      • North RB
      • Kidd DH
      • Farrokhi F
      • Piantadosi SA
      Spinal cord stimulation versus repeated lumbosacral spine surgery for chronic pain: A randomized, controlled trial.

      Outcomes

      The vast majority of studies (94%) clearly identified a single primary outcome measure or multiple primary outcome measures (Table 4). Researchers reported 1 primary outcome in 59% of the studies; accordingly, 35% of the studies reported multiple primary outcomes. The breakdown of the different outcomes can be seen in Table 4. Pain intensity was the most frequent outcome in both those studies that assessed 1 and those that assessed more than 1 primary outcome.
      Table 4Summary of Study Outcomes
      EFFICACYResult
      Primary outcome(s) specified (%)94.1
      Primary outcome pain-related (%)52.9
      Single primary outcome reported (%)58.8
       Pain intensity32.4
       Quantitative sensory testing5.9
       Patient preference5.9
       Pain relief2.9
       PGIC2.9
       Other8.8
      Multiple primary outcomes reported
      Some studies had more than one type of outcome.
      (%)
      35.3
       Pain intensity26.5
       Pain relief5.9
       Patient satisfaction5.9
       No neurological deficit5.9
       Multidimensional pain questionnaire2.9
       Health related quality of life2.9
       PGIC2.9
       Responder analysis2.9
       Other11.8
      Secondary outcomes reported
      Some studies had more than one type of outcome.
      (%)
       Pain intensity52.9
       Function/disability44.1
       Health-related Quality of Life41.2
       Multidimensional pain questionnaire38.2
       Patient satisfaction38.2
       Patient preference29.4
       Depression26.5
       Reduction in opioid use26.5
       PGIC23.5
       Sleep20.6
       Pain relief17.7
       Mood14.7
       Pain catastrophizing14.7
       No neurological deficit11.8
       Quantitative sensory testing5.9
      SAFETYResult
      Adverse event prespecified as an outcome (%)47.1
      Adverse event was primary outcome (%)8.8
      Serious adverse events reported (%)47.1
      Adverse event caused adjustment in regimen
       Unclear (%)64.7
      Reported Adverse Events (%)YES

      Reported but

      NOT Prespecified (%)
      YES

      Prespecified in Methods (%)
      NO (%)
      Neurologic injury11.85.982.4
      Localized pain20.6079.4
      Lead migration/inadequate coverage41.28.850.0
      Pulse generator discomfort32.4067.7
      Infection38.22.958.8
      Fractured electrode14.7085.3
      Hardware malfunction20.65.973.5
      Other44.1055.9
      PGIC, patient global impression of change.
      Except where designated otherwise, denominator is 34 studies.
      low asterisk Some studies had more than one type of outcome.
      Secondary outcomes were also collected at high rates in most of the studies, given the effect of pain on other domains. The list of other outcomes includes physical functioning, health-related quality of life, emotional distress, and opioid use.
      Safety was represented by the authors’ reporting of AEs or side effects. Adverse events were reported as a primary outcome in three studies (Table 4).
      • Al-Kaisy A
      • Palmisani S
      • Pang D
      • Sanderson K
      • Wesley S
      • Tan Y
      • McCammon S
      • Trescott A
      Prospective, randomized, sham-control, double blind, crossover trial of subthreshold spinal cord stimulation at various Kilohertz frequencies in subjects suffering from failed back surgery syndrome (SCS Frequency Study).
      ,
      • Kapural L
      • Yu C
      • Doust MW
      • Gliner BE
      • Vallejo R
      • Sitzman BT
      • Amirdelfan K
      • Morgan DM
      • Brown LL
      • Yearwood TL
      • Bundschu R
      Novel 10-kHz high-frequency therapy (HF10 Therapy) is superior to traditional low-frequency spinal cord stimulation for the treatment of chronic back and leg pain.
      ,
      • Van Havenbergh T
      • Vancamp T
      • Van Looy P
      • Vanneste S
      • De Ridder D
      Spinal cord stimulation for the treatment of chronic back pain patients: 500-Hz vs. 1000-Hz burst stimulation.
      The list of reported AEs is in Table 4. Many trials did not prespecify assessment of SCS-related AEs. Adverse events of note were incidences of lead migration, inadequate paresthesia coverage, or both; 41% of studies reported these findings. Infection and pulse generator discomfort had the next highest rates of reporting. Most studies either did not report SCS-related AEs, or reported that AEs did not occur.

      Methods of Outcome Reporting

      The efficacy and safety results from the studies were reported in different units. The most frequent method of reporting was the mean score of outcomes at a specified time; 41% of studies reported using these units. The remaining units in which primary outcomes were reported are shown in Table 5.
      Table 5Summary of Outcome Reporting
      Method of Reporting Primary Outcomes (%)Result
       Mean score of outcomes at a specified time41.2
       Pain intensity: patients with a specified level at a specified time (responders)17.7
       Mean change in score of outcomes at a specified time17.7
       Pain intensity: difference at a specified time11.8
       Pain relief: patients with a specified level at a specified time (responders)11.8
       Time-to-event designated by the study5.9
       No stated primary outcome measure5.9
       Other11.8
      Method of Reporting Secondary Outcomes (%)
       Pain relief: patients with a specified level at a specified time (responders)67.7
       Time-to-event designated by the study32.4
       Pain intensity: difference at a specified time14.7
       Pain intensity: patients with a specified level at a specified time (responders)11.8
       Mean score of outcomes at a specified time2.9
       Mean change in score of outcomes at a specified time8.8
       No stated primary outcome measure5.9
       Other58.8
      Clinical significance of outcome reported (%)
       Percent change (eg, 30% or 50%)38.2
       Absolute change (eg, point reduction)23.5
       Not defined38.2
      If assessed at multiple timepoints, interval between primary outcome assessments (%)
       Less than 1 day20.6
       1 day up to 1 week11.8
       More than 7 days up to 1 month8.8
       More than 1 Month38.2
       Variable11.7
       Not specified8.8
      Timing of primary outcome endpoint (weeks)
       Median (range)12 (0–208)
      Except where designated otherwise, denominator is 34 studies.
      In the secondary outcomes, responder rates (proportion of participants who met a cut point for improvement) were reported in 68% of the studies.
      Clinical significance of the reported outcome was not discussed in 38% of the studies. When clinical significance was addressed, it was reported as a percent change in 38% of the studies, leaving a reporting of absolute change as the method in 24% of the studies.
      When the primary outcome measure was assessed at multiple timepoints, the intervals between assessments ranged from less than one day to more than one month, with the latter occurring in 38% of studies. The primary outcome endpoints were generally assessed at 12 weeks (median) but ranged from 0 (less than a day) to 208 weeks.

      Study Characteristics and Analysis

      The majority of studies used a cross-over design (59%); the remaining studies used a parallel-group design (Table 6). Open-label (no blinding) was reported in 68% of studies. Where studies were defined as blinded, they were most frequently reported as double blind. In 3 studies, the participant, the interventionist, and the outcome assessor were all blinded.
      • Al-Kaisy A
      • Palmisani S
      • Pang D
      • Sanderson K
      • Wesley S
      • Tan Y
      • McCammon S
      • Trescott A
      Prospective, randomized, sham-control, double blind, crossover trial of subthreshold spinal cord stimulation at various Kilohertz frequencies in subjects suffering from failed back surgery syndrome (SCS Frequency Study).
      ,
      • Thomson SJ
      • Tavakkolizadeh M
      • Love-Jones S
      • Patel NK
      • Gu JW
      • Bains A
      • Doan Q
      • Moffitt M
      Effects of rate on analgesia in Kilohertz frequency spinal cord stimulation: Results of the PROCO randomized controlled trial.
      ,
      • Tjepkema-Cloostermans MC
      • de Vos
      • Wolters CC
      • Dijkstra-Scholten R
      • Lenders MWPM C
      Effect of burst stimulation evaluated in patients familiar with spinal cord stimulation.
      Of the 11 studies that attempted blinding, 2 were assessed as having a high risk of bias, 6 a low risk, and 3 were assessed as having an unclear risk of bias. None of the studies were assessed as having a high risk of randomization bias, and the majority (74%) was assessed as having a low risk of randomization bias.
      Table 6Summary of Study Design, Methods, and Analytic Techniques
      Study Design and Methods SummaryResult
      Design (%)
       Cross-over58.8
       Parallel41.2
      Blinding (%)
       No blinding (open label)67.6
       Single-blind3.0
       Double-blind20.6
       Triple-blind8.8
        Participants blinded26.5
        Investigator23.5
        Outcome assessors20.6
      Adequacy of blinding (%)
       No blinding67.6
       High risk of bias5.9
       Low risk of bias17.7
       Unclear8.9
      Randomization integrity (%)
       High risk of bias0
       Low risk of bias73.5
       Unclear26.5
      Adverse events collection method (%)
       Actively0
       Passively0
       Both actively and passively8.8
       Unclear91.2
      Studies with SCS trial/screening phase (prerandomization) (%)55.9
      Cross-over studies: washout duration (days)
       Mean (SD)1.8 (3.8)
       Median (Range)0 (0–14)
      Study duration in weeks
       Median (Range)12 (0–298)
       Mean (SD)34.7 (49.8)
      Comparison of randomized groups (%)
       Stated/shown to be equal in all arms38.2
       Stated as not equal with explanation or accommodation2.9
       Stated as not equal with no explanation or accommodation0
       Nothing stated about comparability of groups or unclear58.8
      Summary of Analytic Procedures
      Type of analysis (%)
       Superiority52.9
       Noninferiority11.8
       Equivalence2.9
       Not specified32.4
      Required sample size reported (%)61.8
       Required elements for sample size estimation reported (%)
       1. Significance level73.5
       2. Power calculation58.8
       3. Treatment effect size to be detected55.9
      Reported statistical adjustment for multiple outcomes (%)11.8
      4 out of 34 studies reported adjustment (4 of 12 with multiple outcomes).
      Type of analysis
      Adds to more than 100% as some studies performed more than one analysis.
      (%)
       Intention-to-treat35.3
       Modified intention-to-treat2.9
       Per-protocol (completer analysis)64.7
       Not reported2.9
      For Intention-to-Treat and Modified Intention-to-Treat Analyses, Plan for Accommodation of Missing Data (N = 13) (%)
       Last observation carried forward23.1
       Other15.4
       Not reported61.5
      Except where designated otherwise, denominator is 34 studies.
      low asterisk 4 out of 34 studies reported adjustment (4 of 12 with multiple outcomes).
      Adds to more than 100% as some studies performed more than one analysis.
      An initial trial of SCS, after which patients were enrolled and randomized, occurred in 56% of the studies.
      The washout periods reported in the cross-over studies were very short or nonexistent. The range of days for washout was 0 to 14 days, with a median of 0 days.
      For their primary analysis, more than half of the studies employed a superiority analysis. In 32% of the studies, the analytic approach was unclear or not specified. A noninferiority analysis was reported as being used for the primary analysis in 12% of the studies, and only 1 study was reported to be an equivalence study.
      • Thomson SJ
      • Tavakkolizadeh M
      • Love-Jones S
      • Patel NK
      • Gu JW
      • Bains A
      • Doan Q
      • Moffitt M
      Effects of rate on analgesia in Kilohertz frequency spinal cord stimulation: Results of the PROCO randomized controlled trial.
      Researchers reported a sample size calculation in 62% of the studies. Of the elements required to calculate a sample size (ie, power, difference to detect, and alpha), 59% reported the percent power the study aimed to achieve, 56% reported the degree of change in outcome that the trial was designed to detect, and 74% reported the assumed alpha (eg, .05). Sixteen studies (47%) reported a sample size calculation along with all of the required elements for its calculation.
      While approximately one-third of the sample reported assessing multiple primary outcomes, only 12% of all studies (four of the 12 with multiple outcomes) reported use of a statistical adjustment for multiple outcomes. The majority of studies (59%) reported that they employed a per-protocol (completer) analysis only. Eleven studies (32%) employed an intention-to-treat (ITT) or modified ITT analysis, and 2 studies reported both a per-protocol and an ITT analysis. Of those that employed an ITT or modified ITT analysis, 5 of 13 reported using a method to accommodate missing data, most commonly last observation carried forward (LOCF).

      Angina Studies

      Twelve studies (26% of 46) assessed the use of SCS in patients with angina.
      • de Jongste MJ
      • Hautvast RW
      • Hillege HL
      • Lie KI
      Efficacy of spinal cord stimulation as adjuvant therapy for intractable angina pectoris: A prospective, randomized clinical study.
      ,
      • de Jongste MJL
      • Staal MJ
      Preliminary results of a randomized study on the clinical efficacy of spinal cord stimulation for refractory severe angina pectoris.
      ,
      • Di Pede F
      • Zuin G
      • Giada F
      • Pinato G
      • Turiano G
      • Bevilacqua M
      • Cazzin R
      • Raviele A
      Long-term effects of spinal cord stimulation on myocardial ischemia and heart rate variability: Results of a 48-hour ambulatory electrocardiographic monitoring.
      ,
      • Eddicks S
      • Maier-Hauff K
      • Schenk M
      • Müller A
      • Baumann G
      • Theres H
      Thoracic spinal cord stimulation improves functional status and relieves symptoms in patients with refractory angina pectoris: The first placebo-controlled randomised study.
      ,
      • Eldabe S
      • Thomson S
      • Duarte R
      • Brookes M
      • deBelder M
      • Raphael J
      • Davies E
      • Taylor R
      The effectiveness and cost-effectiveness of spinal cord stimulation for refractory angina (RASCAL study): A pilot randomized controlled trial.
      ,
      • Hautvast RW
      • DeJongste MJ
      • Staal MJ
      • van Gilst WH
      • Lie KI
      Spinal cord stimulation in chronic intractable angina pectoris: A randomized, controlled efficacy study.
      ,
      • Jessurun GA
      • DeJongste MJ
      • Hautvast RW
      • Tio RA
      • Brouwer J
      • van Lelieveld ST
      • Crijns HJ
      Clinical follow-up after cessation of chronic electrical neuromodulation in patients with severe coronary artery disease: A prospective randomized controlled study on putative involvement of sympathetic activity.
      ,
      • Lanza GA
      • Grimaldi R
      • Greco S
      • Ghio S
      • Sarullo F
      • Zuin G
      • De Luca A
      • Allegri M
      • Di Pede F
      • Castagno D
      • Turco A
      Spinal cord stimulation for the treatment of refractory angina pectoris: A multicenter randomized single-blind study (the SCS-ITA trial).
      ,
      • Lanza GA
      • Sestito A
      • Sgueglia GA
      • Infusino F
      • Papacci F
      • Visocchi M
      • Ierardi C
      • Meglio M
      • Bellocci F
      • Crea F
      Effect of spinal cord stimulation on spontaneous and stress-induced angina and 'ischemia-like' ST-segment depression in patients with cardiac syndrome X.
      ,
      • Mannheimer C
      • Eliasson T
      • Augustinsson LE
      • Blomstrand C
      • Emanuelsson H
      • Larsson S
      • Norrsell H
      • Hjalmarsson A
      Electrical stimulation versus coronary artery bypass surgery in severe angina pectoris: The ESBY study.
      ,
      • McNab D
      • Khan SN
      • Sharples LD
      • Ryan JY
      • Freeman C
      • Caine N
      • Tait S
      • Hardy I
      • Schofield PM
      An open label, single-centre, randomized trial of spinal cord stimulation vs. percutaneous myocardial laser revascularization in patients with refractory angina pectoris: The SPiRiT trial.
      ,
      • Zipes DP
      • Svorkdal N
      • Berman D
      • Boortz-Marx R
      • Henry T
      • Lerman A
      • Ross E
      • Turner M
      • Irwin C
      Spinal cord stimulation therapy for patients with refractory angina who are not candidates for revascularization.
      The majority of these studies were single site (75%), and only 1 study included US sites.
      • Zipes DP
      • Svorkdal N
      • Berman D
      • Boortz-Marx R
      • Henry T
      • Lerman A
      • Ross E
      • Turner M
      • Irwin C
      Spinal cord stimulation therapy for patients with refractory angina who are not candidates for revascularization.
      Three studies indicated no funding source, but others were industry or government sponsored research, or both. Three studies were publicly registered.
      • Eldabe S
      • Thomson S
      • Duarte R
      • Brookes M
      • deBelder M
      • Raphael J
      • Davies E
      • Taylor R
      The effectiveness and cost-effectiveness of spinal cord stimulation for refractory angina (RASCAL study): A pilot randomized controlled trial.
      ,
      • Lanza GA
      • Grimaldi R
      • Greco S
      • Ghio S
      • Sarullo F
      • Zuin G
      • De Luca A
      • Allegri M
      • Di Pede F
      • Castagno D
      • Turco A
      Spinal cord stimulation for the treatment of refractory angina pectoris: A multicenter randomized single-blind study (the SCS-ITA trial).
      ,
      • Zipes DP
      • Svorkdal N
      • Berman D
      • Boortz-Marx R
      • Henry T
      • Lerman A
      • Ross E
      • Turner M
      • Irwin C
      Spinal cord stimulation therapy for patients with refractory angina who are not candidates for revascularization.
      All studies were in severe angina or angina refractory to other treatments, or both. Three studies (25%) used cross-over designs with no washout phase;
      • Di Pede F
      • Zuin G
      • Giada F
      • Pinato G
      • Turiano G
      • Bevilacqua M
      • Cazzin R
      • Raviele A
      Long-term effects of spinal cord stimulation on myocardial ischemia and heart rate variability: Results of a 48-hour ambulatory electrocardiographic monitoring.
      ,
      • Eddicks S
      • Maier-Hauff K
      • Schenk M
      • Müller A
      • Baumann G
      • Theres H
      Thoracic spinal cord stimulation improves functional status and relieves symptoms in patients with refractory angina pectoris: The first placebo-controlled randomised study.
      ,
      • Lanza GA
      • Sestito A
      • Sgueglia GA
      • Infusino F
      • Papacci F
      • Visocchi M
      • Ierardi C
      • Meglio M
      • Bellocci F
      • Crea F
      Effect of spinal cord stimulation on spontaneous and stress-induced angina and 'ischemia-like' ST-segment depression in patients with cardiac syndrome X.
      all other studies were parallel group designs. Half of studies had some degree of blinding (mostly of outcome assessors) with insufficient detail to assess risk of bias. Bias assessment of randomization procedures was rated as unclear due to insufficient detail in 10 studies; 2 studies were rated as low risk for bias due to randomization.
      • Eldabe S
      • Thomson S
      • Duarte R
      • Brookes M
      • deBelder M
      • Raphael J
      • Davies E
      • Taylor R
      The effectiveness and cost-effectiveness of spinal cord stimulation for refractory angina (RASCAL study): A pilot randomized controlled trial.
      ,
      • McNab D
      • Khan SN
      • Sharples LD
      • Ryan JY
      • Freeman C
      • Caine N
      • Tait S
      • Hardy I
      • Schofield PM
      An open label, single-centre, randomized trial of spinal cord stimulation vs. percutaneous myocardial laser revascularization in patients with refractory angina pectoris: The SPiRiT trial.
      Conventional (paresthesia-based) use of SCS was assessed in all studies; most studies specified that SCS was utilized in addition to other treatments such as nitroglycerin. Half of the trials allowed for adjustments to interventional SCS settings. Two studies compared SCS against surgery (eg, revascularization), 1 against usual care, and 6 against implant without activation or with subthreshold stimulation. Duration of the intervention varied from less than 1 week
      • Di Pede F
      • Zuin G
      • Giada F
      • Pinato G
      • Turiano G
      • Bevilacqua M
      • Cazzin R
      • Raviele A
      Long-term effects of spinal cord stimulation on myocardial ischemia and heart rate variability: Results of a 48-hour ambulatory electrocardiographic monitoring.
      to up to 52 weeks;
      • McNab D
      • Khan SN
      • Sharples LD
      • Ryan JY
      • Freeman C
      • Caine N
      • Tait S
      • Hardy I
      • Schofield PM
      An open label, single-centre, randomized trial of spinal cord stimulation vs. percutaneous myocardial laser revascularization in patients with refractory angina pectoris: The SPiRiT trial.
      ,
      • Zipes DP
      • Svorkdal N
      • Berman D
      • Boortz-Marx R
      • Henry T
      • Lerman A
      • Ross E
      • Turner M
      • Irwin C
      Spinal cord stimulation therapy for patients with refractory angina who are not candidates for revascularization.
      most studies evaluated the intervention for durations of 4 to 8 weeks. Outcomes varied amongst studies but commonly included number of angina attacks and nitroglycerin consumption. Seven studies (58%) assessed health-related quality of life and 7 (58.3%) also assessed physical function/disability.
      Of the 12 studies, 3 prespecified AEs as an outcome,
      • Eldabe S
      • Thomson S
      • Duarte R
      • Brookes M
      • deBelder M
      • Raphael J
      • Davies E
      • Taylor R
      The effectiveness and cost-effectiveness of spinal cord stimulation for refractory angina (RASCAL study): A pilot randomized controlled trial.
      ,
      • McNab D
      • Khan SN
      • Sharples LD
      • Ryan JY
      • Freeman C
      • Caine N
      • Tait S
      • Hardy I
      • Schofield PM
      An open label, single-centre, randomized trial of spinal cord stimulation vs. percutaneous myocardial laser revascularization in patients with refractory angina pectoris: The SPiRiT trial.
      ,
      • Zipes DP
      • Svorkdal N
      • Berman D
      • Boortz-Marx R
      • Henry T
      • Lerman A
      • Ross E
      • Turner M
      • Irwin C
      Spinal cord stimulation therapy for patients with refractory angina who are not candidates for revascularization.
      and 6 (50%) reported serious AEs.
      • de Jongste MJ
      • Hautvast RW
      • Hillege HL
      • Lie KI
      Efficacy of spinal cord stimulation as adjuvant therapy for intractable angina pectoris: A prospective, randomized clinical study.
      ,
      • de Jongste MJL
      • Staal MJ
      Preliminary results of a randomized study on the clinical efficacy of spinal cord stimulation for refractory severe angina pectoris.
      ,
      • Eldabe S
      • Thomson S
      • Duarte R
      • Brookes M
      • deBelder M
      • Raphael J
      • Davies E
      • Taylor R
      The effectiveness and cost-effectiveness of spinal cord stimulation for refractory angina (RASCAL study): A pilot randomized controlled trial.
      ,
      • Lanza GA
      • Grimaldi R
      • Greco S
      • Ghio S
      • Sarullo F
      • Zuin G
      • De Luca A
      • Allegri M
      • Di Pede F
      • Castagno D
      • Turco A
      Spinal cord stimulation for the treatment of refractory angina pectoris: A multicenter randomized single-blind study (the SCS-ITA trial).
      ,
      • McNab D
      • Khan SN
      • Sharples LD
      • Ryan JY
      • Freeman C
      • Caine N
      • Tait S
      • Hardy I
      • Schofield PM
      An open label, single-centre, randomized trial of spinal cord stimulation vs. percutaneous myocardial laser revascularization in patients with refractory angina pectoris: The SPiRiT trial.
      ,
      • Zipes DP
      • Svorkdal N
      • Berman D
      • Boortz-Marx R
      • Henry T
      • Lerman A
      • Ross E
      • Turner M
      • Irwin C
      Spinal cord stimulation therapy for patients with refractory angina who are not candidates for revascularization.
      Specific AEs reported in studies were lead migration (42%), localized pain (25%), infection (25%), pulse generator discomfort (17%), hospitalization (8%), death (8%), battery life (8%), and other cardiac complications.
      Only 3 of the angina studies reported superiority analyses, all of which described sample size calculation and the elements required to calculate sample size.
      • Lanza GA
      • Grimaldi R
      • Greco S
      • Ghio S
      • Sarullo F
      • Zuin G
      • De Luca A
      • Allegri M
      • Di Pede F
      • Castagno D
      • Turco A
      Spinal cord stimulation for the treatment of refractory angina pectoris: A multicenter randomized single-blind study (the SCS-ITA trial).
      ,
      • McNab D
      • Khan SN
      • Sharples LD
      • Ryan JY
      • Freeman C
      • Caine N
      • Tait S
      • Hardy I
      • Schofield PM
      An open label, single-centre, randomized trial of spinal cord stimulation vs. percutaneous myocardial laser revascularization in patients with refractory angina pectoris: The SPiRiT trial.
      ,
      • Zipes DP
      • Svorkdal N
      • Berman D
      • Boortz-Marx R
      • Henry T
      • Lerman A
      • Ross E
      • Turner M
      • Irwin C
      Spinal cord stimulation therapy for patients with refractory angina who are not candidates for revascularization.
      Per-protocol analysis was conducted in 50% of studies, 17% utilized ITT, and 25% utilized both methods of analysis. Two of the studies that reported both types of analyses accommodated missing data with LOCF.
      • McNab D
      • Khan SN
      • Sharples LD
      • Ryan JY
      • Freeman C
      • Caine N
      • Tait S
      • Hardy I
      • Schofield PM
      An open label, single-centre, randomized trial of spinal cord stimulation vs. percutaneous myocardial laser revascularization in patients with refractory angina pectoris: The SPiRiT trial.
      ,
      • Zipes DP
      • Svorkdal N
      • Berman D
      • Boortz-Marx R
      • Henry T
      • Lerman A
      • Ross E
      • Turner M
      • Irwin C
      Spinal cord stimulation therapy for patients with refractory angina who are not candidates for revascularization.
      No other studies described methods for missing data. The median number of patients randomized was 25 (range 10–104), and the mean age was 64.4 (5.0).

      Extension Studies

      Seven extension studies were identified from the search, four of which were extensions of a study included in our main analysis.
      • Kemler MA
      • Barendse GA
      • Van Kleef M
      • de Vet HC
      • Rijks CP
      • Furnee CA
      • Van Den Wildenberg FA
      Spinal cord stimulation in patients with chronic reflex sympathetic dystrophy.
      Most extension studies reported follow-up data from 24 months.
      • Kapural L
      • Yu C
      • Doust MW
      • Gliner BE
      • Vallejo R
      • Sitzman BT
      • Amirdelfan K
      • Morgan DM
      • Yearwood TL
      • Bundschu R
      • Yang T
      Comparison of 10-kHz high-frequency and traditional low-frequency spinal cord stimulation for the treatment of chronic back and leg pain: 24-month results from a multicenter, randomized, controlled pivotal trial.
      ,
      • Kemler MA
      • De Vet HC
      • Barendse GA
      • Van Den Wildenberg FA
      • Van Kleef M
      The effect of spinal cord stimulation in patients with chronic reflex sympathetic dystrophy: Two years' follow-up of the randomized controlled trial.
      ,
      • Kumar K
      • Taylor RS
      • Jacques L
      • Eldabe S
      • Meglio M
      • Molet J
      • Thomson S
      • O'Callaghan J
      • Eisenberg E
      • Milbouw G
      • Buchser E
      The effects of spinal cord stimulation in neuropathic pain are sustained: A 24-month follow-up of the prospective randomized controlled multicenter trial of the effectiveness of spinal cord stimulation.
      ,
      • van Beek M
      • Slangen R
      • Schaper NC
      • Faber CG
      • Joosten EA
      • Dirksen CD
      • van Dongen RT
      • Kessels AG
      • van Kleef M
      Sustained treatment effect of spinal cord stimulation in painful diabetic peripheral neuropathy: 24-month follow-up of a prospective two-center randomized controlled trial.
      Additionally, most extension studies reported follow-up data for the same efficacy and safety outcomes originally described. One study reported efficacy and safety outcomes only from the SCS arm of the original study and identified additional AEs (ie, new pulse generator implantation, lead replacement) with 24-month data.
      • van Beek M
      • Slangen R
      • Schaper NC
      • Faber CG
      • Joosten EA
      • Dirksen CD
      • van Dongen RT
      • Kessels AG
      • van Kleef M
      Sustained treatment effect of spinal cord stimulation in painful diabetic peripheral neuropathy: 24-month follow-up of a prospective two-center randomized controlled trial.
      Of the 4 extension studies from the research conducted by Kemler et al, 1 reported 12-month data for outcomes related to pressure sensibility, warmth and cold sensibility, and mechanical hyperalgesia.
      • Kemler MA
      • Reulen JP
      • Barendse GA
      • van Kleef M
      • de Vet HC
      • van den Wildenberg FA
      Impact of spinal cord stimulation on sensory characteristics in complex regional pain syndrome type IA randomized trial.
      Efficacy data were also presented for 2-year follow-up with added reporting of AEs, such as revision of pulse generator pocket, lead replacement, reimplantation, disturbed urination, cramping, and explantation.
      • Kemler MA
      • De Vet HC
      • Barendse GA
      • Van Den Wildenberg FA
      • Van Kleef M
      The effect of spinal cord stimulation in patients with chronic reflex sympathetic dystrophy: Two years' follow-up of the randomized controlled trial.
      Another extension presented 5-year data for pain intensity.
      • Kemler MA
      • De Vet HC
      • Barendse GA
      • Van Den Wildenberg FA
      • Van Kleef M
      Spinal cord stimulation for chronic reflex sympathetic dystrophy - Five-year follow-up.
      Lastly, an analysis of prognostic factors for successful SCS-related outcomes in an extension study was presented.
      • Van Eijs F
      • Smits H
      • Geurts JW
      • Kessels AG
      • Kemler MA
      • Van Kleef M
      • Joosten EA
      • Faber CG
      Brush-evoked allodynia predicts outcome of spinal cord stimulation in complex regional pain syndrome type 1.

      Discussion

      Where design and methodology from angina studies were similar to non-angina studies, data are summarized for all; where differences exist, findings from angina studies are separated.

      Study Features and Designs

      In studies that reported funding, the majority were supported financially by device manufacturers. However, the sponsor's role in the trial was less clear. Given the expense, it is perhaps not surprising that devices were supplied without cost by the manufacturer in many studies. Industry-sponsored studies have been associated with a higher likelihood of positive outcomes.

      North RB, Shipley J: Chapter 4 - Clinical study designs for neuromodulation, in Krames Elliot S, Peckham P Hunter, Rezai Ali R, (eds.): Neuromodulation, 2nd ed. London, United Kingdom, Academic Press, 2018, pp 41-51.

      Most angina studies were conducted at a single site. Multicenter studies are thought to have greater external validity, but they may have less internal validity, and more measurement error.

      North RB, Shipley J: Chapter 4 - Clinical study designs for neuromodulation, in Krames Elliot S, Peckham P Hunter, Rezai Ali R, (eds.): Neuromodulation, 2nd ed. London, United Kingdom, Academic Press, 2018, pp 41-51.

      Sixteen of the 46 studies were registered on a clinical trial website, typically the National Institute of Health site. Given that this was not established for sponsors and principal investigators to submit protocols or results of clinical studies until 2008, the majority of registered studies are recent. However, many studies published after 2010 did not post a protocol or results, and this did not appear to be associated with whether or not they were conducted in US sites.
      We analyzed pain location based on inclusion criteria. Such criteria were often broad and resulted in potentially heterogeneous patient populations that may have different prognostic factors. The requirement for a minimum duration or severity of disability was uncommon. A requirement for a minimum pain intensity or duration was considerably more common, with cut-offs of at least moderate severity and minimum durations ranging from 2 weeks to 2 years. Chronic pain is routinely defined as pain of 3 months or greater duration; therefore, inclusion of subjects with shorter durations of pain means that many did not meet this criterion. Studies with minimum pain or disability criteria, or that only included patients resistant to other treatments, may more closely reflect candidates for SCS, in that SCS can be seen as a treatment of last resort.
      • Ganty P
      • Sharma M
      Failed back surgery syndrome: A suggested algorithm of care.
      Most studies did not stipulate that patients had to discontinue their current analgesic regimen or keep dosing stable. While this may affect the internal validity, it likely better reflects clinical practice, and may also avoid missing data.
      • Dworkin JD
      • McKeown A
      • Farrar JT
      • Gilron I
      • Hunsinger M
      • Kerns RD
      • McDermott MP
      • Rappaport BA
      • Turk DC
      • Dworkin RH
      • Gewandter JS
      Deficiencies in reporting of statistical methodology in recent randomized trials of nonpharmacologic pain treatments: ACTTION systematic review.
      Overall, study designs were evenly split between parallel group and cross-over design. Cross-over trials generally have smaller sample sizes, are less expensive to conduct, and may increase enrollment.
      • Prasad V
      • Grady C
      The misguided ethics of crossover trials.
      On the other hand, they may take longer to complete. Included cross-over studies frequently had no washout. While the duration, if any, of carryover effect of a previous SCS intervention is unclear, the possibility that this may have an effect on outcomes cannot be ruled out. While not noted, the lack of washout may have been due to concerns about return of pain during this period. To mitigate for this, 1 study offered participants opioid prescriptions for the 1-week washout period.
      • North JM
      • Hong KS
      • Cho PY
      Clinical outcomes of 1 kHz subperception spinal cord stimulation in implanted patients with failed paresthesia-based stimulation: Results of a prospective randomized controlled trial.
      Another study waited for pain to return to 80% of baseline before initiating the alternate intervention.
      • Thomson SJ
      • Tavakkolizadeh M
      • Love-Jones S
      • Patel NK
      • Gu JW
      • Bains A
      • Doan Q
      • Moffitt M
      Effects of rate on analgesia in Kilohertz frequency spinal cord stimulation: Results of the PROCO randomized controlled trial.
      Other studies only collected outcomes in the last few days of a cross-over period to minimize carryover.
      As mentioned, conventional SCS is associated with paresthesia, which may preclude complete patient blinding using a comparator of sham or placebo therapy. It is, therefore, not clear whether investigators or even outcome assessors can be blinded in studies of conventional SCS. Patient blinding with placebo or sham treatment can potentially be achieved in studies where participants receive either HF or burst SCS, as both are reported to not cause paresthesia.
      • Duarte RV
      • McNicol E
      • Colloca L
      • Taylor RS
      • North RB
      • Eldabe S
      Randomised placebo/sham controlled trials of spinal cord stimulation: A systematic review and methodological appraisal.
      Risks of unblinding through rapid draining of the battery in the intervention phase can be attenuated by programming a current leak during the placebo periods. Six studies compared HF or burst SCS with placebo or sham. Of the 17 studies that attempted any type of blinding, 11 blinded participants. Of these, only 5 were assessed as being adequately blinded. Lack of blinding or inadequate blinding is associated with a high risk of study bias. Encouragingly, many newer studies incorporated sham/placebo into their design.
      We assessed 27 studies (59%) as having a low risk of bias related to randomization. This is similar to findings from assessments of randomization in studies of pharmacological interventions.
      • Finnerup NB
      • Attal N
      • Haroutounian S
      • McNicol E
      • Baron R
      • Dworkin RH
      • Gilron I
      • Haanpää M
      • Hansson P
      • Jensen TS
      • Kamerman PR
      Pharmacotherapy for neuropathic pain in adults: Systematic review, meta-analysis and NeuPSIG recommendations.
      Twenty-two studies (48% of 46) incorporated a trial or screening phase, such as testing SCS with a temporary electrode, and assigning participants as having a successful trial if their pain was reduced. The use of a short-term endpoint to determine which participants should continue to the main study phase, a type of enrichment of the study population, has been recommended.
      • Eldabe S
      • Gulve A
      • Thomson S
      • Baranidharan G
      • Duarte R
      • Jowett S
      • Sandhu H
      • Chadwick R
      • Brookes M
      • Tariq A
      • Earle J
      Does a screening trial for spinal cord stimulation in patients with chronic pain of neuropathic origin have clinical utility and cost-effectiveness? (TRIAL-STIM Study): Study protocol for a randomised controlled trial.
      ,

      North RB, Shipley J: Chapter 4 - Clinical study designs for neuromodulation, in Krames Elliot S, Peckham P Hunter, Rezai Ali R, (eds.): Neuromodulation, 2nd ed. London, United Kingdom, Academic Press, 2018, pp 41-51.

      The majority of studies allowed for SCS adjustments during the intervention, most often in signal amplitude. The ability to tailor settings to individual patients has increased, which potentially optimizes the chances for a positive outcome, but also adds to the complexity of conducting a trial, of providing SCS in clinical practice, and to the expertise required for administration.
      • Geurts JW
      • Smits H
      • Kemler MA
      • Brunner F
      • Kessels AG
      • van Kleef M
      Spinal cord stimulation for complex regional pain syndrome type I: A prospective cohort study with long-term follow-up.
      The median duration of intervention and time to assessment of the primary outcome was 12 weeks (angina studies were mostly less than 8 weeks). This is similar to studies of pharmacological interventions for chronic pain.
      • Finnerup NB
      • Attal N
      • Haroutounian S
      • McNicol E
      • Baron R
      • Dworkin RH
      • Gilron I
      • Haanpää M
      • Hansson P
      • Jensen TS
      • Kamerman PR
      Pharmacotherapy for neuropathic pain in adults: Systematic review, meta-analysis and NeuPSIG recommendations.
      This duration may not adequately predict success, in that analgesic efficacy of both pharmacological and nonpharmacological treatments may change in patients with chronic pain. A prospective study of SCS with a 12-year follow-up demonstrated that after three years of treatment, 40% of initially successfully treated (defined as at least 30% pain relief) cases were reassigned as failures, and that this increased to 60% by 12 years.
      • Geurts JW
      • Smits H
      • Kemler MA
      • Brunner F
      • Kessels AG
      • van Kleef M
      Spinal cord stimulation for complex regional pain syndrome type I: A prospective cohort study with long-term follow-up.
      Clearly, there are major challenges to retaining participants in RCTs for prolonged periods. Many studies have tried to offset this by having open-label extension periods.

      Statistical Analysis

      The vast majority of studies (87%) specified a primary outcome(s), most of which were pain-related. This is superior to a review of noninvasive nonpharmacological studies, where only 62% identified a primary outcome.
      • Dworkin JD
      • McKeown A
      • Farrar JT
      • Gilron I
      • Hunsinger M
      • Kerns RD
      • McDermott MP
      • Rappaport BA
      • Turk DC
      • Dworkin RH
      • Gewandter JS
      Deficiencies in reporting of statistical methodology in recent randomized trials of nonpharmacologic pain treatments: ACTTION systematic review.
      Consensus guidelines have recommended 4 chronic pain outcome domains: pain intensity; physical functioning; emotional functioning; and patient ratings of improvement.
      • Turk DC
      • Dworkin RH
      • Allen RR
      • Bellamy N
      • Brandenburg N
      • Carr DB
      • Cleeland C
      • Dionne R
      • Farrar JT
      • Galer BS
      • Hewitt DJ
      Core outcome domains for chronic pain clinical trials: IMMPACT recommendations.
      Pain is by its nature subjective but clinically important to patients. Objective endpoints, such as limb survival in patients with peripheral vascular disease, or reductions in opioid use, were less frequent. Surrogate endpoints, such as sensation and coverage of paresthesia, were common, but do not necessarily translate to pain relief.

      North RB, Shipley J: Chapter 4 - Clinical study designs for neuromodulation, in Krames Elliot S, Peckham P Hunter, Rezai Ali R, (eds.): Neuromodulation, 2nd ed. London, United Kingdom, Academic Press, 2018, pp 41-51.

      Given current concerns with opioids, one might have expected to see more of the recent studies compare opioid use among groups.
      Adverse events were assessed as a primary outcome in 4 studies. Most studies did not specify whether AEs were collected actively (eg, a checklist), passively (eg, open-ended questions), or both. Reporting of paresthesias is of interest, in that studies have assessed them either as an indicator of efficacy or as an AE or side effect, or as both. With the advent of SCS parameters that reduce or eliminate paresthesia, it may be considered as a risk of unblinding. Finally, the majority of studies did not report the number of participants that required SCS adjustments because of AEs. Given recent reports of serious AEs in patients implanted with SCS, and differences in FDA criteria for device approval, more rigorous assessment of such events along with complete and transparent reporting are needed.

      Spinal-Cord Stimulators Help Some Patients, Injure Others. Associated Press. Available at: https://www.apnews.com/86ba45b0a4ad443fad1214622d13e6cb. Accessed March 15, 2019.

      A single noninferiority study has led to FDA approval of at least 2 SCS devices after 2015, according to the premarket approval database. For older SCS devices, additional data were often provided; however, most of these data were not based on RCTs.
      • Deer T
      • Slavin KV
      • Amirdelfan K
      • North RB
      • Burton AW
      • Yearwood TL
      • Tavel E
      • Staats P
      • Falowski S
      • Pope J
      • Justiz R
      Success using neuromodulation with BURST (SUNBURST) study: Results from a prospective, randomized controlled trial using a novel burst waveform.
      ,
      • Deer TR
      • Levy RM
      • Kramer J
      • Poree L
      • Amirdelfan K
      • Grigsby E
      • Staats P
      • Burton AW
      • Burgher AH
      • Obray J
      • Scowcroft J
      Dorsal root ganglion stimulation yielded higher treatment success rate for complex regional pain syndrome and causalgia at 3 and 12 months: A randomized comparative trial.
      ,
      • Kapural L
      • Yu C
      • Doust MW
      • Gliner BE
      • Vallejo R
      • Sitzman BT
      • Amirdelfan K
      • Morgan DM
      • Brown LL
      • Yearwood TL
      • Bundschu R
      Novel 10-kHz high-frequency therapy (HF10 Therapy) is superior to traditional low-frequency spinal cord stimulation for the treatment of chronic back and leg pain.
      ,

      Premarket approval (PMA). U.S. Food & Drug Administration. Available at: https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpma/pma.cfm. Accessed March 29, 2019.

      Almost half of studies did not specify the type of analysis performed. Where specified (or inferred), 21 were superiority studies, 4 were noninferiority, and 1 was an equivalence analysis. It has been suggested that noninferiority analyses may be appropriate for SCS studies, in that new devices may offer more stimulation parameters without necessarily being superior to an existing product;

      North RB, Shipley J: Chapter 4 - Clinical study designs for neuromodulation, in Krames Elliot S, Peckham P Hunter, Rezai Ali R, (eds.): Neuromodulation, 2nd ed. London, United Kingdom, Academic Press, 2018, pp 41-51.

      however, such studies have been criticized for not paying sufficient regard to patients’ interests.
      • Garattini S
      • Bertele V
      Non-inferiority trials are unethical because they disregard patients’ interests.
      In studies that reported analysis of multiple primary outcomes, 5/15 performed multiplicity adjustments to prevent an increase in the probability of type I error (ie, a false positive result). Systematic reviews of pharmacological, interventional, and noninterventional nonpharmacologic treatments identified deficiencies in reporting and, potentially in turn, the execution of multiplicity adjustment.
      • Dworkin JD
      • McKeown A
      • Farrar JT
      • Gilron I
      • Hunsinger M
      • Kerns RD
      • McDermott MP
      • Rappaport BA
      • Turk DC
      • Dworkin RH
      • Gewandter JS
      Deficiencies in reporting of statistical methodology in recent randomized trials of nonpharmacologic pain treatments: ACTTION systematic review.
      ,
      • Gewandter JS
      • Smith SM
      • McKeown A
      • Burke LB
      • Hertz SH
      • Hunsinger M
      • Katz NP
      • Lin AH
      • McDermott MP
      • Rappaport BA
      • Williams MR
      Reporting of primary analyses and multiplicity adjustment in recent analgesic clinical trials: ACTTION systematic review and recommendations.
      Our finding of only 33%, while based on a small number of studies, is broadly in agreement with both reviews.
      In 46% of studies, authors presented sample size, along with all required elements for its calculation, for the primary outcome. By comparison, a 2015 review of both noninvasive pharmacologic and interventional pain treatments, noted that 65% of studies reported at least 1 element of a sample size calculation.
      • McKeown A
      • Gewandter JS
      • McDermott MP
      • Pawlowski JR
      • Poli JJ
      • Rothstein D
      • Farrar JT
      • Gilron I
      • Katz NP
      • Lin AH
      • Rappaport BA
      Reporting of sample size calculations in analgesic clinical trials: ACTTION systematic review.
      The Consolidated Standards of Reporting Trials (CONSORT) organization recommends reporting of all elements; the lack of such has been associated with additional methodological deficiencies.
      Twenty-six studies (56%) analyzed only participants who completed the study and contributed data at all time points. These are similar to those from reviews of pharmacologic and other nonpharmacologic treatments.
      • Dworkin JD
      • McKeown A
      • Farrar JT
      • Gilron I
      • Hunsinger M
      • Kerns RD
      • McDermott MP
      • Rappaport BA
      • Turk DC
      • Dworkin RH
      • Gewandter JS
      Deficiencies in reporting of statistical methodology in recent randomized trials of nonpharmacologic pain treatments: ACTTION systematic review.
      ,
      • Gewandter JS
      • McDermott MP
      • McKeown A
      • Smith SM
      • Pawlowski JR
      • Poli JJ
      • Rothstein D
      • Williams MR
      • Bujanover S
      • Farrar JT
      • Gilron I
      Reporting of intention-to-treat analyses in recent analgesic clinical trials: ACTTION systematic review and recommendations.
      It has been suggested that per-protocol analyses may artificially inflate efficacy of a test intervention.
      • Eldabe S
      • Thomson S
      • Duarte R
      • Brookes M
      • deBelder M
      • Raphael J
      • Davies E
      • Taylor R
      The effectiveness and cost-effectiveness of spinal cord stimulation for refractory angina (RASCAL study): A pilot randomized controlled trial.
      Eighteen studies employed an ITT analysis. Of these, only seven specified a method to accommodate missing data. This was typically achieved by imputation, using LOCF. LOCF analyses are thought to have a relatively high risk of bias, in that they can attribute pain relief to participants who have withdrawn.
      • Higgins JPT
      • Altman DG
      • Sterne JAC
      Chapter 8: Assessing risk of bias in included studies.

      Reporting of Results

      Average enrollment and demographics were similar to studies of pharmacologic treatments for chronic pain.
      • McNicol ED
      • Midbari A
      • Eisenberg E
      Opioids for neuropathic pain.
      It has been suggested that studies enrolling at least 200 participants per arm are required to overcome random effects in estimating treatment effects.
      • Moore RA
      • Gavaghan D
      • Tramer MR
      • Collins SL
      • McQuay HJ
      Size is everything–large amounts of information are needed to overcome random effects in estimating direction and magnitude of treatment effects.
      None of the trials in this review met this criterion.
      Similarity of comparator groups at baseline is essential to prevent confounding. Half of included studies either did not report similarity or did not present sufficient data for comparison. Twenty studies described groups as being comparable at baseline, although they may not have been adequately powered. In 3 studies having groups that were not similar at baseline, 2 provided explanation or methods to accommodate differences.
      This systematic review provides the first overview, to our knowledge, of research methodology and reporting quality of published RCTs of SCS for pain. Grider et al performed a systematic review of effectiveness of SCS in chronic spinal pain, with a quality assessment, but did not review methodology.
      • Grider J
      • Manchikanti L
      • Carayannopoulos A
      • Sharma ML
      • Balog CC
      • Harned ME
      • Grami V
      • Justiz R
      • Nouri KH
      • Hayek SM
      • Vallejo R
      Effectiveness of spinal cord stimulation in chronic spinal pain: A systematic review.
      North et al provided a narrative review of methodology.

      North RB, Shipley J: Chapter 4 - Clinical study designs for neuromodulation, in Krames Elliot S, Peckham P Hunter, Rezai Ali R, (eds.): Neuromodulation, 2nd ed. London, United Kingdom, Academic Press, 2018, pp 41-51.

      We assessed all methods of SCS for any painful conditions. We searched multiple databases and accepted manuscripts published in any language (although no foreign language studies met criteria). Our review also has limitations. We were constrained by what was reported. We attempted to verify data posted on clinical trials websites; however, information from these is not always complete or accurate.

      North RB, Shipley J: Chapter 4 - Clinical study designs for neuromodulation, in Krames Elliot S, Peckham P Hunter, Rezai Ali R, (eds.): Neuromodulation, 2nd ed. London, United Kingdom, Academic Press, 2018, pp 41-51.

      Therefore, for some aspects of study methodology we inferred entries based on what was (or was not) reported. Complete data submitted as part of FDA premarket approval was not reviewed.
      Many of the methodological and reporting deficiencies highlighted in this review are similar to those in studies of other interventions used in the treatment of pain. Among those that are unique to the field of SCS, is the difficulty with performing completely blinded, placebo-controlled studies involving paresthesia-based (conventional) SCS. Although most studies included conventional SCS arms, such interventions are not necessarily homogeneous and have evolved over time, which may be problematic for interpretation of findings both within and between studies. Reporting recommendations are presented to increase transparency and, in turn, shape the design of future clinical trials (Table 7). Improved methodology will increase the validity of findings, facilitate comparisons across studies, and increase the probability that results are clinically valuable so that effective SCS treatments and settings are identified for this evolving method of neuromodulation.
      Table 7Reporting Recommendations for RCTs of SCS for Pain
      Reporting recommendations for randomized clinical trials of spinal cord stimulation for treatment of pain
      The following information should be clearly reported
      Reporting
      • Source of funding and specific role of funder in compensation, study design and analysis
      Study design
      • Parallel group, cross-over, other
      • Posting of a protocol detailing a priori inclusion criteria, outcomes assessed (with clear delineation of primary and secondary endpoints, and if multiple endpoints are primary, methods for multiplicity adjustment) and statistical methods employed on a website such as www.clinicaltrials.gov.
      Study methodology
      • Clinical eligibility criteria
      • Duration of washout in cross-over trials
      • Extent and methodology of blinding
      • Methods of randomization and its concealment
      • Role of screening phase in enrollment of participants
      • Initial settings and adjustment parameters for SCS units
      • Allowance of concurrent treatments
      • Methods to ensure balanced expectation of benefit of both researchers and patients (equipoise) between groups, and also balance of non-intervention treatment between groups (eg, programming time, psychological support, physical activity, rescue meds, etc.)
      Outcomes
      • Primary and secondary outcomes
      • Assessment of adverse events, including what and how these were assessed
      Statistical analysis
      • Number of participants and reasons for withdrawing
      • Similarity of groups at baseline and methods for accommodating differences
      • Type of analysis (superiority, noninferiority, etc.)
      • Sample size calculations, power analyses, and assumed effect size
      • Methods for dealing with missing data
      Interpretation
      • Clinical significance of any statistically significant difference

      Appendix. Supplementary data

      References

        • Al-Kaisy A
        • Palmisani S
        • Pang D
        • Sanderson K
        • Wesley S
        • Tan Y
        • McCammon S
        • Trescott A
        Prospective, randomized, sham-control, double blind, crossover trial of subthreshold spinal cord stimulation at various Kilohertz frequencies in subjects suffering from failed back surgery syndrome (SCS Frequency Study).
        Neuromodulation. 2018; 21: 457-465
        • De Andres J
        • Monsalve-Dolz V
        • Fabregat-Cid G
        • Villanueva-Perez V
        • Harutyunyan A
        • Asensio-Samper JM
        • Sanchis-Lopez N
        Prospective, randomized blind effect-on-outcome study of conventional vs high-frequency spinal cord stimulation in patients with pain and disability due to failed back surgery syndrome.
        Pain Med. 2017; 18: 2401-2421
        • de Jongste MJ
        • Hautvast RW
        • Hillege HL
        • Lie KI
        Efficacy of spinal cord stimulation as adjuvant therapy for intractable angina pectoris: A prospective, randomized clinical study.
        J Am Coll Cardiol. 1994; 23: 1592-1597
        • de Jongste MJL
        • Staal MJ
        Preliminary results of a randomized study on the clinical efficacy of spinal cord stimulation for refractory severe angina pectoris.
        Acta Neurochir. 1993; : 161-164
        • de Ridder Plazier M
        • Kamerling N
        • Menovsky T
        • Vanneste S
        Burst spinal cord stimulation for limb and back pain.
        World Neurosurg. 2013; 80: 642-649
        • de Vos CC
        • Meier K
        • Zaalberg PB
        • Nijhuis HJ
        • Duyvendak W
        • Vesper J
        • Enggaard TP
        • Lenders MW
        Spinal cord stimulation in patients with painful diabetic neuropathy: A multicentre randomized clinical trial.
        Pain. 2014; 155: 2426-2431
        • Deer T
        • Slavin KV
        • Amirdelfan K
        • North RB
        • Burton AW
        • Yearwood TL
        • Tavel E
        • Staats P
        • Falowski S
        • Pope J
        • Justiz R
        Success using neuromodulation with BURST (SUNBURST) study: Results from a prospective, randomized controlled trial using a novel burst waveform.
        Neuromodulation. 2018; 21: 56-66
        • Deer TR
        • Levy RM
        • Kramer J
        • Poree L
        • Amirdelfan K
        • Grigsby E
        • Staats P
        • Burton AW
        • Burgher AH
        • Obray J
        • Scowcroft J
        Dorsal root ganglion stimulation yielded higher treatment success rate for complex regional pain syndrome and causalgia at 3 and 12 months: A randomized comparative trial.
        Pain. 2017; 158 (2017): 669-681
        • Di Pede F
        • Zuin G
        • Giada F
        • Pinato G
        • Turiano G
        • Bevilacqua M
        • Cazzin R
        • Raviele A
        Long-term effects of spinal cord stimulation on myocardial ischemia and heart rate variability: Results of a 48-hour ambulatory electrocardiographic monitoring.
        Ital Heart J. 2001; 2: 690-695
        • Duarte RV
        • McNicol E
        • Colloca L
        • Taylor RS
        • North RB
        • Eldabe S
        Randomised placebo/sham controlled trials of spinal cord stimulation: A systematic review and methodological appraisal.
        Neuromodulation. 2020; 23: 10-18
        • Duarte RV
        • Nevitt S
        • McNicol E
        • Taylor RS
        • North RB
        • Eldabe S
        Are all placebo/sham controls the same? Lessons from a systematic review and meta-analysis of placebo/sham controlled cross-over randomised trials of spinal cord stimulation for neuropathic pain.
        Pain. 2020; 161: 24-35
        • Dworkin JD
        • McKeown A
        • Farrar JT
        • Gilron I
        • Hunsinger M
        • Kerns RD
        • McDermott MP
        • Rappaport BA
        • Turk DC
        • Dworkin RH
        • Gewandter JS
        Deficiencies in reporting of statistical methodology in recent randomized trials of nonpharmacologic pain treatments: ACTTION systematic review.
        J Clin Epidemiol. 2016; 72: 56-65
        • Eddicks S
        • Maier-Hauff K
        • Schenk M
        • Müller A
        • Baumann G
        • Theres H
        Thoracic spinal cord stimulation improves functional status and relieves symptoms in patients with refractory angina pectoris: The first placebo-controlled randomised study.
        Heart. 2007; 93: 585-590
        • Eisenberg E
        • Burstein Y
        • Suzan E
        • Treister R
        • Aviram J
        Spinal cord stimulation attenuates temporal summation in patients with neuropathic pain.
        Pain. 2015; 156: 381-385
        • Eldabe S
        • Gulve A
        • Thomson S
        • Baranidharan G
        • Duarte R
        • Jowett S
        • Sandhu H
        • Chadwick R
        • Brookes M
        • Tariq A
        • Earle J
        Does a screening trial for spinal cord stimulation in patients with chronic pain of neuropathic origin have clinical utility and cost-effectiveness? (TRIAL-STIM Study): Study protocol for a randomised controlled trial.
        Trials. 2018; 19: 633
        • Eldabe S
        • Thomson S
        • Duarte R
        • Brookes M
        • deBelder M
        • Raphael J
        • Davies E
        • Taylor R
        The effectiveness and cost-effectiveness of spinal cord stimulation for refractory angina (RASCAL study): A pilot randomized controlled trial.
        Neuromodulation. 2016; 19: 60-70
        • Finnerup NB
        • Attal N
        • Haroutounian S
        • McNicol E
        • Baron R
        • Dworkin RH
        • Gilron I
        • Haanpää M
        • Hansson P
        • Jensen TS
        • Kamerman PR
        Pharmacotherapy for neuropathic pain in adults: Systematic review, meta-analysis and NeuPSIG recommendations.
        Lancet Neurol. 2015; 14: 162-173
        • Ganty P
        • Sharma M
        Failed back surgery syndrome: A suggested algorithm of care.
        Br J Pain. 2012; 6: 153-161
        • Garattini S
        • Bertele V
        Non-inferiority trials are unethical because they disregard patients’ interests.
        Lancet. 2007; 370: 1875-1877
        • Geurts JW
        • Joosten EA
        • van Kleef M
        Current status and future perspectives of spinal cord stimulation in treatment of chronic pain.
        Pain. 2017; 158: 771-774
        • Geurts JW
        • Smits H
        • Kemler MA
        • Brunner F
        • Kessels AG
        • van Kleef M
        Spinal cord stimulation for complex regional pain syndrome type I: A prospective cohort study with long-term follow-up.
        Neuromodulation. 2013; 16: 523-529
        • Gewandter JS
        • McDermott MP
        • McKeown A
        • Smith SM
        • Pawlowski JR
        • Poli JJ
        • Rothstein D
        • Williams MR
        • Bujanover S
        • Farrar JT
        • Gilron I
        Reporting of intention-to-treat analyses in recent analgesic clinical trials: ACTTION systematic review and recommendations.
        Pain. 2014; 155: 2714-2719
        • Gewandter JS
        • Smith SM
        • McKeown A
        • Burke LB
        • Hertz SH
        • Hunsinger M
        • Katz NP
        • Lin AH
        • McDermott MP
        • Rappaport BA
        • Williams MR
        Reporting of primary analyses and multiplicity adjustment in recent analgesic clinical trials: ACTTION systematic review and recommendations.
        Pain. 2014; 155: 461-466
        • Grider J
        • Manchikanti L
        • Carayannopoulos A
        • Sharma ML
        • Balog CC
        • Harned ME
        • Grami V
        • Justiz R
        • Nouri KH
        • Hayek SM
        • Vallejo R
        Effectiveness of spinal cord stimulation in chronic spinal pain: A systematic review.
        Pain Physician. 2016; 19: E33-E54
        • Hautvast RW
        • DeJongste MJ
        • Staal MJ
        • van Gilst WH
        • Lie KI
        Spinal cord stimulation in chronic intractable angina pectoris: A randomized, controlled efficacy study.
        Am Heart J. 1998; 136: 1114-1120
        • Higgins JPT
        • Altman DG
        • Sterne JAC
        Chapter 8: Assessing risk of bias in included studies.
        in: Higgins JPT Green S Cochrane Handbook for Systematic Reviews of Interventions. The Cochrane Collaboration, 2011 (Version 5.1.0 [updated March 2011]. Available at:) (Accessed July 9, 2020)
        • Jessurun GA
        • DeJongste MJ
        • Hautvast RW
        • Tio RA
        • Brouwer J
        • van Lelieveld ST
        • Crijns HJ
        Clinical follow-up after cessation of chronic electrical neuromodulation in patients with severe coronary artery disease: A prospective randomized controlled study on putative involvement of sympathetic activity.
        Pacing Clin Electrophysiol. 1999; 22: 1432-1439
        • Jivegard LEH
        • Augustinsson LE
        • Holm J
        • Risberg B
        • Ortenwall P
        Effects of spinal cord stimulation (SCS) in patients with inoperable severe lower limb ischaemia: A prospective randomised controlled study.
        Eur J Vasc Surg. 1995; 9: 421-425
        • Kapural L
        • Yu C
        • Doust MW
        • Gliner BE
        • Vallejo R
        • Sitzman BT
        • Amirdelfan K
        • Morgan DM
        • Brown LL
        • Yearwood TL
        • Bundschu R
        Novel 10-kHz high-frequency therapy (HF10 Therapy) is superior to traditional low-frequency spinal cord stimulation for the treatment of chronic back and leg pain.
        Anesthesiology. 2015; 123: 851-860
        • Kapural L
        • Yu C
        • Doust MW
        • Gliner BE
        • Vallejo R
        • Sitzman BT
        • Amirdelfan K
        • Morgan DM
        • Yearwood TL
        • Bundschu R
        • Yang T
        Comparison of 10-kHz high-frequency and traditional low-frequency spinal cord stimulation for the treatment of chronic back and leg pain: 24-month results from a multicenter, randomized, controlled pivotal trial.
        Neurosurgery. 2016; 79: 667-677
        • Kemler MA
        • Barendse GA
        • Van Kleef M
        • de Vet HC
        • Rijks CP
        • Furnee CA
        • Van Den Wildenberg FA
        Spinal cord stimulation in patients with chronic reflex sympathetic dystrophy.
        New England J Med. 2000; 343: 618-624
        • Kemler MA
        • De Vet HC
        • Barendse GA
        • Van Den Wildenberg FA
        • Van Kleef M
        The effect of spinal cord stimulation in patients with chronic reflex sympathetic dystrophy: Two years' follow-up of the randomized controlled trial.
        Ann Neurol. 2004; 55: 13-18
        • Kemler MA
        • De Vet HC
        • Barendse GA
        • Van Den Wildenberg FA
        • Van Kleef M
        Spinal cord stimulation for chronic reflex sympathetic dystrophy - Five-year follow-up.
        NEJM. 2006; 354: 2394-2396
        • Kemler MA
        • Reulen JP
        • Barendse GA
        • van Kleef M
        • de Vet HC
        • van den Wildenberg FA
        Impact of spinal cord stimulation on sensory characteristics in complex regional pain syndrome type IA randomized trial.
        Anesthesiology. 2001; 95: 72-80
        • Klomp HM
        • Spincemaille GH
        • Steyerberg EW
        • Habbema JD
        • van Urk H
        • ESES Study Group
        Spinal-cord stimulation in critical limb ischaemia: A randomised trial.
        Lancet. 1999; 353: 1040-1044
        • Kriek N
        • Groeneweg JG
        • Stronks DL
        • Huygen FJPM
        Preferred frequencies and waveforms for spinal cord stimulation in patients with complex regional pain syndrome: A multicentre, double-blind, randomized and placebo-controlled crossover trial.
        Eur J Pain. 2017; 21: 507-519
        • Kumar K
        • Taylor RS
        • Jacques L
        • Eldabe S
        • Meglio M
        • Molet J
        • Thomson S
        • O'Callaghan J
        • Eisenberg E
        • Milbouw G
        • Buchser E
        The effects of spinal cord stimulation in neuropathic pain are sustained: A 24-month follow-up of the prospective randomized controlled multicenter trial of the effectiveness of spinal cord stimulation.
        Neurosurgery. 2008; 63: 762-770
        • Kumar K
        • Taylor RS
        • Jacques L
        • Eldabe S
        • Meglio M
        • Molet J
        • Thomson S
        • O'Callaghan J
        • Eisenberg E
        • Milbouw G
        • Buchser E
        Spinal cord stimulation versus conventional medical management for neuropathic pain: A multicentre randomised controlled trial in patients with failed back surgery syndrome.
        Pain. 2007; 132: 179-188
        • Kumar V
        • Prusik J
        • Lin Y
        • Hwang R
        • Feustel P
        • Pilitsis JG
        Efficacy of alternating conventional stimulation and high frequency stimulation in improving spinal cord stimulation outcomes: A pilot study.
        Neuromodulation. 2018; 21: 466-471
        • Lanza GA
        • Grimaldi R
        • Greco S
        • Ghio S
        • Sarullo F
        • Zuin G
        • De Luca A
        • Allegri M
        • Di Pede F
        • Castagno D
        • Turco A
        Spinal cord stimulation for the treatment of refractory angina pectoris: A multicenter randomized single-blind study (the SCS-ITA trial).
        Pain. 2011; 152: 45-52
        • Lanza GA
        • Sestito A
        • Sgueglia GA
        • Infusino F
        • Papacci F
        • Visocchi M
        • Ierardi C
        • Meglio M
        • Bellocci F
        • Crea F
        Effect of spinal cord stimulation on spontaneous and stress-induced angina and 'ischemia-like' ST-segment depression in patients with cardiac syndrome X.
        Eur Heart J. 2005; 26: 983-989
        • Lind G
        • Winter J
        • Linderoth B
        • Hellström PM
        Therapeutic value of spinal cord stimulation in irritable bowel syndrome: A randomized crossover pilot study.
        Am J Physiol Regul Integr Comp Physiol. 2015; 308: R887-R894
        • Mailis-Gagnon A
        • Furlan MD PhD AD
        • Sandoval JA
        • Taylor RS
        Spinal cord stimulation for chronic pain.
        Cochrane Database Syst Rev. 2004; (Art. No.: CD003783)
        • Mannheimer C
        • Eliasson T
        • Augustinsson LE
        • Blomstrand C
        • Emanuelsson H
        • Larsson S
        • Norrsell H
        • Hjalmarsson A
        Electrical stimulation versus coronary artery bypass surgery in severe angina pectoris: The ESBY study.
        Circulation. 1998; 97: 1157-1163
        • McKeown A
        • Gewandter JS
        • McDermott MP
        • Pawlowski JR
        • Poli JJ
        • Rothstein D
        • Farrar JT
        • Gilron I
        • Katz NP
        • Lin AH
        • Rappaport BA
        Reporting of sample size calculations in analgesic clinical trials: ACTTION systematic review.
        J Pain. 2015; 16: 199-206
        • McNab D
        • Khan SN
        • Sharples LD
        • Ryan JY
        • Freeman C
        • Caine N
        • Tait S
        • Hardy I
        • Schofield PM
        An open label, single-centre, randomized trial of spinal cord stimulation vs. percutaneous myocardial laser revascularization in patients with refractory angina pectoris: The SPiRiT trial.
        Eur Heart J. 2006; 27: 1048-1053
        • McNicol ED
        • Midbari A
        • Eisenberg E
        Opioids for neuropathic pain.
        Cochrane Database Syst Rev. 2013; (Art. No.: CD006146)
        • Meier K
        • Nikolajsen L
        • Sörensen JC
        • Jensen TS
        Effect of spinal cord stimulation on sensory characteristics: A randomized, blinded crossover study.
        Clin J Pain. 2015; 31: 384-392
        • Melzack R
        • Wall PD
        Pain mechanisms: A new theory.
        Science. 1965; 150: 971-979
        • Miller JP
        • Eldabe S
        • Buchser E
        • Johanek LM
        • Guan Y
        • Linderoth B
        Parameters of spinal cord stimulation and their role in electrical charge delivery: A review.
        Neuromodulation. 2016; 19: 373-384
        • Moore RA
        • Gavaghan D
        • Tramer MR
        • Collins SL
        • McQuay HJ
        Size is everything–large amounts of information are needed to overcome random effects in estimating direction and magnitude of treatment effects.
        Pain. 1998; 78: 209-216
        • Neugebauer EA
        • Rath A
        • Antoine SL
        • Eikermann M
        • Seidel D
        • Koenen C
        • Jacobs E
        • Pieper D
        • Laville M
        • Pitel S
        • Martinho C
        Specific barriers to the conduct of randomised clinical trials on medical devices.
        Trials. 2017; 18: 427
        • North JM
        • Hong KS
        • Cho PY
        Clinical outcomes of 1 kHz subperception spinal cord stimulation in implanted patients with failed paresthesia-based stimulation: Results of a prospective randomized controlled trial.
        Neuromodulation. 2016; 19: 731-737
        • North R
        • Shipley J
        Practice parameters for the use of spinal cord stimulation in the treatment of chronic neuropathic pain.
        Pain Medicine. 2007; 8: S200-S275
        • North RB
        • Brigham DD
        • Khalessi A
        • Calkins SK
        • Piantadosi S
        • Campbell DS
        • Daly MJ
        • Dey PB
        • Barolat G
        • Taylor R
        Spinal cord stimulator adjustment to maximize implanted battery longevity: A randomized, controlled trial using a computerized patient-interactive programmer.
        Neuromodulation. 2004; 7: 13-25
        • North RB
        • Kidd DH
        • Farrokhi F
        • Piantadosi SA
        Spinal cord stimulation versus repeated lumbosacral spine surgery for chronic pain: A randomized, controlled trial.
        Neurosurgery. 2005; 56: 98-107
        • North RB
        • Kidd DH
        • Lee MS
        • Piantodosi S
        A prospective, randomized study of spinal cord stimulation versus reoperation for failed back surgery syndrome: Initial results.
        Stereotactic Funct Neurosurg. 1994; 62: 267-272
        • North RB
        • Kidd DH
        • Petrucci L
        • Dorsi MJ
        Spinal cord stimulation electrode design: A prospective, randomized, controlled trial comparing percutaneous with laminectomy electrodes: Part II–clinical outcomes.
        Neurosurgery. 2005; 57: 990-996
        • North RB
        • Shipley J
        WIKISTIM.org: An on-line database of published neurostimulation studies.
        Neuromodulation. 2018; 21: 828-836
      1. North RB, Shipley J: Chapter 4 - Clinical study designs for neuromodulation, in Krames Elliot S, Peckham P Hunter, Rezai Ali R, (eds.): Neuromodulation, 2nd ed. London, United Kingdom, Academic Press, 2018, pp 41-51.

        • Perruchoud C
        • Eldabe S
        • Batterham AM
        • Madzinga G
        • Brookes M
        • Durrer A
        • Rosato M
        • Bovet N
        • West S
        • Bovy M
        • Rutschmann B
        Analgesic efficacy of high-frequency spinal cord stimulation: A randomized double-blind placebo-controlled study.
        Neuromodulation. 2013; 16: 363-369
        • Prasad V
        • Grady C
        The misguided ethics of crossover trials.
        Contemp Clin Trials. 2014; 37: 167-169
      2. Premarket approval (PMA). U.S. Food & Drug Administration. Available at: https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpma/pma.cfm. Accessed March 29, 2019.

      3. PROSPERO: International prospective register of systematic reviews. National Institute for Health Research. Registered May 2018. Updated November 2018. Available at: http://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42018090412. Accessed July 9, 2020

        • Schu S
        • Slotty PJ
        • Bara G
        • von Knop M
        • Edgar D
        • Vesper J
        A prospective, randomised, double-blind, placebo-controlled study to examine the effectiveness of burst spinal cord stimulation patterns for the treatment of failed back surgery syndrome.
        Neuromodulation. 2014; 17: 443-450
        • Schultz DM
        • Webster L
        • Kosek P
        • Dar U
        • Tan Y
        • Sun M
        Sensor-driven position-adaptive spinal cord stimulation for chronic pain.
        Pain Phys. 2012; 15: 1-2
        • Shealy CN
        • Mortimer JT
        • Reswick JB
        Electrical inhibition of pain by stimulation of the dorsal columns: Preliminary clinical report.
        Anesth Analg. 1967; 46: 489-491
        • Slangen R
        • Schaper NC
        • Faber CG
        • Joosten EA
        • Dirksen CD
        • van Dongen RT
        • Kessels AG
        • van Kleef M
        Spinal cord stimulation and pain relief in painful diabetic peripheral neuropathy: A prospective two-center randomized controlled trial.
        Diabetes Care. 2014; 37: 3016-3024
      4. Spinal-Cord Stimulators Help Some Patients, Injure Others. Associated Press. Available at: https://www.apnews.com/86ba45b0a4ad443fad1214622d13e6cb. Accessed March 15, 2019.

        • Spincemaille GH
        • Klomp HM
        • Steyerberg EW
        • Habbema JD
        Pain and quality of life in patients with critical limb ischaemia: Results of a randomized controlled multicentre study on the effect of spinal cord stimulation.
        Eur J Pain. 2000; 4: 173-184
        • Spincemaille GH
        • Klomp HM
        • Steyerberg EW
        • Habbema JD
        Spinal cord stimulation in patients with critical limb ischemia: A preliminary evaluation of a multicentre trial.
        Acta Chir Austriaca. 2000; 32: 49-51
        • Spincemaille GH
        • Klomp HM
        • Steyerberg EW
        • van Urk H
        • Habbema JD
        Technical data and complications of spinal cord stimulation: Data from a randomized trial on critical limb ischemia.
        Stereotact Funct Neurosurg. 2000; 74: 63-72
        • Suy R
        • Gybels J
        • Van Damme H
        • Martin D
        • Van Maele R
        • Delaporte C
        Spinal cord stimulation for ischemic rest pain. The Belgian randomized study.
        Spinal Cord Stimulation. Steinkopff, 1994: 197-202
        • Thomson SJ
        • Tavakkolizadeh M
        • Love-Jones S
        • Patel NK
        • Gu JW
        • Bains A
        • Doan Q
        • Moffitt M
        Effects of rate on analgesia in Kilohertz frequency spinal cord stimulation: Results of the PROCO randomized controlled trial.
        Neuromodulation. 2018; 21: 67-76
        • Tjepkema-Cloostermans MC
        • de Vos
        • Wolters CC
        • Dijkstra-Scholten R
        • Lenders MWPM C
        Effect of burst stimulation evaluated in patients familiar with spinal cord stimulation.
        Neuromodulation. 2016; 19: 492-497
        • Turk DC
        • Dworkin RH
        • Allen RR
        • Bellamy N
        • Brandenburg N
        • Carr DB
        • Cleeland C
        • Dionne R
        • Farrar JT
        • Galer BS
        • Hewitt DJ
        Core outcome domains for chronic pain clinical trials: IMMPACT recommendations.
        Pain. 2003; 106: 337-345
        • Turner JA
        • Loeser JD
        • Bell KG
        Spinal cord stimulation for chronic low back pain. A systematic literature synthesis.
        Neurosurgery. 1995; 37: 1088-1096
        • van Beek M
        • Slangen R
        • Schaper NC
        • Faber CG
        • Joosten EA
        • Dirksen CD
        • van Dongen RT
        • Kessels AG
        • van Kleef M
        Sustained treatment effect of spinal cord stimulation in painful diabetic peripheral neuropathy: 24-month follow-up of a prospective two-center randomized controlled trial.
        Diabetes Care. 2015; 38: e132-e134
        • van Bussel CM
        • Stronks DL
        • Huygen FJ
        Dorsal column stimulation vs. dorsal root ganglion stimulation for complex regional pain syndrome confined to the knee: Patients’ preference following the trial period.
        Pain Pract. 2018; 18: 87-93
        • Van Eijs F
        • Smits H
        • Geurts JW
        • Kessels AG
        • Kemler MA
        • Van Kleef M
        • Joosten EA
        • Faber CG
        Brush-evoked allodynia predicts outcome of spinal cord stimulation in complex regional pain syndrome type 1.
        Eur J Pain. 2010; 14: 164-169
        • Van Havenbergh T
        • Vancamp T
        • Van Looy P
        • Vanneste S
        • De Ridder D
        Spinal cord stimulation for the treatment of chronic back pain patients: 500-Hz vs. 1000-Hz burst stimulation.
        Neuromodulation. 2015; 18: 9-12
        • Washburn S
        • Catlin R
        • Bethel K
        • Canlas B
        Patient-perceived differences between constant current and constant voltage spinal cord stimulation systems.
        Neuromodulation. 2014; 17: 28-36
      5. Wikistim.org. The Neuromodulation Foundation, Inc. Copyright 2013-2019. Available at: https://www.wikistim.org/. Accessed July 9, 2020

        • Wolter T
        • Kiemen A
        • Porzelius C
        • Kaube H
        Effects of sub-perception threshold spinal cord stimulation in neuropathic pain: A randomized controlled double-blind crossover study.
        Eur J Pain. 2012; 16: 648-655
        • Youn Y
        • Smith H
        • Morris B
        • Argoff C
        • Pilitsis J
        The effect of high-frequency stimulation on sensory thresholds in chronic pain patients.
        Stereotact Funct Neurosurg. 2015; 93: 355-359
        • Zipes DP
        • Svorkdal N
        • Berman D
        • Boortz-Marx R
        • Henry T
        • Lerman A
        • Ross E
        • Turner M
        • Irwin C
        Spinal cord stimulation therapy for patients with refractory angina who are not candidates for revascularization.
        Neuromodulation. 2012; 15: 550-558