Automated Prediction of Risk for Problem Opioid Use in a Primary Care Setting

Published:January 29, 2015DOI:


      • An automated method predictive in identifying clinician-documented problem opioid use was assessed.
      • Analysis developed measures using information from a health plan's electronic health record system.
      • Models showed moderate sensitivity and specificity in predicting problem opioid use.
      • The method may enable more efficient implementation of problem opioid use detection.


      Identification of patients at increased risk for problem opioid use is recommended by chronic opioid therapy (COT) guidelines, but clinical assessment of risks often does not occur on a timely basis. This research assessed whether structured electronic health record (EHR) data could accurately predict subsequent problem opioid use. This research was conducted among 2,752 chronic noncancer pain patients initiating COT (≥70 days' supply of an opioid in a calendar quarter) during 2008 to 2010. Patients were followed through the end of 2012 or until disenrollment from the health plan, whichever was earlier. Baseline risk indicators were derived from structured EHR data for a 2-year period prior to COT initiation. Problem opioid use after COT initiation was assessed by reviewing clinician-documented problem opioid use in EHR clinical notes identified using natural language processing techniques followed by computer-assisted manual review of natural language processing–positive clinical notes. Multivariate analyses in learning and validation samples assessed prediction of subsequent problem opioid use. The area under the receiver operating characteristic curve (c-statistic) for problem opioid use was .739 (95% confidence interval = .688, .790) in the validation sample. A measure of problem opioid use derived from a simple weighted count of risk indicators was found to be comparably predictive of the natural language processing measure of problem opioid use, with 60% sensitivity and 72% specificity for a weighted count of ≥4 risk indicators.


      An automated surveillance method utilizing baseline risk indicators from structured EHR data was moderately accurate in identifying COT patients who had subsequent problem opioid use.

      Key words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic and Personal


      Subscribe to The Journal of Pain
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Birnbaum H.G.
        • White A.G.
        • Schiller M.
        • Waldman T.
        • Cleveland J.M.
        • Roland C.L.
        Societal costs of prescription opioid abuse, dependence, and misuse in the United States.
        Pain Med. 2011; 12: 657-667
        • Brown J.
        • Setnik B.
        • Lee K.
        • Wase L.
        • Roland C.L.
        • Cleveland J.M.
        • Siegal S.
        • Katz N.
        Assessment, stratification, and monitoring of the risk for prescription opioid misuse and abuse in the primary care setting.
        J Opioid Manag. 2011; 7: 467-483
        • Chou R.
        • Fanciullo G.J.
        • Fine P.G.
        • Adler J.A.
        • Ballantyne J.C.
        • Davies P.
        • Donovan M.I.
        • Fishbain D.A.
        • Foley K.M.
        • Fudin J.
        • Gilson A.M.
        • Kelter A.
        • Mauskop A.
        • O'Connor P.G.
        • Passik S.D.
        • Pasternak G.W.
        • Portenoy R.K.
        • Rich B.A.
        • Roberts R.G.
        • Todd K.H.
        • Miaskowski C.
        • American Pain Society-American Academy of Pain Medicine Opioids Guidelines Panel. Opioid Treatment Guidelines
        Clinical guidelines for the use of chronic opioid therapy in chronic noncancer pain.
        J Pain. 2009; 10: 113-130
        • Chou R.
        • Fanciullo G.J.
        • Fine P.G.
        • Miaskowski C.
        • Passik S.D.
        • Portenoy R.K.
        Opioids for chronic noncancer pain: Prediction and identification of aberrant drug-related behaviors: A review of the evidence for an American Pain Society and American Academy of Pain Medicine clinical practice guideline.
        J Pain. 2009; 10: 131-146
        • Cronkite D.
        Estimating Prevalence of Clinician-Labeled Opioid Abuse: Structured vs. Unstructured Clinical Data. HMO Research Network Annual Meeting Presentation.
        2014 (Phoenix, AZ)
        • Dufour R.
        • Joshi A.V.
        • Pasquale M.K.
        • Schaaf D.
        • Mardekian J.
        • Andrews G.A.
        • Patel N.
        The prevalence of diagnosed opioid abuse in commercial and Medicare managed care populations.
        Pain Pract. 2014; 14: E106-E115
        • Dunn K.M.
        • Saunders K.W.
        • Rutter C.M.
        • Banta-Green C.J.
        • Merrill J.O.
        • Sullivan M.D.
        • Weisner C.M.
        • Silverberg M.J.
        • Campbell C.I.
        • Psaty B.
        • Von Korff M.
        Opioid prescriptions for chronic pain and overdose: A cohort study.
        Ann Intern Med. 2010; 152: 85-92
        • Frakt A.
        Painkiller abuse, a cyclical challenge. New York Times.
        2014 (Available at:) (Accessed December 29, 2014)
        • Gourlay D.L.
        • Heit H.A.
        • Almahrezi A.
        Universal precautions in pain medicine: A rational approach to the treatment of chronic pain.
        Pain Med. 2005; 6: 107-112
        • Washington State Agency Medical Directors’ Group
        Interagency Guideline on Opioid Dosing for Chronic Non-cancer Pain: An Educational Aid to Improve Care and Safety With Opioid Treatment.
        Washington State Department of Labor and Industries, Olympia (WA)2010 (Available at:) (Accessed December 28, 2014)
      1. FDA Statement. FDA Commissioner Margaret A Hamburg Statement on Prescription Opioid Abuse. 2014 (Available at:) (Accessed April 28, 2014)
        • Institute of Medicine of the National Academies
        Relieving Pain in America: A Blueprint for Transforming Prevention, Care, Education, and Research.
        National Academies Press, Washington, DC2011
        • Jones T.
        • Moore T.
        • Levy J.L.
        • Daffron S.
        • Browder J.H.
        • Allen L.
        • Passik S.D.
        A comparison of various risk screening methods in predicting discharge from opioid treatment.
        Clin J Pain. 2012; 28: 93-100
        • Krebs E.E.
        • Ramsey D.
        • Miloshoff J.M.
        • Bair M.J.
        Primary care monitoring of long-term opioid therapy among veterans with chronic pain.
        Pain Med. 2011; 12: 740-746
        • Meier B.
        FDA urging a tighter rein on painkillers.
        2013 (Available at:) (Accessed March 30, 2014)
        • Moore T.M.
        • Jones T.
        • Browder J.H.
        • Daffron S.
        • Passik S.D.
        A comparison of common screening methods for predicting aberrant drug-related behavior among patients receiving opioids for chronic pain management.
        Pain Med. 2009; 10: 1426-1433
        • National Institute of Drug Abuse
        Prescription Drug Abuse.
        (Available at:) (Accessed March 31, 2014)
        • Office of National Drug Control Policy
        Epidemic: Responding to America’s Prescription Drug Abuse Crisis.
        2011 ([updated 2011 Apr 11; cited 2011 Jun 10]. Available at:) (Accessed December 28, 2014)
        • Passik S.D.
        • Kirsh K.L.
        • Casper D.
        Addiction-related assessment tools and pain management: Instruments for screening, treatment planning, and monitoring compliance.
        Pain Med. 2008; 9: S145-S166
      2. Prescription Drug Monitoring Program (PDMP) Center of Excellence, Brandeis University. National Rx Drug Abuse Summit Meeting, 22 April 2014

        • Rice J.B.
        • White A.G.
        • Birnbaum H.G.
        • Schiller M.
        • Brown D.A.
        • Roland C.L.
        A model to identify patients at risk for prescription opioid abuse, dependence, and misuse.
        Pain Med. 2012; 13: 1162-1173
        • Rolfs R.T.
        • Johnson E.
        • Williams N.J.
        • Sundwall D.N.
        Utah clinical guidelines on prescribing opioids for treatment of pain.
        J Pain Pall Care Pharm. 2010; 24: 219-235
        • Saunders K.W.
        • Von Korff M.
        • Campbell C.I.
        • Banta-Green C.J.
        • Sullivan M.D.
        • Merrill J.O.
        • Weisner C.
        Concurrent use of alcohol and sedatives among persons prescribed chronic opioid therapy: Prevalence and risk factors.
        J Pain. 2012; 13: 266-275
        • Savova G.K.
        • Masanz J.J.
        • Ogren P.V.
        • Zheng J.
        • Sohn S.
        • Kipper-Schuler K.C.
        • Chute C.G.
        Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): Architecture, component evaluation and applications.
        J Am Med Inform Assoc. 2010; 17: 507-513
        • Sehgal N.
        • Manchikanti L.
        • Smith H.S.
        Prescription opioid abuse in chronic pain: A review of opioid abuse predictors and strategies to curb opioid abuse.
        Pain Physician. 2012; 15: ES67-ES92
        • Starrels J.L.
        • Becker W.C.
        • Alford D.P.
        • Kapoor A.
        • Williams A.R.
        • Turner B.
        Systematic review: Treatment agreements and urine drug testing to reduce opioid misuse in patients with chronic pain.
        Ann Intern Med. 2010; 152: 712-720
        • Sullivan M.D.
        • Edlund M.J.
        • Fan M.Y.
        • Devries A.
        • Brennan Braden J.
        • Martin B.C.
        Risks for possible and probable opioid misuse among recipients of chronic opioid therapy in commercial and medicaid insurance plans: The TROUP Study.
        Pain. 2010; 50: 332-339
        • Trescott C.E.
        • Beck R.M.
        • Seelig M.D.
        • Von Korff M.R.
        Group Health's initiative to avert opioid misuse and overdose among patients with chronic noncancer pain.
        Health Aff. 2011; 30: 1420-1424
        • Turk D.C.
        • Swanson K.S.
        • Gatchel R.J.
        Predicting opioid misuse by chronic pain patients: A systematic review and literature synthesis.
        Clin J Pain. 2008; 24: 497-508
        • US Food and Drug Administration
        letter to Extended Release/Long Acting opioid application holders.
        2013 (Available at:) (Accessed July 10, 2014)
        • Von Korff M.
        • Kolodny A.
        • Deyo R.A.
        • Chou R.
        Long-term opioid therapy reconsidered.
        Ann Intern Med. 2011; 155: 325-328
        • Von Korff M.
        • Merrill J.O.
        • Rutter C.M.
        • Sullivan M.
        • Campbell C.I.
        • Weisner C.
        Time-scheduled vs. pain-contingent opioid dosing in chronic opioid therapy.
        Pain. 2011; 152: 1256-1262
        • Von Korff M.
        • Ormel J.
        • Keefe F.J.
        • Dworkin S.F.
        Grading the severity of chronic pain.
        Pain. 1992; 50: 133-149
        • White A.G.
        • Birnbaum H.G.
        • Mareva M.N.
        • Daher M.
        • Vallow S.
        • Schein J.
        • Katz N.
        Direct costs of opioid abuse in an insured population in the United States.
        J Manag Care Pharm. 2005; 11: 469-479
        • Wilson P.
        • D’Agostino R.
        • Levy D.
        • Belanger A.
        • Silbershatz H.
        • Kannel W.
        Prediction of coronary heart disease using risk factor categories.
        Circulation. 1998; 97: 1837-1847