Advertisement
Original Report| Volume 16, ISSUE 5, P445-453, May 2015

Defining Risk of Prescription Opioid Overdose: Pharmacy Shopping and Overlapping Prescriptions Among Long-Term Opioid Users in Medicaid

Published:February 10, 2015DOI:https://doi.org/10.1016/j.jpain.2015.01.475

      Highlights

      • We examined behavioral indicators of high risk of high opioid overdose.
      • Multistate Medicaid longitudinal opioid user data are used.
      • Pharmacy shopping is associated with high overdose risk.
      • Medicaid should also consider overlapping prescriptions as a risk indicator.

      Abstract

      Use of multiple pharmacies concurrently (pharmacy shopping) and overlapping prescriptions may be indicators of potential misuse or abuse of prescription opioid medications. To evaluate strategies for identifying patients at high risk, we first compared different definitions of pharmacy shopping and then added the indicator of overlapping opioid prescriptions. We identified a cohort of 90,010 Medicaid enrollees who used ≥3 opioid prescriptions for ≥90 days during 2008 to 2010 from a multistate Medicaid claims database. We compared the diagnostic odds ratios for opioid overdose events of 9 pharmacy shopping definitions. Within a 90-day interval, a threshold of 4 pharmacies had the highest diagnostic odds ratio and was used to define pharmacy shopping. The overdose rate was higher in the subgroup with overlapping prescriptions (18.5 per 1,000 person-years [PYs]) than in the subgroup with pharmacy shopping as the sole indicator (10.7 per 1,000 PYs). Among the subgroup with both conditions, the overdose rate was 26.3 per 1,000 PYs, compared with 4.3 per 1,000 PYs for those with neither condition. Overlapping opioid prescriptions and pharmacy shopping measures had adjusted hazard ratios of 3.0 and 1.8, respectively, for opioid overdose. Using these measures will improve accurate identification of patients at highest risk of opioid overdose, the first step in implementing targeted prevention policies.

      Perspective

      Long-term prescription opioid use may lead to adverse events, including overdose. Both pharmacy shopping and overlapping opioid prescriptions are associated with adverse outcomes. This study demonstrates that using both indicators will better identify those at high risk of overdose.

      Key words

      Because of increasing involvement of prescription opioids in drug overdose deaths, overdose has surpassed motor vehicle crashes as a cause of death in the United States.
      • Warner M.
      • Chen L.H.
      • Makuc D.M.
      • Anderson R.N.
      • Minino A.M.
      Drug poisoning deaths in the United States, 1980-2008.
      Programs that strive to attenuate this risk have utilized various measures to identify patients at high risk of prescription opioid overdose. Pharmacy shopping, an indicator of potential opioid misuse or abuse, refers to visiting multiple pharmacies to obtain multiple opioid prescriptions.
      • Buurma H.
      • Bouvy M.L.
      • De Smet P.A.
      • Floor-Schreudering A.
      • Leufkens H.G.
      • Egberts A.C.
      Prevalence and determinants of pharmacy shopping behaviour.
      In many states, Medicaid programs monitor the number of pharmacies that a patient visits to obtain opioid prescriptions as part of patient review and restriction (PRR) programs, also referred to as lock-in programs.
      Patient Review & Restriction Programs: Lessons learned from state Medicaid programs.
      The PRR program was among the interventions recommended by the Centers for Disease Control and Prevention to potentially reduce prescription drug abuse.
      Policy Impact: Prescription painkiller overdoses. Centers for Disease Control and Prevention.
      The rationale for PRR programs comes from prevalence analyses that indicate that Medicaid beneficiaries are a high-risk group.
      Centers for Disease Control and Prevention
      Overdose deaths involving prescription opioids among Medicaid enrollees—Washington, 2004-2007.
      One Centers for Disease Control and Prevention study determined that 45% of people who died from prescription opioid overdose in the state of Washington were Medicaid enrollees.
      Centers for Disease Control and Prevention
      Overdose deaths involving prescription opioids among Medicaid enrollees—Washington, 2004-2007.
      In Arkansas, patients on Medicaid were prescribed opioids at twice the rate of non-Medicaid patients and had 6 times the risk of prescription opioid overdose.
      • Braden J.B.
      • Fan M.Y.
      • Edlund M.J.
      • Martin B.C.
      • DeVries A.
      • Sullivan M.D.
      Trends in use of opioids by noncancer pain type 2000-2005 among Arkansas Medicaid and HealthCore enrollees: Results from the TROUP study.
      PRR programs restrict patients who visit multiple pharmacies over a threshold number within a specific length of time (“pharmacy shoppers”) to one designated pharmacy and/or one designated prescriber.
      Patient Review & Restriction Programs: Lessons learned from state Medicaid programs.
      However, no consensus exists regarding the threshold number of pharmacies or the specified length of time that define pharmacy shopping behavior. For example, the PRR program in Kentucky uses ≥3 pharmacies in 2 consecutive 180-day periods,
      Kentucky Medicaid Provider Resources: Lock-in program update.
      Washington uses ≥4 pharmacies in any 90 days,
      • Washington Administrative Code
      Patient review and restriction criteria.
      and Michigan uses ≥3 pharmacies in a quarter.
      • Michigan Department of Community Health
      Inconsistent definitions of pharmacy shopping are seen in both research and practice.
      • Cepeda M.S.
      • Fife D.
      • Chow W.
      • Mastrogiovanni G.
      • Henderson S.C.
      Opioid shopping behavior: How often, how soon, which drugs, and what payment method.
      • Peirce G.L.
      • Smith M.J.
      • Abate M.A.
      • Halverson J.
      Doctor and pharmacy shopping for controlled substances.
      A consistent, evidence-based definition of high-risk prescription opioid use behavior will help these programs better target patients at risk and, it is hoped, reduce the incidence of unintentional opioid overdose.
      Use of pharmacy shopping as an indicator of opioid misuse or abuse may misidentify patients who have justifiable reasons for using multiple pharmacies. Changes in a patient's insurance coverage, need for medications while traveling, and/or moving to a new residence may be among the reasons a patient might visit multiple pharmacies. Overlapping prescriptions are another mechanism by which misuse of prescription opioids may occur. Including an additional indicator of the criteria used to identify opioid misuse or abuse that captures overlapping prescriptions might help to improve the accuracy of identifying those with higher risk of misuse or abuse of opioids.
      We present an analysis of alternative approaches for identification of prescription opioid users at high risk of opioid overdose. We compare the performance of different definitions of pharmacy shopping and identify a new measure for overlapping opioid prescriptions using a large Medicaid database to identify the useful indicators of high risk of overdose.

      Materials and Methods

      This project was exempted from review by the University of California Davis Institutional Review Board because it involved only secondary analysis of deidentified data.

      Data Source

      We analyzed data from the MarketScan Medicaid 2008–2010 data sets. This commercially available administrative claims data set includes deidentified information on Medicaid enrollees from multiple anonymous states, including demographics (age, race, and gender), Medicaid enrollment duration, diagnoses, and health care utilization (ie, prescription drugs, hospital admissions, emergency department visits, and outpatient care).
      The study population for this analysis was limited to a cohort of Medicaid enrollees aged 18 to 64 years who were long-term opioid users. We used an episode-based definition of use whereby the patient was provided an opioid for 90 days or longer on a continuous basis between January 2008 and December 2010. In addition, the patient had to have received at least 3 opioid prescriptions during the continuous use period with no gaps greater than 31 days between the dispensing date of a prescription and the end date of the previous prescription. This threshold of 90 days or longer has been previously utilized to identify long-term opioid users.
      • Von Korff M.
      • Saunders K.
      • Thomas Ray G.
      • Boudreau D.
      • Campbell C.
      • Merrill J.
      • Sullivan M.D.
      • Rutter C.M.
      • Silverberg M.J.
      • Banta-Green C.
      • Weisner C.
      De facto long-term opioid therapy for noncancer pain.
      A threshold of at least 3 opioid prescriptions over the 90-day period was utilized because the preponderance of prescriptions were acute or episodic fills or refills covering 30 days of supply or less. To focus on outpatient treatment of chronic noncancer pain, we excluded individuals with a history of cancer, except nonmelanoma skin cancer
      • Dunn K.M.
      • Saunders K.W.
      • Rutter C.M.
      • Banta-Green C.J.
      • Merrill J.O.
      • Sullivan M.D.
      • Weisner C.M.
      • Silverberg M.J.
      • Campbell C.I.
      • Psaty B.M.
      • Von Korff M.
      Opioid prescriptions for chronic pain and overdose: A cohort study.
      (International Classification of Diseases, 9th Revision, Clinical Modification [ICD-9-CM] neoplasms 140-239.2) and patients residing in long-term care facilities.

      Opioid Schedule and Morphine Equivalent Dose (MED)

      We included opioid analgesics listed in Schedules II–IV. The lower scheduled drugs have a higher potential for abuse and/or higher risk of dependence. Consistent with previous studies,
      • Bohnert A.S.
      • Valenstein M.
      • Bair M.J.
      • Ganoczy D.
      • McCarthy J.F.
      • Ilgen M.A.
      • Blow F.C.
      Association between opioid prescribing patterns and opioid overdose-related deaths.
      • Dunn K.M.
      • Saunders K.W.
      • Rutter C.M.
      • Banta-Green C.J.
      • Merrill J.O.
      • Sullivan M.D.
      • Weisner C.M.
      • Silverberg M.J.
      • Campbell C.I.
      • Psaty B.M.
      • Von Korff M.
      Opioid prescriptions for chronic pain and overdose: A cohort study.
      • Gomes T.
      • Mamdani M.M.
      • Dhalla I.A.
      • Paterson J.M.
      • Juurlink D.N.
      Opioid dose and drug-related mortality in patients with nonmalignant pain.
      • Sullivan M.D.
      • Edlund M.J.
      • Fan M.Y.
      • Devries A.
      • Brennan Braden J.
      • Martin B.C.
      Trends in use of opioids for non-cancer pain conditions 2000-2005 in commercial and Medicaid insurance plans: The TROUP study.
      • Von Korff M.
      • Saunders K.
      • Thomas Ray G.
      • Boudreau D.
      • Campbell C.
      • Merrill J.
      • Sullivan M.D.
      • Rutter C.M.
      • Silverberg M.J.
      • Banta-Green C.
      • Weisner C.
      De facto long-term opioid therapy for noncancer pain.
      we compared the effects of different opioids using the MED calculated by multiplying the strength by the quantity of the prescription and then adjusting this dose using a conversion factor (expressed in milligrams of morphine equivalents). The conversion factors were based on Von Korff's CONSORT study.
      • Von Korff M.
      • Saunders K.
      • Thomas Ray G.
      • Boudreau D.
      • Campbell C.
      • Merrill J.
      • Sullivan M.D.
      • Rutter C.M.
      • Silverberg M.J.
      • Banta-Green C.
      • Weisner C.
      De facto long-term opioid therapy for noncancer pain.

      Opioid-Related Overdose Events

      Opioid-related overdoses were identified using inpatient and outpatient claims data based on the ICD-9-CM codes described by Dunn et al.
      • Dunn K.M.
      • Saunders K.W.
      • Rutter C.M.
      • Banta-Green C.J.
      • Merrill J.O.
      • Sullivan M.D.
      • Weisner C.M.
      • Silverberg M.J.
      • Campbell C.I.
      • Psaty B.M.
      • Von Korff M.
      Opioid prescriptions for chronic pain and overdose: A cohort study.
      We excluded ICD-10 codes (not available in the study data set) and ICD-9-CM codes E935.0 (adverse effects of heroin). To focus on unintentional overdose events, we excluded episodes of suicidal poisoning (E950.0); poisoning by analgesics, antipyretics, and antirheumatics, undetermined whether accidentally or purposely inflicted (E980.0); and opioid drug addiction and abuse (304.X and 305.X, respectively).
      We defined “definite cases of overdose” as claims with ICD-9-CM codes indicating opioid-related poisoning (965.0, 965.00, 965.02, and 965.09) or accidental poisoning (E850.1 and E850.2). We defined “probable cases of overdose” as claims with ICD-9-CM codes indicating adverse effects of opioid use (E935.1 and E935.2) in addition to at least 1 ICD-9-CM code indicating overdose-related symptoms (such as acute respiratory failure and apnea) on the same day (see Dunn et al for full list of overdose-related symptoms ICD-9-CM codes
      • Dunn K.M.
      • Saunders K.W.
      • Rutter C.M.
      • Banta-Green C.J.
      • Merrill J.O.
      • Sullivan M.D.
      • Weisner C.M.
      • Silverberg M.J.
      • Campbell C.I.
      • Psaty B.M.
      • Von Korff M.
      Opioid prescriptions for chronic pain and overdose: A cohort study.
      ). Both definite and probable cases were included in the analysis.
      • Dunn K.M.
      • Saunders K.W.
      • Rutter C.M.
      • Banta-Green C.J.
      • Merrill J.O.
      • Sullivan M.D.
      • Weisner C.M.
      • Silverberg M.J.
      • Campbell C.I.
      • Psaty B.M.
      • Von Korff M.
      Opioid prescriptions for chronic pain and overdose: A cohort study.
      Patients with overdose were defined as those who had at least 1 overdose event during the Medicaid enrollment period. In the regression analysis, if an individual had multiple overdose events during his or her continuous Medicaid enrollment period, only the initial overdose was included.

      Pharmacy Shopping Definition

      To evaluate the best predictive pharmacy shopping measure of overdose risk for Medicaid enrollees, we examined combinations of 2 variables: time intervals (90 days, 180 days, and 1 year) and number of pharmacies visited during the time interval (≥3, ≥4, and ≥5 pharmacies). The variables were based on definitions of pharmacy shopping used by PRR programs in different states and research studies.
      Kentucky Medicaid Provider Resources: Lock-in program update.
      • Michigan Department of Community Health
      • Washington Administrative Code
      Patient review and restriction criteria.
      Nine possible definitions for pharmacy shopping emerged by combining 1 of 3 time intervals with 1 of 3 thresholds on number of pharmacies: ≥3 pharmacies in a 1-year period (ie, obtaining opioid prescriptions from ≥3 pharmacies over 1-year period), ≥4 pharmacies in a 1-year period, ≥5 pharmacies in a 1-year period, ≥3 pharmacies in any 180-day period, ≥4 pharmacies in any 180-day period, ≥5 pharmacies in any 180-day period, ≥3 pharmacies in any 90-day period, ≥4 pharmacies in any 90-day period, and ≥5 pharmacies in any 90-day period. For each definition, we calculated the number of pharmacy shoppers (ie, patients who met the criteria) and the number of patients who overdosed in both shopper and nonshopper groups. If a patient's overdose occurred before he or she met the definition, he or she was considered as an overdose case in the nonshopper group.
      We calculated the diagnostic odds ratio (DOR) for each definition, considering patients with an overdose event as “disease”-positive subjects. The DOR is defined as the ratio of the odds of testing positive if the subject has a disease relative to the odds of testing positive if the subject does not have the disease.
      • Glas A.S.
      • Lijmer J.G.
      • Prins M.H.
      • Bonsel G.J.
      • Bossuyt P.M.
      The diagnostic odds ratio: A single indicator of test performance.
      We compared the different definitions of pharmacy shopping as a “positive test” and considered the presence of an opioid-related overdose event as disease positive, using the DOR to assess each definition's efficacy to identify high risk of overdose among long-term opioid users. Higher DOR values indicate better test prediction.

      Overlapping Prescription Definition

      Overlapping prescriptions were defined as 2 prescriptions of the same drug type that overlapped by ≥25% of the days prescribed, with the initial dispensed prescription having a supply time of 5 days or longer. The 25% cutoff point was derived from the opinion of an expert panel and the work of Paulozzi et al.
      • Goldberg G.A.
      • Kim S.S.
      • Seifeldin R.
      • Haberman M.
      • Robinson Jr., D.
      Identifying suboptimal management of persistent pain from integrated claims data: A feasibility study.
      • Paulozzi L.J.
      • Kilbourne E.M.
      • Shah N.G.
      • Nolte K.B.
      • Desai H.A.
      • Landen M.G.
      • Harvey W.
      • Loring L.D.
      A history of being prescribed controlled substances and risk of drug overdose death.
      We excluded prescriptions when the initial dispensed medication had a supply time of 5 days or less so as to exclude opioids provided by emergency room providers who may have provided bridge-dosing until a patient was seen by the primary care physician.
      • Wilsey B.L.
      • Fishman S.M.
      • Crandall M.
      • Casamalhuapa C.
      • Bertakis K.D.
      A qualitative study of the barriers to chronic pain management in the ED.
      We restricted this definition to prescriptions for the same opioid type (ie, the same active ingredient, eg, hydrocodone) because patients could have clinically appropriate use of ≥2 different types of opioids. For example, a short-acting opioid could be added for breakthrough pain to a long-acting opioid regimen.

      Combination Measure

      To examine the effects of combining both pharmacy shopping and overlapping prescriptions indicators of assessment of overdose risk, we divided the study population into 4 subgroups based on whether they met the definitions of pharmacy shopping and/or overlapping prescriptions: 1) both pharmacy shopping and overlapping prescriptions, 2) overlapping prescriptions but no pharmacy shopping, 3) pharmacy shopping but no overlapping prescriptions, and 4) neither condition. For each subgroup, we adopted the pharmacy shopping definition found to have the highest DOR among the 9 definitions, as this group had the highest risk of overdosing from opioids. The demographic characteristics, overdose risk, and prescription fill patterns (in terms of prescription frequency, dose level, and drug type by the Drug Enforcement Administration classification) were calculated for each of the 4 subgroups.

      Statistical Analysis

      All data analyses were performed using SAS, version 9.3 (SAS Institute, Cary, NC). Cross-subgroup comparisons were based on Pearson's chi-square test for categorical variables and F test in analysis of variance for continuous variables. We used a Cox proportional hazards model to examine separately the individual association of pharmacy shopping behavior and overlapping prescriptions with an overdose event after controlling for dose level, demographic characteristics, and preexisting history of addictive behaviors. Pharmacy shopping and overlapping prescriptions were included as independent covariates, together with adjustment for age, gender, race, average daily MED, history of depression (based on ICD-9-CM codes), history of alcohol abuse (based on ICD-9-CM codes), and concurrent sedative/hypnotic drug use. In particular, average daily MED was treated as a time-varying covariate. Patient exposure to opioids was classified at 1 of 4 average daily MED levels (0–20, 20–50, 50–100, and >100 mg/d).
      • Dunn K.M.
      • Saunders K.W.
      • Rutter C.M.
      • Banta-Green C.J.
      • Merrill J.O.
      • Sullivan M.D.
      • Weisner C.M.
      • Silverberg M.J.
      • Campbell C.I.
      • Psaty B.M.
      • Von Korff M.
      Opioid prescriptions for chronic pain and overdose: A cohort study.

      Results

      Baseline characteristics of the 90,010 Medicaid beneficiaries in the MarketScan data who met the inclusion criteria are presented in Table 1. The cohort had a mean of 31.3 months of continuous Medicaid enrollment; 70.3% were female, 98.1% were 18 years of age and older, and 67.7% were white. The prevalence of diagnosed depression and alcohol abuse, based on ICD-9-CM codes, was 10.6% and 3.0%, respectively. More than 3 million opioid prescriptions (an average of 36.9 opioid prescriptions per person) were dispensed to the study cohort between 2008 and 2010, with the majority of prescriptions being Schedule III or IV (65.7%).
      Table 1Baseline Characteristics of Medicaid Study Population From MarketScan Medicaid Data 2008–2010
      CharacteristicValue
      N90,010
      Mean months enrolled in Medicaid31.3
      Female, %70.3
      Age, %
       12–171.9
       18–2914.6
       30–4432.0
       ≥4551.5
      Race, %
       White67.7
       Black24.7
       Hispanic1.0
       Other6.6
      Depression diagnosis, %10.6
      Alcohol abuse, %3.01
      Opioid prescriptions, n3,320,489
      Opioid prescription type, %
       Schedule II, long-acting15.1
       Schedule II, short-acting19.2
       Schedule III or IV65.7
      Overdose events, n1,237
      Overdosed patients,
      Patients who had at least 1 opioid-related overdose encounter.
      n
      1,181
      Patients who had at least 1 opioid-related overdose encounter.
      Before comparing the definitions of pharmacy shopping, we looked at the distribution of patients and the percentage of overdose classified by the peak number of pharmacies visited (1, 2, 3, 4, and ≥5 pharmacies) in a given period, as shown in Fig 1. Over a 1-year period, the numbers (percentages) of patients using 3, 4, and ≥5 pharmacies were 15,901 (16.8%), 9,766 (10.8%), and 19,409 (20.6%), respectively. For a time interval of 180 consecutive days, patients using 3, 4, and ≥5 pharmacies changed to 16,806 (18.7%), 9,012 (10.0%), and 12,339 (13.7%), respectively. For 90 consecutive days, the corresponding numbers (percentages) were 15,647 (17.3%), 7,564 (8.4%), and 5,519 (6.1%).
      Figure thumbnail gr1
      Figure 1Patients and percentage of overdosed patients by peak number of pharmacies visited are shown. The study population is divided into subgroups according to peak number of pharmacies that a patient visited in the specified interval. Bars (on left y-axis) represent the number of patients in each subgroup. Dots (on right y-axis) represent the percentage of overdosed patients in each subgroup. The 3-bar set and corresponding dot indicate 3 different lengths of time (ie, 1 year, 180 days, and 90 days) in which the number of pharmacies were counted.
      In all 3 time intervals (90 days, 180 days, and 1 year), the percentage of patients with opioid overdose events increased as the peak number of pharmacies increased. The most dramatic increase was seen in the subgroup using ≥4 pharmacies in a 90-day interval. That is, 3.4 and 5.4% of patients who visited ≥4 and ≥5 pharmacies, respectively, in any 90 consecutive days had at least 1 opioid-related overdose in the study period, compared with .4 to 1.5% among those visiting 1 to 3 pharmacies. In the 180-day and 1-year intervals, we found a smaller difference in the proportion of patients with overdose events between high and low peak numbers of pharmacies.
      A comparison of the 9 different definitions of pharmacy shopping is listed in Table 2. Corresponding to the information provided in Fig 1, nearly half (45,076 [50.1%]) of the study cohort were categorized as pharmacy shoppers when using the definition of ≥3 pharmacies in any 1-year interval, whereas only 13,083 (14.5%) were classified as shoppers when using the definition of ≥4 pharmacies over any 90 consecutive days. Table 2 also shows that the more restrictive definition was less sensitive in identifying overdose events—less than half (47.0%) of patients with overdose events were included in the pharmacy shopping group defined as “≥4 pharmacies in a 90-day period,” compared with nearly 70% (69.9%) when defined as “≥3 pharmacies in a 1-year period.” The DOR for the criterion of “≥4 pharmacies in a 90-day period” had the highest value at 5.40.
      Table 2Comparison of Different Pharmacy Shopping Definitions
      Pharmacy Shopping CriteriaEligible Patients
      “Eligible” means exhibiting pharmacy shopping behavior according to the specified definition.
      , n
      Overdosed Patients (% Among Eligible
      “Eligible” means exhibiting pharmacy shopping behavior according to the specified definition.
      )
      % Overdosed Identified
      Defined as percentage of overdosed patients in the eligible population over overdosed patients in the entire study population.
      DOR
      1-year setting, ≥3 pharmacies45,076825 (1.83)69.862.33
      1-year setting, ≥4 pharmacies29,175620 (2.13)52.502.33
      1-year setting, ≥5 pharmacies19,409433 (2.23)36.662.13
      180-day setting, ≥3 pharmacies38,157818 (2.14)69.263.11
      180-day setting, ≥4 pharmacies21,351599 (2.81)50.723.38
      180-day setting, ≥5 pharmacies12,339390 (3.16)33.023.17
      90-day setting, ≥3 pharmacies28,730788 (2.74)66.724.37
      90-day setting, ≥4 pharmacies13,083555 (4.24)46.995.40
      90-day setting, ≥5 pharmacies5,519297 (5.38)25.155.32
      “Eligible” means exhibiting pharmacy shopping behavior according to the specified definition.
      Defined as percentage of overdosed patients in the eligible population over overdosed patients in the entire study population.
      To identify the group of opioid prescription users at the highest risk of an overdose event (who may be most likely to benefit from a PRR program or other intervention), we examined the effect of combining the pharmacy shopping indicator with the overlapping prescription indicator. We chose the definition of pharmacy shopping with the highest DOR (≥4 pharmacies in 90-day interval) and combined it with the indicator of overlapping opioid prescriptions. As shown in Table 3, among the 90,010 long-term users, 6,024 (6.7%) had both pharmacy shopping behavior and overlapping prescriptions; 3,885 (4.3%) exhibited no pharmacy shopping behavior but did have overlapping prescriptions; 7,059 (7.8%) exhibited shopping behavior but no overlapping prescriptions; and the rest (81.1%) had neither shopping behavior nor overlapping prescriptions.
      Table 3Demographics, Overdose Rates, and Drug Use Patterns of Patients With Different Pharmacy Shopping Characteristics (95% CI)
      CharacteristicNo Shopping, No Overlapping RXNo Shopping, Overlapping RXShopping, No Overlapping RXShopping, Overlapping RX
      Demographics
       n73,0423,8857,0596,024
       Mean age
      P < .001. P values are from chi-square test for categorical variables and from F test for continuous variables.
      44.145.638.042.0
       Male,
      P < .001. P values are from chi-square test for categorical variables and from F test for continuous variables.
      %
      28.232.126.831.2
       Depression diagnosis,
      P < .001. P values are from chi-square test for categorical variables and from F test for continuous variables.
      %
      9.011.416.818.0
       History of alcohol abuse,
      P < .001. P values are from chi-square test for categorical variables and from F test for continuous variables.
      %
      2.42.85.15.6
      Overdosed patients
       n461165188367
       %
      P < .001. P values are from chi-square test for categorical variables and from F test for continuous variables.
      .634.252.666.09
      Overdose incidence
      Only the initial overdose events were counted if multiple overdose events occurred to the same patient. Time after an initial overdose event was not counted into the total person-years.
       Total person-years, n108,0428,91117,53913,934
       Incidence rate, per 1,000 person-years
      P < .001. P values are from chi-square test for categorical variables and from F test for continuous variables.
      4.27 (3.89–4.67)18.51 (15.80–21.57)10.72 (9.24–12.37)26.34 (23.71–29.18)
      Drug use pattern
       Monthly prescriptions
      P < .001. P values are from chi-square test for categorical variables and from F test for continuous variables.
      1.17 (1.14–1.20)2.41 (2.23–2.59)1.33 (1.17–1.49)2.62 (2.52–2.72)
       Average dose, MME/d
      P < .001. P values are from chi-square test for categorical variables and from F test for continuous variables.
      40.2 (40.0–40.4)100.7 (97.0–104.4)53.9 (52.3–55.4)89.2 (87.1–91.4)
       Dose level distribution (MME/d),
      P < .001. P values are from chi-square test for categorical variables and from F test for continuous variables.
      %
      0–2035.414.523.415.2
      20–5044.536.748.638.4
      50–10013.621.317.622.1
      ≥1006.527.610.424.4
       Distribution of opioid prescription type,
      P < .001. P values are from chi-square test for categorical variables and from F test for continuous variables.
      %
      Schedule III or IV69.248.157.045.9
      Schedule II, short-acting17.428.223.831.2
      Schedule II, long-acting13.423.719.222.9
      Abbreviations: RX, opioid prescriptions; MME, milligrams of morphine equivalents.
      P < .001. P values are from chi-square test for categorical variables and from F test for continuous variables.
      Only the initial overdose events were counted if multiple overdose events occurred to the same patient. Time after an initial overdose event was not counted into the total person-years.
      We examined the association between patient characteristics and the 2 measures of pharmacy shopping behavior and overlapping prescriptions. There was no clear trend across the 4 categories in terms of mean age. Men were more frequent in the 2 categories with overlapping prescriptions (32.1% without shopping and 31.2% with shopping) than the 2 without overlapping prescriptions (28.2% without shopping and 26.8% with shopping). Patients classified as pharmacy shopping had a higher prevalence of depression (16.8% without overlapping prescriptions and 18.0% with overlapping prescriptions) and alcohol abuse (5.1% without overlapping prescriptions and 5.6% with overlapping prescriptions).
      About 40% of overdosed patients (461 of 1,181) were in the subgroup of patients without any pharmacy shopping behavior or overlapping prescriptions, whereas the remaining 60% (720 of 1,181) involved either pharmacy shopping or overlapping prescriptions, or both. The percentage of overdosed patients in the group with both shopping behavior and overlapping prescriptions was almost 10 times higher (6.09%/.64% = 9.52) than the group with neither condition. Patients who had overlapping prescriptions only, without pharmacy shopping behavior, still had a higher percentage of overdoses (4.25%) than those who had pharmacy shopping behavior but no overlapping prescriptions (2.66%).
      The incidence rates of overdose (Table 3) reflect similar differences. The group having both pharmacy shopping and overlapping prescriptions was about 6 times more likely to overdose compared with the group with neither condition (26.34 [95% confidence interval (CI) = 23.71–29.18] vs 4.27 [95% CI = 3.89–4.67] per 1,000 person-years); patients with overlapping prescriptions but no shopping behavior had a higher incidence rate (18.51 [95% CI = 15.80–21.57] per 1,000 person-years) than patients with shopping behavior but no overlapping prescriptions (10.72 [95% CI = 9.24–12.37] per 1,000 person-years).
      Patients who had overlapping prescriptions but were not considered pharmacy shoppers, defined as visiting ≥4 pharmacies in any 90-day interval, had the highest average monthly number of prescriptions (2.41 [95% CI = 2.23–2.59] prescriptions per month) and average daily MED dose (100.7 [95% CI = 97.0–104.4] mg MED/d). In comparison, patients who were pharmacy shoppers without overlapping prescriptions only had a moderate increase in the average number of prescriptions and average daily dose (1.33 [95% CI = 1.17–1.49] per month and 53.9 [95% CI = 52.3–55.4] mg MED/d) compared with the group without pharmacy shopping or overlapping prescriptions (1.17 [95% CI = 1.14–1.20] per month and 40.2 [95% CI = 40.0–40.4] mg MED/d) (Table 3). Patients who had overlapping prescriptions, without simultaneous substantiation of pharmacy shopping, were more likely to use high doses (P < .001) and Schedule II opioids (P < .001), especially long-acting formulations, compared with those who did not have overlapping prescriptions.
      Findings from the Cox proportional hazards model (Table 4) show that the adjusted risk of overdose events was 1.80 (95% CI = 1.54–2.10) times more likely in patients who visited ≥4 pharmacies relative to those who did not. Having overlapping prescriptions was associated with an almost threefold (2.96, 95% CI = 2.45–3.68) increase in the adjusted risk of an overdose event compared with not having overlapping prescriptions. These findings indicate that individually, pharmacy shopping and overlapping prescriptions were associated with increased overdose risk after adjusting for MED, concurrent sedative/hypnotic use, and history of alcohol abuse or depression. Including an interaction term for the measures of pharmacy shopping and overlapping prescriptions was not significant, indicating that the effect of the measures was additive.
      Table 4Hazard Ratios for Overdose Risk, Including Indicators of Pharmacy Shopping and Overlapping Prescriptions, From Cox Proportional Model
      CharacteristicHazard Ratio
      Pharmacy shopping (defined as ≥4 pharmacies visited within any 90-day period) and overlapping prescriptions are independent covariates along with covariates that control for a time-varying opioid dose level, demographic characteristics, and history of other addictive behaviors as listed in above table.
      95% CIP Value
      Opioid dose (mg/d)
       1–<201.00
       20–<501.611.24–2.08.0004
       50–1003.062.33–4.02<.0001
       ≥1004.023.07–5.26<.0001
      Gender
       Female1.00
       Male1.02.87–1.18.8444
      Age (y)
       12–17.21.03–1.53.1235
       18–291.00
       30–44.94.74–1.19.593
       ≥45.88.70–1.11.2875
      Race
       White1.00
       Black.60.48–.74<.0001
       Hispanic1.09.57–2.11.7959
       Other1.13.86–1.48.377
      Concurrent sedative/hypnotic use2.511.95–3.20<.0001
      History of alcohol abuse3.072.09–4.50<.0001
      History of depression diagnosis2.902.20–3.82<.0001
      Pharmacy shopping1.801.54–2.10<.0001
      Overlapping prescriptions2.962.45–3.68<.0001
      Pharmacy shopping (defined as ≥4 pharmacies visited within any 90-day period) and overlapping prescriptions are independent covariates along with covariates that control for a time-varying opioid dose level, demographic characteristics, and history of other addictive behaviors as listed in above table.

      Discussion

      Pharmacy shopping is a mechanism for obtaining more opioid prescription drugs than medically necessary.
      • Peirce G.L.
      • Smith M.J.
      • Abate M.A.
      • Halverson J.
      Doctor and pharmacy shopping for controlled substances.
      There have been several studies on the association between pharmacy shopping and prescription drug misuse and abuse.
      • Buurma H.
      • Bouvy M.L.
      • De Smet P.A.
      • Floor-Schreudering A.
      • Leufkens H.G.
      • Egberts A.C.
      Prevalence and determinants of pharmacy shopping behaviour.
      • Katz N.
      • Panas L.
      • Kim M.
      • Audet A.D.
      • Bilansky A.
      • Eadie J.
      • Kreiner P.
      • Paillard F.C.
      • Thomas C.
      • Carrow G.
      Usefulness of prescription monitoring programs for surveillance—analysis of Schedule II opioid prescription data in Massachusetts, 1996-2006.
      • Peirce G.L.
      • Smith M.J.
      • Abate M.A.
      • Halverson J.
      Doctor and pharmacy shopping for controlled substances.
      Peirce et al compared the pharmacy shopping behavior between deceased (from drug-related causes) and living subjects enrolled in a state-run prescription-monitoring program and found that 17.5% of decedents visited ≥4 pharmacies prior to death, compared to 1.3% for living subjects.
      Policy Impact: Prescription painkiller overdoses. Centers for Disease Control and Prevention.
      Our study supports the predictive value of pharmacy shopping for risk of opioid overdose, showing that the subgroups using higher peak numbers of pharmacies consistently had higher proportions of patients with ≥1 overdose event.
      In most states with a prescription-monitoring program, the number of pharmacies visited within a certain period is used to screen patients who are at risk of overuse/misuse of prescription opioids. However, we found no consistency regarding the appropriate threshold number of pharmacies or the specified time interval for defining the optimal measure to evaluate pharmacy shopping behavior as a risk factor. To our knowledge, this study is the first attempt to compare the performance of different pharmacy shopping measures in predicting prescription opioid overdose events. We found that the 90-day interval and the threshold number of ≥4 pharmacies had the highest DOR of the candidate numbers of pharmacies and time intervals. However, about half of the patients with ≥1 overdose events were still missed when this pharmacy shopping criterion was used alone.
      The present study also examined history of overlapping prescriptions as an additional risk factor for opioid overdose. Evaluating overdose risk using this measure in the context of pharmacy shopping yielded some novel findings. We found that having overlapping prescriptions was associated with a high level of daily opioid MED and more frequent opioid prescriptions, even in the absence of pharmacy shopping. Having overlapping prescriptions is a useful adjunct to assess risk of opioid overdose. Compared with patients who had overlapping prescriptions, patients who exhibited pharmacy shopping but had no overlapping prescriptions had a significantly lower opioid MED and lower risk of overdose.
      The present study has several limitations. For one thing, many pharmacies now store smaller quantities of opioids to dispense as a result of the potential for burglary and theft of controlled substances. This has resulted in an increasing proportion of legitimate chronic pain patients who are unable to fill their prescriptions. In one study, 18% of patients were unable to fill prescriptions on at least 1 occasion despite having previously received these medications from the same pharmacy on a previous occasion.
      • Gleason R.M.
      • Kirsh K.L.
      • Passik S.D.
      • Chambers J.L.
      This may have increased the number of pharmacies that a patient may have had to visit in order to fill the prescription completely. On the other hand, an unknown number of pharmacy visits may have been overlooked in our analysis because of licit activity in which the patient paid cash for the prescription. Alternatively, there may have been illicit activity in which diverted prescription opioids were ingested. Therefore, both lawful and black market activities may have limited our ability to measure the number of multiple pharmacies and overlapping prescriptions because of factors beyond our control.
      There are other limitations due to opioid diversion besides measurements of overlapping prescriptions and the use of multiple pharmacies. We focused on opioid overdose events as the key outcome of interest. However, when prescription opioids are diverted for sale to others, overdose events may not be limited to the individual who filled the prescription. A previous study that interviewed drug-involved club- and street-based populations suggested that Medicaid recipients are an important source of controlled drugs in the drug-diversion market.
      • Inciardi J.A.
      • Surratt H.L.
      • Kurtz S.P.
      • Cicero T.J.
      Mechanisms of prescription drug diversion among drug-involved club- and street-based populations.
      Our findings may also underestimate overdose because some episodes involving minor cardiorespiratory embarrassment may have resolved spontaneously and not have come to medical attention or may have been misidentified as falls or syncopal episodes. Further, our estimates were subject to errors resulting from misclassification of diagnostic codes in the MarketScan data set.
      Our study was limited to Medicaid enrollees, who in some cases may have had obstacles to coordinated appointments, which may have led to duplicate and overlapping prescriptions through the provision of fragmented care.
      Office of Inspector General: Mandatory Managed Care.
      Similar research in commercially insured populations is needed to evaluate whether our findings are generalizable, because other insured patients may also benefit from the same monitoring of prescription opioid use. Prescription drug–monitoring programs are one way for prescribers to know their patients' controlled drug prescription histories and accurately assess overdose risk.
      • Edlund M.J.
      • Martin B.C.
      • Fan M.Y.
      • Devries A.
      • Braden J.B.
      • Sullivan M.D.
      Risks for opioid abuse and dependence among recipients of chronic opioid therapy: Results from the TROUP Study.
      Therefore, the potential of providing proactive reports to stakeholders (ie, providers and insurers) on the number of pharmacies visited and overlapping opioid prescriptions provided to patients in order to reduce overdose events deserves investigation.
      • Dormuth C.R.
      • Miller T.A.
      • Huang A.J.
      • Mamdani M.M.
      • Juurlink D.N.
      Canadian Drug Safety and Effectiveness Research Network
      Effect of a centralized prescription network on inappropriate prescriptions for opioid analgesics and benzodiazepines.
      • Wilsey B.L.
      • Prasad H.
      Real-time access to prescription drug monitoring databases.
      The simultaneous dispensing of benzodiazepines with opioids was not studied and is recognized as a limitation of the present study because polysubstance abuse has been found to be a significant predictor of drug overdose.
      • Kerr T.
      • Marsh D.
      • Li K.
      • Montaner J.
      • Wood E.
      Factors associated with methadone maintenance therapy use among a cohort of polysubstance using injection drug users in Vancouver.
      In addition, opioids and benzodiazepines are the most common combination found in overdose deaths, and this mixture may have increased risk for some of the patients in this study.
      • Jones C.M.
      • Mack K.A.
      • Paulozzi L.J.
      Pharmaceutical overdose deaths, United States, 2010.
      Finally, the data set lacked prescriber identification details, preventing the evaluation of another key indicator, doctor shopping, that some PRR programs use to identify opioid misuse or abuse. Consequently, we could not determine if patients sought different types of opioids (eg, morphine, oxycodone, or a fentanyl patch) from different providers. In addition, doctor shopping has been linked to opioid-related overdoses and deaths.
      • Michigan Department of Community Health
      However, using doctor shopping to identify opioid misuse may not have a high specificity. For instance, a study of controlled substances (including opioids) using West Virginia prescription drug–monitoring program data showed that 56% of pharmacy shoppers exhibited doctor shopping, whereas only about 20% of doctor shoppers exhibited pharmacy shopping.
      • Peirce G.L.
      • Smith M.J.
      • Abate M.A.
      • Halverson J.
      Doctor and pharmacy shopping for controlled substances.
      As pointed out by the investigators, an explanation for this finding was that a patient seeking a controlled substance for the purpose of misuse might visit multiple pharmacies to avoid suspicion, whereas a patient receiving legitimate prescriptions from multiple physicians might visit a small number of pharmacies as they do not have nonlegitimate reasons (eg, excessive dose, duplicate fills, or inappropriate drug combination) to hide. Reasons for patients to use multiple providers include 1) changing clinicians, 2) obtaining medications from a practitioner filling in for the patient's customary provider, or 3) appropriately receiving treatment from another practitioner (eg, dentist, emergency room doctor, or orthopedist).
      • Wilsey B.L.
      • Fishman S.M.
      • Gilson A.M.
      • Casamalhuapa C.
      • Baxi H.
      • Zhang H.
      • Li C.S.
      Profiling multiple provider prescribing of opioids, benzodiazepines, stimulants, and anorectics.
      These reasons are supported by the findings of a study using the California prescription drug–monitoring program data, which affirmed the program's policy to investigate only patients who visited more than 5 prescribers per year.
      • Wilsey B.L.
      • Fishman S.M.
      • Gilson A.M.
      • Casamalhuapa C.
      • Baxi H.
      • Lin T.C.
      • Li C.S.
      An analysis of the number of multiple prescribers for opioids utilizing data from the California Prescription Monitoring Program.
      In addition to legitimate patients being falsely flagged as potential abuser/misusers by the indicator of doctor shopping, true abusers or misusers might obtain prescriptions from a small number of “pill mills” to avoid being identified as doctor shoppers and then seek to fill the prescriptions in multiple pharmacies. Considering the number of pharmacies would improve specificity over using the doctor shopping indicator alone.
      Our findings suggest that including overlapping prescriptions in addition to multiple pharmacies as an eligibility criterion for oversight in state Medicaid PRR programs may help identify patients at higher risk of overdose events. For patients whose use of overlapping prescriptions and multiple pharmacies cannot be justified on medical grounds, PRR programs could restrict reimbursement for controlled prescription drugs to a single designated physician and a single designated pharmacy. This might improve the coordination of care and ensure appropriate access to opioids while potentially reducing the risk of overdose. No peer-reviewed studies have been published to date documenting the effect of PRR programs on the risk of overdose episodes. Confirmatory data on the effectiveness of these programs are essential given the severity of the problem and widespread use of these programs to attempt to mitigate it.

      Acknowledgments

      We thank Brian Manns for his contribution to the discussion of PRR program policy. We also thank Dr. Richard Puddy, Katie McLendon, and 3 anonymous reviewers for providing their valuable comments and suggestions on the manuscript.

      References

        • Bohnert A.S.
        • Valenstein M.
        • Bair M.J.
        • Ganoczy D.
        • McCarthy J.F.
        • Ilgen M.A.
        • Blow F.C.
        Association between opioid prescribing patterns and opioid overdose-related deaths.
        JAMA. 2011; 305: 1315-1321
        • Braden J.B.
        • Fan M.Y.
        • Edlund M.J.
        • Martin B.C.
        • DeVries A.
        • Sullivan M.D.
        Trends in use of opioids by noncancer pain type 2000-2005 among Arkansas Medicaid and HealthCore enrollees: Results from the TROUP study.
        J Pain. 2008; 9: 1026-1035
        • Buurma H.
        • Bouvy M.L.
        • De Smet P.A.
        • Floor-Schreudering A.
        • Leufkens H.G.
        • Egberts A.C.
        Prevalence and determinants of pharmacy shopping behaviour.
        J Clin Pharm Ther. 2008; 33: 17-23
        • Centers for Disease Control and Prevention
        Overdose deaths involving prescription opioids among Medicaid enrollees—Washington, 2004-2007.
        MMWR Morb Mortal Wkly Rep. 2009; 58: 1171-1175
        • Cepeda M.S.
        • Fife D.
        • Chow W.
        • Mastrogiovanni G.
        • Henderson S.C.
        Opioid shopping behavior: How often, how soon, which drugs, and what payment method.
        J Clin Pharmacol. 2013; 53: 112-117
        • Dormuth C.R.
        • Miller T.A.
        • Huang A.J.
        • Mamdani M.M.
        • Juurlink D.N.
        • Canadian Drug Safety and Effectiveness Research Network
        Effect of a centralized prescription network on inappropriate prescriptions for opioid analgesics and benzodiazepines.
        Can Med Assoc J. 2012; 184: E852-E856
      1. Drug Scheduling.
        (Available at:) (Accessed December 2, 2014)
        • Dunn K.M.
        • Saunders K.W.
        • Rutter C.M.
        • Banta-Green C.J.
        • Merrill J.O.
        • Sullivan M.D.
        • Weisner C.M.
        • Silverberg M.J.
        • Campbell C.I.
        • Psaty B.M.
        • Von Korff M.
        Opioid prescriptions for chronic pain and overdose: A cohort study.
        Ann Intern Med. 2010; 152: 85-92
        • Edlund M.J.
        • Martin B.C.
        • Fan M.Y.
        • Devries A.
        • Braden J.B.
        • Sullivan M.D.
        Risks for opioid abuse and dependence among recipients of chronic opioid therapy: Results from the TROUP Study.
        Drug Alcohol Depend. 2010; 112: 90-98
        • Glas A.S.
        • Lijmer J.G.
        • Prins M.H.
        • Bonsel G.J.
        • Bossuyt P.M.
        The diagnostic odds ratio: A single indicator of test performance.
        J Clin Epidemiol. 2003; 56: 1129-1135
        • Gleason R.M.
        • Kirsh K.L.
        • Passik S.D.
        • Chambers J.L.
        Current access to opioids—survey of chronic pain patients.
        (Available at:) (Accessed December 11, 2014)
        • Goldberg G.A.
        • Kim S.S.
        • Seifeldin R.
        • Haberman M.
        • Robinson Jr., D.
        Identifying suboptimal management of persistent pain from integrated claims data: A feasibility study.
        Manag Care. 2003; 12: 8-13
        • Gomes T.
        • Mamdani M.M.
        • Dhalla I.A.
        • Paterson J.M.
        • Juurlink D.N.
        Opioid dose and drug-related mortality in patients with nonmalignant pain.
        Arch Intern Med. 2011; 171: 686-691
        • Inciardi J.A.
        • Surratt H.L.
        • Kurtz S.P.
        • Cicero T.J.
        Mechanisms of prescription drug diversion among drug-involved club- and street-based populations.
        Pain Med. 2007; 8: 171-183
        • Jones C.M.
        • Mack K.A.
        • Paulozzi L.J.
        Pharmaceutical overdose deaths, United States, 2010.
        JAMA. 2013; 309: 657-659
        • Katz N.
        • Panas L.
        • Kim M.
        • Audet A.D.
        • Bilansky A.
        • Eadie J.
        • Kreiner P.
        • Paillard F.C.
        • Thomas C.
        • Carrow G.
        Usefulness of prescription monitoring programs for surveillance—analysis of Schedule II opioid prescription data in Massachusetts, 1996-2006.
        Pharmacoepidemiol Drug Saf. 2010; 19: 115-123
      2. Kentucky Medicaid Provider Resources: Lock-in program update.
        (Available at:) (Accessed November 14, 2012)
        • Kerr T.
        • Marsh D.
        • Li K.
        • Montaner J.
        • Wood E.
        Factors associated with methadone maintenance therapy use among a cohort of polysubstance using injection drug users in Vancouver.
        Drug Alcohol Depend. 2005; 80: 329-335
        • Michigan Department of Community Health
        Medicaid Provider Manual. Section 8: Beneficiary monitoring program.
        (Available at:) (Accessed January 24, 2013)
      3. Office of Inspector General: Mandatory Managed Care.
        (Available at:) (Accessed December 11, 2014)
      4. Opioids drive continued increase in drug overdose deaths.
        (Available at:) (Accessed April 9, 2013)
      5. Patient Review & Restriction Programs: Lessons learned from state Medicaid programs.
        (Available at:) (Accessed July 1, 2014)
        • Paulozzi L.J.
        • Kilbourne E.M.
        • Shah N.G.
        • Nolte K.B.
        • Desai H.A.
        • Landen M.G.
        • Harvey W.
        • Loring L.D.
        A history of being prescribed controlled substances and risk of drug overdose death.
        Pain Med. 2012; 13: 87-95
        • Peirce G.L.
        • Smith M.J.
        • Abate M.A.
        • Halverson J.
        Doctor and pharmacy shopping for controlled substances.
        Med Care. 2012; 50: 494-500
      6. Policy Impact: Prescription painkiller overdoses. Centers for Disease Control and Prevention.
        (Available at:) (Accessed January 29, 2013)
        • Sullivan M.D.
        • Edlund M.J.
        • Fan M.Y.
        • Devries A.
        • Brennan Braden J.
        • Martin B.C.
        Trends in use of opioids for non-cancer pain conditions 2000-2005 in commercial and Medicaid insurance plans: The TROUP study.
        Pain. 2008; 138: 440-449
        • Von Korff M.
        • Saunders K.
        • Thomas Ray G.
        • Boudreau D.
        • Campbell C.
        • Merrill J.
        • Sullivan M.D.
        • Rutter C.M.
        • Silverberg M.J.
        • Banta-Green C.
        • Weisner C.
        De facto long-term opioid therapy for noncancer pain.
        Clin J Pain. 2008; 24: 521-527
        • Warner M.
        • Chen L.H.
        • Makuc D.M.
        • Anderson R.N.
        • Minino A.M.
        Drug poisoning deaths in the United States, 1980-2008.
        NCHS Data Brief. 2011; : 1-8
        • Washington Administrative Code
        Patient review and restriction criteria.
        (Available at:) (Accessed January 24, 2013)
        • Wilsey B.L.
        • Fishman S.M.
        • Crandall M.
        • Casamalhuapa C.
        • Bertakis K.D.
        A qualitative study of the barriers to chronic pain management in the ED.
        Am J Emerg Med. 2008; 26: 255-263
        • Wilsey B.L.
        • Fishman S.M.
        • Gilson A.M.
        • Casamalhuapa C.
        • Baxi H.
        • Lin T.C.
        • Li C.S.
        An analysis of the number of multiple prescribers for opioids utilizing data from the California Prescription Monitoring Program.
        Pharmacoepidemiol Drug Saf. 2011; 20: 1262-1268
        • Wilsey B.L.
        • Fishman S.M.
        • Gilson A.M.
        • Casamalhuapa C.
        • Baxi H.
        • Zhang H.
        • Li C.S.
        Profiling multiple provider prescribing of opioids, benzodiazepines, stimulants, and anorectics.
        Drug Alcohol Depend. 2010; 112: 99-106
        • Wilsey B.L.
        • Prasad H.
        Real-time access to prescription drug monitoring databases.
        Can Med Assoc J. 2012; 184: 1767-1768