Advertisement

Revised American Pain Society Patient Outcome Questionnaire (APS-POQ-R) for Quality Improvement of Pain Management in Hospitalized Adults: Preliminary Psychometric Evaluation

      Abstract

      Quality improvement (QI) is a compilation of methods adapted from psychology, statistics, and operations research to identify factors that contribute to poor treatment outcomes and to design solutions for improvement. Valid and reliable measurement is essential to QI using rigorously developed and tested instruments. The purpose of this article is to describe the evolution of the American Pain Society Patient Outcome Questionnaire (APS-POQ) for QI purposes and present a revised version (R) including instrument psychometrics. An interdisciplinary task force of the APS used a step-wise, empiric approach to revise, test, and examine psychometric properties of the society's original POQ. The APS-POQ-R is designed for use in adult hospital pain management QI activities and measures 6 aspects of quality, including (1) pain severity and relief; (2) impact of pain on activity, sleep, and negative emotions; (3) side effects of treatment; (4) helpfulness of information about pain treatment; (5) ability to participate in pain treatment decisions; and (6) use of nonpharmacological strategies. Adult medical-surgical inpatients (n = 299) from 2 hospitals in different parts of the United States participated in this study. Results provide support for the internal consistency of the instrument subscales, construct validity and clinical feasibility.

      Perspective

      This article presents the initial psychometric properties of the APS-POQ-R for quality improvement purposes of hospitalized adult patients. Validation in additional groups of patients will be needed to demonstrate its generalizability.

      Key words

      The need to measure, compare, and improve the quality of pain management is increasingly sought by health care consumers, payers, and professionals alike. However, different definitions and measurement approaches are necessary for distinct purposes such as quality improvement (QI), public reporting (accountability), and research.
      • Donabedian A.
      Explorations in quality assessment and monitoring, Vol 1, The definition of quality and approaches to its assessment.
      Quality measurement has changed significantly in recent years with advances in epidemiology, outcomes research, and information technology.
      • Izakovic M.
      New trends in the management of inpatients in US hospitals-quality measurements and evidence-based medicine in practice.
      • Lohr K.N.
      Rating the strength of scientific evidence: Relevance for quality improvement programs.
      Although evidence to document the effectiveness of a continuous QI approach in pain management is still sparse, it continues to be a recommended strategy to measure and advance pain care processes and outcomes.
      • Gordon D.B.
      • Dahl J.L.
      • Miaskowski C.
      • McCarberg B.
      • Todd K.H.
      • Paice J.A.
      • Lipman A.G.
      • Bookbinder M.
      • Sanders S.H.
      • Turk D.
      • Carr D.B.
      American Pain Society recommendations for improving the quality of acute and cancer pain management.
      QI models including Deming,
      • Walton M.
      The Deming Management Method.
      Juran,
      • Juran D.
      Achieving sustained quantifiable results in an interdepartmental quality improvement project.
      Six Sigma,
      • Black K.
      • Revere L.
      Six Sigma arises from the ashes of TQM with a twist.
      and Lean thinking
      • Kim C.S.
      • Spahlinger D.A.
      • Kin J.M.
      • Billi J.E.
      Lean health care: What can hospitals learn from a world-class automaker.
      provide frameworks and a set of tools to develop initiatives to improve and evaluate the quality of pain management. These models have also been applied to understanding systems, processes of care and practice variations, and systematic evaluations to address patient safety. The purpose of QI data is to provide a better understanding of the extent and nature of the problem, motivation for change, and points for comparison after change have been implemented. The purpose of this article is to describe the evolution of the American Pain Society Patient Outcome Questionnaire (APS-POQ) and present a revised version (R) including instrument psychometrics for QI purposes (Fig 1).
      Figure thumbnail gr1a
      Figure 1Revised APS Patient Outcome Questionnaire (APS-POQ-R).
      Figure thumbnail gr1b
      Figure 1Revised APS Patient Outcome Questionnaire (APS-POQ-R).

      Quality Pain Management and Its Measurement

      High-quality pain management is defined as having several features.
      • Gordon D.B.
      • Dahl J.L.
      • Miaskowski C.
      • McCarberg B.
      • Todd K.H.
      • Paice J.A.
      • Lipman A.G.
      • Bookbinder M.
      • Sanders S.H.
      • Turk D.
      • Carr D.B.
      American Pain Society recommendations for improving the quality of acute and cancer pain management.
      These include appropriate ongoing assessment (eg, screening for the presence of pain, completion of a comprehensive initial assessment when pain is present, and frequent reassessments of patient responses to treatment); interdisciplinary, collaborative care planning that includes patient input; appropriate treatment that is efficacious, cost-conscious, culturally and developmentally appropriate, and safe; and access to specialty care as needed. This definition is consistent with characteristics of quality that encompass structure, process and outcomes, and is applicable to people in need of treatment for acute, cancer, and chronic non-cancer pain. Structure is defined as the physical and organizational properties of the setting where care is delivered. Processes include communication and practice patterns. Pertinent outcome variables include changes in pain severity and frequency, treatment or the disease process causing pain, emotional and physical function, quality of life, adverse effects of pain or pain treatment, and indicators of patient satisfaction.
      The definition and measurement of the quality of pain management for any purpose is difficult. The evaluation of pain care is particularly problematic due to the complex nature of the subjective experience involved and limited knowledge of the relationships among the structures, processes, and outcomes of pain management. The experience of pain is intertwined with its emotional impact (suffering), which can be equally or even more difficult to quantify and compare between individuals. Moreover, many factors affect the quality of care, including practice patterns (assessment and treatment behaviors), decision-making, policies, and patient outcomes.
      • Mechanic D.
      Improving the quality of health care in the United States of America: The need for a multi-level approach.
      Thus, the question of how one measures and improves the quality of care has been a topic of considerable debate and experimentation.
      • Berwick D.
      Disseminating innovations in health care.
      • Gallagher R.M.
      • Rowell P.A.
      Claiming the future of nursing through nursing-sensitive quality indicators.
      • Grol R.
      Improving the quality of medical care: Building bridges among professional pride, payer profit, and patient satisfaction.
      A long-standing criticism of QI has been the lack of scientific rigor in the measurement methodologies used to collect the data that raise questions regarding the reliability, validity, and integrity of the data obtained for QI outcomes. Contributing to these perceptions is the inability to appreciate the fundamental differences among measurement instruments necessary for QI, research, and performance (external accountability). QI data are collected to gain a better understanding of the extent and nature of clinical or systems issues and the motivation for change. These data establish, evaluate and reevaluate targets for improvements. QI processes and measures must be capable of capturing indicators of care, and should be constructed to allow ease of use by clinicians in routine care delivery, meaningful interpretations across repeated time points, and facilitate rapid change in care processes when appropriate. Measurement for QI does not require the same evidentiary rigor across populations as is usually demanded of more general research as it is not primarily intended to generate new knowledge of widely generalizable or universal value.
      • Berwick D.
      The science of improvement.
      However, it is important that measurement instruments for QI meet essential psychometric criteria while retaining feasibility of administration and clinical meaningfulness in its interpretation.
      • Gelinas C.
      • Loiselle C.G.
      • LeMay S.
      • Ranger M.
      • Bouchard E.
      • McCormack D.
      Theoretical, psychometric, and pragmatic issues in pain measurement.

      The APS-POQ

      In 1991, the APS

      American Pain Society Committee on Quality Assurance Standards: American Pain Society quality assurance standards for the relief of acute pain and cancer pain, in Bond MR, Charlton JE, Woolf CJ, (eds): Proceedings of the VIth World Congress on Pain. Seattle, WA, IASP Press, 1991, pp185-188

      published its first APS-POQ as part of quality assurance (QA) standards for the treatment of acute and cancer pain to assist health care organizations to explore patient experiences and outcomes. In 1995, the APS QA standards were revised and published as QI guidelines
      American Pain Society Quality of Care Committee
      Quality improvement guidelines for the treatment of acute pain and cancer pain.
      along with an updated APS-POQ. Updates at that time were based on published reports and clinical experience with the original APS-POQ and on previously validated epidemiologic instruments.
      • Daut R.
      • Cleeland C.
      • Flanery R.
      Development of the Wisconsin Brief Pain Inventory to assess pain in cancer and other diseases.
      • Ward S.E.
      • Goldberg N.
      • Miller-McCauley V.
      • Mueller C.
      • Nolan A.
      • Pawlik-Plank D.
      • Robbins A.
      • Stormoen D.
      • Weissman D.
      Patient related barriers to management of cancer pain.
      • Ware J.
      • Snyder M.
      • Wright W.
      • Davies A.
      Defining and measuring patient satisfaction with care.
      Initial measures included ratings of pain severity, pain interference on daily activities, patient satisfaction, perceived wait time for analgesics, patient beliefs, and clarity of instructions about analgesic use at time of hospital discharge.
      A systematic review of 20 QI studies of pain that utilized the 1995 APS-POQ involving 3527 patients with medical, surgical, and cancer diagnoses in inpatient settings suggested that a number of modifications were needed to the existing tool based on a revised set of core quality indicators to guide measurement of quality for improvement purposes.
      • Gordon D.B.
      • Pellino T.A.
      • Miaskowski C.
      • McNeill J.A.
      • Paice J.A.
      • Laferriere D.
      • Bookbinder M.
      A 10-year review of quality improvement in pain management: Recommendations for standardized outcome measures.
      Six core quality indicators were recommended to provide measurement of the processes and outcomes of pain management. These included categories for (1) use of numeric or descriptive rating scales for pain assessment, (2) documentation of pain intensity at frequent intervals, (3) treatment of pain by a route other than intramuscular; (4) administration of analgesics on a regular schedule, and when possible, use of a multimodal treatment regimen, (5) prevention and control of pain to a degree that facilitates function and quality of life, and (6) provision of adequate information so that patients are knowledgeable about pain management. These indicators require the collection of data from the patient as well as from the nursing and medical records.
      Several specific changes were recommended to the 1995 APS-POQ including the addition of items to quantify the amount of time patients spent in moderate to severe pain, adaptations of the functional interference and satisfaction items, and appraisal of whether the patients perceived that the clinicians involved them in their pain management and in discussing possible options for pain control. The original survey items measuring patient beliefs and perceived wait time were no longer considered to have meaningful relationships to other quality indicators. In 2005, the APS revised and expanded its 1995 guidelines for QI reiterating the need for multiple health care system level changes and how pain-related QI outcomes are measured.
      • Gordon D.B.
      • Dahl J.L.
      • Miaskowski C.
      • McCarberg B.
      • Todd K.H.
      • Paice J.A.
      • Lipman A.G.
      • Bookbinder M.
      • Sanders S.H.
      • Turk D.
      • Carr D.B.
      American Pain Society recommendations for improving the quality of acute and cancer pain management.
      Because formal construct validity testing had not been performed on the original APS-POQ, it became a priority to establish and confirm measurement domains in a new instrument. Although no specific item modifications to the existing POQ tool were recommended, the authors of this report decided that the evaluation of existing items and the construction of new ones should be based on the new QI guidelines by capturing the 5 aspects of (1) pain severity (pain), (2) interference with function (activities), (3) affective experiences (emotional), (4) side effects (safety), and (5) perceptions of care (satisfaction).
      This report presents the systematic processes used to revise the APS-POQ to capture the 5 patient reported aspects of the QI guideline document in a way that was consistent with the recommendations of the APS. We also present the testing procedures used to ensure the reliability and initial validity of the revised instrument. The specific aims for this quality improvement-based study were to (1) add and revise appropriate items relevant to the conduct of QI for pain care; (2) administer the revised instrument to a heterogeneous sample of hospitalized patients; (3) test for internal consistency reliability (Cronbach α), test for item grouping using factor analysis and conduct an initial validation by comparing differences among distinct patient populations within our sample; and (4) determine the relationships among items and subscales, and variables that may predict outcomes.
      Another basic assumption was that the revised instrument should be easy to administer, with understandable components and have a simple scoring system for the subscales to facilitate its use by health care providers in the evaluation of the quality of the pain care provided. If properly designed, the APS-POQ-R should be able to detect differences in outcomes among patient populations on specific items (eg, pain intensity levels) and subscales (eg, interference with function). It will also be possible to determine if nonpharmacological techniques play a role in increasing patients' participation in their care. By comparing the results obtained in different populations, health care professionals should be able to determine what patient characteristics affect the quality of pain care and then make adjustments to that care to improve patient satisfaction.

      Methods

      Development of the Revised APS-POQ (APS-POQ-R)

      Ten interdisciplinary members of the APS (5 nurses, 2 physicians, 2 psychologists, and a pharmacist) with experience in using the APS-POQ and expertise in pain management, QI and instrument development were invited to participate in a task force to revise the APS-POQ. Two distinct phases were undertaken to create the APS-POQ-R. First, items from the original APS-POQ were extensively discussed by the committee. By consensus, items perceived to have not been helpful in QI (eg, perceived wait time and patient beliefs) or poorly designed (eg, adequacy of instruction…) were eliminated. Items were added to address the recommendations put forth in a previous publication reviewing 10 years of experience with the original instrument.
      • Gordon D.B.
      • Pellino T.A.
      • Miaskowski C.
      • McNeill J.A.
      • Paice J.A.
      • Laferriere D.
      • Bookbinder M.
      A 10-year review of quality improvement in pain management: Recommendations for standardized outcome measures.
      The second phase involved administration of the revised instrument to patients in various clinical settings who were experiencing different types of acute and chronic pain, with psychometric testing of the instrument in that population.
      The principal determinations for retaining or deleting items were based on item-to-item intercorrelations, and item-to-subscale correlations by calculating the internal consistency reliability (Cronbach α), with item deletion. Once the final set of items had been determined, a principal component varimax factor rotation was performed to verify the structure. Initial validity testing was also conducted by assessing for significant differences among biologically appropriate subgroup cohorts. Group comparisons were performed using Student t tests and 1-way analyses of variance (ANOVAs) to detect differences among patient populations based on the primary type of pain (acute, chronic cancer-related pain, and chronic nonmalignant pain). Bivariate correlations with Pearson product-moment (r) were conducted to assess the magnitude of relationships between variables and subscales. Linear and logistic regressions were used to identify significant variables and subscale scores that were predictors of satisfaction.

      Phase 1

      An abbreviated process for developing outcomes measures for clinical trials
      • Turk D.C.
      • Dworkin R.H.
      • Burke L.B.
      • Gershon R.
      • Rothman
      • Scott J.
      • Allen R.R.
      • Atkinson J.H.
      • Chandler J.
      • Cleeland C.
      • Cowan P.
      • Dimitrova R.
      • Dionne R.
      • Farrar J.T.
      • Haythornthwaite J.A.
      • Hertz S.
      • Jadad A.R.
      • Jensen M.P.
      • Kellstein D.
      • Kerns R.D.
      • Manning D.C.
      • Martin S.
      • Max M.B.
      • McDermott M.P.
      • McGrath P.
      • Moulin D.E.
      • Nurmikko T.
      • Quessy S.
      • Raja S.
      • Rappaport B.A.
      • Rauschkolb C.
      • Robinson J.P.
      • Royal M.A.
      • Simon L.
      • Stauffer J.W.
      • Stucki G.
      • Tollett J.
      • von Stein T.
      • Wallace M.S.
      • Wernicke J.
      • White R.E.
      • Williams A.C.
      • Witter J.
      • Wyrwich K.W.
      Developing patient-reported outcome measures for pain clinical trials: IMMPACT recommendations.
      was used to guide and build consensus on an updated APS-POQ for field testing. This involved identifying the overall question and theoretical approach, establishing the target population, determining the format and scale properties and methods for analysis, locating gaps in construct (content domains), and developing instructions for instrument administration. Starting with the original 27-item APS-POQ, items that were ambiguous or not useful were removed and new items added to cover the appropriate domains. Although different scalar methods could have been used for some of the questions, the 0 to 10 NRS was chosen for all responses for consistency, evidence of cross cultural consistency
      • Dworkin R.H.
      • Turk D.C.
      • Wyrwich K.W.
      • Beaton D.
      • Cleeland C.S.
      • Farrar J.T.
      • Haythornthwaite J.A.
      • Jensen M.P.
      • Kerns R.D.
      • Ader D.N.
      • Brandenburg N.
      • Burke L.B.
      • Cella D.
      • Chandler J.
      • Cowan P.
      • Dimitrova R.
      • Dionne R.
      • Hertz S.
      • Jadad A.R.
      • Katz N.P.
      • Kehlet H.
      • Kramer L.D.
      • Manning D.C.
      • McCormick C.
      • McDermott M.P.
      • McQuay H.J.
      • Patel S.
      • Porter L.
      • Quessy S.
      • Rappaport B.A.
      • Rauschkolb C.
      • Revicki D.A.
      • Rothman M.
      • Schmader K.E.
      • Stacey B.R.
      • Stauffer J.W.
      • von S.T.
      • White R.E.
      • Witter J.
      • Zavisic S.
      Interpreting the clinical importance of treatment outcomes in chronic pain clinical trials: IMMPACT recommendations.

      Serlin RC, Mendoze TR, Nakamura Y, Edwards KR, Cleeland CS: When is cancer pain mild, moderate, or severe: Grading pain severity by its interference with function. Pain 61:277–228, 1995

      and because a numeric rating scale (NRS) may provide a more sensitive measure.
      • Max M.B.
      • Laska E.M.
      Subsequent drafts were reviewed by email and teleconference.

      Pain Severity

      The original APS-POQ included pain severity ratings (0 = no pain, 10 = worst pain possible) at the time the survey was completed (“right now”), and worst and average rating over the past 24 hours. During the survey revision, “pain now” was deleted because it was felt to be highly influenced by the variable proximity of analgesic dosing to survey completion in the acute hospital setting. The item was replaced with a question about the “least amount of pain.” Least pain has previously been shown to be a useful measure of severity.
      • Keller S.
      • Bann C.M.
      • Dodd S.L.
      • Schein J.
      • Mendoze T.R.
      • Cleeland C.S.
      Validity of the Brief Pain Inventory for use in documenting outcomes of patients with noncancer pain.
      Measuring least and worst pain ratings helps appraise the full range of pain experienced during rest and activity common in acute pain. Because of the importance for QI of the amount of time in pain and to reduce the overall patient burden for the scale, average pain was replaced with an item querying the percentage of time severe pain was experienced over 24 hours in an attempt to capture the “area under the curve” or amount of time being spent with a level of pain known to negatively impact persons.

      Pain Relief

      A measure of pain relief was also added to assess the patient's perception of the pain control provided by the treatments provided. A conventional method of analyzing pain relief in research is to calculate a change in pain rating for each patient and then compare the mean value of the change between the treatment and placebo groups.

      Laska EM, Meisner M, Siegel C: Analytic approaches to quantifying pain scores, in Max M, Portenoy R, Laska E (eds): Advances in Pain Research And Therapy. Vol 18. New York, Raven Press, 1991, pp 675-683

      Analgesic trials have used derived measures to indicate the magnitude of pain relief as summed pain intensity differences (SPID) or total pain relief (TOTPAR) using visual analog scales (VAS) or numeric rating scales (NRS).
      • Max M.B.
      • Laska E.M.
      A reduction of approximately 30% in a NRS in acute and cancer pain has been defined as a clinically important difference.
      • Cepda M.S.
      • Africano J.M.
      • Polo R.
      • Alcala R.
      • Carr D.B.
      What decline in pain intensity is meaningful to patients with acute pain.
      • Farrar J.T.
      • Portenoy R.K.
      • Berline J.A.
      • Kinman J.L.
      • Strom B.L.
      Defining clinically important differences in pain outcome measures.
      • Farrar J.T.
      • Young J.P.
      • Ma Moreaux L.
      • Werth J.L.
      • Poole R.M.
      Clinical importance of changes in chronic pain intensity measured on an 11-point numerical pain rating scale.
      • Farrar J.T.
      • Berlin J.A.
      • Strom B.L.
      Clinically important changes in acute pain outcomes measures: A validation study.
      • Jensen M.P.
      The validity and reliability of pain measures in adults with cancer.
      • Jensen M.P.
      • Chen C.
      • Brugger A.M.
      Interpretation of visual analog scale ratings and change scores: A reanalysis of two clinical trials for postoperative pain.
      While achieving a clinically important improvement in pain ratings is a useful goal, use of these measures requires data collection at prescribed and consistent time intervals and a calculation of the percentage difference that would be more difficult with most QI study resources.
      • Barden J.
      • Edwards J.E.
      • Mason L.
      • McQuay H.J.
      • Moore A.
      Outcomes in acute pain trials: Systematic review of what was reported.
      Pain intensity and relief measures are complex constructs that may best be described as a reflection of a number of internal and external factors and meanings for persons with pain.
      • Williams A.
      • Davies H.T.O.
      • Chadury Y.
      Simple pain rating scales hide complex idiosyncratic meanings.
      In general, QI differences are examined over variable time frames after practice changes for populations and often using historical controls. In addition, relief category scales have been reported to be sensitive to small reductions in pain.
      • Max M.B.
      • Laska E.M.
      Therefore, we chose to include a pain relief scale for the APS-POQ-R that measured with the widely used horizontal scale of a percentage (0% to 100%) adapted from the Brief Pain Inventory (BPI)
      • Daut R.
      • Cleeland C.
      • Flanery R.
      Development of the Wisconsin Brief Pain Inventory to assess pain in cancer and other diseases.
      to measure the degree of pain relief obtained from all of the patient's pharmacological and nonpharmacological treatments combined.

      Side Effects

      Side effects of treatment can significantly compromise safety, quality of life, patient adherence, and ability to re-medicate. Multimodal therapy is encouraged in pain treatment to both reduce pain, minimize side effects when possible, and to treat side effects when they occur as is often the case with analgesic therapy. To date, there is no consistent scale used for the reporting of side effects or adverse events in analgesic trials.
      • Barden J.
      • Edwards J.E.
      • Mason L.
      • McQuay H.J.
      • Moore A.
      Outcomes in acute pain trials: Systematic review of what was reported.
      Therefore a new set of items was added to assess both the presence and intensity of the most common medication side effects. Previous pain QI studies using this approach have demonstrated significant changes in side effect profiles through improved treatment patterns.
      • Illgen R.
      • Pellino T.A.
      • Gordon D.B.
      • Butts S.
      • Heiner J.P.
      Prospective analysis of a novel long-acting oral opioid analgesic regimen for pain control after total hip and knee replacement.
      • Rajpal S.
      • Gordon D.B.
      • Pellino P.
      • Strayer A.
      • Trost G.
      • Zdeblick T.
      • Resnick D.
      Comparison of perioperative oral multimodal analgesia versus IV PCA for spine surgery.

      Pain Interference Scale for Physical and Emotional Function

      The original APS-POQ also included a subset of items modeled after the BPI designed to evaluate how pain interferes with functioning and well-being in the outpatient setting. One factor analysis model of the BPI has yielded 2 underlying interference dimensions: interference with activity (walking, work, general activity, sleep) and interference with affect (mood, enjoyment of life).
      • Jensen M.P.
      • Mardekian J.
      • Lakshminarayanan M.
      • Boye M.E.
      Validity of 24-h recall ratings of pain severity: Biasing effects of “peak” and “end” pain.
      These items were adapted for the APS-POQ-R to focus on the types of activities common to hospitalized patients and included questions to assess the range of positive and negative moods as is recommended when developing a measure of emotional functioning.
      • Turk D.C.
      • Dworkin R.H.
      • Burke L.B.
      • Gershon R.
      • Rothman
      • Scott J.
      • Allen R.R.
      • Atkinson J.H.
      • Chandler J.
      • Cleeland C.
      • Cowan P.
      • Dimitrova R.
      • Dionne R.
      • Farrar J.T.
      • Haythornthwaite J.A.
      • Hertz S.
      • Jadad A.R.
      • Jensen M.P.
      • Kellstein D.
      • Kerns R.D.
      • Manning D.C.
      • Martin S.
      • Max M.B.
      • McDermott M.P.
      • McGrath P.
      • Moulin D.E.
      • Nurmikko T.
      • Quessy S.
      • Raja S.
      • Rappaport B.A.
      • Rauschkolb C.
      • Robinson J.P.
      • Royal M.A.
      • Simon L.
      • Stauffer J.W.
      • Stucki G.
      • Tollett J.
      • von Stein T.
      • Wallace M.S.
      • Wernicke J.
      • White R.E.
      • Williams A.C.
      • Witter J.
      • Wyrwich K.W.
      Developing patient-reported outcome measures for pain clinical trials: IMMPACT recommendations.
      Therefore, 7 new emotional distress items with published evidence for mediating the pain experience
      • Keefe F.J.
      • Lumley M.
      • Anderson T.
      • Lynch T.
      • Carson K.L.
      Pain and emotion: New research directions.
      were added, including “how pain caused the patient to feel,” “anxious,” “angry,” “depressed,” “frightened,” “frustrated,” “helpless,” and “overwhelmed,” along with 2 well-being (positive emotions); “at ease” and “content.” Previous investigators
      • Wells N.
      • Ridner S.H.
      Examining pain-related distress in relation to pain intensity and psychological distress.
      have found differences in pain-related and psychological distress and the amount of variance in interference in daily activity explained by both.

      Participation in Decision-Making and Measurement of Satisfaction

      Measurement of satisfaction with pain management is extremely complex, responses are almost always skewed toward the positive, and results can be difficult to interpret.
      • Dihle A.
      • Helseth S.
      • Kongsgaard U.E.
      • Paul S.M.
      • Miaskowski C.
      Using the American Pain Society's patient outcome questionnaire to evaluate the quality of postoperative pain management in a sample of Norwegian patients.
      • Ward S.E.
      • Gordon D.B.
      Patient satisfaction and pain severity as outcomes in pain management: A longitudinal view of one setting's experience.
      The original APS-POQ focused on perceived “wait time” for pain medications and satisfaction with staff and overall treatment results. Based on the Institute of Medicine (IOM) recommendations to increase patient's participation in treatment planning
      Institute of Medicine Crossing the Quality Chasm, A New Health System for the 21st Century.
      and the work of McNeill
      • McNeill J.A.
      • Sherwood G.
      • Starck P.
      • Thompson C.
      Assessing clinical outcomes: Patient satisfaction with pain management.
      • McNeill J.A.
      • Sherwood G.
      • Starck P.
      • Nieto B.
      Pain management outcomes in hospitalized Hispanic patients.
      and Hansson,
      • Hansson E.
      • Fridlund B.
      • Hallstrom I.
      Developing and testing a questionnaire to assess the quality of pain management in acute care in Sweden.
      a new item was added regarding how much patients were allowed to participate in decision-making.

      Use of Nonpharmacological Interventions

      The work cited above also encouraged the provision of information about the pain and the integration of physical, psychological, and social interventions with pharmacological strategies to improve patients' sense of participation in their care and to improve pain outcomes. Nonpharmacological related therapies are now widely accepted as appropriate for pain care and have demonstrated impact on patient outcomes and satisfaction.
      American Society of Anesthesiologists Task Force on Acute Pain Management
      Practice guidelines for acute pain management in the perioperative settings: An updated report by the American Society of Anesthesiologists Task Force on Acute Pain Management.
      • Miaskowski C.
      • Cleary J.
      • Burney R.
      • Coyne P.
      • Finley R.
      • Foster R.
      • Grossman S.
      • Janjan N.
      • Ray J.
      • Syrjala K.
      • Weisman S.
      • Zahrbock C.
      Guideline for the Management of Cancer Pain in Adults and Children, APS Clinical Practice Guideline Series, No. 3.
      In one study, approximately 40% of people used some form of nonpharmacological techniques consistent with the biopsychosocial model of pain and experienced a greater degree of relief of symptoms.
      • Astin J.A.
      Why patients use alternative medicine: Results of a national study.
      To date, documentation of these strategies is rarely included in patient records and thus medical record audit does not provide an accurate reflection of the gaps or benefits of this component of pain care. Subsequently, 3 new items were added: (1) to assess the helpfulness of pain care information provided, if any, (2) to assess the types of nonpharmacological strategies patients use to control pain, and (3) whether health care providers encouraged the use of nonpharmacological strategies. Because these 3 questions record information about the patient's interaction with their health care providers rather than how they are experiencing pain, they constitute a separate domain and were not included in our primary factor analysis. We refer to them as the informational items.

      Assessment of Patient Understanding of the APS-POQ Questions

      A temporary item was used in field testing that asked patients about any aspect of the survey they did not understand to help reduce response errors that can occur if questions are not interpreted in the manner they were intended. If any consistent misunderstanding of a question was reported, careful consideration was to be given to changing or dropping the question as appropriate.

      Sample

      Between January 28, 2008, and December 16, 2008, 299 adult inpatients were recruited from 2 US academic medical centers: a 471-bed hospital in the Midwest and a 708-bed hospital located in a South Atlantic state. The study protocol was reviewed and given exempt status by the institutional review boards at both sites. All study procedures were in accordance and compliance with the Health Insurance Portability and Accountability Act (HIPPA) regulations and the institution's policies and guidelines for protection of human subjects.
      A convenience sampling method for recruiting patients was used at each hospital. The criteria for inclusion were age over 18 years, English speaking, and being alert enough to respond. A nurse project team member (not directly involved in the care of the patient) approached inpatients at a convenient time, used a letter of invitation to explain the study, and to obtain verbal assent. Survey items were read aloud to avoid questions about literacy. Patients were asked to participate within 72 hours of medical admission or surgery and surveyed regarding the first 24 hours of inpatient pain management they had received. Although the accuracy of pain recall has been challenged,
      • Stone A.A.
      • Broderick J.E.
      • Shiffman S.S.
      • Schwartz J.E.
      Understanding recall of weekly pain from a momentary assessment perspective: Absolute agreement, between-and within-person consistency, and judge change in weekly pain.
      a minimal bias for influencing peak and end pain on validity of using 24-hour recall has been reported.
      • Jensen M.P.
      • Mardekian J.
      • Lakshminarayanan M.
      • Boye M.E.
      Validity of 24-h recall ratings of pain severity: Biasing effects of “peak” and “end” pain.
      This methodology has been used routinely in pain QI and is consistent with feasibility and clinical meaningfulness within this setting.
      Involvement in the study was voluntary and patients were given assurances of confidentiality and that their care would not be affected in any way whether they agreed to participate or not. Project team members had access to the hospital census through their hospital employment or graduate student status. The patient's medical record was also audited for demographics and analgesic treatment. All forms were coded. Individual participation and responses were not shared directly with staff. There were 199 participants recruited from one institution and 100 from the other.

      Data Analysis: Phase 2 Overview

      After data collection and verification was completed descriptive statistics (ie, means, standard deviations, and frequencies) were calculated for all patient characteristics and APS-POQ-R response items. Means and range of responses (minimum and maximum scores) and frequencies were examined for the 23 primary items of the APS-POQ-R that measured the patients experience with pain. The decisions to retain or delete items from this group were based on the results of psychometric testing along with findings from the literature, critical reasoning and expert consensus about the relative contribution of each item to the measurement domains of the quality of pain care. The need for brevity, clarity, and conciseness also influenced decisions regarding the potential value of items to clinicians who would be conducting quality assessments of pain care using this tool.

      Criteria for Item Deletion

      The principal determinations for retaining or deleting items were based on item-to-item intercorrelations, item-to-subscale correlations by calculating the internal consistency reliability (Cronbach α), with item deletion, relevance of the item supported by the literature, factor structures or item groupings elucidated by exploratory principal component factor analyses, and perceived conceptual confusion of an item by experts. Every effort was made to reduce item burden and to retain only those items believed to provide the greatest value to QI activities.
      Once the final set of items had been determined, construct validity was evaluated by both an exploratory principal components oblique and varimax factor rotation for all the items, to examine the factor structure of the instrument. A principal component varimax factor rotation was performed again with items that were retained for the final instrument. Initial validity testing was also conducted by assessing for significant differences among biologically appropriate subgroup cohorts. Group comparisons were performed using Student t tests and 1-way ANOVAs to detect differences among patient populations based on the primary type of pain (acute, chronic cancer-related pain, and chronic nonmalignant pain). Bivariate correlations with Pearson product-moment (r) were conducted to assess the magnitude of relationships between variables and subscales. Linear and logistic regressions were used to identify significant variables and subscale scores that were predictors of satisfaction. All analytical procedures were performed using SPSS V15.0 (SPSS Inc, Chicago, IL).

      Results

      Sample characteristics for respondents (n = 299) are summarized in Table 1. Average age for participants was (55.60 ± 17.1 years; range, 19 to 95 years) and the sample was comprised of comparable numbers of males and females, (males, n = 138; 46%). There was no significant difference (P = .668) in the mean age of participants between study sites, and similar percentages of males (43.5% vs 51%) and females were recruited from each site. Patients recruited for the study were either on a medical (n = 172) or surgical service (n = 127), and patients were further categorized by diagnosis (eg, cancer or no cancer) by service. There were no meaningful differences between participant characteristics by medical services, however a larger percentage of the sample from hospital 1 had surgical oncology patients (19.5%) versus 3% from hospital 2. Hospital 2 had a higher percentage of noncancer surgical patients (35%) compared with hospital 1 (17.5%). Types or sources of pain were not delineated for each patient. Although the treatment types and amount(s) of analgesics administered were also collected on all patients through matched medical record audits, these data are not presented in this report.
      Table 1Sample Characteristics for Respondents (n = 299)
      Variable
      Mean age ± SD55.6 ± 17.1
      Sex
       Male138 (46%)
       Female161 (54%)
      Diagnosis
       Medical cancer59 (19.7%)
       Medical noncancer113 (37.7%)
       Surgical cancer42 (14%)
       Surgical noncancer85 (28.3%)
      Ethnicity
       Caucasian241 (80.3%)
       Hispanic4 (1.3%)
       African-American43 (14.3%)
       Asian3 (1%)
       American Indian1 (.3%)
       Other/missing7 (2.3%)

      Means, Standard Deviations, and Frequencies of the APS-POQ Items

      The modified APS-POQ administered for this study consisted initially of 26 items. There were 23 primary items in which the response was measured by the 0 to 10 NRS which we treated as a continuous scale. Table 2A provides the means and standard deviations for these items. Additional items, referred to as the “Information” items, are reported in (Table 2B). One used a Likert scale with categories 1 = never, 2 = sometimes, and 3 = often, to measure how often a nurse or doctor encouraged nonpharmacological methods. Two other items involved dichotomous yes/no responses; 1 for whether patients received information about their pain treatment and 1 related to the use of nonpharmacological measures for the treatment of pain. A “yes” response was followed by a request for a more specific response. One hundred fifty-eight participants (53.2%) reported that they had received information about their pain treatment options, and the mean score for how helpful this information was to patients was 7.85 ± 2.3 (range, 0 = not at all helpful to 10 = extremely helpful). Of those who indicated that they had used nonpharmacological measures (n = 184; 61.7%), participants identified primarily distraction (56.8%), prayer (49%), deep breathing (35.9%), heat (34.9%), cold pack (29.1%), and relaxation (24.2%) Fewer participants reported using walking (15.3%), massage (14.7%), meditation (12.2%), music (9.7%), and imagery or visualization (5.8%) to relieve pain.
      Table 2AMeans and Standard Deviations for Each of the 23 Primary Continuous Items on the Original APS APS-POQ
      NMinimumMaximumMeanSD
      Least pain in 24 hours2990103.442.9
      Worst pain in 24 hours2990107.602.6
      Estimate of percentage of time in severe pain2980%100%37.8032.5
      Pain interfered or prevented you from activities in bed2980105.373.7
      Pain interfered or prevented you from activities out of bed2910105.533.6
      Pain interfered or prevented you from falling asleep2980104.303.8
      Pain interfered or prevented you from staying asleep2970104.203.8
      How much the pain caused you to feel anxious2990104.373.6
      How much the pain caused you to feel angry2970102.933.6
      How much the pain caused you to feel depressed2980102.803.4
      How much the pain caused you to feel frightened2990102.703.4
      How much the pain caused you to feel frustrated2990105.183.6
      How much the pain caused you to feel helpless2980104.433.9
      How much the pain caused you to feel overwhelmed2990104.123.9
      How much the pain caused you to feel at ease2940103.393.6
      How much the pain caused you to feel content2940103.163.6
      Severity of nausea2990102.553.5
      Severity of drowsiness2990103.753.5
      Severity of itching2990101.903.1
      Severity of dizziness2990101.562.6
      Pain relief in the first 24 hours2980%100%68.4428.2
      Were you allowed to participate in decisions about pain treatment?2970107.913.0
      How satisfied are you with the results of your pain treatment?2970108.192.4
      Table 2BAdditional Items Referred to “Information” Items
      NYesNo
      Did you use any nonmedication methods?298184 61.7%)114 (38.3%)
      Did you receive information about pain treatment options?297158 (53.2%)139 (46.8%)
      NMinimumMaximumMeanSD
      How often did a doctor or nurse encourage nonmedication methods?296131.470.65
      Items not intended to be answered by all patients.
      Interference with cough, turn, and deep breath/physical therapy
      1110104.143.8
      Items not intended to be answered by all patients.
      How helpful information was if received?
      1570107.852.3
      Items not intended to be answered by all patients.

      Item Correlations and Reliability

      The results of correlations and tests for internal consistency reliability (Cronbach α) for the 23 primary items are shown in Table 3. For the majority of items a lower score was associated with a more positive response. Reversed scoring was used for items where a higher score was more favorable. Cronbach α for all 23 items was .85. Table 3 shows the item-to-total correlations and changes in the overall Cronbach α with each item deleted from the questionnaire. Item-to-item correlations for each of the original stem questions were assessed for the magnitude and direction of relationships. Specially, inter-correlation coefficients for items within specific categories or clusters such as those pertaining to interference, affective experiences, and side effects from analgesic therapies were calculated (Table 3). Moderate to high inter-correlation coefficients can be indicative of the interrelatedness of the items that may reflect conceptual overlap or items measuring a similar concept of construct.
      Table 3Subscale Item-to-Total Correlations and Cronbach α
      Subscale Mean if Item DeletedScale Variance If Item DeletedCorrected Item-Subscale Total CorrelationCronbach α If Item Deleted
      Affective Subscale (Cronbach α = .82)
       How much the pain caused you to feel anxious9.9578.56.659.773
       How much the pain caused you to feel depressed11.5181.42.658.773
       How much the pain caused you to feel frightened11.6281.47.685.726
       How much the pain caused you to feel helpless9.8978.41.597.804
      Pain Severity and Sleep Interference Subscale (Cronbach α = .83)
      Least pain in 24 hours19.79120.37.552.836
      Worst pain in 24 hours15.63123.12.570.818
      Estimate of percentage of time in severe pain19.46107.15.676.787
      Pain interfered or prevented you from falling asleep18.9595.25.716.774
      Pain interfered or prevented you from staying asleep19.0397.73.678.787
      Perceptions of Care Subscale (Cronbach α = .70)
       Pain relief in the first 24 hours15.9423.11.454.689
       Were you allowed to participate in decisions about pain treatment?14.8420.14.498.646
       How satisfied are you with the results of your pain treatment?15.5523.12.627.498
      Activity Interference Subscale (Cronbach α = .82)
       Pain interfered or prevented you from activities in bed5.5313.02.694N/A
       Pain interfered or prevented you from activities out of bed5.3213.59.694N/A
      Adverse Effects Subscale (Cronbach α = .63)
       Severity of nausea7.2144.55.461.524
       Severity of drowsiness6.0142.97.491.499
       Severity of itching7.8654.97.282.649
       Severity of dizziness8.2054.15.435.556
      Of the 4 interference items, a high item-to-item correlation existed between interfered or prevented “falling asleep” and “staying asleep” (r = .78; P < .001). However, the sleep literature
      • Brennan M.J.
      • Lieberman 3rd, J.A.
      Sleep disturbances in patients with chronic pain: Effectively managing opioid analgesia to improve outcomes.
      • Green C.R.
      • Ndao-Brumblay S.K.
      • Hart-Johnson T.
      Sleep problems in a racially diverse chronic pain population.
      supports these items as measuring 2 distinct aspects of the sleep experience, and therefore both were retained in the instrument. All 4 had item-to-total correlations ranging from .46 to .63.
      Of the 9 new affective items, high item-to-item correlations were found between “ease” and “content” (r = .85, P < .0001), and low item-to-total correlations (≤.40). Removing each of these items raised the Cronbach α by .02. In addition, approximately 5% of the respondents in our study indicated they did not understand what was being asked with these 2 questions. Although these positively stated emotions were believed at the time of item generation to be of importance to the measurement of quality care, our evaluation of these items led to their elimination. In addition, a bivariate correlation between “frustrated” and “helplessness” showed a moderate correlation (r = .61; P < .001). “Frustrated,” while also moderately correlated with other negative affective experiences (range, r = .46 to .63), was also eliminated as it was found to be ambiguous without any evidence to support its importance as a component for quality of pain care. Similarly, “overwhelmed” and “angry” showed little effect on changes in internal consistency reliability for the negative emotion item set, and therefore these were removed to consolidate questions related to the affective aspects of pain experiences. Moreover, these experiences are not typically included in quality assessments for pain and satisfaction with pain care surveys. Correlational statistics and reliability testing favored 4 of the remaining affective items (“anxious,” “depressed,” “frightened,” and “helpless”) for inclusion in the final revised version of the APS-POQ-R.

      Construct Validity Testing

      Exploratory Factor Analyses

      To evaluate all 23 primary items for construct validity further, an exploratory, principal components factor analysis with both varimax and oblique rotations were performed. The oblique rotation assumes a level of interrelatedness of all items which is not valid in this group of items and did not yield an interpretable component matrix. A varimax orthogonal rotation which assumes a level of independence of the constructs revealed 6 interpretable factors with Eigen values >1 accounting for over 66% of the variance in the measure. Acceptable factor loading coefficients (≥.4) were evident for all of the items on a least 1 of the factors. The positive emotional items, “at ease” and “content” loaded on an isolated factor, and did not have acceptable factor loadings with the other affective items. Given that a number of respondents noted problems in understanding these items and our interest in creating an assessment tool with an ease of administration to patients and interpretability of the results, these items were removed.
      In total, 5 items were dropped from the original 23 primary items. A second exploratory principal components factor analysis was conducted with the 18 remaining primary continuous level items identified for the final version of the APS-POQ-R. This exploratory principal components varimax factor analysis was generated to verify item groupings that would ultimately define the subscales for the instrument. Five factors were identified with Eigen values >1 (range, 5.98 to 1.05), and all items loaded on at least 1 of the factors with an acceptable loading coefficient of ≥.5, except for itching. In general, itching occurs less frequently than the others but was maintained due to the clinical importance of this side effect. Together the items accounted for 64.05% of the variance in the measure. Table 4 shows the rotated factor component matrix and factor loadings for each item. Factor 1 contained 4 items related to the emotional components of pain, the “Affective Subscale,” and explained 31.36% of the variance in the measure. The “Pain Severity and Sleep Interference” subscale accounted for 11.23% of the variance and included “worst” and “least” pain experienced in the first 24 hours, percent time over the past 24 hours in “severe pain,” and the 2 interference with sleep items. “Perceptions of Pain Care,” the third factor, was defined by percent pain relief, degree of participation in decisions, and satisfaction with pain treatment (8.42% of the variance). The “Interference with Activity” subscale (6.9% of the variance) included 2 items related to pain interfering with activities in and out of bed. The fifth subscale, “Adverse Effects,” captured the severity of side effects associated with pain treatments (6.14% of the variance).
      Table 4Rotated Factor Component Matrix and Factor Loadings for Continuous Level Variables
      Factor Components
      12345
      Total Items Retained 18 Name of SubscaleAffectivePain Severity and Sleep InterferencePerceptions of careActivity InterferenceAdverse Effects
      Variance explained Total 64.05%31.36%11.23%8.42%6.9%6.14%
      Cronbach α Overall 0.860.820.830.700.820.63
      Least pain in 24 hours−.035.528−.450.235.162
      Worst pain in 24 hours.200.535−.151.420.130
      Estimate of percentage of time in severe pain.029.627−.406.317.139
      Pain interfered or prevented you from activities in bed.135.229−.074.791.068
      Pain interfered or prevented you from activities out of bed.194.203−.110.807.073
      Pain interfered or prevented you from falling asleep.343.822−.010.089.102
      Pain interfered or prevented you from staying asleep.268.812−.068.090.064
      How much the pain caused you to feel anxious.734.227−.126.224.021
      How much the pain caused you to feel depressed.745.209−.240−.067.129
      How much the pain caused you to feel frightened.804.151−.190.014.098
      How much the pain caused you to feel helpless.673.097−.134.403.102
      Severity of nausea.138.140.079.026.713
      Severity of drowsiness.079.024−.024.304.694
      Severity of itching−.217.013.069.397.460
      Severity of dizziness.138.132−.063−.124.762
      Pain relief in the first 24 hours (%)−.176−.143.699−.117.070
      Were you allowed to participate in decisions about pain treatment?−.167−.017.745−.005−.025
      How satisfied are you with the results of your pain treatment?−.195−.147.794−.028.050

      Construction of Revised Instrument and Subscales

      The item clusters identified reflect important criteria for the measurement of quality indicators for pain care and patient experiences. A mean score was calculated for each of the subscales. The 2 items, time spent in severe pain and percentage of pain relief received, were normalized to 0 to 10 scales. Mean scores and standard deviations for all of the subscales were as follows for subscales with higher scores indicating poorer outcomes: Affective subscale (3.57 ± 2.9), Pain Severity and Sleep Interference subscale (4.68 ± 2.6), Interference with Activity (5.47 ± 3.4), and Adverse Effects subscale (2.44 ± 2.21). For Perceptions of Pain Care where higher scores indicate better outcomes, the mean score was (7.63 ± 2.2).
      Overall Cronbach α for the revised item set was .86. Cronbach α for each subscale were as follows: Affective subscale, α = .82; Pain Severity and Sleep Interference subscale, α = .83; Perceptions of Pain Care subscale, α = .70; Interference with Activity, α = .82; and Adverse Effects subscale, α = .63. Although an alpha ≥.70 demonstrates acceptable internal consistency, lower thresholds for subscales ≥.6 can be considered acceptable as a measurement of internal consistency, especially when there are fewer items in the subscale
      • Nunnally J.C.
      • Bernstein I.H.
      Psychometric Theory.
      and the items are considered to be important for physiologic reasons. A lower Cronbach α for the Adverse Effects subscale may also be explained by the absence of the presence of adverse effects in a substantial portion of respondents. One hundred sixty-seven (55.9%) reported no nausea, 109 (36.5%) had no drowsiness, 195 (65.2%) indicated no itching, and 189 (63.2%) did not experience dizziness.
      An intercorrelation matrix constructed for the subscales is presented in Table 5. None of the subscales was highly correlated suggesting that these constructs may be partially independent of one another and measuring relatively discrete concepts of quality care and patient experiences. Each of the subscales was significantly correlated in a predictable fashion, except for correlation between Adverse Effects and Perception of Care, supporting the interrelated aspects of these outcomes of care.
      Table 5Intercorrelation Matrix for Subscales
      Affective SubscalePain Severity and Sleep Interference SubscalePerceptions of Care SubscaleInterference with Activity SubscaleAdverse Effects Subscale
      Affective Subscale1.534
      Correlation is significant at the P < .001 level (2-tailed).
      −.394
      Correlation is significant at the P < .001 level (2-tailed).
      .390
      Correlation is significant at the P < .001 level (2-tailed).
      .231
      Correlation is significant at the P < .001 level (2-tailed).
      Pain Severity and Sleep Interference Subscale.534
      Correlation is significant at the P < .001 level (2-tailed).
      1−.422
      Correlation is significant at the P < .001 level (2-tailed).
      .546
      Correlation is significant at the P < .001 level (2-tailed).
      .323
      Correlation is significant at the P < .001 level (2-tailed).
      Perceptions of Care Subscale−.394
      Correlation is significant at the P < .001 level (2-tailed).
      −.422
      Correlation is significant at the P < .001 level (2-tailed).
      1−.265
      Correlation is significant at the P < .001 level (2-tailed).
      −.008
      Interference with Activity Subscale.390
      Correlation is significant at the P < .001 level (2-tailed).
      .546
      Correlation is significant at the P < .001 level (2-tailed).
      −.265
      Correlation is significant at the P < .001 level (2-tailed).
      1.296
      Correlation is significant at the P < .001 level (2-tailed).
      Adverse Effects Subscale.231
      Correlation is significant at the P < .001 level (2-tailed).
      .323
      Correlation is significant at the P < .001 level (2-tailed).
      −.008.296
      Correlation is significant at the P < .001 level (2-tailed).
      1
      Correlation is significant at the P < .001 level (2-tailed).

      Additional Construct Validity Testing

      Contrasting Groups

      Overall group comparisons by service indicated that medical patients had a significantly lower mean satisfaction scores (7.88 ± 2.7; P < .01) compared with surgical patients (8.59 ± 1.9). No significant difference was found in participation in decision-making between medical and surgical patients (P = .059). One-way ANOVAs with post hoc analyses using least significant differences were conducted on outcomes among service and diagnosis, but no meaningful differences were detected.
      Student t test for independent groups were used to assess differences between users of nonpharmacological techniques for pain control (n = 184) and those who did not (n = 114) for level of participation in decision-making and, how satisfied patients were with their care and worst pain intensity. No difference was found for participation in decision-making between patients using (7.69 ± 3.1) and not using (8.33 ± 2.7) nonpharmacological strategies (P = .063). However, patients using nonpharmacological strategies were significantly less satisfied than those who reported that they were not using these strategies (mean, 7.91 ± 2.6 vs mean, 8.63 ± 1.9; P = .049, respectively). Reported worst pain levels were significantly higher among users compared with nonusers (8.04 ± 2.3 vs 6.9 ± 2.9; P = .001, respectively).

      Prediction Models

      To determine relationships among items and subscales and variables that may predict outcomes for items and specific subscales, specific aim 4, multiple linear regression analyses were used to determine the relationships of various subscales to how satisfied patients were with their pain treatment (Table 6). Age, sex, percent time spent in severe pain, information received about treatment options (yes/no), amount of pain relief received, and level of participation in care were selected to assess whether 1 or more of these was a significant predictor of patient satisfaction with care. A bivariate correlation of worst pain and time spent in severe pain showed a moderate correlation, r = .55 (P < .001). Only time spent in severe pain was included since it is believed to be more predictive of patient satisfaction. The items including less time in severe pain (P = .023), information received (P = .048), and greater pain relief and level of participation (P< .0001) were each associated with greater satisfaction, and together these variables explained a significant portion of the variance in this outcome (R2 = .403; F = 31.48; P < .0001). Overall, these findings validate that several of the items are important components in the measurement of satisfaction with pain care.
      Table 6Regression Model for Patient Characteristics and Care Outcomes as Predictors of Satisfaction
      Coefficients (a)
      Unstandardized CoefficientsStandardized Coefficients
      BSEβtSig.
      Age.004.007.030.603.547
      Sex−.013.234−.003−.056.956
      Percent time spent in severe pain−.009.004−.122−2.279.023
      Information received.488.246.1011.983.048
      Percent pain relief.025.005.2935.481.000
      Participation in decision-making.294.042.3726.986.000
      Four of the APS-POQ-R subscales Affective,” “Pain Severity and Sleep Interference,” “Interference with Activity,” and “Adverse Effects” were entered into a stepwise linear regression model to verify their contributions to the outcome of satisfaction. A significant portion (adjusted R2 = .182) of the variance in the level of satisfaction was explained by 2 of the subscales (F = 48.28; P < .001); lower Pain Severity and Sleep Interference (F = 29.68; P < .001) and Affective subscale scores (F = 22.84; P = .001) were associated with better satisfaction. Adverse Effects subscale scores were positively associated with better satisfaction (P = .006), but the average value in this population was low and it had a minimal effect in contributing to the total adjusted R2, only raising it by 1.9%. Interference with Activity was not statistically significant and also demonstrating a minimal effect on the variance in the level of satisfaction.

      Discussion

      A systematic analytic approach was used to revise the APS-POQ which resulted in the new version titled the APS-POQ-R. To assess the psychometric properties, the internal consistency reliability and initial construct validity of the APS-POQ-R were found to be good, as was its prediction value for patient satisfaction. Further evaluation of the psychometric properties for the instrument will be needed to support its reliability and validity as an acceptable measure for QI activities in other populations and its value in directing and reevaluating pain care. The APS-POQ-R seems to provide a practical instrument that is clinically feasible with adequate psychometric properties to help clinicians standardize measurement of important aspects of the quality of pain management for QI activities. The instrument is designed to be administered to gather information about the quality of pain management for hospitalized patients during the first 24 hours of care.
      Principal component factor analysis revealed 5 important subscales of the instruments 18 primary continuous items including Affective Distress, Pain Severity and Sleep Interference, Perceptions of Care, Interference with Activity, and Adverse Effects. The results support the instrument as a measure of interrelated yet separate components of the quality of pain management. In addition, the new APS-POQ-R captures adult patients' perceptions of pain control and the degree that pain control facilitates physical function and emotions during hospitalization as well as the adequacy of information received about pain treatment options and ability to participate in those decisions. Additionally, using 3 additional questions, the survey directly collects information about the prevalence and types of nonpharmacological strategies used to manage pain in the hospital. Patient report of nonpharmacological strategies can be combined with information gathered on drug treatment from medical record audits to examine other core indicators of the quality of pain treatment; namely pain is treated by a route other than IM, analgesics are administered on a regular schedule, and when possible treatment is multimodal.
      Item reduction was a key concern with the aim of keeping the patient burden of the survey to a minimum. However, a critical quality indicator in pain management is that pain is prevented or controlled to a degree that facilitates improved function and quality of life.
      • Gordon D.B.
      • Dahl J.L.
      • Miaskowski C.
      • McCarberg B.
      • Todd K.H.
      • Paice J.A.
      • Lipman A.G.
      • Bookbinder M.
      • Sanders S.H.
      • Turk D.
      • Carr D.B.
      American Pain Society recommendations for improving the quality of acute and cancer pain management.
      Although a comprehensive psychological assessment of this domain would entail a large number of items, we found that 4 of the 9 items we added were adequate to cover a majority of the symptoms experienced in our pain patient population. As such, the APS-POQ-R now includes how pain causes patients to feel anxious, depressed, frightened, and helpless. The Affective subscale explained the largest amount of variance in the measure. As expected, patients who spent less time in severe pain reported less affective distress. Higher scores on negative emotions were inversely correlated with patient satisfaction.
      Although positive emotional items were tested (eg, how pain caused one to feel at ease and content) they added little to our measure, patients commented that they did not understand the items and their inclusion only lowered the internal consistency reliability, perhaps related to the fact that most patients with pain have a significant underlying medical problem. Given this assumption, these were excluded from the final instrument.
      It should be noted that our analysis demonstrated a small but statistically significant positive correlation between the Adverse Events subscale and satisfaction. In general, adverse events are an important consideration in a patient's treatment experience and our results would appear to be counter intuitive. However, the overall number and level of adverse events in our sample was quite low, and so would adversely affect only a small number of patients' reports of satisfaction. It is also possible that the low level of the adverse effects is a reflection of patients having received more effective doses of analgesics leading to a higher level of satisfaction. In either case, the influence of adverse effects on satisfaction requires further study in populations who have higher rates of severity.
      To put this study into perspective, the IOM recommends including consumer satisfaction indices in any evaluation of quality.
      • Chassin M.R.
      • Galvin R.W.
      The urgent need to improve health care quality: Institute of Medicine National Roundtable on Health Care Quality.
      However, as has been reported numerous times,
      • Dihle A.
      • Helseth S.
      • Kongsgaard U.E.
      • Paul S.M.
      • Miaskowski C.
      Using the American Pain Society's patient outcome questionnaire to evaluate the quality of postoperative pain management in a sample of Norwegian patients.
      • McNeill J.A.
      • Sherwood G.
      • Starck P.
      • Thompson C.
      Assessing clinical outcomes: Patient satisfaction with pain management.
      • Ward S.E.
      • Goldberg N.
      • Miller-McCauley V.
      • Mueller C.
      • Nolan A.
      • Pawlik-Plank D.
      • Robbins A.
      • Stormoen D.
      • Weissman D.
      Patient related barriers to management of cancer pain.
      patient satisfaction with pain management is highly variable and dependent on many other factors than the experience of pain. Consequently, it is a poor measure of the quality of pain control when used alone. Patient satisfaction is nevertheless an important measure in quality evaluation, and a better understanding of what patients include in their assessment of satisfaction can be achieved through the use of a multi-item questionnaire, such as the APS-POQ-R, to measure the various aspects thought to contribute to satisfaction. The IOM highlighted the phrase “desired health outcomes” in defining quality to mean the health outcomes that patients desire.
      • Chassin M.R.
      • Galvin R.W.
      The urgent need to improve health care quality: Institute of Medicine National Roundtable on Health Care Quality.
      This sheds light on the importance of including patient's goals and their satisfaction with goal attainment, in addition to medical outcomes, in any assessment of the quality of care. Prediction models in our study confirmed that a variety of items impact patient satisfaction including the amount of time in severe pain, the amount of pain relief obtained, and the ability of the person to participate in decisions about pain treatment. The IOM Quality Chasm report
      Institute of Medicine Crossing the Quality Chasm, A New Health System for the 21st Century.
      further elucidated the need to include patient's perspectives when they proposed 10 new design rules for enhancement of quality. Information about safety, evidence-based practice, and patient satisfaction should be shared with patients and families to allow them to make informed decisions when choosing among available treatments. In this study, only 17.5% of the sample reported a level of participation in decisions about pain management (as much as they wanted) ≤5 on a scale of 0 to 10, with10 being very much so. Still, ongoing efforts to improve communication with patients and include them as partners in determining plans of care are important in all clinical settings. It is disconcerting that almost one half (46.8%) of patients reported they had not received any information about pain control. This finding emphasizes the need for targeting this aspect of care for improvement in quality.

      Limitations

      There are a number of limitations to this study. The survey involved a voluntary convenience sample of patients at 2 sites that may not be representative of all inpatient settings. The majority of the participants were Caucasian, thus more data is needed using the APS-POQ-R with other racial or ethnic groups. A population of hospitalized patients who were confused, too ill, or unable to communicate was not included. The focus of the APS-POQ-R is on adult inpatients and results may not be applicable to pediatrics, outpatient or emergency department settings. A potential confounding issue is whether the patients in this study were reporting on their chronic or acute pain. However, one could assume that there is a mix of both acute and chronic noncancer pain problems in both medical and surgical populations. The APS-POQ-R is reflective not so much of the type of pain being treated, but rather the outcome of the acute care setting's response to a patient's primary pain problem. The survey is also intended to measure very different aspects of quality so it is difficult to establish unidimensionality with standard psychometrics typically used to describe what the survey is measuring, in this case multidimensional quality. Finally, the study design did not allow for analysis of test-retest reliability and sensitivity to change. These properties should be investigated in future research.

      Conclusion

      As has been described, the construct of quality pain management is complex. It is dependent on many practical processes of the care provided, the staff behavior, and patient experiences. A critical element of quality evaluation is obtaining direct feedback from patients about their experiences. The updated APS-POQ-R has been found to have adequate psychometrics for QI and to measure 5 important aspects of the patients experience with their pain and a sixth aspect of nonpharmacological therapies. These 6 aspects include (1) pain severity and relief; (2) impact of pain on activity, sleep, and negative emotions; (3) side effects of treatment; (4) helpfulness of information about pain treatment; (5) ability to participate in pain treatment decisions; and (6) use of nonpharmacological strategies. If further work verifies these findings, then hospitals should be encouraged to use this patient survey along with medical record audits to measure and follow-up on improvements in the quality of pain management. Clearly, more quality improvement-based research must be conducted to examine necessary adaptations and the usefulness of the APS-POQ-R in pediatrics, with specific ethnic groups, and in outpatient settings. This measure should help to examine the relationship between analgesic treatment and patient perceptions of quality of care.

      Acknowledgments

      The authors thank Jun Lee, Christen Greenfield, Kate McGettrick, Eva Allen, and Hui-Hsin for assistance with data collection. We also thank Elizabeth Hansson, Winfried Meissner, and Ruth Zaslansky for their review and thoughtful comments on the manuscript. We would also like to recognize the late Mitchell Max for his founding work in pain QI.

      References

      1. American Pain Society Committee on Quality Assurance Standards: American Pain Society quality assurance standards for the relief of acute pain and cancer pain, in Bond MR, Charlton JE, Woolf CJ, (eds): Proceedings of the VIth World Congress on Pain. Seattle, WA, IASP Press, 1991, pp185-188

        • American Pain Society Quality of Care Committee
        Quality improvement guidelines for the treatment of acute pain and cancer pain.
        JAMA. 1995; 274: 1874-1880
        • American Society of Anesthesiologists Task Force on Acute Pain Management
        Practice guidelines for acute pain management in the perioperative settings: An updated report by the American Society of Anesthesiologists Task Force on Acute Pain Management.
        Anesthesia. 2004; 100: 1573-1581
        • Astin J.A.
        Why patients use alternative medicine: Results of a national study.
        JAMA. 1998; 279: 1548-1553
        • Barden J.
        • Edwards J.E.
        • Mason L.
        • McQuay H.J.
        • Moore A.
        Outcomes in acute pain trials: Systematic review of what was reported.
        Pain. 2004; 109: 351-356
        • Berwick D.
        Disseminating innovations in health care.
        JAMA. 2003; 289: 1969-1975
        • Berwick D.
        The science of improvement.
        JAMA. 2008; 299: 1182-1184
        • Black K.
        • Revere L.
        Six Sigma arises from the ashes of TQM with a twist.
        Int J Health Care Qual. 2006; 19: 259-266
        • Brennan M.J.
        • Lieberman 3rd, J.A.
        Sleep disturbances in patients with chronic pain: Effectively managing opioid analgesia to improve outcomes.
        Curr Med Res Opin. 2009; 25: 1045-1055
        • Cepda M.S.
        • Africano J.M.
        • Polo R.
        • Alcala R.
        • Carr D.B.
        What decline in pain intensity is meaningful to patients with acute pain.
        Pain. 2003; 105: 151-157
        • Chassin M.R.
        • Galvin R.W.
        The urgent need to improve health care quality: Institute of Medicine National Roundtable on Health Care Quality.
        JAMA. 1998; 280: 1000-1005
        • Daut R.
        • Cleeland C.
        • Flanery R.
        Development of the Wisconsin Brief Pain Inventory to assess pain in cancer and other diseases.
        Pain. 1983; 17: 197-210
        • Dihle A.
        • Helseth S.
        • Kongsgaard U.E.
        • Paul S.M.
        • Miaskowski C.
        Using the American Pain Society's patient outcome questionnaire to evaluate the quality of postoperative pain management in a sample of Norwegian patients.
        J Pain. 2006; 7: 272-280
        • Donabedian A.
        Explorations in quality assessment and monitoring, Vol 1, The definition of quality and approaches to its assessment.
        Health Administration Press, Ann Arbor, MI1980
        • Dworkin R.H.
        • Turk D.C.
        • Wyrwich K.W.
        • Beaton D.
        • Cleeland C.S.
        • Farrar J.T.
        • Haythornthwaite J.A.
        • Jensen M.P.
        • Kerns R.D.
        • Ader D.N.
        • Brandenburg N.
        • Burke L.B.
        • Cella D.
        • Chandler J.
        • Cowan P.
        • Dimitrova R.
        • Dionne R.
        • Hertz S.
        • Jadad A.R.
        • Katz N.P.
        • Kehlet H.
        • Kramer L.D.
        • Manning D.C.
        • McCormick C.
        • McDermott M.P.
        • McQuay H.J.
        • Patel S.
        • Porter L.
        • Quessy S.
        • Rappaport B.A.
        • Rauschkolb C.
        • Revicki D.A.
        • Rothman M.
        • Schmader K.E.
        • Stacey B.R.
        • Stauffer J.W.
        • von S.T.
        • White R.E.
        • Witter J.
        • Zavisic S.
        Interpreting the clinical importance of treatment outcomes in chronic pain clinical trials: IMMPACT recommendations.
        J Pain. 2008; 9: 105-121
        • Farrar J.T.
        • Portenoy R.K.
        • Berline J.A.
        • Kinman J.L.
        • Strom B.L.
        Defining clinically important differences in pain outcome measures.
        Pain. 2000; 88: 287-294
        • Farrar J.T.
        • Young J.P.
        • Ma Moreaux L.
        • Werth J.L.
        • Poole R.M.
        Clinical importance of changes in chronic pain intensity measured on an 11-point numerical pain rating scale.
        Pain. 2001; 94: 149-158
        • Farrar J.T.
        • Berlin J.A.
        • Strom B.L.
        Clinically important changes in acute pain outcomes measures: A validation study.
        J Pain Symptom Manag. 2003; 25: 406-411
        • Gallagher R.M.
        • Rowell P.A.
        Claiming the future of nursing through nursing-sensitive quality indicators.
        Nurs Admin Q. 2003; 27: 273-284
        • Gelinas C.
        • Loiselle C.G.
        • LeMay S.
        • Ranger M.
        • Bouchard E.
        • McCormack D.
        Theoretical, psychometric, and pragmatic issues in pain measurement.
        Pain Manag Nurs. 2008; 9: 120-130
        • Gordon D.B.
        • Pellino T.A.
        • Miaskowski C.
        • McNeill J.A.
        • Paice J.A.
        • Laferriere D.
        • Bookbinder M.
        A 10-year review of quality improvement in pain management: Recommendations for standardized outcome measures.
        Pain Manag Nurs. 2002; 3: 116-130
        • Gordon D.B.
        • Dahl J.L.
        • Miaskowski C.
        • McCarberg B.
        • Todd K.H.
        • Paice J.A.
        • Lipman A.G.
        • Bookbinder M.
        • Sanders S.H.
        • Turk D.
        • Carr D.B.
        American Pain Society recommendations for improving the quality of acute and cancer pain management.
        Arch Intern Med. 2005; 165: 1574-1580
        • Green C.R.
        • Ndao-Brumblay S.K.
        • Hart-Johnson T.
        Sleep problems in a racially diverse chronic pain population.
        Clin J Pain. 2009; 25: 423-430
        • Grol R.
        Improving the quality of medical care: Building bridges among professional pride, payer profit, and patient satisfaction.
        JAMA. 2001; 286: 2578-2585
        • Hansson E.
        • Fridlund B.
        • Hallstrom I.
        Developing and testing a questionnaire to assess the quality of pain management in acute care in Sweden.
        Pain Manag Nurs. 2005; 6: 91-104
        • Illgen R.
        • Pellino T.A.
        • Gordon D.B.
        • Butts S.
        • Heiner J.P.
        Prospective analysis of a novel long-acting oral opioid analgesic regimen for pain control after total hip and knee replacement.
        J Arthroplasty. 2004; 21: 814-820
      2. Institute of Medicine Crossing the Quality Chasm, A New Health System for the 21st Century.
        National Academy Press, Washington DC2001
        • Izakovic M.
        New trends in the management of inpatients in US hospitals-quality measurements and evidence-based medicine in practice.
        Bratisl Lek Listy. 2007; 108: 117-121
        • Jensen M.P.
        The validity and reliability of pain measures in adults with cancer.
        J Pain. 2003; 4: 2-21
        • Jensen M.P.
        • Chen C.
        • Brugger A.M.
        Interpretation of visual analog scale ratings and change scores: A reanalysis of two clinical trials for postoperative pain.
        J Pain. 2003; 4: 407-414
        • Jensen M.P.
        • Mardekian J.
        • Lakshminarayanan M.
        • Boye M.E.
        Validity of 24-h recall ratings of pain severity: Biasing effects of “peak” and “end” pain.
        Pain. 2008; 137: 422-427
        • Juran D.
        Achieving sustained quantifiable results in an interdepartmental quality improvement project.
        Jt Comm J Qual Improv. 1994; 20: 105-119
        • Keefe F.J.
        • Lumley M.
        • Anderson T.
        • Lynch T.
        • Carson K.L.
        Pain and emotion: New research directions.
        J Clin Psych. 2001; 57: 587-607
        • Keller S.
        • Bann C.M.
        • Dodd S.L.
        • Schein J.
        • Mendoze T.R.
        • Cleeland C.S.
        Validity of the Brief Pain Inventory for use in documenting outcomes of patients with noncancer pain.
        Clin J Pain. 2004; 20: 309-318
        • Kim C.S.
        • Spahlinger D.A.
        • Kin J.M.
        • Billi J.E.
        Lean health care: What can hospitals learn from a world-class automaker.
        J Hosp Med. 2006; 1: 191-199
      3. Laska EM, Meisner M, Siegel C: Analytic approaches to quantifying pain scores, in Max M, Portenoy R, Laska E (eds): Advances in Pain Research And Therapy. Vol 18. New York, Raven Press, 1991, pp 675-683

        • Lohr K.N.
        Rating the strength of scientific evidence: Relevance for quality improvement programs.
        Int J for Qual Health Care. 2004; 16: 9-18
        • Max M.B.
        • Laska E.M.
        Max M. Portenoy R. Laska E. Advances in Pain Management Research and Therapy. Vol. 18. Single-dose analgesic comparisons. Raven Press, Ltd, New York1991
        • McNeill J.A.
        • Sherwood G.
        • Starck P.
        • Thompson C.
        Assessing clinical outcomes: Patient satisfaction with pain management.
        J Pain Symptom Manage. 1998; 16: 29-40
        • McNeill J.A.
        • Sherwood G.
        • Starck P.
        • Nieto B.
        Pain management outcomes in hospitalized Hispanic patients.
        Pain Manag Nurs. 2001; 2: 25-35
        • Mechanic D.
        Improving the quality of health care in the United States of America: The need for a multi-level approach.
        J Health Serv Res Policy. 2002; 7: S35-S39
        • Miaskowski C.
        • Cleary J.
        • Burney R.
        • Coyne P.
        • Finley R.
        • Foster R.
        • Grossman S.
        • Janjan N.
        • Ray J.
        • Syrjala K.
        • Weisman S.
        • Zahrbock C.
        Guideline for the Management of Cancer Pain in Adults and Children, APS Clinical Practice Guideline Series, No. 3.
        American Pain Society, Glenview, IL2005
        • Nunnally J.C.
        • Bernstein I.H.
        Psychometric Theory.
        McGraw-Hill, New York1994
        • Rajpal S.
        • Gordon D.B.
        • Pellino P.
        • Strayer A.
        • Trost G.
        • Zdeblick T.
        • Resnick D.
        Comparison of perioperative oral multimodal analgesia versus IV PCA for spine surgery.
        J Spinal Disord Tech. 2010; 23
      4. Serlin RC, Mendoze TR, Nakamura Y, Edwards KR, Cleeland CS: When is cancer pain mild, moderate, or severe: Grading pain severity by its interference with function. Pain 61:277–228, 1995

        • Stone A.A.
        • Broderick J.E.
        • Shiffman S.S.
        • Schwartz J.E.
        Understanding recall of weekly pain from a momentary assessment perspective: Absolute agreement, between-and within-person consistency, and judge change in weekly pain.
        Pain. 2004; 107: 61-69
        • Turk D.C.
        • Dworkin R.H.
        • Burke L.B.
        • Gershon R.
        • Rothman
        • Scott J.
        • Allen R.R.
        • Atkinson J.H.
        • Chandler J.
        • Cleeland C.
        • Cowan P.
        • Dimitrova R.
        • Dionne R.
        • Farrar J.T.
        • Haythornthwaite J.A.
        • Hertz S.
        • Jadad A.R.
        • Jensen M.P.
        • Kellstein D.
        • Kerns R.D.
        • Manning D.C.
        • Martin S.
        • Max M.B.
        • McDermott M.P.
        • McGrath P.
        • Moulin D.E.
        • Nurmikko T.
        • Quessy S.
        • Raja S.
        • Rappaport B.A.
        • Rauschkolb C.
        • Robinson J.P.
        • Royal M.A.
        • Simon L.
        • Stauffer J.W.
        • Stucki G.
        • Tollett J.
        • von Stein T.
        • Wallace M.S.
        • Wernicke J.
        • White R.E.
        • Williams A.C.
        • Witter J.
        • Wyrwich K.W.
        Developing patient-reported outcome measures for pain clinical trials: IMMPACT recommendations.
        Pain. 2006; 125: 208-215
        • Walton M.
        The Deming Management Method.
        Putnam Publishing Group, New York1986
        • Ward S.E.
        • Goldberg N.
        • Miller-McCauley V.
        • Mueller C.
        • Nolan A.
        • Pawlik-Plank D.
        • Robbins A.
        • Stormoen D.
        • Weissman D.
        Patient related barriers to management of cancer pain.
        Pain. 1993; 52: 319-324
        • Ward S.E.
        • Gordon D.B.
        Patient satisfaction and pain severity as outcomes in pain management: A longitudinal view of one setting's experience.
        J Pain Symptom Manage. 1996; 11: 242-251
        • Ware J.
        • Snyder M.
        • Wright W.
        • Davies A.
        Defining and measuring patient satisfaction with care.
        Eval Program Plann. 1983; 6: 247-263
        • Wells N.
        • Ridner S.H.
        Examining pain-related distress in relation to pain intensity and psychological distress.
        Res Nurs Health. 2008; 31: 52-62
        • Williams A.
        • Davies H.T.O.
        • Chadury Y.
        Simple pain rating scales hide complex idiosyncratic meanings.
        Pain. 2000; 85: 457-463