If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Revised American Pain Society Patient Outcome Questionnaire (APS-POQ-R) for Quality Improvement of Pain Management in Hospitalized Adults: Preliminary Psychometric Evaluation
Center for Biobehavioral Research, University of Pennsylvania School of Nursing and Department of Anesthesiology, University of Pennsylvania, School of Medicine, Philadelphia, Pennsylvania
Quality improvement (QI) is a compilation of methods adapted from psychology, statistics, and operations research to identify factors that contribute to poor treatment outcomes and to design solutions for improvement. Valid and reliable measurement is essential to QI using rigorously developed and tested instruments. The purpose of this article is to describe the evolution of the American Pain Society Patient Outcome Questionnaire (APS-POQ) for QI purposes and present a revised version (R) including instrument psychometrics. An interdisciplinary task force of the APS used a step-wise, empiric approach to revise, test, and examine psychometric properties of the society's original POQ. The APS-POQ-R is designed for use in adult hospital pain management QI activities and measures 6 aspects of quality, including (1) pain severity and relief; (2) impact of pain on activity, sleep, and negative emotions; (3) side effects of treatment; (4) helpfulness of information about pain treatment; (5) ability to participate in pain treatment decisions; and (6) use of nonpharmacological strategies. Adult medical-surgical inpatients (n = 299) from 2 hospitals in different parts of the United States participated in this study. Results provide support for the internal consistency of the instrument subscales, construct validity and clinical feasibility.
Perspective
This article presents the initial psychometric properties of the APS-POQ-R for quality improvement purposes of hospitalized adult patients. Validation in additional groups of patients will be needed to demonstrate its generalizability.
The need to measure, compare, and improve the quality of pain management is increasingly sought by health care consumers, payers, and professionals alike. However, different definitions and measurement approaches are necessary for distinct purposes such as quality improvement (QI), public reporting (accountability), and research.
Although evidence to document the effectiveness of a continuous QI approach in pain management is still sparse, it continues to be a recommended strategy to measure and advance pain care processes and outcomes.
provide frameworks and a set of tools to develop initiatives to improve and evaluate the quality of pain management. These models have also been applied to understanding systems, processes of care and practice variations, and systematic evaluations to address patient safety. The purpose of QI data is to provide a better understanding of the extent and nature of the problem, motivation for change, and points for comparison after change have been implemented. The purpose of this article is to describe the evolution of the American Pain Society Patient Outcome Questionnaire (APS-POQ) and present a revised version (R) including instrument psychometrics for QI purposes (Fig 1).
These include appropriate ongoing assessment (eg, screening for the presence of pain, completion of a comprehensive initial assessment when pain is present, and frequent reassessments of patient responses to treatment); interdisciplinary, collaborative care planning that includes patient input; appropriate treatment that is efficacious, cost-conscious, culturally and developmentally appropriate, and safe; and access to specialty care as needed. This definition is consistent with characteristics of quality that encompass structure, process and outcomes, and is applicable to people in need of treatment for acute, cancer, and chronic non-cancer pain. Structure is defined as the physical and organizational properties of the setting where care is delivered. Processes include communication and practice patterns. Pertinent outcome variables include changes in pain severity and frequency, treatment or the disease process causing pain, emotional and physical function, quality of life, adverse effects of pain or pain treatment, and indicators of patient satisfaction.
The definition and measurement of the quality of pain management for any purpose is difficult. The evaluation of pain care is particularly problematic due to the complex nature of the subjective experience involved and limited knowledge of the relationships among the structures, processes, and outcomes of pain management. The experience of pain is intertwined with its emotional impact (suffering), which can be equally or even more difficult to quantify and compare between individuals. Moreover, many factors affect the quality of care, including practice patterns (assessment and treatment behaviors), decision-making, policies, and patient outcomes.
A long-standing criticism of QI has been the lack of scientific rigor in the measurement methodologies used to collect the data that raise questions regarding the reliability, validity, and integrity of the data obtained for QI outcomes. Contributing to these perceptions is the inability to appreciate the fundamental differences among measurement instruments necessary for QI, research, and performance (external accountability). QI data are collected to gain a better understanding of the extent and nature of clinical or systems issues and the motivation for change. These data establish, evaluate and reevaluate targets for improvements. QI processes and measures must be capable of capturing indicators of care, and should be constructed to allow ease of use by clinicians in routine care delivery, meaningful interpretations across repeated time points, and facilitate rapid change in care processes when appropriate. Measurement for QI does not require the same evidentiary rigor across populations as is usually demanded of more general research as it is not primarily intended to generate new knowledge of widely generalizable or universal value.
However, it is important that measurement instruments for QI meet essential psychometric criteria while retaining feasibility of administration and clinical meaningfulness in its interpretation.
American Pain Society Committee on Quality Assurance Standards: American Pain Society quality assurance standards for the relief of acute pain and cancer pain, in Bond MR, Charlton JE, Woolf CJ, (eds): Proceedings of the VIth World Congress on Pain. Seattle, WA, IASP Press, 1991, pp185-188
published its first APS-POQ as part of quality assurance (QA) standards for the treatment of acute and cancer pain to assist health care organizations to explore patient experiences and outcomes. In 1995, the APS QA standards were revised and published as QI guidelines
along with an updated APS-POQ. Updates at that time were based on published reports and clinical experience with the original APS-POQ and on previously validated epidemiologic instruments.
Initial measures included ratings of pain severity, pain interference on daily activities, patient satisfaction, perceived wait time for analgesics, patient beliefs, and clarity of instructions about analgesic use at time of hospital discharge.
A systematic review of 20 QI studies of pain that utilized the 1995 APS-POQ involving 3527 patients with medical, surgical, and cancer diagnoses in inpatient settings suggested that a number of modifications were needed to the existing tool based on a revised set of core quality indicators to guide measurement of quality for improvement purposes.
Six core quality indicators were recommended to provide measurement of the processes and outcomes of pain management. These included categories for (1) use of numeric or descriptive rating scales for pain assessment, (2) documentation of pain intensity at frequent intervals, (3) treatment of pain by a route other than intramuscular; (4) administration of analgesics on a regular schedule, and when possible, use of a multimodal treatment regimen, (5) prevention and control of pain to a degree that facilitates function and quality of life, and (6) provision of adequate information so that patients are knowledgeable about pain management. These indicators require the collection of data from the patient as well as from the nursing and medical records.
Several specific changes were recommended to the 1995 APS-POQ including the addition of items to quantify the amount of time patients spent in moderate to severe pain, adaptations of the functional interference and satisfaction items, and appraisal of whether the patients perceived that the clinicians involved them in their pain management and in discussing possible options for pain control. The original survey items measuring patient beliefs and perceived wait time were no longer considered to have meaningful relationships to other quality indicators. In 2005, the APS revised and expanded its 1995 guidelines for QI reiterating the need for multiple health care system level changes and how pain-related QI outcomes are measured.
Because formal construct validity testing had not been performed on the original APS-POQ, it became a priority to establish and confirm measurement domains in a new instrument. Although no specific item modifications to the existing POQ tool were recommended, the authors of this report decided that the evaluation of existing items and the construction of new ones should be based on the new QI guidelines by capturing the 5 aspects of (1) pain severity (pain), (2) interference with function (activities), (3) affective experiences (emotional), (4) side effects (safety), and (5) perceptions of care (satisfaction).
This report presents the systematic processes used to revise the APS-POQ to capture the 5 patient reported aspects of the QI guideline document in a way that was consistent with the recommendations of the APS. We also present the testing procedures used to ensure the reliability and initial validity of the revised instrument. The specific aims for this quality improvement-based study were to (1) add and revise appropriate items relevant to the conduct of QI for pain care; (2) administer the revised instrument to a heterogeneous sample of hospitalized patients; (3) test for internal consistency reliability (Cronbach α), test for item grouping using factor analysis and conduct an initial validation by comparing differences among distinct patient populations within our sample; and (4) determine the relationships among items and subscales, and variables that may predict outcomes.
Another basic assumption was that the revised instrument should be easy to administer, with understandable components and have a simple scoring system for the subscales to facilitate its use by health care providers in the evaluation of the quality of the pain care provided. If properly designed, the APS-POQ-R should be able to detect differences in outcomes among patient populations on specific items (eg, pain intensity levels) and subscales (eg, interference with function). It will also be possible to determine if nonpharmacological techniques play a role in increasing patients' participation in their care. By comparing the results obtained in different populations, health care professionals should be able to determine what patient characteristics affect the quality of pain care and then make adjustments to that care to improve patient satisfaction.
Methods
Development of the Revised APS-POQ (APS-POQ-R)
Ten interdisciplinary members of the APS (5 nurses, 2 physicians, 2 psychologists, and a pharmacist) with experience in using the APS-POQ and expertise in pain management, QI and instrument development were invited to participate in a task force to revise the APS-POQ. Two distinct phases were undertaken to create the APS-POQ-R. First, items from the original APS-POQ were extensively discussed by the committee. By consensus, items perceived to have not been helpful in QI (eg, perceived wait time and patient beliefs) or poorly designed (eg, adequacy of instruction…) were eliminated. Items were added to address the recommendations put forth in a previous publication reviewing 10 years of experience with the original instrument.
The second phase involved administration of the revised instrument to patients in various clinical settings who were experiencing different types of acute and chronic pain, with psychometric testing of the instrument in that population.
The principal determinations for retaining or deleting items were based on item-to-item intercorrelations, and item-to-subscale correlations by calculating the internal consistency reliability (Cronbach α), with item deletion. Once the final set of items had been determined, a principal component varimax factor rotation was performed to verify the structure. Initial validity testing was also conducted by assessing for significant differences among biologically appropriate subgroup cohorts. Group comparisons were performed using Student t tests and 1-way analyses of variance (ANOVAs) to detect differences among patient populations based on the primary type of pain (acute, chronic cancer-related pain, and chronic nonmalignant pain). Bivariate correlations with Pearson product-moment (r) were conducted to assess the magnitude of relationships between variables and subscales. Linear and logistic regressions were used to identify significant variables and subscale scores that were predictors of satisfaction.
Phase 1
An abbreviated process for developing outcomes measures for clinical trials
was used to guide and build consensus on an updated APS-POQ for field testing. This involved identifying the overall question and theoretical approach, establishing the target population, determining the format and scale properties and methods for analysis, locating gaps in construct (content domains), and developing instructions for instrument administration. Starting with the original 27-item APS-POQ, items that were ambiguous or not useful were removed and new items added to cover the appropriate domains. Although different scalar methods could have been used for some of the questions, the 0 to 10 NRS was chosen for all responses for consistency, evidence of cross cultural consistency
Serlin RC, Mendoze TR, Nakamura Y, Edwards KR, Cleeland CS: When is cancer pain mild, moderate, or severe: Grading pain severity by its interference with function. Pain 61:277–228, 1995
Max M. Portenoy R. Laska E. Advances in Pain Management Research and Therapy. Vol. 18. Single-dose analgesic comparisons. Raven Press, Ltd,
New York1991
Subsequent drafts were reviewed by email and teleconference.
Pain Severity
The original APS-POQ included pain severity ratings (0 = no pain, 10 = worst pain possible) at the time the survey was completed (“right now”), and worst and average rating over the past 24 hours. During the survey revision, “pain now” was deleted because it was felt to be highly influenced by the variable proximity of analgesic dosing to survey completion in the acute hospital setting. The item was replaced with a question about the “least amount of pain.” Least pain has previously been shown to be a useful measure of severity.
Measuring least and worst pain ratings helps appraise the full range of pain experienced during rest and activity common in acute pain. Because of the importance for QI of the amount of time in pain and to reduce the overall patient burden for the scale, average pain was replaced with an item querying the percentage of time severe pain was experienced over 24 hours in an attempt to capture the “area under the curve” or amount of time being spent with a level of pain known to negatively impact persons.
Pain Relief
A measure of pain relief was also added to assess the patient's perception of the pain control provided by the treatments provided. A conventional method of analyzing pain relief in research is to calculate a change in pain rating for each patient and then compare the mean value of the change between the treatment and placebo groups.
Laska EM, Meisner M, Siegel C: Analytic approaches to quantifying pain scores, in Max M, Portenoy R, Laska E (eds): Advances in Pain Research And Therapy. Vol 18. New York, Raven Press, 1991, pp 675-683
Analgesic trials have used derived measures to indicate the magnitude of pain relief as summed pain intensity differences (SPID) or total pain relief (TOTPAR) using visual analog scales (VAS) or numeric rating scales (NRS).
Max M. Portenoy R. Laska E. Advances in Pain Management Research and Therapy. Vol. 18. Single-dose analgesic comparisons. Raven Press, Ltd,
New York1991
While achieving a clinically important improvement in pain ratings is a useful goal, use of these measures requires data collection at prescribed and consistent time intervals and a calculation of the percentage difference that would be more difficult with most QI study resources.
Pain intensity and relief measures are complex constructs that may best be described as a reflection of a number of internal and external factors and meanings for persons with pain.
In general, QI differences are examined over variable time frames after practice changes for populations and often using historical controls. In addition, relief category scales have been reported to be sensitive to small reductions in pain.
Max M. Portenoy R. Laska E. Advances in Pain Management Research and Therapy. Vol. 18. Single-dose analgesic comparisons. Raven Press, Ltd,
New York1991
Therefore, we chose to include a pain relief scale for the APS-POQ-R that measured with the widely used horizontal scale of a percentage (0% to 100%) adapted from the Brief Pain Inventory (BPI)
to measure the degree of pain relief obtained from all of the patient's pharmacological and nonpharmacological treatments combined.
Side Effects
Side effects of treatment can significantly compromise safety, quality of life, patient adherence, and ability to re-medicate. Multimodal therapy is encouraged in pain treatment to both reduce pain, minimize side effects when possible, and to treat side effects when they occur as is often the case with analgesic therapy. To date, there is no consistent scale used for the reporting of side effects or adverse events in analgesic trials.
Therefore a new set of items was added to assess both the presence and intensity of the most common medication side effects. Previous pain QI studies using this approach have demonstrated significant changes in side effect profiles through improved treatment patterns.
Pain Interference Scale for Physical and Emotional Function
The original APS-POQ also included a subset of items modeled after the BPI designed to evaluate how pain interferes with functioning and well-being in the outpatient setting. One factor analysis model of the BPI has yielded 2 underlying interference dimensions: interference with activity (walking, work, general activity, sleep) and interference with affect (mood, enjoyment of life).
These items were adapted for the APS-POQ-R to focus on the types of activities common to hospitalized patients and included questions to assess the range of positive and negative moods as is recommended when developing a measure of emotional functioning.
were added, including “how pain caused the patient to feel,” “anxious,” “angry,” “depressed,” “frightened,” “frustrated,” “helpless,” and “overwhelmed,” along with 2 well-being (positive emotions); “at ease” and “content.” Previous investigators
have found differences in pain-related and psychological distress and the amount of variance in interference in daily activity explained by both.
Participation in Decision-Making and Measurement of Satisfaction
Measurement of satisfaction with pain management is extremely complex, responses are almost always skewed toward the positive, and results can be difficult to interpret.
Using the American Pain Society's patient outcome questionnaire to evaluate the quality of postoperative pain management in a sample of Norwegian patients.
The original APS-POQ focused on perceived “wait time” for pain medications and satisfaction with staff and overall treatment results. Based on the Institute of Medicine (IOM) recommendations to increase patient's participation in treatment planning
a new item was added regarding how much patients were allowed to participate in decision-making.
Use of Nonpharmacological Interventions
The work cited above also encouraged the provision of information about the pain and the integration of physical, psychological, and social interventions with pharmacological strategies to improve patients' sense of participation in their care and to improve pain outcomes. Nonpharmacological related therapies are now widely accepted as appropriate for pain care and have demonstrated impact on patient outcomes and satisfaction.
American Society of Anesthesiologists Task Force on Acute Pain Management Practice guidelines for acute pain management in the perioperative settings: An updated report by the American Society of Anesthesiologists Task Force on Acute Pain Management.
In one study, approximately 40% of people used some form of nonpharmacological techniques consistent with the biopsychosocial model of pain and experienced a greater degree of relief of symptoms.
To date, documentation of these strategies is rarely included in patient records and thus medical record audit does not provide an accurate reflection of the gaps or benefits of this component of pain care. Subsequently, 3 new items were added: (1) to assess the helpfulness of pain care information provided, if any, (2) to assess the types of nonpharmacological strategies patients use to control pain, and (3) whether health care providers encouraged the use of nonpharmacological strategies. Because these 3 questions record information about the patient's interaction with their health care providers rather than how they are experiencing pain, they constitute a separate domain and were not included in our primary factor analysis. We refer to them as the informational items.
Assessment of Patient Understanding of the APS-POQ Questions
A temporary item was used in field testing that asked patients about any aspect of the survey they did not understand to help reduce response errors that can occur if questions are not interpreted in the manner they were intended. If any consistent misunderstanding of a question was reported, careful consideration was to be given to changing or dropping the question as appropriate.
Sample
Between January 28, 2008, and December 16, 2008, 299 adult inpatients were recruited from 2 US academic medical centers: a 471-bed hospital in the Midwest and a 708-bed hospital located in a South Atlantic state. The study protocol was reviewed and given exempt status by the institutional review boards at both sites. All study procedures were in accordance and compliance with the Health Insurance Portability and Accountability Act (HIPPA) regulations and the institution's policies and guidelines for protection of human subjects.
A convenience sampling method for recruiting patients was used at each hospital. The criteria for inclusion were age over 18 years, English speaking, and being alert enough to respond. A nurse project team member (not directly involved in the care of the patient) approached inpatients at a convenient time, used a letter of invitation to explain the study, and to obtain verbal assent. Survey items were read aloud to avoid questions about literacy. Patients were asked to participate within 72 hours of medical admission or surgery and surveyed regarding the first 24 hours of inpatient pain management they had received. Although the accuracy of pain recall has been challenged,
Understanding recall of weekly pain from a momentary assessment perspective: Absolute agreement, between-and within-person consistency, and judge change in weekly pain.
This methodology has been used routinely in pain QI and is consistent with feasibility and clinical meaningfulness within this setting.
Involvement in the study was voluntary and patients were given assurances of confidentiality and that their care would not be affected in any way whether they agreed to participate or not. Project team members had access to the hospital census through their hospital employment or graduate student status. The patient's medical record was also audited for demographics and analgesic treatment. All forms were coded. Individual participation and responses were not shared directly with staff. There were 199 participants recruited from one institution and 100 from the other.
Data Analysis: Phase 2 Overview
After data collection and verification was completed descriptive statistics (ie, means, standard deviations, and frequencies) were calculated for all patient characteristics and APS-POQ-R response items. Means and range of responses (minimum and maximum scores) and frequencies were examined for the 23 primary items of the APS-POQ-R that measured the patients experience with pain. The decisions to retain or delete items from this group were based on the results of psychometric testing along with findings from the literature, critical reasoning and expert consensus about the relative contribution of each item to the measurement domains of the quality of pain care. The need for brevity, clarity, and conciseness also influenced decisions regarding the potential value of items to clinicians who would be conducting quality assessments of pain care using this tool.
Criteria for Item Deletion
The principal determinations for retaining or deleting items were based on item-to-item intercorrelations, item-to-subscale correlations by calculating the internal consistency reliability (Cronbach α), with item deletion, relevance of the item supported by the literature, factor structures or item groupings elucidated by exploratory principal component factor analyses, and perceived conceptual confusion of an item by experts. Every effort was made to reduce item burden and to retain only those items believed to provide the greatest value to QI activities.
Once the final set of items had been determined, construct validity was evaluated by both an exploratory principal components oblique and varimax factor rotation for all the items, to examine the factor structure of the instrument. A principal component varimax factor rotation was performed again with items that were retained for the final instrument. Initial validity testing was also conducted by assessing for significant differences among biologically appropriate subgroup cohorts. Group comparisons were performed using Student t tests and 1-way ANOVAs to detect differences among patient populations based on the primary type of pain (acute, chronic cancer-related pain, and chronic nonmalignant pain). Bivariate correlations with Pearson product-moment (r) were conducted to assess the magnitude of relationships between variables and subscales. Linear and logistic regressions were used to identify significant variables and subscale scores that were predictors of satisfaction. All analytical procedures were performed using SPSS V15.0 (SPSS Inc, Chicago, IL).
Results
Sample characteristics for respondents (n = 299) are summarized in Table 1. Average age for participants was (55.60 ± 17.1 years; range, 19 to 95 years) and the sample was comprised of comparable numbers of males and females, (males, n = 138; 46%). There was no significant difference (P = .668) in the mean age of participants between study sites, and similar percentages of males (43.5% vs 51%) and females were recruited from each site. Patients recruited for the study were either on a medical (n = 172) or surgical service (n = 127), and patients were further categorized by diagnosis (eg, cancer or no cancer) by service. There were no meaningful differences between participant characteristics by medical services, however a larger percentage of the sample from hospital 1 had surgical oncology patients (19.5%) versus 3% from hospital 2. Hospital 2 had a higher percentage of noncancer surgical patients (35%) compared with hospital 1 (17.5%). Types or sources of pain were not delineated for each patient. Although the treatment types and amount(s) of analgesics administered were also collected on all patients through matched medical record audits, these data are not presented in this report.
Table 1Sample Characteristics for Respondents (n = 299)
Means, Standard Deviations, and Frequencies of the APS-POQ Items
The modified APS-POQ administered for this study consisted initially of 26 items. There were 23 primary items in which the response was measured by the 0 to 10 NRS which we treated as a continuous scale. Table 2A provides the means and standard deviations for these items. Additional items, referred to as the “Information” items, are reported in (Table 2B). One used a Likert scale with categories 1 = never, 2 = sometimes, and 3 = often, to measure how often a nurse or doctor encouraged nonpharmacological methods. Two other items involved dichotomous yes/no responses; 1 for whether patients received information about their pain treatment and 1 related to the use of nonpharmacological measures for the treatment of pain. A “yes” response was followed by a request for a more specific response. One hundred fifty-eight participants (53.2%) reported that they had received information about their pain treatment options, and the mean score for how helpful this information was to patients was 7.85 ± 2.3 (range, 0 = not at all helpful to 10 = extremely helpful). Of those who indicated that they had used nonpharmacological measures (n = 184; 61.7%), participants identified primarily distraction (56.8%), prayer (49%), deep breathing (35.9%), heat (34.9%), cold pack (29.1%), and relaxation (24.2%) Fewer participants reported using walking (15.3%), massage (14.7%), meditation (12.2%), music (9.7%), and imagery or visualization (5.8%) to relieve pain.
Table 2AMeans and Standard Deviations for Each of the 23 Primary Continuous Items on the Original APS APS-POQ
N
Minimum
Maximum
Mean
SD
Least pain in 24 hours
299
0
10
3.44
2.9
Worst pain in 24 hours
299
0
10
7.60
2.6
Estimate of percentage of time in severe pain
298
0%
100%
37.80
32.5
Pain interfered or prevented you from activities in bed
298
0
10
5.37
3.7
Pain interfered or prevented you from activities out of bed
291
0
10
5.53
3.6
Pain interfered or prevented you from falling asleep
298
0
10
4.30
3.8
Pain interfered or prevented you from staying asleep
297
0
10
4.20
3.8
How much the pain caused you to feel anxious
299
0
10
4.37
3.6
How much the pain caused you to feel angry
297
0
10
2.93
3.6
How much the pain caused you to feel depressed
298
0
10
2.80
3.4
How much the pain caused you to feel frightened
299
0
10
2.70
3.4
How much the pain caused you to feel frustrated
299
0
10
5.18
3.6
How much the pain caused you to feel helpless
298
0
10
4.43
3.9
How much the pain caused you to feel overwhelmed
299
0
10
4.12
3.9
How much the pain caused you to feel at ease
294
0
10
3.39
3.6
How much the pain caused you to feel content
294
0
10
3.16
3.6
Severity of nausea
299
0
10
2.55
3.5
Severity of drowsiness
299
0
10
3.75
3.5
Severity of itching
299
0
10
1.90
3.1
Severity of dizziness
299
0
10
1.56
2.6
Pain relief in the first 24 hours
298
0%
100%
68.44
28.2
Were you allowed to participate in decisions about pain treatment?
297
0
10
7.91
3.0
How satisfied are you with the results of your pain treatment?
The results of correlations and tests for internal consistency reliability (Cronbach α) for the 23 primary items are shown in Table 3. For the majority of items a lower score was associated with a more positive response. Reversed scoring was used for items where a higher score was more favorable. Cronbach α for all 23 items was .85. Table 3 shows the item-to-total correlations and changes in the overall Cronbach α with each item deleted from the questionnaire. Item-to-item correlations for each of the original stem questions were assessed for the magnitude and direction of relationships. Specially, inter-correlation coefficients for items within specific categories or clusters such as those pertaining to interference, affective experiences, and side effects from analgesic therapies were calculated (Table 3). Moderate to high inter-correlation coefficients can be indicative of the interrelatedness of the items that may reflect conceptual overlap or items measuring a similar concept of construct.
Table 3Subscale Item-to-Total Correlations and Cronbach α
Subscale Mean if Item Deleted
Scale Variance If Item Deleted
Corrected Item-Subscale Total Correlation
Cronbach α If Item Deleted
Affective Subscale (Cronbach α = .82)
How much the pain caused you to feel anxious
9.95
78.56
.659
.773
How much the pain caused you to feel depressed
11.51
81.42
.658
.773
How much the pain caused you to feel frightened
11.62
81.47
.685
.726
How much the pain caused you to feel helpless
9.89
78.41
.597
.804
Pain Severity and Sleep Interference Subscale (Cronbach α = .83)
Least pain in 24 hours
19.79
120.37
.552
.836
Worst pain in 24 hours
15.63
123.12
.570
.818
Estimate of percentage of time in severe pain
19.46
107.15
.676
.787
Pain interfered or prevented you from falling asleep
18.95
95.25
.716
.774
Pain interfered or prevented you from staying asleep
19.03
97.73
.678
.787
Perceptions of Care Subscale (Cronbach α = .70)
Pain relief in the first 24 hours
15.94
23.11
.454
.689
Were you allowed to participate in decisions about pain treatment?
14.84
20.14
.498
.646
How satisfied are you with the results of your pain treatment?
15.55
23.12
.627
.498
Activity Interference Subscale (Cronbach α = .82)
Pain interfered or prevented you from activities in bed
5.53
13.02
.694
N/A
Pain interfered or prevented you from activities out of bed
Of the 4 interference items, a high item-to-item correlation existed between interfered or prevented “falling asleep” and “staying asleep” (r = .78; P < .001). However, the sleep literature
supports these items as measuring 2 distinct aspects of the sleep experience, and therefore both were retained in the instrument. All 4 had item-to-total correlations ranging from .46 to .63.
Of the 9 new affective items, high item-to-item correlations were found between “ease” and “content” (r = .85, P < .0001), and low item-to-total correlations (≤.40). Removing each of these items raised the Cronbach α by .02. In addition, approximately 5% of the respondents in our study indicated they did not understand what was being asked with these 2 questions. Although these positively stated emotions were believed at the time of item generation to be of importance to the measurement of quality care, our evaluation of these items led to their elimination. In addition, a bivariate correlation between “frustrated” and “helplessness” showed a moderate correlation (r = .61; P < .001). “Frustrated,” while also moderately correlated with other negative affective experiences (range, r = .46 to .63), was also eliminated as it was found to be ambiguous without any evidence to support its importance as a component for quality of pain care. Similarly, “overwhelmed” and “angry” showed little effect on changes in internal consistency reliability for the negative emotion item set, and therefore these were removed to consolidate questions related to the affective aspects of pain experiences. Moreover, these experiences are not typically included in quality assessments for pain and satisfaction with pain care surveys. Correlational statistics and reliability testing favored 4 of the remaining affective items (“anxious,” “depressed,” “frightened,” and “helpless”) for inclusion in the final revised version of the APS-POQ-R.
Construct Validity Testing
Exploratory Factor Analyses
To evaluate all 23 primary items for construct validity further, an exploratory, principal components factor analysis with both varimax and oblique rotations were performed. The oblique rotation assumes a level of interrelatedness of all items which is not valid in this group of items and did not yield an interpretable component matrix. A varimax orthogonal rotation which assumes a level of independence of the constructs revealed 6 interpretable factors with Eigen values >1 accounting for over 66% of the variance in the measure. Acceptable factor loading coefficients (≥.4) were evident for all of the items on a least 1 of the factors. The positive emotional items, “at ease” and “content” loaded on an isolated factor, and did not have acceptable factor loadings with the other affective items. Given that a number of respondents noted problems in understanding these items and our interest in creating an assessment tool with an ease of administration to patients and interpretability of the results, these items were removed.
In total, 5 items were dropped from the original 23 primary items. A second exploratory principal components factor analysis was conducted with the 18 remaining primary continuous level items identified for the final version of the APS-POQ-R. This exploratory principal components varimax factor analysis was generated to verify item groupings that would ultimately define the subscales for the instrument. Five factors were identified with Eigen values >1 (range, 5.98 to 1.05), and all items loaded on at least 1 of the factors with an acceptable loading coefficient of ≥.5, except for itching. In general, itching occurs less frequently than the others but was maintained due to the clinical importance of this side effect. Together the items accounted for 64.05% of the variance in the measure. Table 4 shows the rotated factor component matrix and factor loadings for each item. Factor 1 contained 4 items related to the emotional components of pain, the “Affective Subscale,” and explained 31.36% of the variance in the measure. The “Pain Severity and Sleep Interference” subscale accounted for 11.23% of the variance and included “worst” and “least” pain experienced in the first 24 hours, percent time over the past 24 hours in “severe pain,” and the 2 interference with sleep items. “Perceptions of Pain Care,” the third factor, was defined by percent pain relief, degree of participation in decisions, and satisfaction with pain treatment (8.42% of the variance). The “Interference with Activity” subscale (6.9% of the variance) included 2 items related to pain interfering with activities in and out of bed. The fifth subscale, “Adverse Effects,” captured the severity of side effects associated with pain treatments (6.14% of the variance).
Table 4Rotated Factor Component Matrix and Factor Loadings for Continuous Level Variables
Factor Components
1
2
3
4
5
Total Items Retained 18 Name of Subscale
Affective
Pain Severity and Sleep Interference
Perceptions of care
Activity Interference
Adverse Effects
Variance explained Total 64.05%
31.36%
11.23%
8.42%
6.9%
6.14%
Cronbach α Overall 0.86
0.82
0.83
0.70
0.82
0.63
Least pain in 24 hours
−.035
.528
−.450
.235
.162
Worst pain in 24 hours
.200
.535
−.151
.420
.130
Estimate of percentage of time in severe pain
.029
.627
−.406
.317
.139
Pain interfered or prevented you from activities in bed
.135
.229
−.074
.791
.068
Pain interfered or prevented you from activities out of bed
.194
.203
−.110
.807
.073
Pain interfered or prevented you from falling asleep
.343
.822
−.010
.089
.102
Pain interfered or prevented you from staying asleep
.268
.812
−.068
.090
.064
How much the pain caused you to feel anxious
.734
.227
−.126
.224
.021
How much the pain caused you to feel depressed
.745
.209
−.240
−.067
.129
How much the pain caused you to feel frightened
.804
.151
−.190
.014
.098
How much the pain caused you to feel helpless
.673
.097
−.134
.403
.102
Severity of nausea
.138
.140
.079
.026
.713
Severity of drowsiness
.079
.024
−.024
.304
.694
Severity of itching
−.217
.013
.069
.397
.460
Severity of dizziness
.138
.132
−.063
−.124
.762
Pain relief in the first 24 hours (%)
−.176
−.143
.699
−.117
.070
Were you allowed to participate in decisions about pain treatment?
−.167
−.017
.745
−.005
−.025
How satisfied are you with the results of your pain treatment?
The item clusters identified reflect important criteria for the measurement of quality indicators for pain care and patient experiences. A mean score was calculated for each of the subscales. The 2 items, time spent in severe pain and percentage of pain relief received, were normalized to 0 to 10 scales. Mean scores and standard deviations for all of the subscales were as follows for subscales with higher scores indicating poorer outcomes: Affective subscale (3.57 ± 2.9), Pain Severity and Sleep Interference subscale (4.68 ± 2.6), Interference with Activity (5.47 ± 3.4), and Adverse Effects subscale (2.44 ± 2.21). For Perceptions of Pain Care where higher scores indicate better outcomes, the mean score was (7.63 ± 2.2).
Overall Cronbach α for the revised item set was .86. Cronbach α for each subscale were as follows: Affective subscale, α = .82; Pain Severity and Sleep Interference subscale, α = .83; Perceptions of Pain Care subscale, α = .70; Interference with Activity, α = .82; and Adverse Effects subscale, α = .63. Although an alpha ≥.70 demonstrates acceptable internal consistency, lower thresholds for subscales ≥.6 can be considered acceptable as a measurement of internal consistency, especially when there are fewer items in the subscale
and the items are considered to be important for physiologic reasons. A lower Cronbach α for the Adverse Effects subscale may also be explained by the absence of the presence of adverse effects in a substantial portion of respondents. One hundred sixty-seven (55.9%) reported no nausea, 109 (36.5%) had no drowsiness, 195 (65.2%) indicated no itching, and 189 (63.2%) did not experience dizziness.
An intercorrelation matrix constructed for the subscales is presented in Table 5. None of the subscales was highly correlated suggesting that these constructs may be partially independent of one another and measuring relatively discrete concepts of quality care and patient experiences. Each of the subscales was significantly correlated in a predictable fashion, except for correlation between Adverse Effects and Perception of Care, supporting the interrelated aspects of these outcomes of care.
Overall group comparisons by service indicated that medical patients had a significantly lower mean satisfaction scores (7.88 ± 2.7; P < .01) compared with surgical patients (8.59 ± 1.9). No significant difference was found in participation in decision-making between medical and surgical patients (P = .059). One-way ANOVAs with post hoc analyses using least significant differences were conducted on outcomes among service and diagnosis, but no meaningful differences were detected.
Student t test for independent groups were used to assess differences between users of nonpharmacological techniques for pain control (n = 184) and those who did not (n = 114) for level of participation in decision-making and, how satisfied patients were with their care and worst pain intensity. No difference was found for participation in decision-making between patients using (7.69 ± 3.1) and not using (8.33 ± 2.7) nonpharmacological strategies (P = .063). However, patients using nonpharmacological strategies were significantly less satisfied than those who reported that they were not using these strategies (mean, 7.91 ± 2.6 vs mean, 8.63 ± 1.9; P = .049, respectively). Reported worst pain levels were significantly higher among users compared with nonusers (8.04 ± 2.3 vs 6.9 ± 2.9; P = .001, respectively).
Prediction Models
To determine relationships among items and subscales and variables that may predict outcomes for items and specific subscales, specific aim 4, multiple linear regression analyses were used to determine the relationships of various subscales to how satisfied patients were with their pain treatment (Table 6). Age, sex, percent time spent in severe pain, information received about treatment options (yes/no), amount of pain relief received, and level of participation in care were selected to assess whether 1 or more of these was a significant predictor of patient satisfaction with care. A bivariate correlation of worst pain and time spent in severe pain showed a moderate correlation, r = .55 (P < .001). Only time spent in severe pain was included since it is believed to be more predictive of patient satisfaction. The items including less time in severe pain (P = .023), information received (P = .048), and greater pain relief and level of participation (P< .0001) were each associated with greater satisfaction, and together these variables explained a significant portion of the variance in this outcome (R2 = .403; F = 31.48; P < .0001). Overall, these findings validate that several of the items are important components in the measurement of satisfaction with pain care.
Table 6Regression Model for Patient Characteristics and Care Outcomes as Predictors of Satisfaction
Four of the APS-POQ-R subscales “Affective,” “Pain Severity and Sleep Interference,” “Interference with Activity,” and “Adverse Effects” were entered into a stepwise linear regression model to verify their contributions to the outcome of satisfaction. A significant portion (adjusted R2 = .182) of the variance in the level of satisfaction was explained by 2 of the subscales (F = 48.28; P < .001); lower Pain Severity and Sleep Interference (F = 29.68; P < .001) and Affective subscale scores (F = 22.84; P = .001) were associated with better satisfaction. Adverse Effects subscale scores were positively associated with better satisfaction (P = .006), but the average value in this population was low and it had a minimal effect in contributing to the total adjusted R2, only raising it by 1.9%. Interference with Activity was not statistically significant and also demonstrating a minimal effect on the variance in the level of satisfaction.
Discussion
A systematic analytic approach was used to revise the APS-POQ which resulted in the new version titled the APS-POQ-R. To assess the psychometric properties, the internal consistency reliability and initial construct validity of the APS-POQ-R were found to be good, as was its prediction value for patient satisfaction. Further evaluation of the psychometric properties for the instrument will be needed to support its reliability and validity as an acceptable measure for QI activities in other populations and its value in directing and reevaluating pain care. The APS-POQ-R seems to provide a practical instrument that is clinically feasible with adequate psychometric properties to help clinicians standardize measurement of important aspects of the quality of pain management for QI activities. The instrument is designed to be administered to gather information about the quality of pain management for hospitalized patients during the first 24 hours of care.
Principal component factor analysis revealed 5 important subscales of the instruments 18 primary continuous items including Affective Distress, Pain Severity and Sleep Interference, Perceptions of Care, Interference with Activity, and Adverse Effects. The results support the instrument as a measure of interrelated yet separate components of the quality of pain management. In addition, the new APS-POQ-R captures adult patients' perceptions of pain control and the degree that pain control facilitates physical function and emotions during hospitalization as well as the adequacy of information received about pain treatment options and ability to participate in those decisions. Additionally, using 3 additional questions, the survey directly collects information about the prevalence and types of nonpharmacological strategies used to manage pain in the hospital. Patient report of nonpharmacological strategies can be combined with information gathered on drug treatment from medical record audits to examine other core indicators of the quality of pain treatment; namely pain is treated by a route other than IM, analgesics are administered on a regular schedule, and when possible treatment is multimodal.
Item reduction was a key concern with the aim of keeping the patient burden of the survey to a minimum. However, a critical quality indicator in pain management is that pain is prevented or controlled to a degree that facilitates improved function and quality of life.
Although a comprehensive psychological assessment of this domain would entail a large number of items, we found that 4 of the 9 items we added were adequate to cover a majority of the symptoms experienced in our pain patient population. As such, the APS-POQ-R now includes how pain causes patients to feel anxious, depressed, frightened, and helpless. The Affective subscale explained the largest amount of variance in the measure. As expected, patients who spent less time in severe pain reported less affective distress. Higher scores on negative emotions were inversely correlated with patient satisfaction.
Although positive emotional items were tested (eg, how pain caused one to feel at ease and content) they added little to our measure, patients commented that they did not understand the items and their inclusion only lowered the internal consistency reliability, perhaps related to the fact that most patients with pain have a significant underlying medical problem. Given this assumption, these were excluded from the final instrument.
It should be noted that our analysis demonstrated a small but statistically significant positive correlation between the Adverse Events subscale and satisfaction. In general, adverse events are an important consideration in a patient's treatment experience and our results would appear to be counter intuitive. However, the overall number and level of adverse events in our sample was quite low, and so would adversely affect only a small number of patients' reports of satisfaction. It is also possible that the low level of the adverse effects is a reflection of patients having received more effective doses of analgesics leading to a higher level of satisfaction. In either case, the influence of adverse effects on satisfaction requires further study in populations who have higher rates of severity.
To put this study into perspective, the IOM recommends including consumer satisfaction indices in any evaluation of quality.
Using the American Pain Society's patient outcome questionnaire to evaluate the quality of postoperative pain management in a sample of Norwegian patients.
patient satisfaction with pain management is highly variable and dependent on many other factors than the experience of pain. Consequently, it is a poor measure of the quality of pain control when used alone. Patient satisfaction is nevertheless an important measure in quality evaluation, and a better understanding of what patients include in their assessment of satisfaction can be achieved through the use of a multi-item questionnaire, such as the APS-POQ-R, to measure the various aspects thought to contribute to satisfaction. The IOM highlighted the phrase “desired health outcomes” in defining quality to mean the health outcomes that patients desire.
This sheds light on the importance of including patient's goals and their satisfaction with goal attainment, in addition to medical outcomes, in any assessment of the quality of care. Prediction models in our study confirmed that a variety of items impact patient satisfaction including the amount of time in severe pain, the amount of pain relief obtained, and the ability of the person to participate in decisions about pain treatment. The IOM Quality Chasm report
further elucidated the need to include patient's perspectives when they proposed 10 new design rules for enhancement of quality. Information about safety, evidence-based practice, and patient satisfaction should be shared with patients and families to allow them to make informed decisions when choosing among available treatments. In this study, only 17.5% of the sample reported a level of participation in decisions about pain management (as much as they wanted) ≤5 on a scale of 0 to 10, with10 being very much so. Still, ongoing efforts to improve communication with patients and include them as partners in determining plans of care are important in all clinical settings. It is disconcerting that almost one half (46.8%) of patients reported they had not received any information about pain control. This finding emphasizes the need for targeting this aspect of care for improvement in quality.
Limitations
There are a number of limitations to this study. The survey involved a voluntary convenience sample of patients at 2 sites that may not be representative of all inpatient settings. The majority of the participants were Caucasian, thus more data is needed using the APS-POQ-R with other racial or ethnic groups. A population of hospitalized patients who were confused, too ill, or unable to communicate was not included. The focus of the APS-POQ-R is on adult inpatients and results may not be applicable to pediatrics, outpatient or emergency department settings. A potential confounding issue is whether the patients in this study were reporting on their chronic or acute pain. However, one could assume that there is a mix of both acute and chronic noncancer pain problems in both medical and surgical populations. The APS-POQ-R is reflective not so much of the type of pain being treated, but rather the outcome of the acute care setting's response to a patient's primary pain problem. The survey is also intended to measure very different aspects of quality so it is difficult to establish unidimensionality with standard psychometrics typically used to describe what the survey is measuring, in this case multidimensional quality. Finally, the study design did not allow for analysis of test-retest reliability and sensitivity to change. These properties should be investigated in future research.
Conclusion
As has been described, the construct of quality pain management is complex. It is dependent on many practical processes of the care provided, the staff behavior, and patient experiences. A critical element of quality evaluation is obtaining direct feedback from patients about their experiences. The updated APS-POQ-R has been found to have adequate psychometrics for QI and to measure 5 important aspects of the patients experience with their pain and a sixth aspect of nonpharmacological therapies. These 6 aspects include (1) pain severity and relief; (2) impact of pain on activity, sleep, and negative emotions; (3) side effects of treatment; (4) helpfulness of information about pain treatment; (5) ability to participate in pain treatment decisions; and (6) use of nonpharmacological strategies. If further work verifies these findings, then hospitals should be encouraged to use this patient survey along with medical record audits to measure and follow-up on improvements in the quality of pain management. Clearly, more quality improvement-based research must be conducted to examine necessary adaptations and the usefulness of the APS-POQ-R in pediatrics, with specific ethnic groups, and in outpatient settings. This measure should help to examine the relationship between analgesic treatment and patient perceptions of quality of care.
Acknowledgments
The authors thank Jun Lee, Christen Greenfield, Kate McGettrick, Eva Allen, and Hui-Hsin for assistance with data collection. We also thank Elizabeth Hansson, Winfried Meissner, and Ruth Zaslansky for their review and thoughtful comments on the manuscript. We would also like to recognize the late Mitchell Max for his founding work in pain QI.
References
American Pain Society Committee on Quality Assurance Standards: American Pain Society quality assurance standards for the relief of acute pain and cancer pain, in Bond MR, Charlton JE, Woolf CJ, (eds): Proceedings of the VIth World Congress on Pain. Seattle, WA, IASP Press, 1991, pp185-188
American Society of Anesthesiologists Task Force on Acute Pain Management
Practice guidelines for acute pain management in the perioperative settings: An updated report by the American Society of Anesthesiologists Task Force on Acute Pain Management.
Using the American Pain Society's patient outcome questionnaire to evaluate the quality of postoperative pain management in a sample of Norwegian patients.
Laska EM, Meisner M, Siegel C: Analytic approaches to quantifying pain scores, in Max M, Portenoy R, Laska E (eds): Advances in Pain Research And Therapy. Vol 18. New York, Raven Press, 1991, pp 675-683
Max M. Portenoy R. Laska E. Advances in Pain Management Research and Therapy. Vol. 18. Single-dose analgesic comparisons. Raven Press, Ltd,
New York1991
Serlin RC, Mendoze TR, Nakamura Y, Edwards KR, Cleeland CS: When is cancer pain mild, moderate, or severe: Grading pain severity by its interference with function. Pain 61:277–228, 1995
Understanding recall of weekly pain from a momentary assessment perspective: Absolute agreement, between-and within-person consistency, and judge change in weekly pain.
00351-2&id=gr1a.jpg){kind=link}
00351-2&id=gr1b.jpg){kind=link}