Study design. The study was conducted in 2 phases. Phase I involved developing a new measurement tool and evaluating its content validity; phase II involved psychometric validation of Ostomy-Q. A schematic overview of the study design is shown in Figure 1.
Participant sample. Recruitment of all participants for both phases took place in the UK over 6 months. Participants were recruited from the manufacturer end-user panel, an online forum maintained by Coloplast A/S, Denmark (“the manufacturer”) for contact with users (N = 488). Potential participants were screened online to ensure they met the eligibility criteria, which stipulated they should be a resident of the UK, 18 to 85 years old, have an ileostomy or colostomy, have used ostomy appliances for a minimum of 3 months, handled ostomy appliances themselves (ie, without help from others), and able to complete the online survey or interview. Participants were excluded if they had an acute illness or cognitive impairment that in the opinion of the investigator would interfere with the study requirements. All participants gave online informed consent before participating.
Protocol. A protocol was developed for both phases of the study that was granted ethical approval by the Salus Institutional Review Board (United States).
Literature review. A targeted literature review was undertaken in order to understand the evidence regarding the impact of ostomies and appliances on people’s health-related quality of life (HRQL). EMBASE and PubMed were searched using the terms ostomy, colostomy, ileostomy, ostomy and quality of life, and health-related quality of life. This search identified 12 relevant studies that detailed physical (eg, skin irritation1), emotional, and relationship issues.
Dabirian et al1 conducted a qualitative study in which 14 patients with ostomies were interviewed about their QoL; 9 themes emerged, including physical problems (such as rash, lack of sleep), psychological problems (low mood), relationships (family life), costs, nutrition, physical activity, travel, sexual relationships, and religion. Using a questionnaire, Nugent et al2 had participants (N = 391) assess postoperative care, QoL issues, and equipment issues; major problems noted included rashes (51%), leakage (36%), and ballooning, and the latter 2 were found to cause embarrassment, distress, and sleep disturbance. The fear of unpleasant gases and general use of ostomy appliances can have a negative effect on social relationships and inhibit participation in leisure activities.1,2 Additional research3,8,9 including observational studies has highlighted concerns regarding leakage, ballooning, and inability to conceal the pouch.
Additional psychological problems reported by people with an ostomy include reduced confidence, anxiety, depression, and stigma often related to risk of appliance leakage.3,10-12 To explore practical methods that can be taken to lessen a patient’s fear of embarrassment and ridicule after the surgery, Noone10 described the case of a woman who underwent stoma surgery. Danielsen et al12 conducted focus groups with 15 people with permanent ostomies to understand the affect of ostomies on daily living; participants reported they wanted control and more education from health care professionals regarding their new lives with an ostomy. Participants also mentioned isolating themselves to avoid disclosing their stoma to people.
Existing assessment instruments. The literature also was reviewed to examine existing scales, and several scales were identified that were designed to assess the affect of ostomy care. These included: Stoma-QoL13 Ostomy Adjustment Scale,7 City of Hope Quality of Life: Ostomy Instrument,14 and the Ostomy Adjustment Inventory-23 (OAI-23).15 Most of these instruments evaluated different aspects of the impact of having an ostomy, such as quality of life and physical activity. The Stoma-QoL was of key interest because it was designed to specifically assess HRQL associated with using an ostomy appliance. Therefore, it was reviewed and used as a base for the current research.
The Stoma-QoL was developed to assess QoL for individuals with a colostomy or ileostomy based upon detailed qualitative research (N = 169). This 20-item, unidimensional HRQL instrument is based on Maslow’s Hierarchy of Needs theory,16 a Rasch-based model. The Stoma-QoL captures relevant QoL-related concepts for people with an ostomy, including social interaction, anxiety, and body image, in a single score. Although the Stoma-QoL is built upon a sound foundation of qualitative research, the current authors believe the use of the unidimensional structure derived from the Rasch model is overly restrictive in terms of its assumptions. Therefore, the authors sought to develop a multidimensional scale that relaxed the model assumptions. The content of the Stoma-QoL was used as a starting point for the development of a multidimensional scale to assess the affect of an ostomy appliance on some specific aspects of QoL.
Instrument development and testing. The literature review was used to guide the development of items for the Ostomy-Q. The items in the Stoma-QoL measure were reviewed by the study team (the study authors and 2 speciality nurses from the manufacturer panel). The review was done by assessing items against the team members’ clinical experience and knowledge to see if anything relevant was missing. New items were drafted based on the findings from the literature review and the reviewers regarding what had been reported as important. Five (5) ostomy appliance users from the manufacturer user panel reviewed the first draft of the survey to determine whether the proposed content included important issues. The interviews also were designed to gather general feedback from participants regarding how well they understood each item.
Content validity of the draft Ostomy-Q was evaluated by conducting cognitive debriefing/exploratory telephone interviews with the same 5 individuals with an ileostomy or colostomy. The participants were asked to evaluate item comprehension and interpretation, completeness of item coverage, relevance, clarity, and readability of Ostomy-Q. The instruction recall period of 1 week and respondent burden (in terms of length and complexity of the items) also were assessed. The interviews involved a think-aloud and retrospective approach that enabled participants to speak freely about how well they understood an item and how it could be improved. The interviewer then recorded the answers. The interview provided flexibility to allow the interviewer to adapt the questions to suit each individual participant; this enabled the interviewer to understand the responses specific to each person.17
The interviews were analyzed using an interview grid developed for the study that included each person’s response to each question to evaluate the information gathered; minor revisions to the new instrument were made based on the interviews. Two (2) additional interviews were conducted with new users from the panel to assess user understanding and interpretability of the revised instrument during which participants were asked to explain what they interpreted to be the meaning of each item and its response. The tested version of the Ostomy-Q measure included a total of 23 items.
Participants completed a sociodemographic form, clinical background form, and the questionnaires. Following completion of the interviews, participants were offered remuneration in the form of points that could be used in an online shop for ostomy users.
The tested version of the Ostomy-Q included 23-items rated on a 5-point Likert scale ranging from 1 = strongly agree to 5 = strongly disagree. The items were divided into 4 domains — Discreetness, Comfort, Confidence, and Social life — as identified from the literature review as important to ostomy users. These 4 issues or concepts reflected specific aspects of QoL, but they were not designed to describe all domains of QoL.
Phase II. In phase II, the Ostomy-Q underwent psychometric validation. A total of 200 participants with an ileostomy or colostomy participated in an online survey. In order to evaluate convergent validity, participants were asked to complete the Stoma-QoL and the anxious preoccupation and social engagement domains of the Ostomy Adjustment Inventory-23 (OAI-23), which measured the validity of the new instrument against existing instruments. A subset of participants (N = 82) was invited to complete the survey again approximately 2 weeks later to assess test-retest reliability. Furthermore, a series of tests were applied to assess the psychometric performance of Ostomy-Q (see Figure 2).
Measures. Participants completed sociodemographic and clinical background forms that included demographic and clinical data such as age, date of diagnosis, and treatment (see Table 1A and 1B). They also completed the new Ostomy-Q instrument (Version 1), including transition items (items relating to each of the new instrument domains discreetness, comfort, confidence, and social life) to assess test-retest reliability, the revised Ostomy-Q instrument, Stoma-QoL, and the 2 domains from OAI-23 (anxious preoccupation and social engagement). Participants took 45 minutes to 1 hour to complete the exercise.
Analysis. All data were collected and stored electronically in locked files. Descriptive statistics were used to summarize demographic/clinical data. All instruments were scored and summarized for each participant. A series of different aspects of the psychometric performance of the instrument was explored.
Item variability and conceptual structure. The total number of responses and the percentage of the total responses for each item were calculated. Any floor and ceiling effects for each item were evaluated and defined to occur when 50% of the responses were in the lowest or highest response category for any item.18,19 If floor and ceiling effects occur, it is difficult to measure the influence of the appliance on the given item.
A principal component analysis (PCA) was undertaken to explore the extent to which the selected items naturally grouped into the 4 domains (ie, the extent to which the items load or share variance with the hypothesized domain). An exploratory approach was used because the conceptual framework was only hypothesized at this stage. This analysis included an oblique rotation, which allows the emergent domains to naturally correlate with each other.
Internal consistency. Internal consistency helps determine the homogeneity of the items within each of the conceptual domains. Cronbach’s alpha20 was used to assess the internal consistency reliability of each conceptual domain, and minimum alpha values of 0.70 (acceptable consistency21) or 0.80 (good consistency22) were provided as guidelines to determine a domain or total score as internally consistent.
Item-total correlation. The item-total correlation (ie, the correlation between an individual item and the overall domain score) was used to assess the homogeneity of the Ostomy-Q. Usually, items should have a significant correlation ≥0.2023 (although a higher requirement of 0.30 has been proposed24); this was used to interpret the performance of the Ostomy-Q. Factor analysis and PCA were conducted in order to understand the relationship between the items and overall structure of the instrument. PCA looks at a set of observations from a large dataset and converts them into a smaller set of values (principal components). The proposed items of each domain should cluster together to show consistency with the domain.
Test-retest reliability. The intraclass correlation coefficient (ICC) was used to test for test-retest reliability25 and was assessed by correlating domain scores between baseline and 2 weeks after baseline in participants reporting no change in their use of ostomy appliances and those reporting no change on the relevant global change items (N = 82).
The ICC ranges from 0 to +1.0 and can be interpreted as the proportion of within-user variability. Although no wide agreement exists regarding benchmarks to help interpret the ICC, scale-level ICCs of ≥0.80 have been proposed.22 For the purposes of this study, the following thresholds were used: scale-level ICC <0.6 = poor test-retest reliability; 0.6 to 0.69 = moderate; 0.7 to 0.79 = good; 0.8 to 1.0 = very good. Pearson’s correlation coefficients and t-tests also were conducted to assess the stability of the measure over time.
Convergent validity. Convergent validity refers to the extent a measure relates to other measures or variables based on theoretical content or the expected relationship with the variable chosen. Convergent validity of the Ostomy-Q was evaluated using 2 validated instruments: the original Stoma-QoL and the anxious preoccupation and social engagement domains of the OAI-23. Because the current measure was somewhat based upon the Stoma-QoL, these 2 measures were compared for consistency. Associations between these measures were explored using parametric and/or nonparametric correlations as appropriate. Convergent validity was considered supported if correlation coefficients between related scales were >0.40.
To explore the responsiveness of the Ostomy-Q over time, data from a clinical trial26 involving 129 people with an ostomy were used. This trial employed a cross-over design and explored the performance of a newly developed ostomy appliance compared with the participant’s current ostomy appliance. To explore these issues, the data from both trial arms from the clinical trial and current study were merged. Responsiveness was assessed in terms of sensitivity and minimal important difference (MID).
Sensitivity and MID. Sensitivity assesses the extent to which the measure’s subscale scores reflect changes in users’ experience of the underlying constructs. Sensitivity was estimated in terms of effect size and standardized response mean (SRM). Mean scores for people using their own ostomy appliance (period 1) were compared to mean scores where they tried the test product (period 2) for 4 weeks. The results from the sensitivity analysis also are interpreted in qualitative terms (eg, moderate or high) using established criteria.27
An instrument’s MID can be analyzed in different ways. Statistical methods based on variance and dispersion have been shown to provide useful indicators of MID in the literature, including observational studies and clinical trials.28 An anchor-based approach also can be used to determine the minimal change over time, but this relies on having access to a suitable anchor or marker of change. In a previous trial conducted by the manufacturer (trial CP232), no explicit anchor was included. Some analyses have been explored in which a 25% and 50% reduction in leakage onto clothes is used as a benchmark. In the second phase of the trial, the current study investigated whether the Ostomy-Q is sensitive to the presence or absence of any output leaking onto clothes.
Distribution-based methods offer an alternative and rely on expressing an effect in terms of the underlying distribution of the results.27 The standard error of measurement (SEM) and half a standard deviation are both widely used and accepted methods for estimating MID.29 In the current study, both estimates were considered in order to settle on a single value by averaging the 2 estimates.
Three (3) different anchors were tested in the study — 2 within-group MIDs were estimated in terms of a 25% and a 50% reduction in leaks onto clothes. In addition, a between-group estimate of MID was defined in terms of the presence of absence of any leak onto clothes.