The Satisfaction with Communication in Everyday Speaking Situations (SCESS) scale: An overarching outcome measure of treatment effect
Introduction
International health care policies universally specify that randomized controlled trials are the gold standard of health care evidence, with the highest attainable level of evidence coming from a systematic meta-analytic review of many such trials (Centre for Evidence Based Medicine, 2009; Cochrane, n.d.; National Health & Medical Research Council, 1999). There is an increasing acceptance of the CONSORT statement as a universal reporting standard for stuttering treatment, as shown in recent comparative trials of stuttering treatment (Arnott et al., 2014; De Sonneville-Koedoot, Stolk, Rietveld, & Franken, 2015; Mallick, Kathard, Thabane, & Pillay, 2018; Murza & Nye, 2009; Onslow, Jones, O’Brian, Menzies, & Packman, 2008).
It is pertinent to stuttering treatment research that the CONSORT statement (CONSORT, 2010) strongly suggests only one primary outcome in a randomized trial, defined as “the pre-specified outcome considered to be of greatest importance to relevant stakeholders such as patients, policy makers, clinicians, and funders”. To have more than one primary outcome “incurs the problems of interpretation associated with multiplicity of analyses and is not recommended” and might lead to “selective outcome reporting”. The issue that this presents for stuttering treatment research is that outcomes of interest span a range of behavioral and nonbehavioral (psychosocial) domains. However, in addition to one outcome measure, the CONSORT statement states that researchers can use as many secondary outcome measures as needed for the study purpose.
It is well known that adults who stutter can often have complex psychosocial experiences as a result of stuttering, (Craig & Tran, 2014; Iverach et al., 2009; Stein, Baird, & Walker, 1996). Consequently, treatment targets for those clients will be behavioral or nonbehavioral targets, or both. It is possible to do a systematic review of a set of different treatments for adults who stutter, each of which has a different primary outcome dealing with a different treatment altogether. But meta-analysis, which provides an estimate of clinical effect size that is more powerful than those of the individual trials included in the analysis, would not be possible. Table 1, Table 2 demonstrate some examples of the most-frequently used stuttering-related outcome measures that have been used in treatment studies during the last two decades.
To compare effects sizes obtained with stuttering treatments that have disparate behavioral and nonbehavioral treatment goals, a single overarching measure of outcome is needed. The need for doing so was expressed by Baxter et al. (2015) in a review of stuttering treatment:
The review summarizes evidence from a substantial body of work and indicates the potential for positive outcomes from a range of interventions. It highlights the need to identify key agreed outcomes that should be used both to evaluate and to compare the effectiveness of interventions … (p. 677).
Ideally, any such outcome measure needs to take account of client needs and to document how the treatment has dealt with the concerns that prompted the client to seek professional help (Baer, 1988). In other words, a measure of what is personally significant to the client (Bothe & Richardson, 2011; Ingham, Ingham, & Bothe, 2012).
Bothe and Richardson (2011) make the point that such information needs to be elicited from clients by questioning and raises the prospect of surveying client satisfaction for that purpose. This notion has been incorporated within stuttering outcome measures to date. Section Four of the Overall Assessment of the Speaker’s Experience of Stuttering (OASES) scale (Yaruss & Quesal, 2006) includes four specific items pertaining to “how much does stuttering interfere with your satisfaction with communication … in general … at work … in social situations …at home”. Within the self-report section of the Stuttering Severity Instrument (SSI-4; Riley, 2009) but not included within the composite numerical index is the question “how satisfied are you with the treatment you have received?” with responses from 1 = very satisfied to 9 = very dissatisfied. Huinck and Rietveld (2007) reported a treatment outcome scale that incorporates a measure of speech satisfaction on a 10-point scale from 1 = very bad to 10 = excellent. In addition to including the notion of client satisfaction, an overarching outcome index for chronic stuttering would have additional features that enhance its utility. In order to facilitate its use by clinical researchers who conduct clinical trials, it needs to be brief and easy to administer, with no requisite training or equipment. Its brevity would allow it to be freely available without cost. An issue with commercially available test materials such as the OASES, Wright and Ayre Stuttering Self-Rating Profile (WASSP; Wright & Ayre, 2000) and SSI-4 is that their cost can be seen as an impediment to clinical trials, and their commercial status impedes their widespread use because their translation to many other languages is not commercially viable. Translation to many languages would enable an outcome measure to be used internationally so that treatment effect sizes could be compared for not only different styles of treatments, but treatments conducted within different cultures and health care settings. The time required to administer these measures is another consideration. The WASSP has 26 items, the OASES-A has 100 items, and the SSI-4 takes considerable time to administer because it involves speech transcription and analysis.3 Another issue with the above measures is that they have not been validated for use as overarching primary measures to compare effect sizes of different stuttering treatments. They have been developed and their content and construct validity have been tested for different purposes such as measuring the impact of stuttering on personal life or assessing behavioral aspects of stuttering. Theoretically, a measure might be valid and relevant to a construct but not for every purpose. For example, a measure that is suitable for screening for a disorder might be unsuitable for testing the magnitude of that disorder. Similarly, a scale or questionnaire which is developed for clinical purposes might be unsuitable for use in research settings. Therefore, no such an overarching primary outcome measure has yet been reported to compare effects of different stuttering treatment programs, although some valid and reliable stuttering-related measures are available for other purposes.
Life satisfaction has been assumed to be one of the main three components of well-being (Arthaud-Day, Rode, Mooney, & Near, 2005) and also the most appropriate and adequate concept associated with quality of life (Moons, Budts, & De Geest, 2006). Life satisfaction reflects overall evaluation of life (Pavot & Diener, 2008), while domain satisfaction deals with judgment about specific aspects of life, such as occupation (Stubbe, Posthuma, Boomsma, & De Geus, 2005). The relationship between life satisfaction and domain satisfaction is not clearly understood. However, Schimmack and Oishi (2005), comparing different top-down and bottom-up models of satisfaction, concluded that ‘… domain satisfaction is the most proximal determinant of life satisfaction, and examining the determinants of domain satisfaction can provide important information about the determinants of life satisfaction’(p. 404). Therefore, it seems that measuring satisfaction with communication with those who stutter will provide valuable information about quality of life, and even about their general well-being.
During the last three decades, patient satisfaction measures have been widely reported in diverse fields of health care, including speech-language therapy (Keilmann, Braun, & Napiontek, 2004; Pershey & Reese, 2003).
No generally accepted hypothesis is currently available that describes the underlying process by which people might feel more or less satisfied with treatment in health care settings (Hudak, Hogg-Johnson, Bombardier, McKeever, & Wright, 2004). However, the “overall clinical outcome” hypothesis attributes better clinical outcomes to greater patient satisfaction. “Foresight expectation” and “hindsight expectation” hypotheses attribute satisfaction to fulfilling predicted expectations and receiving better than expected outcomes, respectively (Hudak et al., 2004). Therefore, the client’s satisfaction seems highly related to the outcomes that they expected.
We propose a nine-point scale where respondents answer a single, simple question: “considering all the issues associated with your stuttering, how satisfied are you with your communication in everyday speaking situations at the present time?” The SCESS scale has 1 = extremely satisfied as the most positive response and 9 = extremely dissatisfied as the most negative response and there are no descriptors of other scale scores. The scale is presented in Fig. 1 and can be reproduced for use during clinical trial outcome assessment.
The present report develops the SCESS scale by determining its content and construct validity and reliability.
Section snippets
SCESS content validity
Content validity is defined as “the degree to which elements of an assessment instrument are relevant to and representative of the targeted construct for a particular assessment purpose” (Haynes, Richard, & Kubany, 1995, p. 238). Therefore, several steps were taken to establish the content validity of the SCESS. First, a comprehensive review of literature published in English was conducted for interventions reporting developmental stuttering interventions, outcome measures of developmental
Participants stuttering severity
A Kolmogrov-Smirnov test showed that stuttering severity distribution of the participants was non-normal for %SS (p = .001) and self-rated severity scores (p = .02). This result is similar to what has usually been reported for stuttering populations; stuttering severity distribution is positively skewed, with more speakers with lower %SS and severity rating scores than with higher ones as shown in Fig. 2 (O’Brian, Packman, Onslow, & O’Brian, 2004). The mean %SS score was 4.6 (SD = 5.8, range
Discussion
During randomized controlled trials one primary outcome is desirable (CONSORT, 2010). We argue that a systematic meta-analytic review of many trials can occur if this primary outcome reflects a fundamental treatment gain that overarches all stuttering treatments, supplemented with secondary outcomes reflecting the intended positive behavioral or non-behavioural changes. This paper proposes and evaluates the SCESS as a single overarching measure of treatment effect for adult stuttering treatment
Acknowledgements
This research was supported by the National Health and Medical Research Council (Program Grant #633007).
Dr. Hamid Karimi completed his PhD at the Australian Stuttering Research Center, The University of Sydney in 2013. He is an associate professor at the Isfahan University of Medical Sciences, Iran. He has worked for more than 10 years in the area of stuttering as a clinician, teacher and researcher.
References (73)
Relapse following treatment for stuttering: A critical review and correlative data
Journal of Fluency Disorders
(1998)- et al.
Trait and social anxiety in adults with chronic stuttering: Conclusions following meta-analysis
Journal of Fluency Disorders
(2014) - et al.
A welfare economic approach to measure outcomes in stuttering: Comparing willingness to pay and quality adjusted life years
Journal of Fluency Disorders
(2012) - et al.
A standalone Internet cognitive behavior therapy treatment for social anxiety in adults who stutter: CBTpsych
Journal of Fluency Disorders
(2014) - et al.
A pilot study into a possible relationship between diet and stuttering
Journal of Fluency Disorders
(2017) - et al.
Death anxiety and its role in psychopathology: Reviewing the status of a transdiagnostic construct
Clinical Psychology Review
(2014) - et al.
Prevalence of anxiety disorders among adults seeking speech therapy for stuttering
Journal of Anxiety Disorders
(2009) - et al.
A cross-cultural, long-term outcome evaluation of the ISTAR Comprehensive Stuttering Program across Dutch and Canadian adults who stutter
Journal of Fluency Disorders
(2006) - et al.
Personality dysfunction in adults who stutter: Another look
Journal of Fluency Disorders
(2013) - et al.
Critique on the conceptualization of quality of life: A review and evaluation of conceptual approaches
International Journal of Nursing Studies
(2006)
The reliability, validity and utility of the SAD and FNE scales for anxiety disorder patients
Personality and Individual Differences
Anxiety of children and adolescents who stutter: A review
Journal of Fluency Disorders
Validity of the Social Avoidance and Distress and Fear of Negative Evaluation scales
Behaviour Research and Therapy
Overall Assessment of the Speaker’s Experience of Stuttering (OASES): Documenting multiple outcomes in stuttering treatment
Journal of Fluency Disorders
Group lidcombe program treatment for early stuttering: A randomized controlled trial
Journal of Speech Language and Hearing Research
The subjective well-being construct: A test of its convergent, discriminant, and factorial validity
Social Indicators Research
What is this thing called measurement error?
Statistical methods for assessing measurement error (reliability) in variables relevant to sport medicine
Sports Medicine
If you know why you’re changing a behavior, you’ll know when you’ve changed it enough
Behavioral Assessment
Anxiety and its disorders
The state of the art in non-pharmacological interventions for developmental stuttering. Part 1: A systematic review of effectiveness
International Journal of Language & Communication Disorders
The Beck Depression Inventory-II
Treatment of chronic stuttering: Outcomes from a student training clinic
International Journal of Language & Communication Disorders
Intensive stuttering modification therapy: A multidimensional assessment of treatment outcomes
Journal of Speech Language, and Hearing Research
Statistical, practical, clinical, and personal significance: Definitions and applications in speech-language pathology
American Journal of Speech-Language Pathology
Randomized controlled non-inferiority trial of a telehealth treatment for chronic stuttering: The Camperdown Program
International Journal of Language & Communication Disorders
Oxford Centre for Evidence-based Medicine–Levels of evidence (March 2009)
Camperdown Program for adults who stutter: A student training clinic Phase I trial
International Journal of Language & Communication Disorders
CONSORT statement
Randomized controlled trial of video self-modeling following speech restructuring treatment for stuttering
Journal of Speech Language, and Hearing Research
Self-modelling as a relapse intervention following speech-restructuring treatment for stuttering
International Journal of Language & Communication Disorders
Direct versus indirect treatment for preschool children who stutter: The RESTART randomized trial
PLoS One
Stand-alone Internet speech restructuring treatment for adults who stutter: A Pilot study
Journal of Clinical Practice in Speech-Language Pathology
Standalone Internet speech restructuring treatment for adults who stutter: A phase I study
International Journal of Speech-Language Pathology
G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences
Behavior Research Methods
Cited by (10)
The good, the bad, and the ugly: Unpacking the pros and cons associated with change for adults who stutter
2022, Journal of Fluency DisordersStuttering interventions for children, adolescents, and adults: a systematic review as a part of clinical guidelines
2022, Journal of Communication DisordersCitation Excerpt :Moreover, the primary measure of the studies is too often the percentage of syllables stuttered (%SS), which reflects only the overt aspects of stuttering in a specific situation, not the speaker's experience of stuttering. For PWS, the holistic experience and treatment of stuttering may be more important than the exact amount of overt stuttering (see e.g., Karimi et al., 2018; Rodgers & Gerlach, 2021). It is important to include both the results of effectiveness research and experience-based knowledge, ethical factors, contextual factors, and clients’ perceptions to build a total evidence and knowledge approach (McCurtin et al., 2019).
What works for whom? Multidimensional individualized stuttering therapy (MIST)
2020, Journal of Communication DisordersCitation Excerpt :This category contains, among others, relaxation in general, taking enough speech breaks, concentration on speaking manner, concentration on even and relaxed breathing while speaking, and reduction of speech tempo. Perceived speaking ability and confidence in communication have been found to be important clinical factors (Hayhow, Cray, & Enderby, 2002; Karimi et al., 2018; Siew, Pelczarski, Yaruss, & Vitevitch, 2017). These perceptions relate to, among others, the individual’s perception of his or her stuttering, speaking abilities, speech naturalness and knowledge about treatment options.
What do people search for in stuttering therapy: Personal goal-setting as a gold standard?
2020, Journal of Communication DisordersCitation Excerpt :This was very closely associated with an eagerness to learn better strategies for dealing with stuttering, and/or to cope with stressful feelings associated with social, academic or work-related settings. Participants’ perspectives on speaking ability and stuttering interference in communication were identified as central factors, as described in the previous literature (Karimi et al., 2018; Siew et al., 2017). As a consequence of communication difficulties, people who stutter may experience that social activity is challenged (Bricker-Katz et al., 2013; Craig et al., 2009; Erickson & Block, 2013; O’Brian et al., 2011), as confirmed by the present study’s participants.
Further development in measuring communicative participation: identifying items to extend the applicability of the communicative participation item bank
2023, Journal of Patient-Reported OutcomesSleep Problems, Social Anxiety and Stuttering Severity in Adults Who Do and Adults Who Do Not Stutter
2023, Journal of Clinical Medicine
Dr. Hamid Karimi completed his PhD at the Australian Stuttering Research Center, The University of Sydney in 2013. He is an associate professor at the Isfahan University of Medical Sciences, Iran. He has worked for more than 10 years in the area of stuttering as a clinician, teacher and researcher.
Professor Mark Onslow is the foundation Director of the Australian Stuttering Research Centre at The University of Sydney. His research interests are epidemiology of early stuttering, mental health and stuttering, measurement of stuttering, and clinical trials for the disorder.
Dr Mark Jones is a Senior Lecturer in Biostatistics at the School of Public Health, University of Queensland. He obtained his PhD at the Australian Stuttering Research Centre, University of Sydney, and has a strong research interest in stuttering.
Dr Sue O’Brian is a Senior Researcher at the Australian Stuttering Research Centre. She has extensive experience in the field of stuttering treatment and research. Her current interests include the effectiveness of early stuttering intervention in community settings, development of treatments for adults who stutter and stuttering measurement.
Associate Professor Ann Packman is a Principal Research Fellow at the Australian Stuttering Research Centre. She has worked for more than 30 years in the area of stuttering as a clinician, teacher and researcher. One of her current interests is theories of the cause of stuttering.
Associate Professor Ross Menzies is a clinical psychologist with an interest in the origins and management of anxiety. He has developed cognitive behaviour therapy packages for the treatment of obsessive compulsive disorders and published theories of the origins of phobias. He is currently the director of the Anxiety Clinic at The University of Sydney.
Professor Sheena Reilly is the Pro Vice Chancellor (Health) at Griffith University, having held numerous senior positions within the academic, health and research sector over the past 20 years. Professor Reilly is a leading international expert, renowned for her research that has transformed our current-day understanding, clinical practice and policy of speech, language and literacy problems in children. Her extensive research track record is evidenced by more than 200 peer-reviewed articles, in excess of $50 million in competitive grants and is the recipient of prestigious awards including the inaugural Fellow of the Australian Academy of Health and Medical Sciences.
Martin Sommer graduated from Aachen Medical School, Germany. After studies in Toulouse, France, and at the NIH, Bethesda, Maryland, USA, he went to Goettingen Medical School, Germany, for his residency in Neurology and for further scientific training. There he is consultant and professor at the Department of Clinical Neurophysiology. His clinical and research focus is the (patho) physiology of movements, particularly in Parkinson’s disease and dystonia. Himself stuttering, he employs imaging and neurophysiological tools such as transcranial magnetic stimulation in research elucidating the pathophysiology of persistent stuttering. He currently serves as president of the German Stuttering Association (BVSS).
Suzana Jelčić-Jakšić is a Speech pathologist at the Children's Hospital Zagreb, Croatia. She is President of the Croatian Association for People Who Stutter "Hinko Freund" and Director of Logopedski Centar, Zagreb. She has a Bachelor and a Masters degree, and a PhD, in speech pathology from the University of Zagreb.
- 1
Present address: Communication Disorders Research Center, Isfahan University of Medical Sciences, Isfahan Province, Iran.
- 2
Present address: Australian Stuttering Research Centre, University of Technology Sydney, Ultimo, Australia.