Fluency Bank: A new resource for fluency research and practice

https://doi.org/10.1016/j.jfludis.2018.03.002Get rights and content

Highlights

  • The National Institutes of Health and National Science Foundation have funded a new data repository (FluencyBank) and tools to permit greater datasharing among fluency researchers.

  • FluencyBank also hosts a new teaching resource for instructors of university-level fluency disorders classes.

  • FluencyBank has developed one new computational tool (FluCalc) for fluency researchers and clinicians, and offers additional computational utilities for clinical appraisal of patients/clients with fluency disorders.

  • FluencyBank has implemented a new, standard set of fluency codes to enable datasharing across language communities with differing orthographies.

Introduction

The National Science Foundation (NSF) and the National Institute on Deafness and Other Communication Disorders (NIDCD) have recently provided funding to establish FluencyBank (https://fluency.talkbank.org) as a new component of the larger TalkBank system (https://talkbank.org). The purpose of this article is to explain how FluencyBank will work to extend our understanding of the nature and development of typical and disordered fluency in both children and adults. To ground our discussion, we review the overarching organization of TalkBank and its component databases, and describe common features of TalkBank datasets. We then address the relation of FluencyBank to the overall TalkBank project. In doing this, we will discuss the specific funded research goals of FluencyBank. Finally, we will describe the resources that TalkBank and FluencyBank provide to fluency researchers, instructors, and clinicians with interests in typical and disordered fluency.

TalkBank is the world’s largest open-access repository of data on spoken language. For an extensive summary of the technical aspects of TalkBank, see MacWhinney (in press). The TalkBank initiative began in 2000 as an extension of the Child Language Data Exchange System (CHILDES), established in 1984 by Brian MacWhinney and Catherine Snow (see MacWhinney & Snow, 1990). In the first years of development of the CHILDES system, most corpora were represented only in the form of computerized transcripts, although a few had accompanying media. Currently, new TalkBank corpora include transcripts linked to media (audio and video) on the utterance level, as well as extensive annotations for morphology, syntax, phonology, gesture, and other features of spoken language. All TalkBank corpora can be browsed online, or downloaded for additional annotation and analysis. As we note, many continue to be used to generate new research findings on an ongoing basis.

An important principle underlying the TalkBank approach is that all data are transcribed in a single consistent format, called CHAT (MacWhinney, 2000). This format has been developed over the years to accommodate the needs of a wide range of research communities and disciplinary perspectives. TalkBank also makes available an extensive and free set of analysis programs, called CLAN, that rely on the fact that all TalkBank data use the CHAT transcription format. The CLAN programs and manuals, along with related morphosyntactic taggers that automatically insert part-of-speech and grammatical analysis into transcripts, are freely available and downloadable from the website at https://talkbank.org, for PC, Mac and Unix platforms.

The use of standard formats and codes is particularly important for the field of fluency studies. These conventions include a detailed set of fluency codes that permit automatic computational analysis across data from any language community. For example, blocking is marked with the Unicode symbol ≠ and sound iterations are marked by the Unicode character ↫. Entry of these characters is facilitated through keyboard shortcuts and a dropdown menu. These codes replace the various idiosyncratic codes developed in numerous separate labs to mark stuttering and other forms of disfluency. Use of these standards allows users to convert historical datasets to a standard transcription format with good fidelity of fluency marking. We discuss this issue in greater detail later in this article.

To facilitate use of CLAN analysis programs by researchers or clinicians who have employed other methods of transcription, CLAN includes a series of free programs to convert to CHAT from SALT (saltsoftware.com), Praat (praat.org), Phon (childes.talkbank.org/phon), ELAN (tla.mpi.nl/tools/elan), and LENA (lenafoundation.org) formats, among others.

TalkBank includes over a dozen specialized open-access language banks, all using the same transcription format and standards. These banks include CHILDES for child language acquisition, AphasiaBank for aphasia and other neurodegenerative language conditions, PhonBank for the study of phonological development and disorder, TBIBank for language in traumatic brain injury, DementiaBank for language in dementia, HomeBank for daylong audio- and video-recordings in the home, CABank for Conversation Analysis, SLABank for second language acquisition, ClassBank for studies of language in the classroom, BilingBank for the study of bilingualism and code-switching, and additional smaller banks that are under development. As noted in the Introduction, the most recently funded initiative is FluencyBank, for the study of the development of fluency and disfluency across the lifespan. Each of these components of TalkBank can be accessed from the overall TalkBank index page at https://talkbank.org.

The current size of the TalkBank text database is 800MB, with an additional 5TB of media data. New data are being added continuously. The majority of data in the various components of TalkBank are freely open for browsing, downloading and analysis. However, access to the research data in the clinical banks such as AphasiaBank and FluencyBank requires a password, and access to the data in HomeBank requires further attention to methods for safe-guarding the use of untranscribed day-long audio gathered in fully naturalistic settings.

These language banks have had a substantial impact on wide areas of research, as measured by the large number of publications that have used the data and programs. To date, CHILDES, which is the oldest and most widely recognized database, has been used to provide data for over 7000 published articles. PhonBank has been used in almost 500 articles, and AphasiaBank has been referenced in over 200 publications in only a decade since creation. To donate data to TalkBank can be rewarding: The contributors of TalkBank corpora benefit from bibliographic attribution and citation in these publications. To systematize the citation process, each corpus is assigned a DOI (Digital Object Identifier) number which users are required to cite. In addition, each corpus is described on a web page that includes links for downloading data and media, DOI information, corpus documentation, photos and contact information for the contributors, and articles to be cited when using the data.

TalkBank is an international and cross-linguistic project. Transcription is supported for all orthographies. The free CLAN program provides morphological/syntactic tagging (annotation for part-of-speech and grammatical analysis) for Cantonese, Chinese, Dutch, English, French, German, Hebrew, Japanese, Italian, and Spanish. These quickly and automatically insert information about each word’s part of speech, grammatical function, and additional linguistic information (such as case, number, tense and gender marking for all words in the transcript). Both the availability of numerous free language parsers and automatic tagging (rather than hand-coding, as in alternatives such as SALT), make CLAN uniquely useful to both researchers and clinicians. To demonstrate, the simple command MOR (short for “morphological analysis”) applied to a sentence such as the italicized ones below, produces the added information immediately below it, for one or multiple transcripts, almost instantaneously. The clinician or researcher only needs to type the speaker’s intended words and to annotate perceived disfluencies; no linguistic knowledge is necessary to produce this level of analysis:

*SLP: do you think that intensive programs like the Hollins program might be more useful now that you’re older?

%mor: mod|do pro|you v|think pro:dem|that adj|intensive n|program-PL prep|like art|the n:prop|Hollins n|program mod|might cop|be qn|more adj|use&dn-FULL adv|now rel|that pro|you∼aux|be&PRES adj|old-CP?

*CLI: and &-you_know I've read up on what I think <that> [/] that &-um &-you_knowhhow s:tuttering can s:ometimes be cured through &-umpsps:ychological counseling.

%mor: coord|and pro:sub|I∼aux|have v|read&ZERO adv|up prep|on pro:int|what pro:sub|I v|think pro:rel|that adv:int|how n:gerund|stutter-PRESP mod|can adv|sometimes aux|be part|cure-PASTP prep|through adj|psychological n:gerund|counsel-PRESP.

A full, linked, browsable transcript with accompanying media can be found for readers to view at https://fluency.talkbank.org/browser/index.php?url=Examples/Tom.cha We reiterate that the transcriber noted only what the speaker said, and noted disfluencies, since this level of detection is not capable of automation at present. Then the transcript was treated using the simple one-word MOR command, and resulted in the browsable version at the link above. We realize that the grammatical annotations in these transcripts can resemble gobbledygook for those not entrenched in linguistic analysis, and encourage readers to consult the listing of common abbreviations used in syntactic analysis starting on page 19 of the MOR manual at the Talkbank site (https://talkbank.org/manuals/MOR.pdf). We have included definitions for many of them in Appendix 1. The program automatically reads proper names and modifiers via capitalization, and disregards fluency notations that interrupt the citation forms of lexical entries in assigning part of speech and morphological inflections.

MOR is an excellent example of a powerful free analytical program supported by TalkBank (and thus FluencyBank). One might ask: What is the value of the MOR parsers and their resulting grammatical annotation? The answer is that this first level of analysis is critical to most analyses of spoken or written language. Even basic measures of language use require analysis of complex words (those consisting of multiple inflections), and running tallies of unique word forms over all words used in a speech sample. Because TalkBank morphosyntactic analyzers all use a parallel technology and output format, CLAN commands can be applied to each of these 10 languages for uniform computation of indices such as utterance length and complexity, vocabulary diversity, formulation errors, pause duration, and various measures of disfluency. For English, this development has enabled the development of two new and powerful clinical language analysis “bundles” (EVAL and KidEval), as well as a new fluency calculator (FluCalc), that can greatly improve and speed both clinical and research analysis of spoken and written language samples. We describe these in greater detail in Sections 1.4. and 4, below.

Critically, however, CLAN is an open, programmable system that users can adapt to their specific needs. CLAN includes a wide variety of user customizable search and analysis routines that have been extremely fruitful in evaluating theoretical claims and models. Such user-tailored evaluations have been important in understanding children’s acquisition of morphology and syntax, in areas such as the English past tense (Marcus et al., 1992, Pinker & Prince 1988, MacWhinney & Leinbach 1991) or finite verb marking (Wexler, 1998, Freudenthal, Pine, Aguado‐Orea, & Gobet, 2007). Emergentists (Pine & Lieven 1997) have used CHILDES data to explore theories of how children learn to use determiners, and generativists (Valian, Solt, & Stewart, 2009) have used the same data to argue for the presence of innate categories that guide children’s acquisition of syntax. CHILDES data and CLAN programs have also been used to explore the contribution of adult language models and interaction profiles in children’s language development (e.g., the many publications stemming from Snow, Tabors and Dickinson’s Home-School Study of Language and Literacy Development [HSLLD] corpus, and the large number of investigations of the “learnability” of child-directed speech based on the Bernstein corpus). In these debates, and many others, the availability of a shared open database has been crucial in the development of analysis and theory, as have CLAN’s powerful and flexible computing resources. We hope that the same benefits accrue to fluency researchers and clinicians.

After many years as primarily a research resource, TalkBank entered the clinical arena with the creation of the AphasiaBank initiative, funded in 2007 by the US National Institutes of Health, and directed by Audrey Holland and Brian MacWhinney (see summary in MacWhinney, Fromm, Forbes, & Holland (2011) and Forbes, Fromm & MacWhinney, 2014). AphasiaBank currently has 436 video recordings of people with aphasia and 226 non-aphasic controls performing the AphasiaBank protocol, which includes a uniform set of discourse, narrative, and processing tasks. Using the interactive EVAL program, researchers and clinicians can automatically compute in-depth language sample analysis across 32 measures, with reference values for typical adult and aphasic performance (in English) on each task, stratified by age, gender and diagnosis. AphasiaBank also includes smaller amounts of protocol data from Spanish, German, Italian, Mandarin and Cantonese. The framework of the EVAL program for “bundled” analysis of clinical data (rather than having to specify each type of analysis separately) was then extended to other age groups through the construction of KidEval for child language data. The goal of KidEval is to facilitate faster, more accurate and more informative child language sample analysis, by both researchers and practicing clinicians.

Child language sample analysis (LSA) for either clinical or research purposes can be quite time-consuming (Overton & Wren 2014). After spending hours of work to create a basic transcript, clinicians and researchers must then devote further time to compute measures such as Developmental Sentence Score (DSS; Lee & Canter, 1971; Long & Channell 2001; Cochran & Masterson 1995) or the Index of Productive Syntax (IPSYN) (Scarborough 1990). As a result, LSA is not widely used to inform child language assessment, let alone assessment of fluency clients (Bernstein Ratner & MacWhinney, 2016). Although we know that computer-assisted LSA can save time, and improve accuracy and depth of analysis (Heilmann, 2010; Price, Hendricks & Cook, 2010; Miller, 2001; Hassanali, Liu, Iglesias, Solorio, & Dollaghan, 2014), it is only infrequently used in practice. It is also under-exploited in research on children who stutter, when compared to standardized testing (see Ntourou, Conture & Lipsey, 2011); most studies have stopped with simple measures such as mean length of utterance (MLU). Unfortunately, MLU has limited ability to discriminate among child language profiles after the ages of 3–4 years, or an average MLU of 4.0 (Brown, 1973; Bernstein Ratner & MacWhinney, 2016), while other measures are more informative. Moreover, these measures require even more expertise and time expenditure if done by hand.

Fortunately, the use of free utilities such as CLAN, that can link transcription to the audio- or video-recorded record of the client’s actual speech sample, can greatly improve the accuracy and informativeness of language sample analysis. When combined with the high accuracy of the automatic morphological parser for English, a simple typed transcript (with no need for overt, clinician coding of morphology, as in systems such as SALT) can be immediately annotated for morphological and grammatical features. These, in turn, can feed programs such as EVAL and KidEval. Each produces dozens of counts and proportions of a wide array of features relevant to language sample analysis. For example, both EVAL and KidEval compute: Mean Length of Utterance (in words and morphemes) of utterances pre-screened for eligibility using Brown’s 1973 conventions; multiple alternative computations of vocabulary diversity (such as TTR, MATTR [moving average type-token ratio], number of different words [NDW] and VOCD), and indices of syntactic complexity (such as verbs/utterance). For many purposes, either EVAL or KidEval can be used for language analysis, depending upon the clinician or researcher’s desired measures. Given typical concerns in acquired language disorder, EVAL, which was originally written for analysis of language use by people with aphasia, computes distribution of major parts of speech [POS]), while KidEval adds information listing 14 major morphemes in tracked in assessment of English child language development (more commonly known as “Brown’s morphemes), etc. For more details, readers are invited to consult the CliniciansGuide to CLAN at the TalkBank site (https://talkbank.org/manuals/Clin-CLAN.pdf). The KidEval program also prescreens utterances using the sometimes complex and difficult-to-understand rules for inclusion in computations of MLU, DSS, and IPSYN (Sagae, Davis, Lavie, MacWhinney,& Wintner (2007)). It computes these measures automatically, avoiding the additional labor and computational error that will arise, if done manually. All EVAL and KidEval measures can also be computed for samples of written language, if they are composed in MS-Word or plain text. A somewhat abridged example of KidEval output is shown in Appendix 2. Moreover, these facilities are now available for an increasingly large number of languages other than English, including French, Spanish, Chinese, and Japanese with the extension to other languages in preparation. CLAN’s computational power can greatly benefit clinical assessment, therapy planning, and measurement of therapeutic progress in clinical work in fluency disorders. Media-linked transcripts also preserve data in a single integrated, annotatable format that can easily facilitate post hoc hypothesis testing and data exploration.

Section snippets

Why we need FluencyBank

We think that it’s important to note that FluencyBank development was supported both by the NIDCD, with its clinical focus on research, as well as the National Science Foundation, which has a focus on understanding typical speech/language production and comprehension. Because spoken language production is less amenable to controlled study than is comprehension, fluency is under-represented in psycholinguistic research (Altmann, 2001; Fromkin & Bernstein Ratner, 1998). Disfluency in speech is

The components and resources of FluencyBank

As with past TalkBank initiatives, the FluencyBank initiative emerged out of several years of discussion among researchers in typical speech/language development, stuttering, language disorders and second language acquisition. As with the other TalkBank projects and sites, its primary components are a database, transcription and analytical tools, and teaching resources.

FluencyBank analysisTools

FluencyBank is working to provide new, more accurate methods for fluency analysis based on transcription-media linkage. These include preconfigured analysis tools, and support for program interoperability (movement between transcription programs, such as CLAN, SALT, ELAN, etc., and other analysis platforms, such as Praat and Phon).

User support

Thanks to longstanding NIH and NSF funding, the FluencyBank and TalkBank projects enable a large array of user support services, such as lab- or project-specific instruction in file linking and transcription, use of programs for research and clinical purposes, email support services, and trouble-shooting of problems with data or program use. We currently provide three free manuals for typical user purposes: the CHAT transcription manual (https://talkbank.org/manuals/CHAT.pdf), and the CLAN

Conclusion

Understanding the bases of fluency, disfluency and stuttering is central to both theory and clinical practice. As demonstrated by the success of CHILDES, AphasiaBank, PhonBank, and HomeBank, data sharing speeds the discovery of knowledge in a discipline and provides power to analyses as well as ensuring greater generalizability of findings. The US NIH currently requires applicants to specify a data sharing plan for all grant applications for a reason. Notably, CHILDES, which has resulted in

Acknowledgments

This work was supported by the National Institutes of Health [NIDCD: 1 R01 DC015494-01] and the National Science Foundation [BCS-1626300/1626294].

Nan Bernstein Ratner is Professor, Hearing and Speech Sciences, at the University of Maryland, College Park. She has frequently published in the area of fluency and fluency disorders. She has been recognized for her research by the International Fluency Association (of which she is President-Elect) and the American Speech-Language-Hearing Association (of which she is a Fellow and Honors Recipient). Together with Brian MacWhinney, she co-directs the new FluencyBank project, funded by NIH and NSF.

First page preview

First page preview
Click to open first page preview

References (115)

  • L.J. MacGregor et al.

    Listening to the sound of silence: Disfluent silent pauses in speech have consequences for listeners

    Neuropsychologia

    (2010)
  • B. MacWhinney et al.

    Implementations are not conceptualizations: Revising the verb learning model

    Cognition

    (1991)
  • R.C. Martin et al.

    Planning in sentence production: Evidence for the phrase as a default planning scope

    Cognition

    (2010)
  • S. Pinker et al.

    On language and connectionism: Analysis of a parallel distributed processing model of language acquisition

    Cognition

    (1988)
  • B.P. Ryan

    A longitudinal study of articulation, language, rate, and fluency of 22 preschool children who stutter

    Journal of Fluency Disorders

    (2001)
  • G.T. Altmann

    The language machine: Psycholinguistics in review

    British Journal of Psychology

    (2001)
  • J.E. Arnold et al.

    Disfluencies signal theee, um: New information

    Journal of Psycholinguistic Research

    (2003)
  • J.E. Arnold et al.

    If you say thee uh you are describing something hard: The on-line attribution of disfluency during reference comprehension

    Journal of Experimental Psychology. Learning, Memory, and Cognition

    (2007)
  • K. Bakker

    Technical solutions for quantitative and qualitative assessments of speech fluency

    Seminars in Speech and Language

    (1999)
  • N. Bernstein Ratner et al.

    Your laptop to the rescue: Using the CHILDES Archive and CLAN programs to improve language sample analysis

    Seminars in Speech and Language

    (2016)
  • N. Bernstein Ratner et al.

    Use of big data child language assessment

  • O. Bloodstein et al.

    A handbook on stuttering

    (2008)
  • K. Bock et al.

    Language production: Grammatical encoding

  • P. Boersma et al.

    Praat, a system for doing phonetics by computer

    Tech. Rep.

    (1996)
  • M.G. Boltz

    Temporal dimensions of conversational interaction: The role of response latencies and pauses in social impression formation

    Journal of Language and Social Psychology

    (2005)
  • H. Bortfeld et al.

    Disfluency rates in conversation: Effects of age, relationship, topic, role, and gender

    Language and Speech

    (2001)
  • B. Boscolo et al.

    Fluency of school-aged children with a history of specific expressive language impairment: An exploratory study

    American Journal of Speech Language Pathology

    (2002)
  • R. Brown

    A first language

    (1973)
  • C. Byrd et al.

    The disfluent speech of bilingual Spanish–English children: Considerations for differential diagnosis of stuttering

    Language, Speech, and Hearing Services in Schools

    (2015)
  • H.-M. Chow et al.

    Speech-language development trajectories may predict persistent developmental stuttering during childhood

    (2015)
  • P.S. Cochran et al.

    NOT using a computer in language assessment/intervention: In defense of the reluctant clinician

    Language, Speech, and Hearing Services in Schools

    (1995)
  • A.K. Cordes et al.

    The reliability of observational data: II. Issues in the identification and measurement of stuttering events

    Journal of Speech and Hearing Research

    (1994)
  • A.K. Cordes et al.

    Time-interval measurement of stuttering: Establishing and modifying judgment accuracy

    Journal of Speech and Hearing Research

    (1996)
  • A.K. Cordes et al.

    Time-interval analysis of interjudge and intrajudge agreement for stuttering event judgments

    Journal of Speech and Hearing Research

    (1992)
  • A.K. Cordes

    Individual and consensus judgments of disfluency types in the speech of persons who stutter

    Journal of Speech, Language, and Hearing Research

    (2000)
  • P.S. Dale et al.

    Illusory recovery: Are recovered children with early language delay at continuing elevated risk?

    American Journal of Speech-Language Pathology

    (2014)
  • N.H. De Jong et al.

    Praat script to detect syllable nuclei and measure speech rate automatically

    Behavior Research Methods

    (2009)
  • T. Derwing et al.

    The relationship between L1 fluency and L2 fluency development

    Studies in Second Language Acquisition

    (2009)
  • N. de Jong

    Second language learning of grammar: Output matters too

  • D. Finneran et al.

    Speech disruptions in the sentence formulation of school-age children with specific language impairment

    Language and Communication Disorders

    (2009)
  • J.E. Fox Tree

    The effects of false starts and repetitions on the processing of subsequent words in spontaneous speech

    Journal of Memory and Language

    (1995)
  • S.H. Fraundorf et al.

    Alice's adventures in um-derland: Psycholinguistic dimensions of variation in disfluency production. Language

    Cognition and Neuroscience

    (2014)
  • D. Freudenthal et al.

    Modeling the developmental patterning of finiteness marking in English, Dutch, German, and Spanish using MOSAIC

    Cognitive Science

    (2007)
  • V. Fromkin et al.

    Speech production

  • V. Fromkin

    Slips of the tongue

    (1973)
  • M.F. Garrett

    Syntactic processes in sentence production

    New Approaches to Language Mechanisms

    (1976)
  • F. Goldman-Eisler

    The predictability of words in context and the length of pauses in speech

    Language and Speech

    (1958)
  • S. Gottwald et al.

    Helping graduate students acquire knowledge and skills in fluency disorders

    Paper presented at the sixth international world congress on fluency disorders, rio de janiero

    (2009)
  • L.-Y. Guo et al.

    Speech disruptions in the narratives of English-speaking children with specific language impairment

    Journal of Speech, Language, and Hearing Research

    (2008)
  • N.E. Hall et al.

    Relationship between language and fluency in children with developmental language disorders

    Journal of Speech and Hearing Research

    (1993)
  • Cited by (70)

    View all citing articles on Scopus

    Nan Bernstein Ratner is Professor, Hearing and Speech Sciences, at the University of Maryland, College Park. She has frequently published in the area of fluency and fluency disorders. She has been recognized for her research by the International Fluency Association (of which she is President-Elect) and the American Speech-Language-Hearing Association (of which she is a Fellow and Honors Recipient). Together with Brian MacWhinney, she co-directs the new FluencyBank project, funded by NIH and NSF.

    Brian MacWhinney is Professor of Psychology, Carnegie-Mellon University. He has published extensively in child language acquisition, computational analysis of language and psycholinguistics. He was the first recipient of the International Association for Child Language Roger Brown award for distinguished contributions to language acquisition research. He is founder of the TalkBank project, and its director. With Nan Bernstein Ratner, he co-directs the new FluencyBank project, funded by NIH and NSF.

    View full text