Abstract
The human voice is capable of making a wide variety of sounds. From a psychological point of view, what is interesting is that only some configurations of sounds are meaningful to others. There are the words we choose, which are combinations of sounds that symbolically represent various concepts we are trying to communicate. But even those sounds that compose words have variations that can impart meaning independent of those words — even to the point of changing their meaning entirely. The phrase “It’s not what you said, but how you said it” exists in our parlance because we have come to recognize the reality that the nonverbal elements that accompany the spoken word are as important as the actual words in creating meaning. In fact, when we speak we unleash three distinct types of information upon listeners through our voices, of which one is verbal, while the other two are nonverbal. Thus we can separate the voice channel into three different subchannels. The first subchannel is the verbal subchannel, and consists of the actual words we speak. The second subchannel is the speech style subchannel, which consists of the patterns of pausing and other irregularities of speech that accompany the words spoken. The third subchannel is the speech tone subchannel, which consists of the acoustic properties of speech such as loudness and pitch.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Abberton, E., & Fourcin, A. J. (1978). Intonation and speaker identification. Language and Speech, 21(4), 305–318. doi: 10.1177/002383097802100405.
Andersen, P. A. (1999). Nonverbal communication: Forms and functions. Mountain View, CA: Mayfield.
Apple, W., Streeter, L. A., & Krauss, R. M. (1979). Effects of pitch and speech rate on personal attributions. Journal of Personality and Social Psychology, 37(5), 715–727. doi: 10.1037/0022–3514.37.5.715.
Bachorowski, J. A. (1999). Vocal expression and perception of emotion. Current Directions in Psychological Science, 8(2), 53–57. doi: 10.1111/1467–8721.00013.
Banse, R., & Scherer, K. R. (1996). Acoustic profiles in vocal emotion expression. Journal of Personality and Social Psychology, 70(3), 614–636. doi: 10.1037/00223514.70.3.614.
Bartlett, M. S., Littlewort, G., Frank, M. G., Lainscsek, C., Fasel, I., & Movellan, J. R. (2006). Automatic recognition of facial actions in spontaneous expressions. Journal of Multimedia, 1(6), 22–35.
Batliner, A., Steidl, S., Schuller, B., Seppi, D., Vogt, T., Wagner, J., … Amir, N. (2011). Whodunnit — Searching for the most important feature types signalling emotion-related user states in speech. Computer Speech & Language, 25(1), 4–28. doi: 10.1016/j.csl.2009.12.003.
Boersma, P., & Weenink, D. (2013). PRAAT: Doing phonetics by computer (Version 5.3.65) [Computer software]. Retrieved from http://www.praat.org/.
Bryant, G. A., & Barrett, H. C. (2008). Vocal emotion recognition across disparate cultures. Journal of Cognition and Culture, 8(1–2), 135–148. doi: 10.1163/156770908x289242.
Cappella, J. N. (1985). Production principles for turn-taking rules in social interaction: Socially anxious vs. socially secure persons. Journal of Language and Social Psychology, 4(3–4), 193–212. doi: 10.1177/0261927x8543003.
Collins, S. A., & Missing, C. (2003). Vocal and visual attractiveness are related in women. Animal Behaviour, 65(5), 997–1004. doi: 10.1006/anbe.2003.2123.
Daniloff, R., Schuckers, G., & Feth, L. (1980). The physiology of speech and hearing: An introduction. Englewood Cliffs, NJ: Prentice-Hall.
Darwin, C. (1998). The expression of the emotions in man and animals. New York: Oxford University Press (Original work published 1872).
DePaulo, B. M., Lindsay, J. J., Malone, B. E., Muhlenbruck, L., Charlton, K., & Cooper, H. (2003). Cues to deception. Psychological Bulletin, 129(1), 74–118. doi: 10.1037/0033–2909.129.1.74.
Duncan, S. (1969). Nonverbal communication. Psychological Bulletin, 72(2), 118–137. doi: 10.1037/h0027795.
Duncan, S., & Fiske, D. W. (1977). Face-to-face interaction: Research methods and theory. Hillsdale, NJ: Erlbaum.
Ekman, P. (2003). Emotions revealed (2nd ed.). New York: Times Books.
Ekman, P., & Friesen, W. V. (1978). The facial action coding system. Palo Alto, CA: Consulting Psychologists.
Fant, G. (1960). Acoustic theory of speech production. The Hague, The Netherlands: Mouton.
Fernald, A. (1992). Human maternal vocalizations to infants as biologically relevant signals: An evolutionary perspective. In J. H. Barkow, L. Cosmides, & J. Tooby (Eds.), The adapted mind: Evolutionary psychology and the generation of culture (Vol. xii, pp. 391–428). New York: Oxford University Press.
Frank, M. (2005). Research methods in detecting deception research. In J. A. Harrigan, R. Rosenthal, & K. R. Scherer (Eds.), The new handbook of methods in nonverbal behavior research (pp. 341–368). Oxford: Oxford University Press.
Frank, M. G. (2003). Getting to know your patient: How facial expression reveals true emotion. In M. Katsikitis (Ed.) The clinical application of facial measurement: Methods and meaning (pp. 255–283). Dordrecht: Kluwer.
Fridlund, A. J., Ekman, P., & Oster, H. (1987). Facial expression of emotion. In A. Siegman & S. Feldstein (Eds.), Nonverbal behavior and communication (pp. 143–224). Hillsdale, NJ: Erlbaum.
Fry, D. B. (1979). The physics of speech. Cambridge, UK: Cambridge University Press.
Furness, J. B. (2006). The enteric nervous system. Malden, MA: Blackwell Publishing, Inc.
Gallois, C., Ogay, T., & Giles, H. (2005). Communication accommodation theory: A look back and a look ahead. In W. Gudykunst (Ed.), Theorizing about international communication (pp. 121–148). Thousand Oaks, CA: Sage.
Giles, H., & Powesland, P. F. (1975). Speech style and social evaluation. London: Academic Press.
Glasgow, G. M. (1952). A semantic index of vocal pitch. Communication Monographs, 19(1), 64–68.
Goldman-Eisler, F. (1968). Psycholinguistics: Experiments in spontaneous speech. London, New York: Academic Press.
Gorham-Rowan, M. M., & Laures-Gore, J. (2006). Acoustic-perceptual correlates of voice quality in elderly men and women. Journal of Communication Disorders, 39(3), 171–184. doi: 10.1016/j.jcomdis.2005.11.005.
Grammer, K., & Eibl-Eibesfeldt, I. (1990). The ritualisation of laughter. In W. A. Koch (Ed.), Naturlichkeit der Sprache und der Kultur (pp. 192–214). Bochum, Germany: Brockmeyer.
Greene, J. O., & Ravizza, S. M. (1995). Complexity effects on temporal characteristics of speech. Human Communication Research, 21(3), 390–421. doi: 10.1111/j.1468–2958.1995.tb00352.x.
Hall, J. A., Coats, E. J., & Smith LeBeau, L. (2005). Nonverbal behavior and the vertical dimension of social relations: A meta-analysis. Psychological Bulletin, 131(6), 898–924. doi: 10.1037/0033–2909.131.6.898.
Harms, L. S. (1961). Programmed learning for the field of speech. Communication Education, 10(3), 215–219.
Harnsberger, J. D., Brown Jr., W. S., Shrivastav, R., & Rothman, H. (2010). Noise and tremor in the perception of vocal aging. Journal of Voice, 24, 523–530. doi: 10.1016/j.jvoice.2009.01.003.
Harrigan, J. A., Suarez, I., & Hartman, J. S. (1994). Effect of speech errors on observers’ judgments of anxious and defensive individuals. Journal of Research in Personality, 28(4), 505–529. doi: 10.1006/jrpe.1994.1036.
Harrigan, J. A., Wilson, K, & Rosenthal, R. (2004). Detecting state and trait anxiety from auditory and visual cues: A meta-analysis. Personality and Social Psychology Bulletin, 30(1), 56–66. doi: 10.1177/0146167203258844.
Harrison, R. P. & Knapp, M. L. (1972). Toward an understanding of nonverbal communication systems. The Journal of Communication, 22(4), 339–352.
Hatfield, E., Hsee, C. K., Costello, J., Weisman, M. S., & Denney, C. (1995). The impact of vocal feedback on emotional experience and expression. Journal of Social Behavior and Personality, 10(2), 293–313.
Hollien, H., & Harnsberger, J. (2006). Voice stress analyzer instrumentation evaluation. Final Report CIFA Contract — FA 4814–04-0011. Gainesville: University of Florida.
Hollien, H., Harnsberger, J. D., Martin, C. A., Hill, R., & Alderman, G. A. (2009). Perceiving the effects of ethanol intoxication on voice. Journal of Voice, 23, 552–559. doi: 10.1016/j.jvoice.2007.11.005.
Hollien, H., Majewski, W., & Doherty, E. T. (1982). Perceptual identification of voices under normal, stress and disguise speaking conditions. Journal of Phonetics, 10(2), 139–148.
Hummert, M. L., Mazloff, D., & Henry, C. (1999). Vocal characteristics of older adults and stereotyping. Journal of Nonverbal Behavior, 23(2), 111–132. doi: 10.1023/a:l021483409296.
Johnson, W. E., Emde, R. N., Scherer, K. R., & Klinnert, M. D. (1986). Recognition of emotion from vocal cues. Archives of General Psychiatry, 43(3), 280–283.
Juslin, P. N., & Laukka, P. (2003). Communication of emotions in vocal expression and music performance: Different channels, same code? Psychological Bulletin. 129(5), 770–814. doi: 10.1037/0033–2909.129.5.770.
Kasl, S. V., & Mahl, G. F. (1965). Relationship of disturbances and hesitations in spontaneous speech to anxiety. Journal of Personality and Social Psychology, 1(5), 425–433. doi: 10.1037/h0021918.
Kenny, M. A., & Williams, J. M. G. (2007). Treatment-resistant depressed patients show a good response to mindfulness-based cognitive therapy. Behaviour Research and Therapy, 45, 617–625.
Kim, Y. C., Narayanan, S., & Nayak, K. S. (2009). Accelerated 3D upper airway MRI using compressed sensing. Magnetic Resonance in Medicine, 61, 1434–1440. doi: 10.1002/mrm.21953.
Kitchen, D. M., Cheney, D. L., & Seyfarth, R. M. (2003). Female baboons’ responses to male loud calls. Ethology, 109(5), 401–412. doi: 10.1046/j.1439-Q310.2003.00878.x.
Kitzing, P. (1985). Stroboscopy — A pertinent laryngological examination. The Journal of Otolaryngology, 14(3), 151–157.
Klineberg, O. (1940). Social psychology (Vol. 5). New York: Holt.
Ko, S. J., Judd, C. M., & Blair, I. V. (2006). What the voice reveals: Within- and between-category stereotyping on the basis of voice. Personality and Social Psychology Bulletin, 32(6), 806–819. doi: 10.1177/0146167206286627.
Krauss, R. M., Freyberg, R., & Marsella, E. (2002). Inferring speakers’ physical attributes from their voices. Journal of Experimental Social Psychology, 38(6), 618–625. doi: 10.1016/s0022–1031(02)00510–3.
Kraut, R. E., & Poe, D. (1980). Behavioral roots of person perception: The deception judgments of customs inspectors and laymen. Journal of Personality & Social Psychology, 39(5), 784–798.
Kuroda, I., Fujiwara, O., Okamura, N., & Utsuki, N. (1979). Method for determining pilot stress through analysis of voice communication. Aviation, Space, and Environmental Medicine, 47(5), 528–533.
Lallgee, M. G., & Cook, M. (1969). An experimental investigation of the function of filled pauses in speech. Language and Speech, 12(1), 24–28.
Lass, N. J., Hughes, K. R., Bowyer, M., Waters, L. T., & Bourne, V. T. (1976). Speaker sex identification from voiced, whispered, and filtered isolated vowels. Journal of the Acoustical Society of America, 59(3), 675–678.
Laukka, P. (2005). Categorical perception of vocal emotion expressions. Emotion, 5(3), 277–295. doi: 10.1037/1528–3542.5.3.277.
Laukka, P., & Elfenbein, H. A. (2012). Emotion appraisal dimensions can be inferred from vocal expressions. Social Psychological and Personality Science, 3(5), 529–536. doi: 10.1177/1948550611428011.
Leigh, T. W., & Summers, J. (2002). Effects of salespersons’ use of nonverbal cues. Journal of Personal Selling and Sales Management, 22(1), 41–53.
Lieberman, P. (1961). Perturbations in vocal pitch. Journal of the Acoustical Society of America, 33(5), 597–603.
Lippa, R. (1998). Gender-related individual differences and the structure of vocational interests: The importance of the people — Things dimension. Journal of Personality and Social Psychology, 74(4), 996–1009. doi: 10.1037/0022–3514.74.4.996.
Masaki, N., & Seiji, N. (2008). Changes in speaking: Fundamental frequency characteristics with aging. Folia Phoniatrica et Logopaedica, 60(3), 120–127.
Matsumoto, D., Frank, M., & Hwang, H. S. (2013). Reading people: Introduction to the world of nonverbal behavior. In D. Matsumoto, M. G. Frank, & H. S. Hwang (Eds.), Nonverbal communication: Science and applications (pp. 3–14). Thousand Oaks, CA: Sage.
Matsumoto, D., & Hwang, H. S. (2013). Facial expressions. In D. Matsumoto, M. G. Frank, & H. S. Hwang (Eds.), Nonverbal communication: Science and applications (pp. 15–52). Thousand Oaks, CA: Sage.
McAdams, S., & Bregman, A. (1979). Hearing musical streams. Computer Music Journal, 3(4), 26–60.
Miller, N., Maruyama, G., Beaber, R. J., & Valone, K. (1976). Speed of speech and persuasion. Journal of Personality and Social Psychology, 34(4), 615–624. doi: 10.1037/0022–3514.34.4.615.
Mortensen, C. D. (2007). Communication theory (2nd ed.). New Brunswick, NJ: Transaction Publishers.
Mortillaro, M., Mehu, M., & Scherer, K. R. (2011). Subtly different positive emotions can be distinguished by their facial expressions. Social Psychological and Personality Science, 2(3), 262–271. doi: 10.1177/1948550610389080.
Murphy, P. J. (2000). Spectral characterization of jitter, shimmer, and additive noise in synthetically generated voice signals. The Journal of the Acoustical Society of America, 107(2), 978–988. doi: 10.1121/1.428272.
Neumann, R., & Strack, F. (2000). “Mood contagion”: The automatic transfer of mood between persons. Journal of Personality and Social Psychology, 79(2), 211–223. doi: 10.1037/0022–3514.79.2.211.
Nolan, F. (1983). The phonetic bases of speaker recognition. Cambridge studies in speech science and communication. Cambridge, UK: Cambridge University Press.
Owren, M. J., Rendall, D., & Bachorowski, J. A. (2003). Nonlinguistic vocal communication. In D. Maestripieri (Ed.), Primate psychology (pp. 359–394). Cambridge, MA: Harvard University Press.
Patel, S., & Scherer, K. (2013). Vocal behavior. In J. A. Hall & M. L. Knapp (Eds.), Nonverbal communication. Berlin, Germany: De Gruyter Mouton.
Patel, S., & Shrivastav, R. (2011). A preliminary model of emotional prosody using multidimensional scaling. Proceedings of interspeech (pp. 2957–2960). Florence, Italy.
Pear, T. H. (1931). Voice and personality. Great Britain: Chapman & Hall.
Pittam, J., & Scherer, K. R. (1993). Vocal expression and communication of emotion. In M. L. Lewis & J. Haviland (Eds.), Handbook of emotions (pp. 185–197). New York, London: Guilford Press.
Puts, D. A., Gaulin, S. J. C., & Verdolini, K. (2006). Dominance and the evolution of sexual dimorphism in human voice pitch. Evolution and Human Behavior, 27(4), 283–296. doi: 10.1016/j.evolhumbehav.2005.11.003.
Rinn, W. E. (1984). The neuropsychology of facial expression: A review of the neurological and psychological mechanisms for producing facial expressions. Psychological Bulletin, 95, 52–77. doi: 10.1037/0033–2909.95.1.52.
Robinson, W. P. (1972). Language and social behaviour. Harmondsworth, Middlesex: Penguin Books.
Rochester, S. R. (1973). The significance of pauses in spontaneous speech. Journal of Psycholinguistic Research, 2(1), 51–81. doi: 10.1007/BF01067111.
Rosenthal, R., & Rubin, D. B. (1989). Effect size estimation for one-sample multiple choice-type data: Design, analysis, and meta-analysis. Psychological Bulletin, 106(2), 332–337. doi: 10.1037/0033–2909.106.2.332.
Salamin, H., & Vinciarelli, A. (2012). Automatic role recognition in multiparty conversations: An approach based on turn organization, prosody, and conditional random fields. IEEE Transactions on Multimedia, 99, 1–4. doi: 10.1109/TMM.2011.2173927.
Sauter, D., & Scott, S. (2007). More than one kind of happiness: Can we recognize vocal expressions of different positive states? Motivation and Emotion, 31(3), 192–199. doi: 10.1007/s11031–007-9065-x.
Sauter, D. A., Eisner, E., Calder, A. J., & Scott, S. K. (2010). Perceptual cues in nonverbal vocal expressions of emotion. The Quarterly Journal of Experimental Psychology, 63(11), 2251–2272. doi: 10.1080/17470211003721642.
Scherer, K. R. (1984). On the nature and function of emotion: A component process approach. In K. R. Scherer & P. Ekman (Eds.), Approaches to emotion (pp. 293–317). Hillsdale, NJ: Erlbaum.
Scherer, K. R. (1986). Vocal affect expression: A review and a model for future research. Psychological Bulletin, 99(2), 143–165. doi: 10.1037/00332909.99.2.143.
Scherer, K. R. (1989). Vocal correlates of emotional arousal and affective disturbance. In H. Wagner & A. Manstead (Eds.), Handbook of social psychophysiology (pp. 165–197). Oxford, England: John Wiley & Sons.
Scherer, K. R. (2003). Vocal communication of emotion: A review of research paradigms. Speech Communication, 40(1–2), 227–256. doi: 10.1016/s0167- 6393(02)00084–5.
Scherer, K. R., & Bänziger, T. (2010). On the use of actor portrayals in research on emotional expression. In K. R. Scherer, T. Bänziger and E. B. Roesch (Eds.), Blueprint for affective computing: A sourcebook (pp. 166–178). Oxford: Oxford University Press.
Scherer, K. R., Banse, R., Wallbott, H. G., & Goldbeck, T. (1991). Vocal cues in emotion encoding and decoding. Motivation and Emotion, 15(2), 123–148. doi: 10.1007/bf00995674.
Siegman, A. W. (1987). The telltale voice: Nonverbal messages of verbal communication. In A. W. Siegman & S. Feldstein (Eds.), Nonverbal behavior and communication (2nd ed., pp. 351–433). Hillsdale, NJ, England: Erlbaum.
Siegman, A. W., & Feldstein, S. (1979). Of speech and time. Temporal speech patterns in interpersonal contexts. Hillsdale, NJ: Lawrence Erlbaum Associates.
Sillars, A. L., Coletti, S. E., Parry, D., & Rogers, M. A. (1982). Coding verbal conflict tactics: Nonverbal and perceptual correlates of the “avoidance-distributive-integrative” distinction. Human Communication Research, 9(1), 83–95. doi: 10.llll1j.1468–2958.1982.tb00685.x.
Simon-Thomas, E. R., Keltner, D. J., Sauter, D., Sinicropi-Yao, L., & Abramson, A. (2009). The voice conveys specific emotions: Evidence from vocal burst displays. Emotion, 9(6), 838–846. doi: 10.1037/a0017810.
Simpson, A. P. (2009). Phonetic differences between male and female speech. Language and Linguistics Compass, 3(2), 621–640. doi: 10.1111/j.1749–818X. 2009.00125.x.
Singh, L., Nestor, S., Parikh, C., & Yull, A. (2009). Influence of infant-directed speech on early word recognition. Infancy, 14, 654–666.
Sundberg, J., Patel, S., Bjorkner, E., & Scherer, K. R. (2011). Interdependencies among voice source parameters in emotional speech. IEEE Transactions on Affective Computing, 2(3), 162–174. doi: 10.1109/T-AFFC.2011.14.
Tanaka, A., Koizumi, A., Imai, H., Hiramatsu, S., Hiramoto, E., & de Gelder, B. (2010). I feel your voice: Cultural differences in the multisensory perception of emotion. Psychological Science, 21(9), 1259–1262. doi: 10.1177/0956797610380698.
Trager, G. L. (1961). The typology of paralanguage. Anthropological Linguistics, 3(1), 17–21.
van Lancker, D., Kreiman, J., & Emmorey, K. (1985). Familiar voice recognition: Patterns and parameters. Part I: Recognition of backward voices. Journal of Phonetics, 13(1), 19–38.
Vingerhoets, A. J. J. M., & Cornelius, R. R. (2001). Adult crying: A biopsychosocial approach (Vol. 3). Philadelphia: Routledge.
Vukovic, J., Jones, B. C., Feinberg, D. R., DeBruine, L. M., Smith, F. G., Welling, L. L., & Little, A. C. (2011). Variation in perceptions of physical dominance and trustworthiness predicts individual differences in the effect of relationship context on women’s preferences for masculine pitch in men’s voices. British Journal of Psychology, 102(1), 37–48. doi: 10.1348/000712610X498750.
Walton, J. H., & Orlikoff, R. F. (1994). Speaker race identification from acoustic cues in the vocal signal. Journal of Speech & Hearing Research, 37(4), 738–745.
Wiemann, J. M., & Knapp, M. L. (1975). Turn taking in conversations. Journal of Communication, 25(2), 75–92.
Zemlin, W. R. (1968). Speech and hearing science; anatomy and physiology. Englewood Cliffs, NJ: Prentice-Hall.
Zuckerman, M., Koestner, R., & Driver, R. (1981). Beliefs about cues associated with deception. Journal of Nonverbal Behavior, 6(2), 105–114. doi: 10.1007/bf00987286.
Zwan, P., Szczuko, P., Kostek, B., & Czy Zewski, A. (2007). Automatic singing voice recognition employing neural networks and rough sets. In M. Kryszkiewicz, J. Peters, H. Rybinski, & A. Skowron (Eds.), Lecture notes in artificial intelligence: Proceedings of the international conference on rough sets and intelligent systems paradigms (pp. 793–802). doi: 10.1007/978–3-540–73451-2_83.
Editor information
Editors and Affiliations
Copyright information
© 2015 Mark G. Frank, Darrin J. Griffin, Elena Svetieva, and Andreas Maroulis
About this chapter
Cite this chapter
Frank, M.G., Griffin, D.J., Svetieva, E., Maroulis, A. (2015). Nonverbal Elements of the Voice. In: Kostić, A., Chadee, D. (eds) The Social Psychology of Nonverbal Communication. Palgrave Macmillan, London. https://doi.org/10.1057/9781137345868_5
Download citation
DOI: https://doi.org/10.1057/9781137345868_5
Publisher Name: Palgrave Macmillan, London
Print ISBN: 978-1-349-46669-6
Online ISBN: 978-1-137-34586-8
eBook Packages: Palgrave Social Sciences CollectionSocial Sciences (R0)