Memory and context effects on representations of variation in phrasal intonation

NSF BCS-1944773
Investigator: Jennifer Cole

Understanding how an intonation system works requires a detailed description of the intonation patterns that speakers produce, and an account of the intonational features and regions of he sentence that are perceptually salient and critical for conveying discourse meaning. The goal of this project is to identify the elements of phrasal intonation that are perceived in heard speech and encoded in long-term memory representations. These representations are tapped through tasks where listeners reproduce the intonation of a heard utterance.

Variation in the production of English nuclear tunes

Investigators: Jennifer Cole, Eleanor Chodroff, Stefanie Shattuck-Hufnagel (MIT)

The “nuclear tune” refers to the pitch melody spanning 1 or more syllables from the primary sentence-level prominence to the end of the phrase.  Current models of the intonational phonology of  English specify the nuclear tune in terms of two or more phonologically distinct tonal components, e.g., a pitch accent, phrase accent and boundary tone in the Autosegmental-Metrical model. We are investigating patterns of variation in the phonological and phonetic expression of nuclear tunes, in a series of speech production experiments using a generalized speech imitation paradigm.

Prosody and information status

Investigators: Jennifer Cole, Eleanor Chodroff, Stefan Baumann (U Cologne), José I. Hualde (UIUC), Suyeon Im (UIUC), Tatiana Luchkina (Central Connecticut State U)

The discourse meaning of an utterance is based in part on the information status of the words it comprises. Information status distinguishes words that have contextually restricted reference (focus) from non-focused words, and words that are lexically or referentially new from those that are accessible or given in the prior discourse. This project investigates the prosodic correlates of information status in languages that use prosody as the primary means of expressing information status (English, German) and in languages that use syntax alone or in combination with prosodic marking of information status (Russian, Hindi). Our current work on English examines effects of information status on the assignment of nuclear and prenuclear pitch accents, and on the acoustic correlates of pitch accent and phrasal prominence, through evidence from speech production and perception.

Learning approaches to the study of intonational form-function mapping

Investigators: Jennifer Cole, Timo Roettger

An utterance conveys the communicative intention of a speaker through the choice of lexical items and morphosyntactic structures, and also by its intonational form. Different parts of the intonation contour signal different functions related to pragmatic meaning, yet the mapping between intonation and meaning, and its interaction with lexical and morphosyntactic content is still not well understood. This project investigates how adult language users learn how to map parts of an intonation contour onto meaning, utilizing the artificial language learning paradigm. We investigate how speakers learn novel intonational form-function mappings and how they generalize the acquired patterns to new prosodic, syntactic, and lexical contexts.

Constituents and Heads in Prosody Perception

NSF BCS 12-51343; 2013-2017
Investigators: Jennifer Cole (PI), José I. Hualde (co-PI, UIUC), Caroline Smith (co-PI, U New Mexico)
Graduate research assistants (UIUC):  Christopher Eager, Suyeon Im, Tim Mahrt

This project investigates the relationship between prosodic phrasing (constituents) and prosodic prominence (heads) in a comparative study of English, Spanish and French. These languages are known to differ not only in their prosody (eg., intonation and rhythmic patterns), but also in the associations linking prosody to syntax and semantics. Experiments using the Rapid Prosody Transcription method, developed in our prior work to investigate prosody perception among ordinary (untrained, non-expert) listeners, show how listeners perceive the prosodic phrasing and prominence patterns of an utterance when presented with speech samples that differ in their phonetic properties (pitch and timing), and in syntactic and semantic features. Findings shed light on the interplay of acoustic cues and top-down features from the syntactic, semantic and discourse context in the perception of prosody in these languages, and will contribute to our understanding of cross-linguistic variation in the role prosody plays in conveying linguistic meaning.

In other research on this project we are using a sentence imitation paradigm to investigate the status of pre-nuclear accents in marked and default patterns in American English.

Prosodic and Gestural Entrainment

Funding from Volkswagen Stiftung; 2013-2015
Investigators: Jennifer Cole (PI); Uwe Reichel (Hungarian Academy of Sciences, Budapest)
Graduate Research assistant (UIUC): Suyeon Im, Stephanie Landblom

This project is part of an international, multi-site project led by Jonathan Harrington (LMU, Munich) to develop an experimental paradigm for investigating prosodic and gestural entrainment across typologically diverse languages, in lab and field settings. My lab used the project’s Tangram Games paradigm to collect audio-visual recordings from speakers of American and Indian English. With these data we are investigating evidence for entrainment between interlocutors in their use of prosody, gesture and dialog acts. Entrainment measures are compared for cooperative vs. competitive modes of interaction, and for speakers from matched vs. mismatched dialects.

Prosody in Mother-Child Interactions

USDA National Institute of Food and Agriculture (Hatch Project: # ILLU-793-362); NSF SMA 14-16791 2014-2017
Investigators: Nancy McElwain (PI, UIUC); Dan Berry (co-PI, U MN); Jennifer Cole (co-PI)
Graduate research assistants: Tim Mahrt, Bill Bryce

This project crosses developmental psychology and linguistics to investigate prosody in mother-child interactions. We are examining links between mom’s moment-to-moment use of prosody and measures of mom’s and child’s affect. The goal of this study is to understand the role of maternal prosody in the child’s development of self-regulation, and in parent-child attachment.

Individual Differences in Prosody Perception

NSF BCS 12-51343; 2013-2017
Investigators: Jennifer Cole (PI), Joseph Roy (UIUC)
Graduate research assistant (UIUC): Tim Mahrt

This project examines the prosodic annotations of individual listeners in data collected for our project on Constituents and Heads in Prosody Perception. We use statistical modeling with Generalized Additive Mixed Models to explore the effect of acoustic cues on the assignment of prosodic prominence for individual listeners, comparing annotations from untrained listeners performing Rapid Prosody Transcription, with ToBI annotations from linguists with expertise in prosodic analysis.