Sign Language Phonology

Diane Brentari

DOI: https://doi.org/10.1002/9781444343069.ch21

2011-09-16

Abstract:This chapter contains sections titled: Introduction Structure Modality Effects (The Effects of Communication Mode) Iconicity Effects on Phonological Representation Conclusion Introduction Structure Modality Effects (The Effects of Communication Mode) Iconicity Effects on Phonological Representation Conclusion

What problem does this paper attempt to address?

Ten things you should know about sign languages

Karen Emmorey

DOI: https://doi.org/10.1177/09637214231173071

Abstract:The ten things you should know about sign languages are the following. 1) Sign languages have phonology and poetry. 2) Sign languages vary in their linguistic structure and family history, but share some typological features due to their shared biology (manual production). 3) Although there are many similarities between perceiving and producing speech and sign, the biology of language can impact aspects of processing. 4) Iconicity is pervasive in sign language lexicons and can play a role in language acquisition and processing. 5) Deaf and hard-of-hearing children are at risk for language deprivation. 6) Signers gesture when signing. 7) Sign language experience enhances some visual-spatial skills. 8) The same left hemisphere brain regions support both spoken and sign languages, but some neural regions are specific to sign language. 9) Bimodal bilinguals can code-blend, rather code-switch, which alters the nature of language control. 10) The emergence of new sign languages reveals patterns of language creation and evolution. These discoveries reveal how language modality does and does not affect language structure, acquisition, processing, use, and representation in the brain. Sign languages provide unique insights into human language that cannot be obtained by studying spoken languages alone.
Visual boundaries in sign motion: processing with and without lip-reading cues

Julia Krebs,Evie A. Malaia,Ronnie B. Wilbur,Dietmar Roehm

DOI: https://doi.org/10.3765/elm.2.5336

2023-01-27

Experiments in Linguistic Meaning

Abstract:Sign languages demonstrate a higher degree of iconicity than spoken languages. Studies on a number of unrelated sign languages show that the event structure of verb signs is reflected in the phonological form of the signs (Wilbur (2008), Malaia & Wilbur (2012), Krebs et al. (2021)). Previous research showed that hearing non-signers (with no prior exposure to sign language) can use the iconicity inherent in the visual dynamics of a verb sign to correctly identify its event structure (telic vs. atelic). In two EEG experiments, hearing non-signers were presented with telic and atelic verb signs unfamiliar to them, which they had to classify in a two-choice lexical decision task in their native language. The first experiment assessed the timeline of neural processing mechanisms in non-signers processing telic/atelic signs without access to lip-reading cues in their native language, to understand the pathways for incorporation of physical perceptual motion features into linguistic processing. The second experiment further probed the impact of visual information provided by lip-reading (speech decoding based on visual information from the face of the speaker, most importantly, the lips) on the processing of telic/atelic signs in non-signers.
Facing and challenging language ideologies towards a more inclusive understanding of language in mathematics education research—the case of sign languages

Christina M. Krause

DOI: https://doi.org/10.1007/s11858-023-01526-y

2023-10-12

ZDM - International Journal on Mathematics Education

Abstract:Research on language in mathematics education is largely dominated by a 'normalcy' of spoken languages. This modal hegemony does not only affect a whole group of learners in failing to provide access that is epistemologically equitable—those using sign language as their preferred mode for mathematical discourse—it also obscures our view on the roles language can play in mathematical thinking and learning. As a field, we can only win from seeking to understand Deaf learners of mathematics beyond a disability, as learners of mathematics with a specific linguistic background that influences mathematical thinking and learning in peculiar ways. In this contribution, I suggest a shift in mindset towards a more inclusive view on language in mathematics education research and practice. I propose basic principles to inform a perspective for reconsidering the role of language in mathematics thinking and learning, inspired by work of philosopher Francois Jullien. This perspective counters a perspective that merely integrates sign language into existing research and instead searches for dialogue between linguistic modalities in learning mathematics, looking beyond language as spoken or written. This approach will be illustrated by the case of the modal affordance of iconicity foregrounded in signed mathematical discourse, its role in Deaf students' mathematics thinking and learning, and how this can inform existing research and practice dealing with language in mathematics education.
Tutorial: Sign Language Linguistics

Fabian Bross

DOI: https://doi.org/10.1007/978-3-030-98479-3_2

2022-01-01

Abstract:This tutorial offers a brief overview of linguistic research into sign languages. The tutorial’s target audiences are people with some background in linguistics of spoken languages. For the sake of brevity, I will only concentrate on some major topics. I will briefly introduce sign languages, discuss some basics of phonological structure of these types of languages (including the use of space), discuss some new findings on the syntax of sign languages, and, finally, will briefly address some methodological issues. The majority of data will come from German Sign Language, although data from other sign languages is also included.
Iconicity as structure mapping

Karen Emmorey

DOI: https://doi.org/10.1098/rstb.2013.0301

2014-09-19

Abstract:Linguistic and psycholinguistic evidence is presented to support the use of structure-mapping theory as a framework for understanding effects of iconicity on sign language grammar and processing. The existence of structured mappings between phonological form and semantic mental representations has been shown to explain the nature of metaphor and pronominal anaphora in sign languages. With respect to processing, it is argued that psycholinguistic effects of iconicity may only be observed when the task specifically taps into such structured mappings. In addition, language acquisition effects may only be observed when the relevant cognitive abilities are in place (e.g. the ability to make structural comparisons) and when the relevant conceptual knowledge has been acquired (i.e. information key to processing the iconic mapping). Finally, it is suggested that iconicity is better understood as a structured mapping between two mental representations than as a link between linguistic form and human experience.
More Than Looks: Exploring Methods to Test Phonological Discrimination in the Sign Language Kata Kolok

Hannah Lutzenberger,Marisa Casillas,Paula Fikkert,Onno Crasborn,Connie de Vos

DOI: https://doi.org/10.1080/15475441.2023.2277472

2024-02-17

Abstract:The lack of diversity in the language sciences has increasingly been criticized as it holds the potential for producing flawed theories. Research on (i) geographically diverse language communities and (ii) on sign languages is necessary to corroborate, sharpen, and extend existing theories. This study contributes a case study of adapting a well-established paradigm to study the acquisition of sign phonology in Kata Kolok, a sign language of rural Bali, Indonesia. We conducted an experiment modeled after the familiarization paradigm with child signers of Kata Kolok. Traditional analyses of looking time did not yield significant differences between signing and non-signing children. Yet, additional behavioral analyses (attention, eye contact, hand behavior) suggest that children who are signers and those who are non-signers, as well as those who are hearing and those who are deaf, interact differently with the task. This study suggests limitations of the paradigm due to the ecology of sign languages and the sociocultural characteristics of the sample, calling for a mixed-methods approach. Ultimately, this paper aims to elucidate the diversity of adaptations necessary for experimental design, procedure, and analysis, and to offer a critical reflection on the contribution of similar efforts and the diversification of the field.
Language as Description, Indication, and Depiction

Lindsay Ferrara,Gabrielle Hodge

DOI: https://doi.org/10.3389/fpsyg.2018.00716

2018-05-23

Abstract:Signers and speakers coordinate a broad range of intentionally expressive actions within the spatiotemporal context of their face-to-face interactions (Parmentier, 1994; Clark, 1996; Johnston, 1996; Kendon, 2004). Varied semiotic repertoires combine in different ways, the details of which are rooted in the interactions occurring in a specific time and place (Goodwin, 2000; Kusters et al., 2017). However, intense focus in linguistics on conventionalized symbolic form/meaning pairings (especially those which are arbitrary) has obscured the importance of other semiotics in face-to-face communication. A consequence is that the communicative practices resulting from diverse ways of being (e.g., deaf, hearing) are not easily united into a global theoretical framework. Here we promote a theory of language that accounts for how diverse humans coordinate their semiotic repertoires in face-to-face communication, bringing together evidence from anthropology, semiotics, gesture studies and linguistics. Our aim is to facilitate direct comparison of different communicative ecologies. We build on Clark's (1996) theory of language use as 'actioned' via three methods of signaling: describing, indicating, and depicting. Each method is fundamentally different to the other, and they can be used alone or in combination with others during the joint creation of multimodal 'composite utterances' (Enfield, 2009). We argue that a theory of language must be able to account for all three methods of signaling as they manifest within and across composite utterances. From this perspective, language-and not only language use-can be viewed as intentionally communicative action involving the specific range of semiotic resources available in situated human interactions.
Communication, Language, and Modality in the Education of Deaf Students

Connie C. Mayer,Beverly J. Trezek

DOI: https://doi.org/10.3390/educsci13101033

2023-10-16

Education Sciences

Abstract:In the history of deaf education, questions attending communication, language, and modality have generated much discussion, and even heated debate. This should not be surprising as these questions touch on a fundamental issue that is central to policy and practice in the field—how to provide early, ready, and meaningful linguistic access. While one point of agreement is that such access is vital for age-appropriate language and literacy development, there is less consensus on how this access should be realized. This focus has heightened consequences and significance in the current context in which auditory access to spoken language is possible for the majority of deaf children. With a goal of reframing the conversation, the focus of this article will be on making the critical distinctions between language and modality that can inform understandings as to how access can be best achieved for an increasingly diverse population of deaf children and their families.
Characterizing language production across modalities

Marc Gimeno-Martínez,Cristina Baus

DOI: https://doi.org/10.1080/02643294.2024.2315823

IF: 3.75

2024-02-22

Cognitive Neuropsychology

Abstract:This study investigates factors influencing lexical access in language production across modalities (signed and oral). Data from deaf and hearing signers were reanalyzed (Baus and Costa, 2015, On the temporal dynamics of sign production: An ERP study in Catalan Sign Language (LSC). Brain Research , 1609 (1), 40-53. https://doi.org/10.1016/j.brainres.2015.03.013; Gimeno-Martínez and Baus, 2022, Iconicity in sign language production: Task matters. Neuropsychologia , 167 , 108166. https://doi.org/10.1016/j.neuropsychologia.2022.108166) testing the influence of psycholinguistic variables and ERP mean amplitudes on signing and naming latencies. Deaf signers' signing latencies were influenced by sign iconicity in the picture signing task, and by spoken psycholinguistic variables in the word-to-sign translation task. Additionally, ERP amplitudes before response influenced signing but not translation latencies. Hearing signers' latencies, both signing and naming, were influenced by sign iconicity and word frequency, with early ERP amplitudes predicting only naming latencies. These findings highlight general and modality-specific determinants of lexical access in language production.

psychology, experimental
Effects Of Phonology And Meaning On Processing Chinese Sign Language Sentences

Dengfeng Yao,Minghu Jiang,Abudoukelimu Abulizi

DOI: https://doi.org/10.1109/FSKD.2015.7382103

2015-01-01

Abstract:The investigation on sentence processing has been the focus of language psychologists. The factors that affect sentence processing continue to be explored. However, Chinese Sign Language (CSL), which is a natural language for the deaf community, has not been analyzed on whether similar processing mechanisms underlie the procession of sentences of the spoken language. We examined the electrophysiological correlates of online sentence processing to clarify the role of phonology and meaning in CSL sentences. Electroencephalogram was recorded while 18 native signers observed the CSL sentences for comprehension. The subjects were presented with sentences in which semantic expectancy and phonological form were systematically manipulated to create four types of violations. These four conditions of interest were contrasted to a baseline sentence with a preferred semantic ending. We observed evidence that the phonetic and semantic related signs shared similar time course and effect direction verified that the semantic and phonological properties access already existed in the CSL sentence comprehension and generated a similar online processing cost.
New Perspectives on the Neurobiology of Sign Languages

Karen Emmorey

DOI: https://doi.org/10.3389/fcomm.2021.748430

Abstract:The first 40 years of research on the neurobiology of sign languages (1960-2000) established that the same key left hemisphere brain regions support both signed and spoken languages, based primarily on evidence from signers with brain injury and at the end of the 20th century, based on evidence from emerging functional neuroimaging technologies (positron emission tomography and fMRI). Building on this earlier work, this review focuses on what we have learned about the neurobiology of sign languages in the last 15-20 years, what controversies remain unresolved, and directions for future research. Production and comprehension processes are addressed separately in order to capture whether and how output and input differences between sign and speech impact the neural substrates supporting language. In addition, the review includes aspects of language that are unique to sign languages, such as pervasive lexical iconicity, fingerspelling, linguistic facial expressions, and depictive classifier constructions. Summary sketches of the neural networks supporting sign language production and comprehension are provided with the hope that these will inspire future research as we begin to develop a more complete neurobiological model of sign language processing.
Sign and Spoken Language Processing Differences in the Brain: A Brief Review of Recent Research

Hayley Bree Caldwell

DOI: https://doi.org/10.1177/09727531211070538

2022-02-15

Annals of Neurosciences

Abstract:Background: It is currently accepted that sign languages and spoken languages have significant processing commonalities. The evidence supporting this often merely investigates frontotemporal pathways, perisylvian language areas, hemispheric lateralization, and event-related potentials in typical settings. However, recent evidence has explored beyond this and uncovered numerous modality-dependent processing differences between sign languages and spoken languages by accounting for confounds that previously invalidated processing comparisons and by delving into the specific conditions in which they arise. However, these processing differences are often shallowly dismissed as unspecific to language. Summary: This review examined recent neuroscientific evidence for processing differences between sign and spoken language modalities and the arguments against these differences’ importance. Key distinctions exist in the topography of the left anterior negativity (LAN) and with modulations of event-related potential (ERP) components like the N400. There is also differential activation of typical spoken language processing areas, such as the conditional role of the temporal areas in sign language (SL) processing. Importantly, sign language processing uniquely recruits parietal areas for processing phonology and syntax and requires the mapping of spatial information to internal representations. Additionally, modality-specific feedback mechanisms distinctively involve proprioceptive post-output monitoring in sign languages, contrary to spoken languages’ auditory and visual feedback mechanisms. The only study to find ERP differences post-production revealed earlier lexical access in sign than spoken languages. Themes of temporality, the validity of an analogous anatomical mechanisms viewpoint, and the comprehensiveness of current language models were also discussed to suggest improvements for future research. Key message: Current neuroscience evidence suggests various ways in which processing differs between sign and spoken language modalities that extend beyond simple differences between languages. Consideration and further exploration of these differences will be integral in developing a more comprehensive view of language in the brain.
How (and why) does iconicity effect lexical access: An electrophysiological study of American sign language

Meghan E McGarry,Katherine J Midgley,Phillip J Holcomb,Karen Emmorey

DOI: https://doi.org/10.1016/j.neuropsychologia.2023.108516

IF: 3.054

2023-05-03

Neuropsychologia

Abstract:Prior research has found that iconicity facilitates sign production in picture-naming paradigms and has effects on ERP components. These findings may be explained by two separate hypotheses: (1) a task-specific hypothesis that suggests these effects occur because visual features of the iconic sign form can map onto the visual features of the pictures, and (2) a semantic feature hypothesis that suggests that the retrieval of iconic signs results in greater semantic activation due to the robust representation of sensory-motor semantic features compared to non-iconic signs. To test these two hypotheses, iconic and non-iconic American Sign Language (ASL) signs were elicited from deaf native/early signers using a picture-naming task and an English-to-ASL translation task, while electrophysiological recordings were made. Behavioral facilitation (faster response times) and reduced negativities were observed for iconic signs (both prior to and within the N400 time window), but only in the picture-naming task. No ERP or behavioral differences were found between iconic and non-iconic signs in the translation task. This pattern of results supports the task-specific hypothesis and provides evidence that iconicity only facilitates sign production when the eliciting stimulus and the form of the sign can visually overlap (a picture-sign alignment effect).
Iconicity in Signed and Spoken Vocabulary: A Comparison Between American Sign Language, British Sign Language, English, and Spanish

Marcus Perlman,Hannah Little,Bill Thompson,Robin L Thompson

DOI: https://doi.org/10.3389/fpsyg.2018.01433

2018-08-14

Abstract:Considerable evidence now shows that all languages, signed and spoken, exhibit a significant amount of iconicity. We examined how the visual-gestural modality of signed languages facilitates iconicity for different kinds of lexical meanings compared to the auditory-vocal modality of spoken languages. We used iconicity ratings of hundreds of signs and words to compare iconicity across the vocabularies of two signed languages - American Sign Language and British Sign Language, and two spoken languages - English and Spanish. We examined (1) the correlation in iconicity ratings between the languages; (2) the relationship between iconicity and an array of semantic variables (ratings of concreteness, sensory experience, imageability, perceptual strength of vision, audition, touch, smell and taste); (3) how iconicity varies between broad lexical classes (nouns, verbs, adjectives, grammatical words and adverbs); and (4) between more specific semantic categories (e.g., manual actions, clothes, colors). The results show several notable patterns that characterize how iconicity is spread across the four vocabularies. There were significant correlations in the iconicity ratings between the four languages, including English with ASL, BSL, and Spanish. The highest correlation was between ASL and BSL, suggesting iconicity may be more transparent in signs than words. In each language, iconicity was distributed according to the semantic variables in ways that reflect the semiotic affordances of the modality (e.g., more concrete meanings more iconic in signs, not words; more auditory meanings more iconic in words, not signs; more tactile meanings more iconic in both signs and words). Analysis of the 220 meanings with ratings in all four languages further showed characteristic patterns of iconicity across broad and specific semantic domains, including those that distinguished between signed and spoken languages (e.g., verbs more iconic in ASL, BSL, and English, but not Spanish; manual actions especially iconic in ASL and BSL; adjectives more iconic in English and Spanish; color words especially low in iconicity in ASL and BSL). These findings provide the first quantitative account of how iconicity is spread across the lexicons of signed languages in comparison to spoken languages.
Sociological and Cultural Aspects of the Language Perception of the Deaf

Tatiana Kozlova,Ludmila Dementyeva,Elena Ilyina,Natalya Didenko

DOI: https://doi.org/10.2991/assehr.k.211025.002

2021-01-01

Abstract:The article examines the basic principles of language perception, thinking and socialization of deaf people and outlines the range of problems of learning and the social role of sign language. The sociological aspect of linguistic thinking is considered in connection with the meaning of the sign language of the deaf. The authors promote the cultural-situational...
Situating Language in the Real-World: The Role of Multimodal Iconicity and Indexicality

Margherita Murgiano,Yasamin Motamedi,Gabriella Vigliocco

DOI: https://doi.org/10.5334/joc.113

2021-01-01

Journal of Cognition

Abstract:In the last decade, a growing body of work has convincingly demonstrated that languages embed a certain degree of non-arbitrariness (mostly in the form of iconicity, namely the presence of imagistic links between linguistic form and meaning). Most of this previous work has been limited to assessing the degree (and role) of non-arbitrariness in the speech (for spoken languages) or manual components of signs (for sign languages). When approached in this way, non-arbitrariness is acknowledged but still considered to have little presence and purpose, showing a diachronic movement towards more arbitrary forms. However, this perspective is limited as it does not take into account the situated nature of language use in face-to-face interactions, where language comprises categorical components of speech and signs, but also multimodal cues such as prosody, gestures, eye gaze etc. We review work concerning the role of context-dependent iconic and indexical cues in language acquisition and processing to demonstrate the pervasiveness of non-arbitrary multimodal cues in language use and we discuss their function. We then move to argue that the online omnipresence of multimodal non-arbitrary cues supports children and adults in dynamically developing situational models.
Exploring Strategies for Modeling Sign Language Phonology

Lee Kezar,Riley Carlin,Tejas Srinivasan,Zed Sehyr,Naomi Caselli,Jesse Thomason

2023-09-30

Abstract:Like speech, signs are composed of discrete, recombinable features called phonemes. Prior work shows that models which can recognize phonemes are better at sign recognition, motivating deeper exploration into strategies for modeling sign language phonemes. In this work, we learn graph convolution networks to recognize the sixteen phoneme "types" found in ASL-LEX 2.0. Specifically, we explore how learning strategies like multi-task and curriculum learning can leverage mutually useful information between phoneme types to facilitate better modeling of sign language phonemes. Results on the Sem-Lex Benchmark show that curriculum learning yields an average accuracy of 87% across all phoneme types, outperforming fine-tuning and multi-task strategies for most phoneme types.

Computation and Language,Computer Vision and Pattern Recognition
Homesign Research, Gesture Studies, and Sign Language Linguistics: The Bigger Picture of Homesign and Homesigners

Marie Coppola

DOI: https://doi.org/10.1111/tops.12766

IF: 3.265

2024-11-13

Topics in Cognitive Science

Abstract:Studies of homesigns have shed light on the human capacity for language and on the challenging problem of language acquisition. The study of homesign has evolved from a perspective grounded in gesture studies and child development to include sign language linguistics and the role of homesigns in language emergence at the community level. One overarching finding is that homesigns more closely resemble sign languages used by linguistic communities than they resemble the gestures produced by hearing people along with spoken language. Homesigns may not exhibit all of the linguistic properties of community languages, but the properties they do exhibit are language properties, and for the people who use them, homesigns are their language. Further, the linguistic structures in homesigns are innovated by the deaf people who use them and are imperfectly learned by their hearing communication partners. I close with a call to action: We cannot celebrate discoveries about the mind made possible by studies of homesigns and emerging languages while ignoring the pervasiveness of language deprivation among deaf people, and the relative lack of deaf participation in science, even in studies of sign languages. While the scientific community learns much from studying homesigns and sign languages, we also have a responsibility to work toward ensuring that every deaf person has access to language, communication, and education.

psychology, experimental
Iconicity in the lab: a review of behavioral, developmental, and neuroimaging research into sound-symbolism

Gwilym Lockwood,Mark Dingemanse

DOI: https://doi.org/10.3389/fpsyg.2015.01246

IF: 3.8

2015-08-24

Frontiers in Psychology

Abstract:This review covers experimental approaches to sound-symbolism-from infants to adults, and from Sapir's foundational studies to twenty-first century product naming. It synthesizes recent behavioral, developmental, and neuroimaging work into a systematic overview of the cross-modal correspondences that underpin iconic links between form and meaning. It also identifies open questions and opportunities, showing how the future course of experimental iconicity research can benefit from an integrated interdisciplinary perspective. Combining insights from psychology and neuroscience with evidence from natural languages provides us with opportunities for the experimental investigation of the role of sound-symbolism in language learning, language processing, and communication. The review finishes by describing how hypothesis-testing and model-building will help contribute to a cumulative science of sound-symbolism in human language.

psychology, multidisciplinary
Cross-modal translation priming and iconicity effects in deaf signers and hearing learners of American Sign Language

Megan Mott,Katherine J Midgley,Phillip J Holcomb,Karen Emmorey

DOI: https://doi.org/10.1017/S1366728919000889

Abstract:This study used ERPs to a) assess the neural correlates of cross-linguistic, cross-modal translation priming in hearing beginning learners of American Sign Language (ASL) and deaf highly proficient signers and b) examine whether sign iconicity modulates these priming effects. Hearing learners exhibited translation priming for ASL signs preceded by English words (greater negativity for unrelated than translation primes) later in the ERP waveform than deaf signers and exhibited earlier and greater priming for iconic than non-iconic signs. Iconicity did not modulate translation priming effects either behaviorally or in the ERPs for deaf signers (except in a 800-1000 ms time window). Because deaf signers showed early translation priming effects (beginning at 400ms-600ms), we suggest that iconicity did not facilitate lexical access, but deaf signers may have recognized sign iconicity later in processing. Overall, the results indicate that iconicity speeds lexical access for L2 sign language learners, but not for proficient signers.

Sign Language Phonology

Ten things you should know about sign languages

Visual boundaries in sign motion: processing with and without lip-reading cues

Facing and challenging language ideologies towards a more inclusive understanding of language in mathematics education research—the case of sign languages

Tutorial: Sign Language Linguistics

Iconicity as structure mapping

More Than Looks: Exploring Methods to Test Phonological Discrimination in the Sign Language Kata Kolok

Language as Description, Indication, and Depiction

Communication, Language, and Modality in the Education of Deaf Students

Characterizing language production across modalities

Effects Of Phonology And Meaning On Processing Chinese Sign Language Sentences

New Perspectives on the Neurobiology of Sign Languages

Sign and Spoken Language Processing Differences in the Brain: A Brief Review of Recent Research

How (and why) does iconicity effect lexical access: An electrophysiological study of American sign language

Iconicity in Signed and Spoken Vocabulary: A Comparison Between American Sign Language, British Sign Language, English, and Spanish

Sociological and Cultural Aspects of the Language Perception of the Deaf

Situating Language in the Real-World: The Role of Multimodal Iconicity and Indexicality

Exploring Strategies for Modeling Sign Language Phonology

Homesign Research, Gesture Studies, and Sign Language Linguistics: The Bigger Picture of Homesign and Homesigners

Iconicity in the lab: a review of behavioral, developmental, and neuroimaging research into sound-symbolism

Cross-modal translation priming and iconicity effects in deaf signers and hearing learners of American Sign Language