Thirty Years Beyond Sign Language Computing:Retrospect and Prospect

Deng-Feng YAO,Ming-Hu JIANG,Hong BAO,Han-Jing LI,Abulizi ABUDOUKELIMU

DOI: https://doi.org/10.11897/SP.J.1016.2019.00111

2019-01-01

Chinese Journal of Computers

Abstract:The natural language processing of sign language is an important task in the field of artificial intelligence and information processing.Currently, with the development of information technology, the focus on the information processing of spoken language and written language, is gradually shifting from the word coding and input method to the grammatical level, and then to depth computing.However, sign language information processing is seriously lagging behind and remains at the starting stage.The main reason for this situation is that no ready-made sign language corpus resources can be used for machine learning and deep learning.Sign language machine translation, sign language question-answering system, information retrieval and information processing cannot be applied because of the lack of research foundation.The essential difference between sign language computing and traditional language computing is spatial modeling and it leads to that the core task of sign language computing is to convert single-channel representation to multi-channel representation, while the fundamental task of the traditional language computing is the disambiguation of single-channel representation.From the lexical, syntactic, semantic, pragmatic, and applied levels, sign language computing is reviewed, and the sign language machine translation and classifier predicates in computing are emphatically introduced.Classifier predicates are the key of sign language computing, and it is the breakthrough point of sign language computing.New technologies, such as the emergence of somatosensory devices, the rise of cognitive neuroscience and the progress of deep learning, have brought new opportunities to sign language computing in the Internet age.From the perspective of outlook, sign language computing is compared with spoken language computing.The trend of sign language computing and the breakthrough points are analyzed.The cognitive computing of sign language has been regarded as a mapping conversion process from the physical characteristics of gestures to semantic representation.The trend of sign language computing is to fill these intermediate steps, such as phonological features and semantic units.It avoids the semantic concepts obtained directly from the underlying physical features, focuses on the machine learning on the relationship between sign language behavior and language features, and establishes the statistical learning model of fusion spatial features.These breakthrough points include resource construction, text-to-scene and metaphor understanding.Among them, the text-to-scene in the sign language is helpful to realize the spatial information extraction including spatial orientation, object position, and the ambiguity of natural language can be eliminated by combining with the knowledge base, so as the threedimensional scene construction can be achieved and creates a breakthrough in understanding the spatial relationship and generating the virtual scene.It is pointed out that the sign language computing is from the embryonic period to the development period.Driven by the interdisciplinary, sign language computing may make a substantial breakthrough.The astonishing progress of traditional language computing has promoted artificial intelligence and human-computer interaction to develop further.If a series of problems about sign language computing can be solved in fields of theory, technology and engineering, it will greatly speed up the development of artificial intelligence and natural language processing.