Intelligent Human-Computer Interaction(IHCI)
The IHCI group/team is mainly engaged in the research of multi-perception
technology. Multi-perception technology is a comprehensive advanced
human-computer interface technology which integrate speech, gesture,
feeling (sensibility), and an important means for pervading computing
technologies into the human society. It realizes the imprecise natural
human-computer interaction similar to the interaction between humans.
Multi-perception machines implemented by using multi-perception
technology integrate with human natural language - speech, text
and body language - sign language, face, facial expression, lip
reading, head gesture and hand gesture, etc., and encode, compress
and integratethe information in these modalities including multimedia
information such as image, audio, video and text. The research goal
is to improve the communication ability between human and computer,
and to enable the present computers - blind, deaf and mute- to see,
to listen and to speak and to communicate with human naturally.
The current research topic of multi-perception technology includes
Chinese sign language recognition (SLR) and sign language synthesis
(SLS).
The IHCI group/team has made a lot of original achievements in Chinese
SLR and SLS, which has won a good reputation in the research area
both at home and abroad. A series of inventions, including the HMM
method of state tying has been proposed and one patent has been
granted. and More than 200 papers have been published in the international
& domestic journals and conferences,among them, 11 papers are
accepted by SCI, 8 of themcited 12 times, 57 papers accepted by
EI, 26 papers accepted by ISTP and 25 papers accepted by Chinese
Science Citation Database (CSCD),and 26 papers are cited 40 times.
Comparing with the international and domestic related research,
our research work mainly has following characteristics. In the research
area, we still try to solve the problem of natural interaction,
which belongs to multi-perception or multimodal-interface. For the
research direction and topic, however, most researchers focus on
the speech problem and only few researchers pay attention to sign
language communication. Furthermore , in sign language recognition
(SLR), the gesture vocabulary to be recognized is limited to the
small scale and belongs to the set of control command yet. Continuous
Chinese sign language recognition over a vocabulary of more than
5000 sign words is first implemented by our team.
In the aspect of enhancing the capability of understanding the spoken
and written language for deaf people, we have completed CSL synthesis
system based on models, which is able to synthesize 5500 gesture
and fingerspelling words and can be applied in the Internet environment
to make the hearing impaired understand the language of hearing-abled
people.
In summary, our system is the first to complete dialogue between
hearing impaired and hearing people, which can improve the system
performance in the research area from the experimental level on
the small scale vocabulary to the pratical level on the large scale
vocabulary.
|
Project Director
Prof. GAO Wen, Chief scientist.
Chinese sign language recognition system is a comprehensive application
system of multi-perception technology, which makes the communication
between the hearing impaired and hearing people more convenient. Currently,
the system can realize the recognition task on a vocabulary of 5177
sign words, which is the largest vocabulary size in the reported literature.
Former Members ( of the CSL Recognition Research Team)
Prof. Xilin Chen, Dr. Jiangqin Wu, Dr. Jiyong Ma and MS. Xiujuan
Gao
Current Members
Ph.D Candidates: Chunli Wang, Gaolin Fang, and Liangguo Zhang.
Zhen Li, Yan Ma
Chinese sign language synthesis system as one of an important
parts of deaf dialogue system, is to translate spoken language or
written language into sign language that can be understood by deaf
people.
The advanced sensor devices including 6DOF trackers and Datagloves
are used to capture sign data and build the CSL vocabulary database.
and then the synthesis technology is applied to automatically synthesize
the human body motion data with the given text sentences or a segment
of speech. Finally, computer human body animation technology is
employed to drive the virtual human to perform sign language using
the captured motion data.
Technologies in this project can be extended and applied to other
related virtual human synthesis applications, such as human-computer
interaction (HCI), motion representation, ergonomics, video compression,
game, entertainment, and military drills.
Former Members
Prof. Yibo Song, Prof. Baocai Yin, Dr. Jie Yan, Dr. Lin Xu and
MS. Zhiguo Li
Current Members
Prof. Zhaoqi WANG, Dr. Yiqiang Chen, MS. Changshui Yang and Ph.D
Candidate: Dalong Jiang
|
