Today (September 23) is the annual International Sign Language Day. The Chinese University’s research team has cooperated with Google, the Japanese consortium and Kansai Gakuin University to develop the world’s first multilingual network that combines artificial intelligence and sign language linguistics. The game "Sign Language Village".
Players need to use sign language as the main communication language to complete tasks in the virtual town, including ordering at a restaurant. The artificial intelligence recognition system will analyze the accuracy of the player’s sign language and explain the meaning and culture of “sign language”.
From left, Shi Wanping, deputy director of the CUHK Sign Language and Deaf Research Center, and Cheng Jiayao, assistant computer director of the CUHK Sign Language and Deaf Research Center.
(Provided by CUHK)
Pave the way to develop an automatic translation system for Hong Kong-Japanese sign language
CUHK, Google, the Japanese Consortium and Kansai Gakuin University jointly launched the "Shuwa Project" in 2020, hoping to develop an automatic sign language translation system starting from Hong Kong Sign Language and Japanese Sign Language.
After more than a year of hard work, the team designed the first machine learning model that uses camera lens tracking and artificial intelligence to analyze hand movements, body movements and facial expression characteristics, and develops the world's first multilingual combination of artificial intelligence and sign language linguistics. The online game "Sign Language Village" allows players to learn Hong Kong Sign Language and Japanese Sign Language in the form of games.
The research team of the Chinese University has cooperated with Google, the Japanese consortium and Kansai Gakuin University to develop a multilingual online game "Sign Language Village".
(Photo provided by CUHK)
Artificial intelligence recognition system interprets the meaning of sign language
"Sign Language Village" launched a test version as early as May this year. It was used by more than 8,500 people in Hong Kong and Japan, and it was officially launched today.
Players need to use sign language as the main communication language to complete tasks in the virtual town, including ordering in restaurants, finding hotel accommodations, etc. The artificial intelligence recognition system will analyze the accuracy of the player’s sign language and explain the meaning and culture of different sign languages.
The team explained that sign languages are different in different parts of the world, but the characteristics of hand shape, direction and movement are the same, and the changes are limited. In the future, the "Sign Language Village" will add more scenes, sign language vocabulary and more local sign languages. It is expected to develop an automatic sign language translation in the long term. The model enables common computer or mobile phone lenses to translate sign language into spoken language, provides a platform for recording and learning sign language, so that deaf people can sign language to participate in social, economic and cultural activities, and build a barrier-free society.
01News
Saigon pop-up "Drifting Classroom" Interpreting "Pineapple Buns" to promote the world of sign language and sign language | The United States advocates immersive learning mainstream middle school language elective subject has the world of sign language picking | Mission Tokyo Paralympics | The traditional sign language interpreter within the frame is both an actor and an encyclopedia Tokyo Paralympic | Carrie Lam: Hong Kong and Taiwan will broadcast the first time the lyrics of the national anthem in sign language for the opening of the National Committee of the Chinese People's Political Consultative Conference. "Flesh" turned out to be the sign that the theater reopened on the seventh day of the month | "Should I put out your phone?" There is a new word brainwashing song with hidden sign language!
/cloudfront-eu-central-1.images.arcpublishing.com/prisa/INU3EC6I6VF6NEMPKWATA22NNA.jpg)
