Frontier Science

Emotional interaction between human and machine

Project Profile

52060-Computer Applications
Project Summary

Project Content

Issue Overview:
With the popularization of intelligent devices in recent years, pure imperative operation and its mechanical response gradually cannot meet the needs of natural human-computer interaction. People begin to pursue higher-level human-computer interaction technology. EEG can establish a direct connection between human beings and external devices, and does not convey brain thoughts to the outside world through direct language and actions, so that machines can naturally carry out "perception" interaction like human beings. The development of EEG inevitably arouses people's thinking. The communication between people is natural and emotional. How to make the machines interacting with people have similar ability to observe, understand and generate emotional characteristics is a scientific problem to be solved. Human-computer emotional interaction enables computers to recognize emotional information conveyed by users in a natural way, aiming at improving human-computer interaction and making it more natural and comfortable. With the continuous development of emotional computing and other technologies, emotional interaction has become the main development trend of human-computer interaction in the future super-information age. At present, emotional interaction has become a hot direction in the field of artificial intelligence, and many team enterprises at home and abroad have begun the research and development of human-computer emotional interaction technology.
Emotional interaction is an interdisciplinary subject, involving neuroscience, psychology, computer science, linguistics and other disciplines. In the field of computer, emotional interaction is also the focus of research. In this field, emotional interaction is also called emotional recognition, emotional calculation or automatic emotional recognition. In order to carry out quantitative research on emotions, it is first necessary to quantify and define the basic emotions of human beings. These works have been deeply studied in the fields of psychology and computers [1] [2]. In terms of emotional interaction methods, early emotional interaction mostly discriminates the emotional information of users in a single way. These interaction methods include language [3], facial expression [4] [5] [6] and body posture [7] [8] [9], etc. However, these interaction methods based on a single pattern have limitations, such as specific application scenarios, inability to make full use of multiple semantic clues and low performance of interaction systems. With the progress of mechanical algorithms and the popularization and development of motion acquisition equipment, the advantages of multi-mode emotional interaction methods have gradually emerged. Some representative works, such as Wollmer et al. [10], use the method of two-way long-term and short-term memory network to recognize users' emotions from voice and face video data. Kahou et al. [11] used recurrent-convolution neural network (CNN-RNN) to model facial expressions, and used support vector machine (SVM) to process audio features, and proposed a feature fusion technology based on neural network to predict the emotions of characters in video. However, the research on multi-mode human-computer emotional interaction is still in its infancy, and there are few related research methods and achievements [12] [13]. The future development direction is to combine multi-mode interaction with machine learning methods more closely, and collect massive data through Internet crowdsourcing and other methods, so as to better mine the potential correlation between various modes and emotions.
Although emotional interaction has achieved gratifying results in the field of research, However, it still faces the following key problems and challenges in the future development: First, the current physical interaction mode of human-computer emotional interaction is single. The ideal human-computer emotional interaction should combine various interaction modes such as expression, voice, eyes, gestures, etc., and use bionic robots with face and body as physical media of emotional interaction. In addition, the current human-computer emotional interaction software is relatively simple, and more advanced algorithms are needed to complete emotional perception and communication in the future. How to adjust the interaction mode to better provide services for users, how to associate the feedback of users with the operation of computers, and how to perceive and identify the emotional information conveyed by users in a comfortable and reliable way are the challenges faced by human-computer emotional interaction software. Finally, there are many kinds of information processing methods in human-computer emotional interaction at present, among which there are no unified standards for emotional expression, emotional description, emotional acquisition methods and processing procedures. In the future, unified standards should be formulated at the physical and software levels to provide a good environment for the technical research and development and industrial development of emotional interaction.
Emotional interaction is committed to making machines try to perceive and transmit emotional information, bringing emotional wisdom to the digital world. This will be a milestone innovation in the field of human-computer interaction and even in the field of electronic information, and will also lead many industries to make revolutionary progress. In the field of consumption and service, the new customer service machine with emotional interaction function may completely replace manual customer service in the future. In the industrial field, robots with emotional interaction function will obviously improve production and work efficiency.
At present, most devices still interact through pure commands or simple limb movements. The solution of emotional interaction problems can provide intelligent and comfortable emotional interaction interfaces and bring more humanized interaction experience. This is not only a major innovation in the field of human-computer interaction, but also greatly promotes the development of sensor technology, motion capture technology, artificial intelligence, production and manufacturing and other fields. The revolutionary renewal of emotional interaction technology will bring immeasurable economic benefits to various industries. For example, in the service industry, emotional interaction technology can provide convenient and accurate services for human beings through intelligent service systems, such as nursing robots in hospitals and nursing homes; In advertising and commerce, emotional computing robots analyze human views and attitudes towards certain products or things by observing and recording human emotional changes and testing human excitement points, so as to promote the development of business orientation. At the same time, the influence of emotional interaction technology on future social life cannot be underestimated. For example, the technology is applied to detect the driver's emotional and behavioral state during driving and prevent dangerous conditions such as fatigue driving, thus reducing traffic accidents to a certain extent. Even in the medical field, monitoring and rehabilitation of the emotions of patients with autism and other diseases can reduce the social burden. Therefore, the emergence and application of intelligent systems with "emotional intelligence" will bring a better tomorrow for economic development and social life.

Referral Information

Recommended Organization
China Simulation Society
Recommended Expert
Reason for recommendation

Author's Introduction

Xia Shihong
Unit:Institute of Computing Technology, Chinese Academy of Sciences Professional Title: