Emotion Connect

Introduction

• In many studies, some autistic patients have difficulty in socializing due to emotional recognition barriers. We hope to design an application platform that can assist emotional recognition and evaluate identification ability through the assistance of facial emotion recognition technology.
• First, in order to accurately record users' real emotional responses, we constructed an emotion recording platform (A+B+C). During the process of users watching videos, the system records and analyzes their facial emotions, generating analytical logs for researchers to further analyze. This is a more indirect recording method that allows users to record their emotions without their knowledge. The system can record users' emotional responses more authentically, without affecting the true emotional responses of the test subjects due to the awareness of being tested.
• Next, we designed a supporting app (D) to provide emotional recognition information and also serve as a training tool for emotional recognition.

System Architecture

Components

A. Video Emotion Labele System

• A tool platform that can perform emotional tagging for online video clips.
• languages：javascript、css、html
• screenshot

B. Emotion Tracker

• This is a real-time emotion recognition system. The system captures faces in the environment through a camera and performs emotional recognition. The recognition results can be output as a json format file.
• We use FER-2013(https://www.kaggle.com/datasets/msambare/fer2013) as the training data source to construct a emotion recognition model using CNN.
• language：python
• screenshot

C. Audience Emotion Analysis

• Aggregate the information generated by modules A (Video Emotion Labele System) and B (Emotion Tracker), and generate analytical information on video segments corresponding to user responses.
• screenshot

D. IOS/Android App

• Apple Store：https://apps.apple.com/tw/app/em-touch/id6448122198
• This app helps the user improve social interaction through of both expression and recognition of emotions.
• features
  • Utilize the mobile phone’s camera to analyze the facial expressions of the speaker in real-time, providing assistance in emotion recognition.
  • Through a database of facial images and audio data, offer users a mini-game for practicing emotion identification.
• screenshot