Project involving emotional recognition using electroencephelographic (EEG) signals. EEG .mat files provided courtesy of Queen Mary University of London, from their DEAP dataset.
The objective is to ascertain viability of EEG signals as a means of discerning different emotional states, motivated by the idea of 'affective computing'.
The dataset contains physiological data from 32 participants. Each participant viewed 40 one-minute-long excerpts of music videos and in the duration of each viewing, the physiological signals were recorded. Participants then rated the experience of each video viewing based on valence, arousal, liking and dominance.
The physiological data consists of 40 features — 32 channels of EEG readings; and a further 8 peripheral readings such as skin temperature, respiration amplitude, electrooculogram (EOG), electrocardiogram (ECG), galvanic skin response (GSR), blood volume by plethysmograph, and electromyograms (EMG) of Zygomaticus and Trapezius muscles are also included but all are superfluous to this project.
The EEG recordings were conducted according to the 10-20 system and the 32 EEG channels are recorded following to the standard.
The EEG data obtained from the dataset was already pre-processed at the source. It was down-sampled from 512Hz to 128Hz, a bandpass frequency filter from 4.0 – 45.0Hz was applied and EOG artefacts (interference from the electrical activity generated by eye movement) were removed.
A total of 1280 samples were obtained – 32 participants, each viewing 40 videos. Each sample is composed of 8064 numerical features – a 63-second recording, consisting of 3-second pre-trial baseline, followed by 60 seconds of video viewing, sampled at 128Hz. Each sample contained 40 channels, only 32 of which were pertinent to EEG. The 32 channels correspond to the 32 electrode positions in the 10- 20 system.
The relevant EEG information was extracted and consolidated and no further pre-processing was made to the EEG data.
The participants provided numerical ratings for each video regarding valence, arousal, dominance, liking and familiarity. Valence, arousal, dominance and liking were rated directly after each trial on a continuous 9-point scale, while familiarity was rated after the end of the experiment on a 5-point integer scale. This project utilises the two-dimensional valence-arousal model to classify emotion and only the ratings for valence and arousal were extracted.
The ratings for valence and arousal were further simplified to binary values ’High’ and ‘Low’, in accordance with the valence-arousal model. Ratings between 0 and 4.5 were labelled ‘Low’ and anything above 4.5 was labelled ‘High’. This simplifies the classification process and will provide significant improvement to the accuracy of the model.
Of the 32 EEG channels available, only channels corresponding to Fp1, Fp2, F3 and F4 were selected. For each channel, a bandpass butterworth filter was applied to obtain the alpha (8-12Hz) and beta (12-30Hz) frequency bands of each channel.
Principal component analysis was performed to reduce each set of features to between 1 to 25. Linear discriminant and Bayesian models were then trained with this range of features.
Linear discriminant and bayesian models were trained with varying predictor variables and number of principal components used. For each combination, the experiment was repeated 10 times and the data analysed was randomly divided into a training subset and a testing subset, in the ratio 80:20. As the accuracy of the model is dependent on the training data, randomisation of the data and averaging over multiple runs would be a fairer measure of the performance of the model.
The maximum, minimum and mean performance of each model was recorded.
| Feature Set | Linear Discriminant Classifier (Accuracy %) | Naive Bayes' Classifier (Accuracy %) |
|---|---|---|
| F3/F4 Beta Power | 69.5 | 67.0 |
| Fp1, Fp2 Beta Freq. | 70.3 | 71.1 |
| Feature Set | Linear Discriminant Classifier (Accuracy %) | Naive Bayes' Classifier (Accuracy %) |
|---|---|---|
| Fp1, Fp2 Alpha, Beta Power | 69.5 | 67.0 |
| Fp1, Fp2 Alpha Freq. | 68.9 | 69.9 |
| Fp1, Fp2 Beta Freq. | 67.6 | 69.5 |
| F3/F4 Alpha, Beta Power | 68.4 | 68.4 |
| F3/F4 Beta Power | 66.4 | 66.4 |
| F3/F4 Alpha Freq. | 68.4 | 63.6 |
The results proved sufficient in providing evidence that electroencephalography is a viable method of recognising human emotion. There are various factors that affect the efficacy of EEG readings for emotion recognition, chiefly being the positions on the scalp from which the readings are taken, and the exact features of the readings that are taken. As this field is still relatively new, the full extent of the capabilities of this technology is not fully known. However, ascertaining its viability is the first step to any future development and serves as crucial progress to the betterment of many other fields.
Further exploration can be done in the areas of signal processing and data analysis. The performance of the models is highly dependent on both the data being operated on and the statistical model used. Further research can be done to ascertain the best methods to process and filter the EEG data, as well as to identify the best statistical model to use for emotion analysis.