Advanced EEG Analysis Based Emotion Recognition: A Deep Learning Classifier with Hybrid Feature Selection and Artifact Reduction

International Journal of Electrical and Electronics Engineering

© 2023 by SSRG - IJEEE Journal

Volume 10 Issue 11

Year of Publication : 2023

Authors : T. Manoj Prasath, R. Vasuki

10.14445/23488379/IJEEE-V10I11P112

How to Cite?

T. Manoj Prasath, R. Vasuki, "Advanced EEG Analysis Based Emotion Recognition: A Deep Learning Classifier with Hybrid Feature Selection and Artifact Reduction," SSRG International Journal of Electrical and Electronics Engineering, vol. 10,  no. 11, pp. 128-141, 2023. Crossref, https://doi.org/10.14445/23488379/IJEEE-V10I11P112

Abstract:

Emotion recognition from Electroencephalogram (EEG) signals has emerged as a crucial area of research with a broad spectrum of applications. To enhance the accuracy and effectiveness of emotion classification, this study presents an innovative approach that combines advanced signal processing and deep learning techniques. The proposed methodology is structured into several distinct phases for optimal performance. Initially, EEG signals are preprocessed using a Fractional Order Butterworth (FOB) filter, which exhibits flexibility, allowing for precise control over the trade-off between preserving relevant emotional information and mitigating unwanted interference. The Short Time Fourier Transform (STFT) is applied to extract time-frequency representations from the preprocessed EEG signals. This transformation captures dynamic changes in spectral content, providing a comprehensive view of the emotional state over time. A hybrid approach is employed to optimize the feature set and enhance the efficiency of emotion classification. This approach combines the Improved Artificial Fish Swarm algorithm with Particle Swarm Optimization (IAFS-PSO). Combining these algorithms efficiently navigates the solution space to select the most informative features, ensuring the subsequent classification process is based on a highly relevant and discriminative set of features. Emotion recognition is accomplished using a hybrid attention-based Convolutional Neural Network combined with Long Short-Term Memory (CNN-LSTM) networks. The CNN component captures spatial features in the EEG data, while the LSTM component is adapted to modelling temporal dependencies. The hybrid architecture is further enhanced with attention mechanisms, allowing the model to focus on critical segments of the EEG data, thereby improving classification accuracy. The evaluation of this approach is conducted rigorously, and the results of this study highlight the proposed methodology’s effectiveness and efficiency in accurately classifying emotional states from EEG signals.

Keywords:

EEG, Fractional Order Butterworth filter, Hybrid attention CNN-LSTM, Hybrid IAFS-PSO, Short Time Fourier Transform.

References:

[1] Theerawit Wilaiprasitporn et al., “Affective EEG-Based Person Identification Using the Deep Learning Approach,” IEEE Transactions on Cognitive and Developmental Systems, vol. 12, no. 3, pp. 486–496, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[2] Md. Rabiul Islam et al., “Emotion Recognition from EEG Signal Focusing on Deep Learning and Shallow Learning Techniques,” IEEE Access, vol. 9, pp. 94601–94624, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Rania Alhalaseh, and Suzan Alasasfeh, “Machine-Learning-Based Emotion Recognition System Using EEG Signals,” Computers, vol. 9, no. 4, pp. 1-15, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Aya Hassouneh, A.M. Mutawa, and M. Murugappan, “Development of a Real-Time Emotion Recognition System Using Facial Expressions and EEG Based on Machine Learning and Deep Neural Network Methods,” Informatics in Medicine Unlocked, vol. 20, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[5] Alireza Samavat et al., “Deep Learning Model with Adaptive Regularization for EEG-Based Emotion Recognition Using Temporal and Frequency Features,” IEEE Access, vol. 10, pp. 24520–24527, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[6] Wei Tao et al., “EEG-Based Emotion Recognition via Channel-Wise Attention and Self-Attention,” IEEE Transactions on Affective Computing, vol. 14, no. 1, pp. 382-393, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[7] Syahrull Hi Fi Syam et al., “Comparing Common Average Referencing to Laplacian Referencing in Detecting Imagination and Intention of Movement for Brain Computer Interface,” MATEC Web of Conferences, 2017 International Conference on Emerging Electronic Solutions for IoT (ICEESI 2017), vol. 140, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[8] Vasilisa Mishuhina, and Xudong Jiang, “Feature Weighting and Regularization of Common Spatial Patterns in EEG-Based Motor Imagery BCI,” IEEE Signal Processing Letters, vol. 25, no. 6, pp. 783–787, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[9] Simon Geirnaert, Tom Francart, and Alexander Bertrand, “Fast EEG-Based Decoding of the Directional Focus of Auditory Attention Using Common Spatial Patterns,” IEEE Transactions on Biomedical Engineering, vol. 68, no. 5, pp. 1557–1568, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[10] P. Rithwik, V.K. Benzy, and A.P. Vinod, “High Accuracy Decoding of Motor Imagery Directions from EEG-Based Brain Computer Interface Using Filter Bank Spatially Regularised Common Spatial Pattern Method,” Biomedical Signal Processing and Control, vol. 72, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[11] Sebastian Michelmann et al., “Data-Driven Re-Referencing of Intracranial EEG Based on Independent Component Analysis (ICA),” Journal of Neuroscience Methods, vol. 307, pp. 125–137, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[12] Souvik Phadikar, Nidul Sinha, and Rajdeep Ghosh, “Automatic EEG Eyeblink Artifact Identification and Removal Technique Using Independent Component Analysis in Combination with Support Vector Machines and Denoising Autoencoder,” IET Signal Processing, vol. 14, no. 6, pp. 396–405, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[13] Fiorenzo Artoni, Arnaud Delorme, and Scott Makeig, “Applying Dimension Reduction to EEG Data by Principal Component Analysis Reduces the Quality of Its Subsequent Independent Component Decomposition,” NeuroImage, vol. 175, pp. 176–187, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[14] Qiang Gao et al., “EEG Based Emotion Recognition Using Fusion Feature Extraction Method,” Multimedia Tools and Applications, vol. 79, pp. 27057–27074, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[15] Peiyang Li et al., “EEG Based Emotion Recognition by Combining Functional Connectivity Network and Local Activations,” IEEE Transactions on Biomedical Engineering, vol. 66, no. 10, pp. 2869–2881, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[16] Xin Chai et al., “A Fast, Efficient Domain Adaptation Technique for Cross-Domain Electroencephalography (EEG)-Based Emotion Recognition,” Sensors, vol. 17, no. 5, pp. 1-21, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[17] Yong Zhang, Suhua Zhang, and Xiaomin Ji, “EEG-Based Classification of Emotions Using Empirical Mode Decomposition and Autoregressive Model,” Multimedia Tools and Applications, vol. 77, pp. 26697–26710, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[18] Vipin Gupta, Mayur Dahyabhai Chopda, and Ram Bilas Pachori, “Cross-Subject Emotion Recognition Using Flexible Analytic Wavelet Transform from EEG Signals,” IEEE Sensors Journal, vol. 19, no. 6, pp. 2266–2274, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[19] Arti Anuragi, Dilip Singh Sisodia, and Ram Bilas Pachori, “EEG-Based Cross-Subject Emotion Recognition Using Fourier-Bessel Series Expansion Based Empirical Wavelet Transform and NCA Feature Selection Method,” Information Sciences, vol. 610, pp. 508–524, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[20] Jingxia Chen, Dongmei Jiang, and Yanning Zhang, “A Common Spatial Pattern and Wavelet Packet Decomposition Combined Method for EEG-Based Emotion Recognition,” Journal of Advanced Computational Intelligence and Intelligent Informatics, vol. 23, no. 2, pp. 274–281, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[21] Kranti S. Kamble, and Joydeep Sengupta, “Ensemble Machine Learning-Based Affective Computing for Emotion Recognition Using Dual-Decomposed EEG Signals,” IEEE Sensors Journal, vol. 22, no. 3, pp. 2496–2507, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[22] S. Thejaswini et al., “EEG Based Emotion Recognition Using Wavelets and Neural Networks Classifier,” Cognitive Science and Artificial Intelligence, Springer Briefs in Applied Sciences and Technology, pp. 101–112, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[23] Heng Cui et al., “EEG-Based Emotion Recognition Using an End-to-End Regional-Asymmetric Convolutional Neural Network,” Knowledge-Based Systems, vol. 205, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[24] Stamos Katsigiannis, and Naeem Ramzan, “DREAMER: A Database for Emotion Recognition through EEG and ECG Signals from Wireless Low-Cost Off-the-Shelf Devices,” IEEE Journal of Biomedical & Health Informatics, vol. 22, no. 1, pp. 98–107, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[25] Wei-Long Zheng et al., “Emotionmeter: A Multimodal Framework for Recognizing Human Emotions,” IEEE Transactions on Cybernetics, vol. 49, no. 3, pp. 1110–1122, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[26] Tengfei Song et al., “MPED: A Multi-Modal Physiological Emotion Database for Discrete Emotion Recognition,” IEEE Access, vol. 7, pp. 12177–12191, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[27] Shalabh Kumar Mishra, Maneesha Gupta, and Dharmendra Kumar Upadhyay, “Active Realization of Fractional Order Butterworth Lowpass Filter Using DVCC,” Journal of King Saud University-Engineering Sciences, vol. 32, no. 2, pp. 158–165, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[28] Mi Li et al., “Emotion Recognition from Multichannel EEG Signals Using K-Nearest Neighbor Classification,” Technology and Health Care, vol. 26, no. S1, pp. 509–519, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[29] Wei-Long Zheng et al., “EEG-Based Emotion Classification Using Deep Belief Networks,” 2014 IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6, 2014.
[CrossRef] [Google Scholar] [Publisher Link]
[30] Adnan Mehmood Bhatti et al., “Emotion Recognition and Analysis in Response to Audio Music Using Brain Signals,” Computers in Human Behavior, vol. 65, pp. 267–275, 2016.
[CrossRef] [Google Scholar] [Publisher Link]