ABSTRACT
Speech Emotion Recognition (SER) stands at the forefront of computer science, holding great promise and significance. Emotions, conveyed through speech tones, are pivotal in professions like surgery and military command, where emotional control is paramount. Yet, finding emotions in speech is a complex task, marked by the unique tonal and intonational variations among individuals. This research looks to decide the most precise method for classifying emotions in spoken language. Our journey traverses the realm of machine learning, with a comparative analysis of two robust classifiers: the Support Vector Machine (SVM) and the Multilayer Perceptron (MLP) Classifier. SVM excels in processing clean sound input but falters in the presence of noisy data due to its reliance on a single decision boundary. In contrast, the MLP Classifier, embedded within the domain of artificial neural networks, adeptly manages intricate time series data, making it a more adaptable and scalable solution for emotion recognition. Beyond its applications in speech analysis, SER finds increasing relevance in the realm of mental health. Continuous analysis of emotional states through speech aids in the early detection and monitoring of mental health conditions. This technology empowers individuals and professionals to raise awareness and offer prompt support. This article explores the dynamic interplay between machine learning and emotion recognition, highlighting the strengths and limitations of SVM and the MLP Classifier. It underscores SER's pivotal role, not only in various industries but also in the critical domain of mental health support, making the bridge between technology and emotions increasingly vital.
KEYWORDS : Neural Networks, Speech Emotion Recognition (SER), Tone and pitch, MLP- Classifier, RAVDESS, MFCC, Mel Spectrogram Frequency, Tonnetz, Decimal encoding, Accuracy, Audio parameters, Human-computer interaction.
INTRODUCTION
Speech Emotion Recognition (SER) has evolved into a central focus within the expansive realm of computer science, signifying its continuous evolution and increasing significance. It pertains to the recognition and interpretation of emotions conveyed through speech, a capability that holds pivotal importance in a diverse array of professional domains, ranging from surgery to military command. In these high-stakes arenas, the ability to master emotional regulation becomes an indispensable skill. Nevertheless, the endeavour of understanding and predicting emotions communicated through speech is by no means straightforward. The challenge lies in the distinctive tonal and intonational variations that define each individual's voice, making the task complex and multifaceted. Amid the vast spectrum of human emotions, which spans from joy and anger to neutrality, sorrow, and astonishment, this research embarks on an ambitious journey to navigate the intricate landscape of emotion recognition in speech. The central aim is to find the most precise method for classifying these nuanced emotional states within given speech samples. Fulfil this mission, our expedition takes us deep into the domain of machine learning, with a particular focus on the potential of the multilayer perceptron (MLP). Within the context of emotion classification, we engage in a comprehensive comparative analysis of two robust classifiers: the Support Vector Machine (SVM) and the MLP Classifier. Our aim is to dissect their unique strengths and limitations, offering valuable insights into their utility within the context of emotion recognition. The Support Vector Machine, renowned for its ability in handling clean sound input, excels in scenarios where the data is pristine and free from distortions. However, its predictive accuracy diminishes notably when confronted with noisy input. This limitation is rooted in the SVM's reliance on a single decision boundary, referred to as a 'plane,' which, by nature, is less adaptable to the intricate nuances present within speech patterns. While SVM stays robust in specific scenarios, it meets challenges when dealing with the complex, time-series- based nature of speech data. Our exploration into these two classifiers reveals an intriguing paradox. While SVM achieves commendable accuracy in its predictions, it does so at the cost of increased computational demands, making it a resource-intensive choice. In contrast, the MLP Classifier, running within the realm of artificial neural networks, shows a unique ability to navigate intricate time series data with finesse. Within the context of emotion recognition, the MLP Classifier appears as a more adaptable and scalable choice, poised to capture the subtleties and intricacies inherent in the expression of human emotions. In the vast tapestry of emotion recognition, our exploration extends beyond the boundaries of speech analysis. It stretches into the realm of mental health applications, where continuous analysis of emotional states through speech supplies invaluable insights for early detection and monitoring of mental health conditions. This technology empowers individuals and professionals to raise awareness and offer prompt support. Consequently, SER serves as a promising tool, not only across diverse industries but also in the critical arena of mental health support. This article embarks on a thorough investigation into the dynamic interplay between machine learning and the intricate landscape of human emotions. Our journey through emotion recognition illuminates the comparative strengths and limitations of SVM and the MLP Classifier, offering valuable insights into their applicability within the multifaceted realm of speech emotion recognition. In summary, the intersection of technology and emotions is not only a growing field of innovation but also a significant contributor to advancements in various professional domains and mental health support.
METHODOLOGY
In our pursuit of harnessing the power of Speech Emotion Recognition (SER) for mental health applications, a robust and systematic method is paramount. This section delineates the step-by-step process, tools, and procedures we employ to effectively integrate SER into the domain of mental health support.
A. Dataset Utilization
The foundation of any SER-based application lies in the quality and relevance of the dataset used. In this endeavour, we use the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS). This database features 24 actors, equally divided between male and female, and covers a diverse range of emotions, including sad, happy, neutral, angry, disgust, surprised, fearful, and calm expressions. Given our focus on speech-based emotion recognition, we train our model solely on audio data from this extensive dataset.
B. Feature Extraction
Unravel the emotional content embedded in speech, the first step involves the extraction of pertinent features. Our feature extraction process encompasses the following five crucial elements:
1. Mel Frequency Cepstral Coefficients (MFCC): MFCCs are employed to transform audio signals into a format amenable for analysis. This process involves the use of distinct hop lengths and HTK-style Mel frequencies, anchored by a reference point set at 1 kHz tone and 40 dB over the perceptual audible threshold. The MFCC supplies a Discrete Cosine Transform (DCT) of the natural logarithm of the short-term energy displayed on the Mel frequency scale, thereby serving as a foundational feature in our method.
2. Mel Spectrogram Frequency: The Mel scale, with its ability to relate clear pitch to actual estimated frequency, plays an indispensable role. The Mel scale empowers our features to capture the intricacies of human auditory perception, effectively highlighting minute pitch changes at lower frequencies, which aligns closely with human auditory capabilities.
3. Chroma Feature: The Chroma feature is instrumental in characterizing the tonal content of the audio signal in a condensed manner. It enables a more comprehensive understanding of the tonal and harmonic aspects of the emotional content in speech. A high-quality Chroma feature enhances the performance of elevated-level semantic analysis, such as chord recognition or harmonic similarity measurement, which are invaluable in our mental health application.
4. Tonnetz: The Tonnetz feature is a pitch space defined by the network of relationships between musical notes. This feature excels in capturing the harmony and melody aspects of audio data, making it a vital part in recognizing emotional nuances within speech. The Tonnetz feature effectively models the relative distances between pitch intervals, offering valuable insights into emotional expression.
C. Neural Network Architecture
The core of our SER-based mental health application lives in the design of a Multilayer Perceptron (MLP) Classifier. This neural network architecture is tailored to accommodate the specific requirements of our application:
1. Input Layer: Our MLP Classifier features an input layer designed to receive the extracted audio features, which are essential for discerning emotional cues embedded within the speech.
2. Hidden Layers: The hidden layers, in our case, are structured with (40, 80, 40) neurons. The choice of hidden layers is a critical element in the network's ability to process and analyse the intricate time series data present in emotional speech.
3. Activation Function: We employ the logistic activation function within the hidden layers, easing the network's ability to act upon the input data and perform critical processing tasks.
4. Output Layer: The output layer is the pinnacle of the network, responsible for deciphering the emotional content within the speech. It classifies and outputs the predicted emotion, a crucial element in our mental health application's decision-making process.
D. Training and Learning
The success of our SER-based mental health application hinges on the training phase. During training, the MLP Classifier learns the intricate correlations between the input audio features and the corresponding emotional states. Our method employs Backpropagation as the learning algorithm, a fundamental technique that drives the network to adjust model parameters, including weights and biases, to minimize the prediction error. The error minimization process is essential in refining the network's predictive accuracy and enhancing its ability to recognize emotions within speech.
E. Multi-Layer Perceptron Classifier (MLP Classifier)
Our SER-based mental health application leans on the Multi-Layer Perceptron Classifier, which uses an underlying Neural Network to perform classification. This integral part undergoes the following stages:
1. Initialization: The MLP Classifier is initialized by defining and configuring the required parameters, setting the stage for later steps.
2. Training: The Neural Network is trained using the extracted audio features and corresponding emotional labels. This phase empowers the MLP Classifier to learn the complex relationships between audio features and emotions.
3. Prediction: Post-training, the MLP Classifier is employed to predict emotional states in real-time, supplying a valuable tool for analysing and watching emotional well-being.
4. Accuracy Assessment: The predictions generated by the MLP Classifier are rigorously assessed to gauge the accuracy of the model, a critical step in confirming the efficacy of our SER-based mental health application. This comprehensive method underpins the seamless integration of SER technology into the domain of mental health. By following these rigorous steps, we aim to use the power of speech-based emotion recognition to enhance early detection, monitoring, and support for mental health conditions. Our method is designed to foster the constructive interaction between technology and emotional intelligence, bridging the gap between the digital realm and the well-being of individuals.
CONCLUSION
In the ever-evolving landscape of technology and human emotions, Speech Emotion Recognition (SER) appears as a compelling fusion of human expression and artificial intelligence. This journey has untraveled the potential for SER to bridge the divide between the digital and the emotional, offering promising applications that extend far beyond the confines of scientific exploration. SER, at its core, delves into the intricate melodies of human emotion, where the cadence of speech reveals the depths of our feelings. From the harmonious dance of joy to the thunderous rhythms of anger, the nuances within our voice form a mosaic of emotions. The path we've trodden, grounded in the Multilayer Perceptron (MLP) Classifier and fuelled by the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS), has been an odyssey of understanding and harnessing these emotional cues. Our method, marked by the precision of feature extraction and the sophistication of neural network design, illuminates the intricacies of emotion recognition. Backpropagation and the Multi-Layer Perceptron Classifier serve as the enablers of this technology, driving it toward the realm of practicality. As our journey concludes, the broader implications of SER become clear. Beyond the boundaries of research, this technology converges with the pragmatic arena of mental health. The continuous analysis of emotional states through speech opens doors to early detection and ongoing monitoring of mental well-being. SER empowers individuals and professionals, supplying a lifeline of emotional awareness in the complex landscape of mental health support.
In this confluence of technology and emotions, the line between the digital and the human blurs. SER, beyond its industrial applications, finds a profound purpose in mental health support. This constructive collaboration between technological prowess and empathy heralds a new era of innovative solutions, bridging the gap between the digital realm and our profound emotional experiences. In summary, our exploration reaffirms that with every step forward, we inch closer to a world where the eloquence of speech fortifies emotional well-being. Speech Emotion Recognition is not just a technological innovation; it's a testament to our evolving relationship with technology and emotions, offering a promising avenue for enhancing the human experience.
