Key focus: Fundamentals of signal processing for machine learning. Speaker identification is taken as an example for introducing supervised learning concepts.
Signal Processing
A signal, mathematically a function, is a mechanism for conveying information. Audio, image, electrocardiograph (ECG) signal, radar signals, stock price movements, electrical current/voltages etc.., are some of the examples.
Signal processing is an engineering discipline that focuses on synthesizing, analyzing and modifying such signals. Some of the applications of signal processing are
● Converting one signal to another – filtering, decomposition, denoising
● Information extraction and interpretation – computer vision, speech recognition, Iris recognition, finger print recognition
● Error control and source coding – low density parity codes (LDPC), turbo coding, linear prediction coding, JPG, PNG
● Detection – SONAR, RADAR
Machine Learning (ML)
Machine learning is a science that deals with the development of algorithms that learn from data. According to Arthur Samuel (1959)[1] machine learning is a “Field of study that gives computers the ability to learn without being explicitly programmed”. Kevin Murphy, in his seminal book [2], defines machine learning as a collection of algorithms that automatically detect patterns in data that use the uncovered patterns to predict future data or other outcomes of interest.
Essentially, a machine learning algorithm may learn from data to
● learn from data to recognize patterns – example: recognizing text patterns in a set of spam emails
● classify data into different categories – example: classifying the emails into spam or non-spam emails.
● predict a future outcome – example: predicting whether the incoming email is spam or not
Machine learning algorithms are divided into three main types
● Supervised learning – a predictive learning approach where the goal is to learn from a labeled set of input-output pairs. The labeled set provides the training examples for further classification or prediction. In machine learning jargon, inputs are called ‘features’ and outputs are called ‘response variables’.
● Unsupervised learning – A kind of less well defined knowledge discovery process, the goal is to learn structured patterns in the data by separating them from pure unstructured noise
● Reinforced learning – is learning by interacting with an environment in order to make decision making tasks
Based on the discussion so far, we can start to recognize how the synergy between the fields of signal processing and machine learning can provide a new perspective to approach many problems.
Speaker identification – an application of ML algorithms in signal processing
Speaker identification (Figure 1) is the identification of a person from the analysis of voice characteristics. In this supervised classification application, a labeled training set of voice samples (from a set of speakers) are used in the learning process.
Voice samples/recordings cannot be used as such in the learning process. For further processing, it may require sampling, cleaning (removal of noise or invalid samples etc..,) or re-formatting the samples to suitable format. This step is called ‘data pre-processing‘.
Also, we may have to transform the data specific to the ML algorithm and the knowledge of the problem. To train the ML model recognize the patterns in the voice samples, feature extraction on voice samples is performed using signal processing. In this case, the features that are used to train the ML model are pitch and Mel-Frequency Cepstrum Coefficients (MFCC) [3] extracted from the voice samples.
Generally, the available dataset (set of input voice samples) is split into two sets: one set for training the model and the other set for testing needs (typically in 75%-25% ratio). The training set is used to train the ML model and the test set is used to evaluate the effectiveness and performance of the ML algorithm.
The training process should attempt to generalize the underlying relationship between the feature vectors (input to the supervised learning algorithm) and the class labels (supervised learner’s output). Cross-validation is one of the verification technique for evaluating the generalization ability of the ML model.
The training process should also avoid overfitting, which may cause poor generalization and erroneous classification in the execution phase. If the performance of the algorithm needs improvement, we need to go back and make changes to the previous steps. Metrics such as accuracy, recall, confusion matrix are typically used to evaluate the effectiveness and performance of the ML algorithm.
After the ML model is adequately trained to provide satisfying performance, we move on to the execution phase. In the execution phase, when an unlabeled instance of an voice sample is presented to the trained classifier, it identifies the person to which it belongs to.
Rate this article: 
(16 votes, average: 3.31 out of 5)
References
[1] Samuel, Arthur L. “Some Studies in Machine Learning Using the Game of Checkers,” IBM Journal of Research and Development 44:1.2 (1959): 210–229.↗
[2] Kevin P. Murphy, “Machine Learning – A Probabilistic Perspective”, ISBN 978-0262018029, The MIT Press, Cambridge, UK.↗
[3] P. M. Chauhan and N. P. Desai, “Mel Frequency Cepstral Coefficients (MFCC) based speaker identification in noisy environment using wiener filter,” 2014 International Conference on Green Computing Communication and Electrical Engineering (ICGCCEE), Coimbatore, 2014, pp. 1-5.↗
Articles in this series
| [1] Introduction to Signal Processing for Machine Learning |
| [2] Generating simulated dataset for regression problems - sklearn make_regression |
| [3] Hands-on: Basics of linear regression |
Books by the author
 Wireless Communication Systems in Matlab Second Edition(PDF) Note: There is a rating embedded within this post, please visit this post to rate it. |  Digital Modulations using Python (PDF ebook) Note: There is a rating embedded within this post, please visit this post to rate it. |  Digital Modulations using Matlab (PDF ebook) Note: There is a rating embedded within this post, please visit this post to rate it. |
| Hand-picked Best books on Communication Engineering Best books on Signal Processing |
||