Author: Mathuranathan

Mathuranathan Viswanathan, is an author @ gaussianwaves.com that has garnered worldwide readership. He is a masters in communication engineering and has 12 years of technical expertise in channel modeling and has worked in various technologies ranging from read channel, OFDM, MIMO, 3GPP PHY layer, Data Science & Machine learning.

Convolutional Codes – sliding window and shift register

Key focus : Convolutional codes: Error-correcting codes effective in noisy channels and burst error correction. Know about sliding window & shift registers and how they are used in a convolutional encoder.

Linear block codes

Linear block codes, like Hamming codes, are mainly employed in hard decision decoding, utilizing the inherent algebraic structure of the code based on finite field properties.

Hard decision decoding of these codes creates a model resembling a binary symmetric channel. This model includes the essential components: the binary modulator, waveform channel, and optimal binary detector.

The decoder’s task for these codes is to find the codeword with the shortest Hamming distance from the binary symmetric channel’s output. When designing effective linear block codes, the primary aim is to discover the code with the highest minimum distance, considering the specific values of \(n\) and \(k\), where \(n\) is the number of output bits and \(k\) is the number of input bits.

As we saw, linear block codes are easily described in terms of algebraic equations. Convolutional codes are another class of error-correcting codes that are described using graphs and trellises.

Convolutional codes

Convolutional codes are a type of error-correcting code used in digital communication and data storage systems to detect and correct errors that may occur during transmission or storage. They are particularly well-suited for channels that introduce burst errors, such as noisy communication channels or storage media with localized errors.

Unlike the linear blocks, where the message bits and parity bits are transmitted, in the convolutional coding scheme, we do not send the message bits but only send the parity bits. Which message bits participate in generation of the parity bits is determined by a sliding window.

Convolutional codes are defined by a set of shift registers and associated feedback connections. The fundamental principle involves encoding input data via a sliding bit window (Figure 1), utilizing bitwise operations to generate the encoded result. The encoded bits are the parity bits. The encoded bits are a linear combination of the current input bits and previous input bits in the shift registers.

Let’s see some definitions with respect to convolutional codes:

Rate: Convolutional codes are often described by their rate, which is the ratio of the number of input bits to the number of output bits (encoded bits). A common notation for convolutional codes is given as \(n,k,m\) where \(n\) is the number of output bits (equivalently the number of generator polynomials), \(k\) is the number of input bits (equivalently the number of bits shifted by the shifter register at a time), and \(m\) is the number of shift registers. The code rate is given by \(R_c = k/n\).

Constraint Length: The constraint length of a convolutional code is the number of shift registers in the encoder. It determines the memory of the code and affects its error-correction capabilities. Larger the constraint length, greater is the redundancy and it may provide better error correction capabilities.

Generator Polynomials: The feedback connections and the bitwise operations performed on the shift registers are defined by generator polynomials. These polynomials determine the encoding process and the redundancy (parity bits) introduced into the data. Code rate is simply calculated as \(R_c = k/n\), where \(n\) is the number of generator polynomials in the encoder structure and \(k\) is the number of input bits at a time. More generator polynomials may improve the error correction capability of the code, it effectively decreases the code rate (more overhead).

Encoding: Convolutional encoding involves shifting the input bits through the shift registers and applying bitwise operations based on the generator polynomials. The encoder bits are just parity bits. In the convolutional encoding scheme, we only transmit the parity bits and not the message itself. If we use multiple generator polynomials, we interleave the output bits of each generate and serialize them when transmitting.

Decoding: At the receiver end, a decoding algorithm (such as the Viterbi algorithm) is used to analyze the received bits and recover the original data. The decoding process tries to find the most likely sequence of input bits given the received encoded bits and the code’s properties. We will come to this process in a later post.

Figure 1 shows a \(n=2, k=1,m=3\) convolutional encoder. The symbol D represents the delay as the input bits are processed through the shift registers. The generator polynomial is specified as \(g_0 = [1,1,1]\) and \(g_1=[1,0,1]\). The generator polynomials connect the appropriate cells in the shift register and performs XOR (modulo-2) operation on the bits stored in the shift register cells. This particular convolutional code generates 2 output bits for every input bit processed. The two output bit streams are designated as \(c_0\) and \(c_1\).

Since there are two generator polynomials, the code rate is \(R_c = 1/2\)

Given an input message \(x[n]\) the equation for generating the parity bit streams is given by

\[\begin{align} c_0[n] &= x[n] + x[n − 1] + x[n − 2] & \quad \quad (modulo-2) \\ c_1[n] &= x[n] + x[n − 2] & \quad \quad (modulo-2) \end{align}\]
n=2, k=1, m=3 Convolutional encoder
Figure 1: \(n=2, k=1, m=3\) Convolutional encoder

Figure 2 shows the sliding window representation of the shift register arrangement shown in Figure 1. Figure 2 shows a 3-bit (\(m\)) sliding window shift register that advances 1-bit (\(k\)) at a time. The number of output bits (\(n\)) depend on how these bits are processes using the generator polynomials.

Figure 2: Generating convolutional codes using sliding window operation and shift register

Sample encoding operation

Figure 2 visualizes the encoding process for a \(n=2, k=1, m=3\) convolutional code. The generator polynomial is specified as \(g_0 =[1,1,1]\) and \(g_1 = [1, 0, 1]\). The generator polynomials connect the appropriate cells in the shift register and performs XOR (modulo-2) operation on the bits in the shift register cells.

These two generator polynomials generate the corresponding output bits \(c_0\) and \(c_1\). The input message to encode is \([1,0,1,1,0,0]\). The output generated by this particular convolutional encoder is \([11,10,00,01,01,11]\).

Figure 2: Encoding process for convolutional codes

Encoder state machine

As illustrated in Figure 2, the constraint length \(m\) determines the number of memory cells in the shift register structure. The input bit and the \(m-1\) bits of current state determine the encoder state on the next clock cycle.

Because the encoder’s output relies on both its input and current state, state diagram is a more concise illustration than the trellis. Essentially, the state diagram represents a graphical depiction of the encoder’s potential states and the potential transition between these states.

From the state diagram, we can immediately recognize the fact : convolutional encoder is indeed a Markov chain.

To know more about Markov chain refer here. Implementing a Markov chain in python is discussed here.

Figure 4: Encoder state machine for convolutional code shown in Figure 1

Figure 4 depicts the state diagram for the convolutional encoder described in Figure 1. For a convolutional encoder with constraint length m, there will be \(2^{m-1}\) possible states. For this specific case, there are \(2^{3-1} = 4\) possible states.

The current state is represented as \(x[n-1]\) and \(x[n-2]\) (corresponding to the values of the memory cells \(D_1\) and \(D_2\). The output of the encoder is determined by the current input bit \(x[n]\) and the current state. Since there are two generator polynomials \(g_0\) and \(g_1\), they generate two outputs \(c_0[n]\) and \(c_1[n]\) respectively. For an input bit \(x[n]\), the encoder transits from one state to the next state, it emits \(c_0[n]\) and \(c_1[n]\) as output. This is depicted as \(x[n]/c_0[n]c_1[n]\).

Figure 5, shows the encoding process when the input message is \([1, 0, 1, 1, 0, 0]\). The encoder transits through various states and generates the output \([11, 10, 00, 01, 01, 11]\).

Figure 5: Encoder state machine processing the input message [1,0,1,1,0,0]

Orthogonality of OFDM

OFDM, known as Orthogonal Frequency Division Multiplexing, is a digital modulation technique that divides a wideband signal into several narrowband signals. By doing so, it elongates the symbol duration of each narrowband signal compared to the original wideband signal, effectively minimizing the impact of time dispersion caused by multipath delay spread.

OFDM is categorized as a form of multicarrier modulation (MCM), where multiple user symbols are transmitted simultaneously through distinct subcarriers having overlapping frequency bands, ensuring they remain orthogonal to each other.

OFDM implements the same number of channels as the traditional Frequency Division Multiplexing (FDM). Since the channels (subcarriers) are arranged in overlapping manner, OFDM significantly reduces the bandwidth requirement.

OFDM equation

Consider an OFDM system that transmits a user symbol stream \(s_i\) (rate \(R_u\)) over a set of \(N\) subcarriers. Therefore, the symbol rate of each subcarrier is \(R_s = \frac{R_u}{N}\) and the symbol duration is \(T_s = \frac{N}{R_u}\).

The incoming symbol stream is split into \(N\) symbols streams and each of the symbol stream is multiplied by a function \(\Phi_k\) taken from a family of orthonormal functions \(\Phi_k, k \in \left\{0,1, \cdots, N-1 \right\}\)

In OFDM, these orthogonormal functions are complex exponentials

\[\Phi_k (t) = \begin{cases} e^{j 2 \pi f_k t}, & \quad for \; t \in \left[ 0, T_s\right] \\ 0, & \quad otherwise \end{cases} \quad \quad \quad (1) \]

For simplicity lets assume BPSK modulation for the user symbol \(s_i \in \left\{-1,1 \right\} \) and \(g_i\) is the individual gain of each subchannels. The OFDM symbol is formed by multiplexing the symbols on each subchannels and combining them.

\[S (t) =\frac{1}{N} \sum_{k=0}^{N-1} s_k \cdot g_k \cdot \Phi_k(t) \quad \quad \quad (2)\]

The individual subcarriers are

\[s_n (t) = s_k \cdot g_k \cdot e^{j 2 \pi f_k t} \quad \quad \quad (3)\]

For a consecutive stream of input symbols \(m = 0,1, \cdots\) the OFDM equation is given by

\[S(t) = \sum_{m = 0 }^{ \infty} \left[\frac{1}{N} \sum_{k=0}^{N-1} s_{k,m} \cdot g_{k,m} \cdot \Phi_{k}(t – m T_s) \right] \quad \quad \quad (4)\]

With \(g_{k,m} = 1\), the OFDM equation is given by

\[S(t) = \sum_{m = 0 }^{ \infty} \left[\frac{1}{N} \sum_{k=0}^{N-1} s_{k,m} \cdot e^{j 2 \pi f_k \left(t – m T_s \right)} \right] \quad \quad \quad (5)\]

Orthogonality

The functions \(\Phi\) by which the symbols on the subcarriers are multiplied are orthonormal over the symbol period \(T_s\). That is

\[ \left< \Phi_p (t), \Phi_q (t) \right> = \frac{1}{T_s} \int_{0}^{Ts} \Phi_p (t) \cdot \Phi^*_q (t) dt = \delta_{p,q} \quad \quad \quad (6) \]

where, \(\delta_{p,q}\) is the Kronecker delta given by

\[\delta_{p,q} = \begin{cases} 1, & \quad p=q \\ 0, & \quad otherwise \end{cases} \]

The right hand side of equation (5) will be equal to 0 (satisfying orthogonality) if and only if \(2 \pi \left(f_p−f_q \right)T_s=2 \pi k\) where \(k\) is a non-zero integer. This implies that the distance between the two subcarriers, for them to be orthogonal, must be

\[\Delta f = f_p – f_q = \frac{k}{T_s} \quad \quad (7)\]

Hence, the smallest distance between two subcarriers, for them to be orthogonal, must be

\[ \Delta f = \frac{1}{T_s} \quad \quad (8)\]

This implies that each subcarrier frequency experiences \(k\) additional full cycles per symbol period compared to the previous carrier. For example, in Figure 1 that plots the real and imaginary parts of three OFDM subcarriers (with \(k=1\)), each successive subcarrier contain additional full cycle per symbol period compared to the previous carrier.

Figure 1: Three orthogonal subcarriers of OFDM

With \(N\) subcarriers, the total bandwidth occupied by one OFDM symbol will be \(B \approx N \cdot \Delta f \; (Hz) \)

Figure 2: OFDM spectrum illustrating 12 subcarriers

Benefits of orthogonality

Orthogonality ensures that each subcarrier’s frequency is precisely spaced and aligned with the others. This property prevents interference between subcarriers, even in a multipath channel, which greatly improves the system’s robustness against fading and other channel impairments.

The orthogonality property allows subcarriers to be placed close together without causing mutual interference. As a result, OFDM can efficiently utilize the available spectrum, enabling high data rates and maximizing spectral efficiency, making it ideal for high-speed data transmission in wireless communication systems.

Reference

[1] Chakravarthy, A. S. Nunez, and J. P. Stephens, “TDCS, OFDM, and MC-CDMA: a brief tutorial,” IEEE Radio Commun., vol. 43, pp. 11-16, Sept. 2005.

5G standardization – 3GPP and ITU

5G standardization: Collaborative effort between 3GPP and ITU, ensuring global interoperability and compatibility for next-gen mobile networks.

ITU

ITU stands for the International Telecommunication Union. It is a specialized agency of the United Nations responsible for the development and coordination of international telecommunications standards and regulations. The ITU plays a crucial role in ensuring global interoperability and compatibility of telecommunication networks and services.

One of the key contributions of the ITU is the development of international standards for various aspects of telecommunications, including radio communication, satellite systems, network protocols, and multimedia coding. These standards enable different countries and regions to establish compatible communication systems, ensuring seamless connectivity and effective global communication.

3GPP

3GPP stands for the 3rd Generation Partnership Project. It is a collaboration between several telecommunications standards organizations, including the European Telecommunications Standards Institute (ETSI), the Alliance for Telecommunications Industry Solutions (ATIS), the China Communications Standards Association (CCSA), the Association of Radio Industries and Businesses (ARIB) in Japan, and the Telecommunications Standards Development Society, India (TSDSI). Through their respective regional standards organizations, over 600 companies have become members of 3GPP.

The primary goal of 3GPP is to develop and maintain standards for mobile communication systems. It has played a crucial role in the development of various generations of mobile networks, including 3G (UMTS – Universal Mobile Telecommunications Service), 4G (LTE – Long Term Evolution), and 5G (NR – New Radio). 3GPP specifications define the technical specifications, protocols, and interfaces that enable interoperability and compatibility among different vendors’ network equipment and devices.

3GPP is not a part of the ITU, but it works closely with the ITU to ensure global interoperability and compatibility of mobile networks.

The evolution of IMT standards

Standardization: 3GPP is responsible for developing technical specifications and standards for mobile communication systems, including 2G (GSM), 3G (UMTS), 4G (LTE), and 5G (NR). These standards address various aspects such as radio access networks, core networks, protocols, interfaces, and services. The ITU recognizes and considers these standards for potential adoption as international standards.

ITU Recognition: Once 3GPP completes the development of a new standard or specification, it submits the documentation to the ITU for consideration as an international standard. The ITU reviews 3GPP’s work and may incorporate it into its global standards portfolio.

IMT standards: The ITU has defined frameworks for the International Mobile Telecommunications (IMT) systems. Different IMT standards and its corresponding 3GPP releases are shown below:

Table 1: ITU standards and 3GPP releases

3GPP’s standards, particularly for 3G (UMTS), 4G (LTE), 5G (NR) align with the ITU’s IMT-2000, IMT-Advanced and IMT-2020 frameworks respectively. This ensures that 3GPP-developed technologies conform to the ITU’s global standards for mobile communications.

Collaboration and Cooperation: The ITU and 3GPP collaborate closely to ensure compatibility and harmonization between the ITU’s global standards and the technical specifications developed by 3GPP. They work together to address technical challenges, promote interoperability, and facilitate the deployment of mobile networks worldwide.

We will explore more about IMT-2020 in the coming article.

Bandwidth Part (BWP) in 5G NR

Key focus: Bandwidth Part (BWP): Allocates segments of spectrum for flexible resource allocation in 5G NR networks, enhancing efficiency and adaptability. Know the difference between bandwidth part and transmission bandwidth

Introduction

The 3rd Generation Partnership Project (3GPP), in its Release 15, specified the new radio-access technology called 5G New Radio (5G NR). The 5G NR continues to evolve in the subsequent releases that addresses performance improvements and new use cases.

In the release 15, new features for 5G NR were introduced so as to support a varied range of devices, services and deployments. Once such new basic feature in 5G NR is the Bandwidth Part (BWP).

5G NR Frequency ranges

3GPP defines the frequency ranges for 5G NR operation, as specified in Table 1 (illustrated in Figure 1).

Table 1: Definition of frequency ranges as given in TS 38.104 [1]
Figure 1: 5G NR frequency ranges

Operating bands

Within each Frequency Range (FR), the 3GPP defines operational bands. 3GPP 5G NR defines 60 operating bands in FR1 and 7 bands in FR2 (refer section 5.2 in [1]). These operating bands represent specific frequency ranges that come with their own unique radio frequency requirements (for example: some of these bands are designated for use in Frequency Division Duplexing (FDD), Time Division Duplexing (TDD), Supplemental Up Link (SUL), Supplemental Down Link (SDL) communication). The width of these operating bands can vary significantly, ranging from just a few megahertz to several gigahertz. Different mobile network operators might be allocated (typically by respective governments through spectrum auction) different portions of the available spectrum within a given operational band.

To cater to the diverse spectrum allocation scenarios while keeping implementation complexities in check, the New Radio (NR) technology supports a wide range of channel bandwidths (a.k.a Base station (BS) channel bandwidth) spanning from 5 MHz to 2 GHz. In this context, “channel bandwidth” refers to the width of an NR carrier, the fundamental component of the 5G system.

Channel bandwidth

The base station’s channel bandwidth enables the use of a single NR RF carrier in either the uplink or downlink. The user equipment (UE) may support different channel bandwidth than the BS.

The UE connects to the BS. The UE receives information about the cell’s channel bandwidth from the network. The network also provides the UE with details regarding the location and extent of a Bandwidth Part (BWP). The core concept is that a User Equipment (UE) can employ a wide-bandwidth when a significantly large data is required to be transmitted/received, but operate with a limited bandwidth during other periods of limited activity.

Bandwidth Part (BWP)

As we know, 5G NR defines scalable Orthogonal Frequency Division Multiplexing (OFDM) using numerology \(\mu = \left\{ 0,1,2,3,4 \right\}\) that defines the subcarrier spacings (SCS) as \( \Delta f = 2^{\mu} \times 15 \; kHz\). A resource element is the smallest time-frequency resource over one subcarrier of a single OFDM symbol. A resource block (RB) is a block of \( N_{sc}^{RB} = 12 \) subcarriers over which the transmissions are scheduled (Figure 2).

Figure 2: Resource element, resource block and resource grid in 5G NR

In a broad sense, a BWP can be described as a collection of contiguous Resource Blocks (RBs) configured within a given channel bandwidth. The BWP provides a flexible mechanism for adapting to varied bandwidth ranges in order to aid in reduction of power consumption.

Another reason is to cater to devices with varying bandwidth capabilities by setting up different BWPs for each device. A Base Station (BS) might have the ability to accommodate an exceptionally broad channel bandwidth, even though certain User Equipment (UEs) may not have the capability to handle it. The concept of Bandwidth Part (BWP) offers a way to dynamically allocate radio resources, ensuring that a UE’s signals are limited to a portion of the BS’s channel bandwidth that aligns with the UE’s capabilities.

Point A, illustrated in Figure 3, functions as a standardized reference across all resource grids within the frequency. It acts as the central marker for subcarrier 0 within a common resource block 0 of the lowest resource grid and it can reside outside the carrier’s bandwidth. Each individual resource grid (RG) adheres to a distinct 5G NR numerology and commences at a frequency offset from point A.

As shown in Figure 3, the bandwidth part consists of contiguous set of RBs for a particular numerology on a given carrier and starts at an offset RB_{start} from the reference Point A.

Figure 3: Spectrum configuration in 5G NR illustrating channel bandwidth and bandwidth part (BWP)

Transmission bandwidth

5G NR defines another concept called transmission bandwidth that is closely related with bandwidth part. While BWP refers to a portion of the overall channel bandwidth that is allocated for specific communication purposes, transmission bandwidth refers to the range of frequencies that are used to transmit signals. It encompasses the total frequency range used for communication, including both the data-carrying portion and any guard bands or additional spectrum used for various purposes.

In short, the BWP refers to a specific allocated portion of the transmission bandwidth, while the transmission bandwidth encompasses the entire range of frequencies used for communication.

Figure 4: Transmission bandwidth and Channel bandwidth in 5G NR
\[\]

References

[1] 3GPP Specification TS 38.104 (version 17.7.0 Release 17) – 5G NR Base Station (BS) radio transmission and reception

Machine Learning Basics: A Comprehensive Introduction for Beginners

Key focus: machine learning, introduction, basics, beginners, algorithms, applications, concepts

Introduction

Machine learning has emerged as a groundbreaking technology that is transforming industries and reshaping our interaction with technology. From personalized recommendations to autonomous vehicles, machine learning algorithms play a pivotal role in these advancements. If you’re new to the field, this comprehensive beginner’s guide will provide you with a solid introduction to machine learning, covering its fundamental concepts, practical applications, and key techniques.

Understanding Machine Learning:

Machine learning is a subset of artificial intelligence (AI) focused on developing algorithms and models that can learn from data and make predictions or decisions without explicit programming. Instead of relying on fixed instructions, these algorithms extract patterns and insights from available data, enabling them to generalize and make accurate predictions on unseen examples.

The core machine learning algorithms are categorized into three types:

  • Supervised learning
  • Unsupervised learning
  • Reinforcement learning

Supervised Learning:

Supervised learning is a crucial branch of machine learning. In this approach, algorithms are trained on labeled datasets, where each example consists of input features and corresponding target labels. By learning from these labeled examples, algorithms can map inputs to correct outputs, identifying underlying patterns. Linear regression, decision trees, and support vector machines are common supervised learning algorithms.

Unsupervised Learning:

Unsupervised learning tackles unlabeled data. Its goal is to discover hidden patterns, structures, or relationships without prior knowledge of labels. Clustering and dimensionality reduction are prominent techniques within unsupervised learning. Clustering algorithms group similar data points, while dimensionality reduction methods aim to reduce feature dimensions while retaining relevant information.

Reinforcement Learning:

Reinforcement learning takes inspiration from how humans learn through trial and error. In this learning paradigm, an agent interacts with an environment and learns to maximize a reward signal by taking appropriate actions. Through repeated interactions, the agent explores the environment, receives feedback, and adjusts its actions to optimize its performance. Reinforcement learning has been successfully applied in areas such as robotics, gaming, and autonomous systems.

Key Steps in Machine Learning:

  1. Data Collection: Machine learning relies on quality data. Gathering relevant and representative data is a crucial initial step. It can come from various sources, including structured databases, APIs, or unstructured text and images.
  2. Data Preprocessing: Raw data often contains noise, missing values, or inconsistencies. Data preprocessing involves cleaning, transforming, and normalizing the data to ensure it is suitable for analysis and model training.
  3. Feature Engineering: Feature engineering involves selecting, extracting, or creating meaningful features from the available data. Good features can significantly impact the performance of a machine learning model.
  4. Model Training: This step involves feeding the prepared data into a machine learning algorithm to create a model. The algorithm learns from the data and adjusts its internal parameters to make accurate predictions or decisions.
  5. Model Evaluation: Evaluating the performance of a trained model is essential to assess its accuracy and generalization capabilities. Various metrics, such as accuracy, precision, recall, and F1 score, are used to measure the model’s performance.
  6. Model Deployment and Monitoring: Once the model is deemed satisfactory, it can be deployed in real-world applications. Continuous monitoring is crucial to ensure the model’s performance remains optimal and to address any issues that may arise.

Business use cases:

Businesses are increasingly leveraging machine learning to gain a competitive edge, improve operational efficiency, and enhance decision-making processes. Here are some common ways in which businesses are using machine learning:

  • Customer Insights and Personalization: Machine learning enables businesses to analyze customer data, such as purchase history, browsing behavior, and demographic information, to gain valuable insights. This information can be used to personalize marketing campaigns, recommend relevant products or services, and improve customer experiences.
  • Fraud Detection and Risk Management: Machine learning algorithms can identify patterns and anomalies in large volumes of transactional data, helping businesses detect fraudulent activities and mitigate risks. These algorithms learn from historical data to spot fraudulent patterns and predict potential risks, enabling proactive measures to safeguard businesses and their customers.
  • Demand Forecasting and Inventory Management: By analyzing historical sales data, market trends, and external factors, machine learning algorithms can predict future demand for products or services. This helps businesses optimize inventory levels, minimize stock-outs, reduce costs, and improve overall supply chain management.
  • Predictive Maintenance: Machine learning models can analyze sensor data from machinery and equipment to detect patterns indicating potential failures or maintenance needs. By identifying issues before they occur, businesses can schedule maintenance proactively, minimize downtime, and optimize equipment performance.
  • Natural Language Processing (NLP) for Customer Support: NLP techniques powered by machine learning are employed in chatbots and virtual assistants to automate customer support processes. These systems can understand and respond to customer queries, provide relevant information, and assist with common issues, improving response times and enhancing customer satisfaction.
  • Sentiment Analysis and Social Media Monitoring: Machine learning algorithms can analyze social media data and other online sources to gauge public sentiment and monitor brand reputation. This information helps businesses understand customer opinions, identify emerging trends, and respond effectively to customer feedback.
  • Supply Chain Optimization: Machine learning algorithms optimize supply chain operations by analyzing data related to logistics, transportation, and inventory management. These models can identify bottlenecks, streamline routes, optimize scheduling, and reduce costs, ultimately improving the overall efficiency of the supply chain.
  • Credit Scoring and Risk Assessment: Financial institutions employ machine learning algorithms to assess creditworthiness, predict default probabilities, and automate the loan approval process. By analyzing a range of variables, such as credit history, income, and demographics, these algorithms provide more accurate risk assessments and streamline lending processes.
  • Image and Speech Recognition: Machine learning models have advanced image and speech recognition capabilities. Businesses can leverage these technologies for various applications, such as facial recognition for security purposes, automatic image tagging, voice-controlled virtual assistants, and automated document analysis.
  • Data Analytics and Business Intelligence: Machine learning algorithms assist in analyzing large volumes of data to extract insights, identify patterns, and make data-driven decisions. By leveraging machine learning techniques, businesses can uncover hidden trends, gain a deeper understanding of their operations, and drive informed strategies.

These are just a few examples of how businesses are utilizing machine learning to improve their operations and decision-making processes. As machine learning continues to evolve, its applications across various industries and business functions are expected to expand, unlocking even greater opportunities for organizations.

Products/Services using machine learning

Machine learning has been integrated into a wide range of products and services across various industries. Here is a list of products that utilize machine learning:

  • Virtual Assistants: Virtual assistants like Amazon Alexa, Google Assistant, and Apple Siri use machine learning to understand and respond to user queries, perform tasks, and provide personalized recommendations.
  • Recommendation Systems: Platforms such as Netflix, Amazon, and Spotify leverage machine learning to analyze user preferences and behavior, providing personalized recommendations for movies, products, and music.
  • Fraud Detection Systems: Financial institutions and online payment processors employ machine learning algorithms to detect and prevent fraudulent activities by analyzing patterns and anomalies in transactions and user behavior.
  • Autonomous Vehicles: Self-driving cars and autonomous vehicles rely on machine learning algorithms to perceive and interpret the environment, make real-time decisions, and navigate safely on the roads.
  • Image and Speech Recognition: Products like Google Photos, Facebook’s automatic photo tagging, and voice assistants utilize machine learning algorithms for image and speech recognition tasks, enabling features such as automatic tagging and voice-controlled interactions.
  • Language Translation: Machine learning plays a significant role in language translation tools like Google Translate and Microsoft Translator, enabling accurate and automated translation between different languages.
  • Social Media News Feed Ranking: Social media platforms like Facebook, Twitter, and Instagram employ machine learning algorithms to rank and personalize users’ news feeds, showing relevant content based on their interests and preferences.
  • Customer Service Chatbots: Many companies use machine learning-powered chatbots to provide automated customer support, answer common queries, and assist with basic tasks without the need for human intervention.
  • Email Filtering: Email service providers such as Gmail utilize machine learning algorithms to automatically filter and categorize incoming emails, separating spam from legitimate messages and prioritizing important emails.
  • Medical Diagnosis Systems: Machine learning is applied in medical diagnosis systems to analyze patient data, medical images, and electronic health records, aiding in accurate disease diagnosis and treatment planning.
  • Smart Home Devices: Smart home devices like smart thermostats, security systems, and voice-controlled assistants incorporate machine learning to learn user preferences, automate tasks, and optimize energy consumption.
  • E-commerce Product Search and Recommendations: E-commerce platforms like Amazon and eBay employ machine learning to enhance product search capabilities, provide personalized recommendations, and optimize product listings.
  • Predictive Maintenance Systems: Industrial equipment and machinery are monitored using machine learning algorithms to predict maintenance needs, detect anomalies, and minimize downtime through proactive maintenance.
  • Financial Trading Systems: Machine learning algorithms are utilized in financial trading systems to analyze market data, identify patterns, and make automated trading decisions.
  • Online Advertising: Platforms such as Google Ads and Facebook Ads leverage machine learning to optimize ad targeting, personalize advertisements, and improve campaign performance.

These are just a few examples of the many products and services that incorporate machine learning to provide enhanced functionalities, intelligent automation, and personalized experiences across various industries.

Implementing Markov Chain in Python

Keywords: Markov Chain, Python, probability, data analysis, data science

Markov Chain

Markov chain is a probabilistic models that describe a sequence of observations whose occurrence are statistically dependent only on the previous ones. This article is about implementing Markov chain in Python

Markov chain is described in one of the earlier posts. For better understanding of the concept, review the post before proceeding further.

We will model a car’s behavior using the same transition matrix and starting probabilities described in the earlier post for modeling the corresponding Markov chain model (refer Figure 1). The matrix defines the probabilities of transitioning between different states, including accelerating, maintaining a constant speed, idling, and braking.

Figure 1: Modeling a car’s behavior using Markov chain model

The starting probabilities indicate that the car starts in the break state with probability 1, which means it is already stopped and not moving.

Python implementation

Here’s the sample code in Python that implements the above model:

import random

# Define a transition matrix for the Markov chain
transition_matrix = {
    'accelerate': {'accelerate': 0.3, 'constant speed': 0.2, 'idling': 0 , 'break': 0.5 },
    'constant speed': {'accelerate': 0.1, 'constant speed': 0.4, 'idling': 0 , 'break': 0.5 },
    'idling': {'accelerate': 0.8, 'constant speed': 0, 'idling': 0.2 , 'break': 0 },
    'break': {'accelerate': 0.4, 'constant speed': 0.05, 'idling': 0.5 , 'break': 0.05 },
}

# Define starting probabilities for each state
starting_probabilities = {'accelerate': 0, 'constant speed': 0, 'idling': 0, 'break': 1}

# Choose the starting state randomly based on the starting probabilities
current_state = random.choices(
    population=list(starting_probabilities.keys()),
    weights=list(starting_probabilities.values())
)[0]

# Generate a sequence of states using the transition matrix
num_iterations = 10
for i in range(num_iterations):
    print(current_state)
    next_state = random.choices(
        population=list(transition_matrix[current_state].keys()),
        weights=list(transition_matrix[current_state].values())
    )[0]
    current_state = next_state

In this example, we use the random.choices() function to choose the starting state randomly based on the starting probabilities. We then generate a sequence of 10 states using the transition matrix, and print out the sequence of states as they are generated. A sample output of the program is given below.

>>> exec(open('markov_chain.py').read()) #Python 3 syntax
break
idling
accelerate
break
accelerate
break
accelerate
constant speed
break
accelerate

The Most Important Topics to Learn in Machine Learning

Keywords: machine learning, topics, probability, statistics, linear algebra, data preprocessing, supervised learning, unsupervised learning, deep learning, reinforcement learning, model evaluation, cross-validation, hyperparameter tuning.

Why the buzz ?

Machine learning has been generating a lot of buzz in recent years due to its ability to automate tasks that were previously thought to be impossible or required human-level intelligence. Here are some reasons why there is so much buzz in machine learning:

  1. Improved Data Processing: Machine learning algorithms can process vast amounts of data quickly and accurately. With the advent of big data, there is now more data available than ever before, and machine learning algorithms can analyze this data to extract meaningful insights.
  2. Automation: Machine learning can automate tasks that were previously done by humans, such as image recognition, natural language processing, and even decision making. This has the potential to increase efficiency and reduce costs in many industries.
  3. Personalization: Machine learning can be used to personalize experiences for users. For example, recommendation systems can use machine learning algorithms to suggest products or services that are relevant to a user’s interests.
  4. Predictive Analytics: Machine learning can be used to make predictions about future events based on historical data. This is particularly useful in industries like finance, healthcare, and marketing.
  5. Advancements in Technology: Advancements in technology have made it easier to collect and store data, which has made it possible to train more complex machine learning models. Additionally, the availability of cloud computing has made it easier for companies to implement machine learning solutions.

Overall, the buzz in machine learning is due to its ability to automate tasks, process vast amounts of data, and make predictions about future events. As machine learning continues to evolve, it has the potential to transform many industries and change the way we live and work.

The most important topics to learn in machine learning

There are several important topics to learn in machine learning that are crucial for building effective machine learning models. Here are some of the most important topics to learn:

  1. Probability and Statistics: Probability and statistics are the foundation of machine learning. It is important to have a solid understanding of concepts like probability distributions, statistical inference, hypothesis testing, and Bayesian methods.
  2. Linear Algebra: Linear algebra is used extensively in machine learning algorithms, especially in deep learning. Topics like matrices, vectors, eigenvectors, and eigenvalues are important to understand.
  3. Data Preprocessing: Data preprocessing is the process of cleaning and transforming raw data into a format that can be used by machine learning algorithms. It includes tasks like feature scaling, feature selection, data normalization, and data augmentation.
  4. Supervised Learning: Supervised learning is a type of machine learning where the model learns from labeled data to make predictions or classifications on new, unseen data. This includes topics like regression, classification, decision trees, and support vector machines.
  5. Unsupervised Learning: Unsupervised learning is a type of machine learning where the model learns from unlabeled data to discover patterns and relationships in the data. This includes topics like clustering, dimensionality reduction, and anomaly detection.
  6. Deep Learning: Deep learning is a subset of machine learning that involves training artificial neural networks with multiple layers. It is used for tasks like image recognition, natural language processing, and speech recognition.
  7. Reinforcement Learning: Reinforcement learning is a type of machine learning where an agent learns to take actions in an environment to maximize a reward signal. It is used for tasks like game playing, robotics, and autonomous driving.
  8. Model Evaluation and Selection: Model evaluation and selection is the process of selecting the best machine learning model for a given task. It includes topics like cross-validation, bias-variance tradeoff, and hyperparameter tuning.

Overall, these are some of the most important topics to learn in machine learning. However, it is important to note that the field of machine learning is constantly evolving, and there may be new topics and techniques to learn in the future.

Spectrogram Analysis using Python

Keywords: Spectrogram, signal processing, time-frequency analysis, speech recognition, music analysis, frequency domain, time domain, python

Introduction

A spectrogram is a visual representation of the frequency content of a signal over time. Spectrograms are widely used in signal processing applications to analyze and visualize time-varying signals, such as speech and audio signals. In this article, we will explore the concept of spectrograms, how they are generated, and their applications in signal processing.

What is a Spectrogram?

A spectrogram is a two-dimensional representation of the frequency content of a signal over time. The x-axis of a spectrogram represents time, while the y-axis represents frequency. The color or intensity of each point in the spectrogram represents the magnitude of the frequency content at that time and frequency.

How are Spectrograms Generated?

Spectrograms are typically generated using a mathematical operation called the short-time Fourier transform (STFT).

The STFT is a variation of the Fourier transform that computes the frequency content of a signal in short, overlapping time windows. The resulting frequency content is then plotted over time to generate the spectrogram.

\[ STFT(x(t), f, \tau)(\omega, \tau) = \int_{-\infty}^{\infty} x(t)w(t – \tau)e^{-i\omega t}dt \]

where \(STFT(x(t),f \tau)\) is the STFT of the signal \(x(t)\) with respect to frequency f and time shift \(\tau\), \(\omega\) is the frequency variable, \(w(t-\tau)\) is the window function, and \(e^{-j\omega t}\) is the complex exponential that represents the frequency component at \(\omega\).

The STFT is computed by dividing the signal \(x(t)\) into overlapping windows of length \(\tau\) and applying the Fourier transform to each window. The window function \(w(t-\tau)\) is used to taper the edges of the window and minimize spectral leakage. The resulting complex-valued STFT is a function of both frequency and time, and can be visualized as a spectrogram.

In practice, the STFT is computed using a sliding window that moves over the signal in small increments. The size and overlap of the window determine the frequency and temporal resolution of the spectrogram. A larger window size provides better frequency resolution but poorer temporal resolution, while a smaller window size provides better temporal resolution but poorer frequency resolution.

The equation for computing spectrogram can be expressed as:

\[S(f, t) = |STFT(x(t), f, \tau)|^2\]

where \(S(f, t)\) is the spectrogram, STFT is the short-time Fourier transform, \(x(t)\) is the input signal, f is the frequency, and \(\tau\) is the time shift or window shift.

The STFT is calculated by dividing the signal into overlapping windows of length \(\tau\) and computing the Fourier transform of each window. The magnitude squared of the resulting complex-valued STFT is then used to compute the spectrogram. The spectrogram provides a time-frequency representation of the signal, where the magnitude of the STFT at each time and frequency point represents the strength of the signal at that particular time and frequency.

Spectrogram using python

To generate a spectrogram in Python, we can use the librosa library which provides an easy-to-use interface for computing and visualizing spectrograms. Here’s an example program that generates a spectrogram for an audio signal:

import librosa
import librosa.display
import numpy as np
import matplotlib.pyplot as plt

# Load audio file
y, sr = librosa.load('audio_147793__setuniman__sweet-waltz-0i-22mi.hq.ogg')

# Compute spectrogram
spec = librosa.feature.melspectrogram(y=y, sr=sr)

# Convert power to decibels
spec_db = librosa.power_to_db(spec, ref=np.max)

# Plot spectrogram
fig, ax = plt.subplots(nrows = 1, ncols = 1)
img = librosa.display.specshow(spec_db, x_axis='time', y_axis='mel', ax = ax)
fig.colorbar(img, ax = ax, format='%+2.0f dB')
ax.set_title('Spectrogram')
fig.show()
Figure 1: Spectrogram of an example audio file using librosa python library

In this program, we first load an audio file using the librosa.load() function, which returns the audio signal (y) and its sampling rate (sr). We then compute the spectrogram using the librosa.feature.melspectrogram() function, which computes a mel-scaled spectrogram of the audio signal.

To convert the power spectrogram to decibels, we use the librosa.power_to_db() function, which scales the spectrogram to decibels relative to a maximum reference power. We then plot the spectrogram using the librosa.display.specshow() function, which displays the spectrogram as a heatmap with time on the x-axis and frequency on the y-axis.

Finally, we add a colorbar and title to the plot using the fig.colorbar() and ax.set_title() functions, respectively, and display the plot using the fig.show() function.

Note that this program assumes that the audio file is in OGG format and is located in the current working directory. If the file is in a different format or location, you will need to modify the librosa.load() function accordingly. The sample audio file can be obtained from the librosa github repository.

Applications of Spectrograms in Signal Processing

Spectrograms are widely used in signal processing applications, particularly in speech and audio processing. Some common applications of spectrograms include:

  1. Speech Analysis: Spectrograms are used to analyze the frequency content of speech signals, which can provide insight into the characteristics of the speaker, such as gender, age, and emotional state.
  2. Music Analysis: Spectrograms are used to analyze the frequency content of music signals, which can provide information about the genre, tempo, and instrument composition of the music.
  3. Noise Reduction: Spectrograms can be used to identify and remove noise from a signal by filtering out the frequency components that correspond to the noise.
  4. Voice Activity Detection: Spectrograms can be used to detect the presence of speech in a noisy environment by analyzing the frequency content of the signal.

Conclusion

Spectrograms are a powerful tool in signal processing for analyzing and visualizing time-varying signals. They provide a detailed view of the frequency content of a signal over time, enabling accurate analysis and interpretation of complex signals such as speech and audio signals. With their numerous applications in speech and audio processing, spectrograms are an essential tool for researchers and practitioners in these fields.

Volterra series – RF power amplifier model

Key focus: Volterra series is an useful tool for modeling nonlinear time invariant systems in general, and they are commonly used to model RF power amplifiers with memory effects

Introduction

Power amplifier are nonlinear devices responsible for compensating for the signal power loss between a transmitter and receiver. The input power levels to the power amplifiers can be increased to compensate for the power loss during transmission. At low input power regime, an ideal power amplifier provides constant gain to the output signal.

However, in the high input power regime, nonlinear effects of the power amplifier come to play, the power of the output signal shows diminished gain (compressed) and starts to saturate.

1-dB compression point is the most important measure of quantifying the beginning of non-linearity of the power amplifier (Figure 1). It is the point where the RF output power level decreases 1 dB from the constant value. Therefore, 1-dB compression point is a measure of the beginning of distortion, where a perfect sine wave fed at the input of the power amplifier do not maintain the perfect shape at the output of the power amplifier.

Figure 1: 1-dB compression point

It is therefore critical to model and understand the power amplifier in order to correct for the non-linear distortion it creates and not break the requirements defined by the agreed-upon standards like 3GPP.

Memoryless models: Taylor series

Memoryless models place emphasis on the power amplifier with memoryless nonlinearity, which means that the current output is determined solely by the current input via a nonlinear method. A nonlinear memoryless system can be modeled using Taylor series. For example, a discrete time system with input x[n] and output y[n], the nonlinearity is incorporated using nonlinear terms with corresponding coefficients/weights (ɑi) for each term.

\[ y[n] = \alpha_1 x[n] + \alpha_2 x^2[n] + \alpha_3 x^3[n]+ \cdots + \alpha_ x^n[n] + \cdots \quad \quad (1) \]

Memory models : Volterra series

The bandwidth of the signals, such as in 3GPP LTE and 3GPP 5G NR, gets wider. When the wideband signals are input to high power amplifiers (such as in base stations), the power amplifiers start to exhibit memory effects. As a consequence, the power amplifier’s current output is affected not only by the current input, but also by previous input values. Volterra series and its derivatives are commonly used to model power amplifiers with memory.

Volterra series is an useful tool for modeling nonlinear time invariant systems in general, and they are commonly used to model RF power amplifiers with memory effects. The Volterra series is a polynomial based mathematical model that captures the nonlinear terms and memory effects present in these devices.

For a discrete time system with input x[n] and output y[n], the Volterra series with nonlinearity order K and memory depth Q is given by

\[y[n] = \sum_{k=1}^K y_k[n] \quad \quad (2)\]

where,

\[ \begin{align} y_k[n] &= \sum_{i_1=0}^{Q-1} \cdots \sum_{i_k=0}^{Q-1} h_k \left(i_1, \cdots, i_k \right) x[n-i_1]x[n-i_2]\cdots x[n-i_k] \\ & = \sum_{i_1=0}^{Q-1} \cdots \sum_{i_k=0}^{Q-1} h_k \left(i_1, \cdots, i_k \right) \prod_{r=1}^{k} x[n-i_r] \\ \end{align}\]

hk(i1, …, ik) are the nth order Volterra Kernels. It can be understood that it models any system by using the system’s impulse responses. Because of the high complexity and high resource requirements, Volterra series cannot be used in practical applications.

For practical applications, Volterra series has to be truncated in such a way that it has the least impact on the models performance. This can be accomplished by omitting some of the terms in equation (2) for each application. Special versions of Volterra series allow for use in power amplifier behavioral modeling. Memory polynomial model is one of the derivative of Volterra series that is easier to implement. We will discuss this model with an example simulation model in the next post.

5G NR Resource block

Key focus: 5G NR resource block : a block of 14 OFDM symbols (1 slot across time domain) with the corresponding 12 subcarriers for those symbols

5G NR protocol stack

In the 5G New Radio (NR), the protocol architecture can be separated into two categories: user plane protocol stack and control plane protocol stack. The user plane protocol stack architecture is responsible for delivering user data and the control plane architecture is responsible for setting up the connection, maintaining mobility and providing end-to-end security.

The user plane protocol stack for 5G New Radio (NR) is shown in Figure 1. We see that at the physical layer interface (air interface) between the gNB and UE, the transmission occurs in the form of radio frames.

Figure 1: 5G NR user plane protocol stack architecture

Radio frame structure

Looking at the 5G New Radio (NR) frame structure from the time domain perspective, the radio transmissions are categorized into radio frames, subframes, slots and mini-slots (Figure 2).

A radio frame is for a duration of 10 ms and it comprises of 10 subframes of duration 1 ms each. Each subframe may consist of one or multiple adjacent slots with each slot having 14 OFDM (Orthogonal Frequency Division Multiplexing) symbols. The possibility of transmission over a fraction of a slot is referred to as mini-slot.

Figure 1 shows the radio frame structure for the supported transmission numerologies (μ = 0, 1, 2, 3, 4) as per Table 1.

Figure 2: 5G NR frame structure

Because the duration of an OFDM signal is inversely proportional to its subcarrier spacing, the time duration of a slot scales with the selected numerology (Table 1).

Table 1: 5G NR – supported transmission numerologies

From the frequency domain perspective, an OFDM symbol is comprised of 12 subcarriers, each subcarrier may be spaced according to the scalable numerology as per Table 1.

Time-Frequency resource

A resource element is the smallest time-frequency resource over one subcarrier of a single OFDM symbol. It is identified as (k,l)p,μ where k is the index of the subcarrier in the frequency domain, l is the OFDM symbol position in time domain, p is the antenna port and μ is the subcarrier spacing configuration defined in Table 1 above.

A resource block (a.k.a physical resource block – PRB) is a block of N RBsc = 12 subcarriers over which the transmissions are scheduled. 5G NR physical layer uses time-frequency resource (physical resource block) for transmission.

A resource grid consists of N size,μgrid,x  subcarriers and N subframe,μ symb number of OFDM symbols (Table 2 and 3).

Figure 3: Resource element, Resource block, Resource grid in 5G NR
Table 2: Number of OFDM symbols per slot, slots per subframe and OFDM symbols per subframe for normal cyclic prefix configuration
Table 3: Number of OFDM symbols per slot, slots per subframe and OFDM symbols per subframe for extended cyclic prefix configuration