Basic operations on signal sequences – Addition

Key focus: How to implement the basic addition operation on two discrete time signal sequences. Python code for signal addition is provided.

Signal addition

Given two discrete-time sequences \(x_1[n]\) and \(x_2[n]\), the addition of these two sequences is represented as \(x_1[n]+ x_2[n]\). The start of the sample at \(n=0\) can be different for these two sequences.

For example, if

\[ \begin{align} x_1[n] &= \left\{ -3, 12, -4, \underset{\uparrow}{9}, 2, 15\right\} \\ x_2[n] &= \left\{ -6, \underset{\uparrow}{4}, -2, 10\right\} \end{align} \]

the position of the sample at \(n=0\) is different in these sequences. Also, the length of these sequences are different.

The addition operation should take these differences into account. The following python code adjusts for these differences before computing the addition operation. It creates two sequences and of equal length that spans the minimum and maximum indices of and . The sequences and are first filled with zeros and then the sequences & are copied over to & at corresponding positions. The final result is computed as the sum of and .

import numpy as np

def signal_add(x1, n1, x2, n2):
    '''
    Computes y(n) = x1(n) + x2(n)
    ---------------------------------
    [y, n] = signal_add(x1, n1, x2, n2)
    
    Parameters
    ----------
    n1 = indices of first sequence x1
    n2 = indices of second sequence x2
    x1 = first sequence defined for indices n1
    x2 = second sequence defined for indices n2
    Returns
    -------
    y = output sequence defined over indices n that
    covers whole range of n1 and n2
    '''

    n_start = min(min(n1), min(n2))
    n_end = max(max(n1), max(n2))
    n = np.arange(n_start, n_end + 1)  # duration of y(n)

    y1 = np.zeros_like(n, dtype='complex_')
    y2 = np.zeros_like(n, dtype='complex_')

    mask1 = (n >= n1[0]) & (n <= n1[-1])
    mask2 = (n >= n2[0]) & (n <= n2[-1])

    y1[np.where(mask1)[0]] = x1[np.where(n1 == n[mask1])]
    y2[np.where(mask2)[0]] = x2[np.where(n2 == n[mask2])]

    y = y1 + y2

    return y, n

The following code snippet uses the function above to compute the addition of two sequences \(x_1[n]\) and \(x_2[n]\), defined as

\[\begin{align} x_1[n] &= cos \left( 0.03 \pi n \right), & 0 \leq n \leq 50 \\ x_2[n] &= e^{0.06 n}, & -30 \leq 0 \leq n \end{align}\]
n1 = np.arange(0,50)
n2 = np.arange(-30,10)
x1 = np.cos(0.03*np.pi*n1)
x2 = np.exp(0.06*n2)
[y,n] = signal_add(x1, n1, x2, n2)

These discrete sequences are plotted (Figure 1) using the code below.

import matplotlib.pyplot as plt
f, (ax1, ax2, ax3) = plt.subplots(3, 1, sharex=True, sharey=True)

ax1.stem(n1, x1, 'k', label = '$x_1[n]$')
ax2.stem(n2, x2, 'b', label = '$x_2[n]$')
ax3.stem(n, y, 'r', label = '$y[n] = x_1[n] + x_2[n]$')

ax1.legend(loc="upper right")
ax2.legend(loc="upper right")
ax3.legend(loc="upper right")

ax1.set_xlim((min(n), max(n)+1))
ax1.set_title('Addition of signals')
plt.show()
Addition of discrete signals in signals and systems. Signal addition
Figure 1: Addition of discrete signals

Complex-valued exponential sequence

In digital signal processing, we utilize various elementary sequences for the purpose of analysis. In this series, we will see such sequences. One such elementary sequence is the real-valued exponential sequence. (see the articles on unit sample sequence, unit step sequence, real-valued exponential sequence)

A complex-valued exponential sequence in signals and systems is a discrete-time sequence that exhibits complex exponential behavior. It is characterized by complex numbers raised to the power of the index. The general form of a complex-valued exponential sequence is given by:

\[x[n] = e^{ \alpha n }e^{ j \omega n } = e^{\left( \alpha + j \omega \right) n } = cos \left[ \left( \alpha + j \omega \right) n \right] + j sin \left[ \left( \alpha + j \omega \right) n \right], \; \forall n \]

where:

  • x[n] is the value of the sequence at index n.
  • \(\alpha\) acts as an attenuation factor if \(\alpha \lt 0\) or as an amplification factor for \(\alpha \gt 0\)
  • \(j\) is the indeterminate satisfying \(j^2 = -1\) (imaginary unit).
  • \(\omega\) is the angular frequency in radians per sample.

The complex nature is indicated by the presence of the indeterminate \(j\) in the exponent.

The python function to generate a complex exponential function is given below

import numpy as np
import matplotlib.pyplot as plt

def complex_exponential_sequence(n, alpha, omega):
    return  np.exp((alpha + 1j * omega) * n)

n = np.linspace(0, 40, 1000)
alpha = 0.025; omega = 0.75
x = complex_exponential_sequence(n, alpha, omega)

Plot of real and imaginary parts of the sequence generated for various values of \(\alpha\) and \(\omega\) is given next

Figure 1: Complex exponential sequence for various values of \(\alpha\) and \(\omega\)

From Figure 1, we see that the variable \(\alpha\) governs the decay or growth of the sequence in time and the \(\omega\) controls the oscillation frequency on a circle in the complex plane.

The 3D views of the complex sequence for various values of \(\alpha\) and \(\omega\) are illustrated next.

When (\(\alpha=0\)), the sequence remains the on circle in the complex plane.

Figure 2: A neutral sequence (\(\alpha =0\) and \(\omega = 1\))

When \(\alpha \gt 0 \), the sequence grows exponentially and it spirals out.

Figure 3: A growing sequence (\(\alpha >0\) and \(\omega = 0.75 \))

When \(\alpha \lt 0 \), the sequence decays exponentially.

Figure 4: A decaying sequence (\(\alpha <0\) and \(\omega = 2.5 \))

Applications

Complex exponential sequences have various applications in modeling and signal processing. Some of the key applications include:

Signal Analysis and Representation: Complex exponential sequences form the basis for Fourier analysis, which decomposes a signal into a sum of sinusoidal components. The complex exponential sequence (\(e^{j\omega n}\)) serves as the building block for representing and analyzing signals in the frequency domain.

System Modeling and Analysis: These sequences play a fundamental role in modeling and analyzing linear time-invariant (LTI) systems. By applying complex exponential inputs to a system and observing the resulting outputs, one can determine the system’s frequency response and characterize its behavior in terms of amplitude and phase shifts at different frequencies.

Digital Filtering: Complex exponential sequences are utilized in digital filtering algorithms, such as the Fourier transform-based frequency domain filtering and the Z-transform-based discrete-time filtering. These sequences help design filters for various applications, such as noise removal, equalization, and signal enhancement.

Modulation Techniques: These sequences are fundamental in various modulation schemes used in communication systems. For instance, in amplitude modulation (AM), frequency modulation (FM), and phase modulation (PM), the modulating signals are typically expressed as complex exponential sequences that are mixed with carrier signals to encode information.

Control Systems: Complex exponential sequences are relevant in control system analysis and design. In control theory, the Laplace transform, which involves complex exponentials, is used to analyze system dynamics, stability, and transient response. The concept of the complex plane, where complex exponentials reside, is crucial in control system design and stability analysis.

Digital Signal Processing (DSP): These sequences find extensive use in various DSP applications, including digital audio processing, image processing, speech recognition, and data compression. Techniques like the discrete Fourier transform (DFT) and fast Fourier transform (FFT) exploit complex exponentials to efficiently analyze signals in the frequency domain.

Real-valued exponential sequence

In digital signal processing, we utilize various elementary sequences for the purpose of analysis. In this series, we will see such sequences. One such elementary sequence is the real-valued exponential sequence. (see the articles on unit sample sequence, unit step sequence, complex exponential sequence)

An exponential sequence in signals and systems is a discrete-time sequence that exhibits exponential growth or decay. It is characterized by a constant base raised to the power of the index. The general form of the sequence is given by:

\[x[n] = A \cdot r^n, \quad \forall n; \quad r \in \mathbb{R}\]

where:

  • \(x[n]\) is the value of the sequence at index \(n\).
  • \(A\) is the initial amplitude or value at \(n = 0\).
  • \(r\) is the constant base, which determines the growth or decay behavior.
  • \(n\) represents the index of the sequence.

If the value of r is greater than 1, the sequence grows exponentially as n increases, resulting in exponential growth. Conversely, if r is between 0 and 1, the sequence decays exponentially, approaching zero as n increases.

Exponential sequences find various applications in fields such as finance, physics, and telecommunications. In signal processing and system analysis, these sequences are fundamental components used to model and analyze various system behaviors, such as stability, convergence, and frequency response.

Python code that implements a real-valued exponential sequence for various values of \(r\).

import matplotlib.pyplot as plt
import numpy as np

def exponential_sequence(n, A, r):
    return A * np.power(r, n)

# Define the range of n
n = np.arange(0, 25)

# Define the values of r
r_values = [0.5, 0.8, 1.2]

# Plotting the exponential sequences for various values of r
fig, axs = plt.subplots(len(r_values), 1, figsize=(8, 6))

for i, r in enumerate(r_values):
    # Generate the exponential sequence for the current value of r
    x = exponential_sequence(n, 1, r)

    # Plot the exponential sequence in the current subplot
    axs[i].stem(n, x, 'k', use_line_collection=True)
    axs[i].set_xlabel('n')
    axs[i].set_ylabel(f'x[n], r={r}')
    axs[i].set_title(f'Exponential Sequence, r={r}')
    axs[i].grid(True)

# Adjust spacing between subplots
plt.tight_layout()

# Display the plot
plt.show()
Figure 1: Real-valued exponential sequence \(x[n] = A \cdot r^n\) for \(r = 0.5, 0.8, 1.5\)

Applications

Real-valued exponential sequences have various applications in different fields, including:

Signal Processing: Exponential sequences are used to model and analyze signals in fields like audio processing, image processing, and telecommunications. They are fundamental in Fourier analysis, frequency response analysis, and filter design.

System Analysis: Exponential sequences are essential in understanding and characterizing the behavior of linear time-invariant (LTI) systems. They help analyze system stability, impulse response, and frequency response.

Finance: Exponential sequences find applications in finance and economics for modeling compound interest, population growth, investment returns, and other exponential growth/decay phenomena.

Physics: In physics, exponential sequences are used to describe natural phenomena such as radioactive decay, charging/discharging of capacitors, and decay of electrical or mechanical systems.

Control Systems: Exponential sequences play a crucial role in control systems engineering. They are used to model system dynamics, analyze stability, and design controllers for desired response characteristics.

Probability and Statistics: Exponential sequences are utilized in probability and statistics to model various distributions, including the exponential distribution, which represents events occurring randomly and independently over time.

Machine Learning: Exponential sequences are used in machine learning algorithms for tasks such as feature scaling, regularization, and gradient descent optimization.

These are just a few examples of the broad range of applications where real-valued exponential sequences are utilized. Their ability to represent exponential growth or decay makes them a valuable tool for modeling and understanding dynamic systems and phenomena in various disciplines.

Unit Step Sequence

In digital signal processing, we utilize various elementary sequences for the purpose of analysis. In this series, we will see such sequences. One such elementary sequence is the unit step sequence (see the articles on unit sample sequence, unit step sequence, real-valued exponential sequence, complex exponential sequence).

Unit Step Sequence

A unit step sequence is a discrete-time signal that represents a step function. In discrete-time signal processing, a sequence is a set of values indexed by integers. A unit step sequence, denoted as u[n], is defined as follows:

\[ u[n] = \begin{cases} 1 , & \quad n \geq 0 \\ 0, & \quad n \lt 0\end{cases} = \left\{ \cdots,0, 0, \underset{\uparrow}{1}, 1, 1, 1, \cdots\right\} \]

In this definition, the value of \(u[n]\) is 0 for negative values of \(n\) (\(n \lt 0\)), and it is 1 for non-negative values of \(n (n \geq 0)\). The up-arrow indicates the sample at \(n=0\)

Graphically, the sequence looks like a step function, where the value remains zero for negative indices and abruptly jumps to 1 at n=0, continuing at that value for positive indices.

In the equation above, the sequence value of 1 start at index 0. In signal processing, the starting place (where the value 1 starts) can be shifted. The generalized formula for generated a shifted sequence will be

\[ u[n – n_0] = \begin{cases} 1 , & \quad n \geq n_0 \\ 0, & \quad n \lt n_0\end{cases}\]

The following python code implements a unit step sequence over the interval \(n_1 \leq n \leq n_2\)

import matplotlib.pyplot as plt
import numpy as np

def unit_step_sequence(n0, n1, n2):
    """
    Generate unit step sequence u(n - n0); n1<=n<=n
    n0 = number of samples to offset/shift
    n1 = starting number to generate the sequence index
    n2 = ending number to generate the sequence index
    """
    n = np.arange(n1,n2+1)
    u = np.zeros_like(n)
    u[n >= n0] = 1
    return u

# Generate the shifted unit step sequence for a given range and shift value
n0 = 2  # Shift value
n1 = -3
n2 = 9
u = unit_step_sequence(n0, n1, n2)

fig = plt.figure()
ax = fig.add_subplot(1, 1, 1)

# Plotting the shifted unit step sequence
plt.stem(np.arange(n1,n2+1), u,'r', markerfmt='o', basefmt='k', use_line_collection=True)
plt.xlabel('n')
plt.ylabel(r'$u[n-n_0]$')
plt.title(r'Discrete-time Unit Step Sequence $u[n-2]$')
plt.grid(True)
plt.show()
Figure 1: Discrete unit step sequence \(u[n-n_0]\); shifted by \(n_0=2\), generated over the interval \(-3 <= n <= 9\)

Applications

The unit step sequence is often used as a fundamental building block in signal processing and system analysis. It serves as a basic reference for studying and analyzing other signals and systems. By manipulating the sequence, one can derive other useful sequences and functions, such as shifted step sequences, ramp sequences, and more complex signals.

This sequence is particularly important in the field of discrete-time systems, where it is used to analyze system behavior and characterize properties like stability, causality, and linearity. It is also employed in various applications, such as digital filters, signal modeling, and signal reconstruction.

References

[1] Prof. Alan V. Oppenheim, Lecture 2: Discrete-Time Signals and Systems, Part 1, RES.6-008, MIT OCW, Spring 2011

Unit sample sequence

In digital signal processing, we utilize various elementary sequences for the purpose of analysis. In this series, we will see such sequences. One such elementary sequence is the unit sample sequence (see the articles on unit sample sequence, unit step sequence, real-valued exponential sequence, complex exponential sequence)

A unit sample sequence, also known as an impulse sequence or delta sequence, is a discrete sequence that consists of a single sample with the value of 1 at a specific index, and all other samples are zero. It is commonly represented as a discrete-time impulse function or delta function.

Mathematically, a unit sample sequence can be defined as:

\[x[n] = \delta[n – n_0] = \begin{cases} 1 & \quad n = n_0 \\ 0 & \quad n \neq n_0 \end{cases} \]

Where:

  • \(x[n]\) represents the value of the sequence at index \(n\).
  • \(δ[n]\) is the discrete-time impulse function or delta function.
  • \(n_0\) is the index at which the impulse occurs. At this index, \(x[n]\) has the value of 1, and for all other indices, \(x[n]\) is zero.

This sequence represents a localized impulse or sudden change at that particular index. The unit sample sequence is commonly used in signal processing and system analysis to study the response of systems to impulses or abrupt changes. It serves as a fundamental tool for representing and analyzing signals and systems.

In python, the scipy.signal.unit_impulse function can be used to generate an impulse sequence in SciPy. For 1-D signals, the first argument to the function is the number of samples requested for generating the unit_impulse and the second argument is the offset (i.e, \(n_0\))

import matplotlib.pyplot as plt
import numpy as np
from scipy import signal

n = 11 #number of samples
n0 = 5 # Offset (Shift index)
imp = signal.unit_impulse(n, n0) #unit impulse

fig = plt.figure()
ax = fig.add_subplot(1, 1, 1)

plt.stem(np.arange(0,n),imp,'r', markerfmt='ro', basefmt='k')
plt.xlabel('Sample #')
plt.ylabel(r'$\delta[n-n_0]$')
plt.title('Discrete-time Unit Impulse')
plt.show()
Figure 1: Discrete-time unit impulse

Applications of unit sample sequences

Unit sample sequences, also known as impulse sequences, have several applications in digital signal processing. Here are a few common applications:

System Analysis: Unit sample sequences are used to analyze and characterize the behavior of systems, such as filters and signal processors, in response to impulses or sudden changes.

Convolution: Unit sample sequences are essential in the mathematical operation of convolution, which is used for filtering, signal analysis, and processing tasks.

Signal Reconstruction: Unit sample sequences are employed in reconstructing continuous signals from their sampled versions using techniques like impulse sampling and interpolation.

System Identification: Unit sample sequences can be utilized to estimate the impulse response of a system, allowing for system identification and modeling.

These are just a few examples of the diverse applications of unit sample sequences in digital signal processing. They serve as fundamental tools for analyzing signals, characterizing systems, and performing various signal processing operations.

References

[1] Prof. Alan V. Oppenheim, Lecture 2: Discrete-Time Signals and Systems, Part 1, RES.6-008, MIT OCW, Spring 2011

Analog and Discrete signals

In the context of signal and systems, analog and discrete signals are two different types of signals that convey information.

Analog signal

An analog signal is a continuous signal that varies smoothly over time. It can take on any value within a certain range. Analog signals are represented by physical quantities such as voltage, current, or sound waves. For example, the varying voltage produced by a microphone when recording sound is an analog signal. Analog signals are typically represented as continuous waveforms.

Let’s consider an example of a simple analog signal:

\[x(t) = A \cdot sin \left(2 \pi f t + \phi \right)\]

    In this equation:

    • x(t) represents the value of the analog signal at time t.
    • A is the amplitude of the signal, which determines its maximum value.
    • f is the frequency of the signal, which represents the number of cycles per unit of time.
    • φ is the phase of the signal, which represents the offset or starting point of the waveform.

    This equation describes a sinusoidal analog signal, where the value of the signal varies continuously over time. The signal can have an infinite number of values at any given instant.

    Discrete signal

    On the other hand, a discrete signal is a signal that is defined only at specific instances of time and takes on a finite set of values. Discrete signals are often derived from analog signals by a process called sampling, where the continuous analog signal is measured or sampled at regular intervals. Each sample represents the value of the signal at a particular instant. These samples can be stored and processed using digital systems. Examples of discrete signals include digital audio, digital images, and the output of a digital sensor.

    Discrete signals are commonly used in digital signal processing and can be represented using mathematical equations.

    The general equation for a discrete signal can be written as:

    \[x[n] = f(n)\]

    In this equation:

    • x[n] represents the value of the discrete signal at time instance n.
    • f(n) is the function that determines the value of the signal at each specific time instance.

    The function f(n) can take various forms depending on the specific characteristics of the discrete signal. For example, let’s start with the equation for the analog sinusoidal signal:

    \[x(t) = A \cdot sin \left(2 \pi f t + \phi \right)\]

    To obtain the discrete version of this signal, we need to sample it at regular intervals. The sampling process involves measuring the analog signal at equidistant points in time.

    Let’s define the sampling period as \(T_s\), which represents the time between two consecutive samples. The sampling rate is the inverse of the sampling period and is denoted as \(f_s = 1 / T_s\).

    Now, we can express the discrete version of the sinusoidal signal as:

    \[x[n] = x(n T_s) = A \cdot sin(2 \pi f n T_s + \phi)\]

    In this equation:

    • x[n] represents the value of the discrete signal at sample index n.
    • f is the frequency of the sinusoidal signal in hertz
    • n represents the sample index, indicating which sample we are considering.
    • \(T_s\) is the sampling period.
    • \(f_s\) is the sampling frequency, which is the reciprocal of the sampling period.

    By substituting \(nT_s\) for t in the analog sinusoidal signal equation, we obtain the discrete version of the sinusoidal signal. The discrete signal represents the sampled values of the original analog signal at each specific time instance, \(nT_s\).

    It’s important to note that the accuracy of the discrete signal representation depends on the sampling rate. According to the Nyquist-Shannon sampling theorem, for real signals, the sampling rate should be at least twice the maximum frequency of the analog signal to avoid aliasing and accurately reconstruct the signal from its samples.

    Python code

    Following is an example Python code that simulates an analog sinusoidal signal, samples it to obtain a discrete version, and overlays the two signals for comparison

    import numpy as np
    import matplotlib.pyplot as plt
    
    # Parameters for the analog signal
    amplitude = 1.0  # Amplitude of the signal
    frequency = 2.0  # Frequency of the signal in Hz
    phase = 0.0  # Phase of the signal in radians
    
    # Parameters for the discrete signal
    sampling_rate = 10  # Number of samples per second
    num_samples = 20
    
    # Time arrays for the analog and discrete signals
    t_analog = np.linspace(0, num_samples / sampling_rate, num_samples * 10)  # Higher resolution for analog signal
    n_discrete = np.arange(num_samples)
    
    # Generate the analog signal
    analog_signal = amplitude * np.sin(2 * np.pi * frequency * t_analog + phase)
    
    # Sample the analog signal to obtain the discrete signal
    discrete_signal = amplitude * np.sin(2 * np.pi * frequency * n_discrete / sampling_rate + phase)
    
    # Plot the analog and discrete signals
    plt.plot(t_analog, analog_signal, label='Analog Signal')
    plt.stem(n_discrete / sampling_rate, discrete_signal, 'r', markerfmt='ro', basefmt=' ', label='Discrete Signal')
    plt.xlabel('Time')
    plt.ylabel('Amplitude')
    plt.title('Analog and Discrete Sinusoidal Signals')
    # Move the legend outside the figure
    plt.legend(loc='upper right', bbox_to_anchor=(1.1, 1))
    plt.grid(True)
    plt.show()

    Resulting plot

    Figure 1: Simulated analog and discrete sinusoidal signals

    Exploring Orthogonality: From Vectors to Functions

    Keywords: orthogonality, vectors, functions, dot product, inner product, discrete, Python programming, data analysis, visualization

    Orthogonality

    Orthogonality is a mathematical principle that signifies the absence of correlation or relationship between two vectors (signals). It implies that the vectors or signals involved are mutually independent or unrelated.

    Two vectors (signals) A and B are said to be orthogonal (perpendicular in vector algebra) when their inner product (also known as dot product) is zero.

    \[ A \perp B \Leftrightarrow \left<A.B \right> = A_1 \cdot B_1 + A_2 \cdot B_2 + \cdots A_n \cdot B_n = 0\]

    Example: Let’s show that the two vectors \(\overrightarrow{A} = \binom{-2}{3}\) and \(\overrightarrow{B} = \binom{3}{2}\) are orthogonal

    \[\overrightarrow{A} \cdot \overrightarrow{B} = A_x B_x + A_y B_y = (-2)(3) + (3)(2) = 0 \]

    Let verify if the angle between the vectors is \(90^{\circ}\)

    \[ \theta = cos^{-1} \left( \frac{\overrightarrow{A} \cdot \overrightarrow{B}}{|\overrightarrow{A} | |\overrightarrow{B}|} \right) = cos^{-1}(0) = 90 ^{\circ} \]
    Figure 1: Two vectors exhibiting orthogonality

    To find the dot product of two vectors, you need to multiply their corresponding components and then sum the results. Here’s the general formula (in matrix notation) for checking the orthogonality of two complex valued vectors \(\vec{a}\) and \(\vec{b}\):

    \[\vec{a} \perp \vec{b} \Rightarrow \left< \vec{a}, \vec{b} \right> = \begin{bmatrix} a_1^* & a_2^* & \cdots & a_n^* \\ \end{bmatrix} \begin{bmatrix} b_1 \\ b_2\\ \vdots \\ b_n \\ \end{bmatrix} = 0 \]

    Here’s an example code snippet in Python that demonstrates to check if two vectors given as lists are orthogonal.

    import numpy as np
    import matplotlib.pyplot as plt
    
    def dot_product(vector1, vector2):
        if len(vector1) != len(vector2):
            raise ValueError("Vectors must have the same length.")
        return sum(x * y for x, y in zip(vector1, vector2))
    
    def are_orthogonal(vector1, vector2):
        result = dot_product(vector1, vector2)
        return result == 0
    
    # Example vectors
    vectorA = [-2, 3]
    vectorB = [3, 2]
    
    # Check if vectors are orthogonal
    if are_orthogonal(vectorA, vectorB):
        print("The vectors are orthogonal.")
    else:
        print("The vectors are not orthogonal.")
    
    # Plotting the vectors
    origin = [0], [0]  # Origin point for the vectors
    
    plt.quiver(*origin, vectorA[0], vectorA[1], angles='xy', scale_units='xy', scale=1, color='r', label='Vector A')
    plt.quiver(*origin, vectorB[0], vectorB[1], angles='xy', scale_units='xy', scale=1, color='b', label='Vector B')
    
    plt.xlim(-5, 5)
    plt.ylim(-5, 5)
    plt.xlabel('x')
    plt.ylabel('y')
    plt.title('Plot of Vectors')
    plt.grid(True)
    plt.legend()
    plt.show()

    Orthogonality of Continuous functions

    Orthogonality, in the context of functions, can be seen as a broader concept akin to the orthogonality observed in vectors. Geometrically, orthogonal vectors are perpendicular to each other since their dot product equals zero.

    When computing the dot product of two vectors, their components are multiplied and summed. However, when considering the “dot” product of functions, a similar approach is taken. Functions are treated as if they were vectors with an infinite number of components, and the dot product is obtained by multiplying the functions together and integrating over a specific interval.

    Let f(t) and g(t) are two continuous functions (imagined as two vectors) on the closed interval [a,b] (i.e a ≤ t ≤ b). For the functions to be orthogonal in the given interval, their dot product should be zero

    \[ \left<f,g\right> = \int_a^b f(t) g(t) dt = 0 \Rightarrow \text{f(t) and g(t) are orthogonal}\]

    Here is a small python script to check if two given functions are orthogonal

    Python Script

    import sympy
    import numpy as np
    import matplotlib.pyplot as plt
    
    plt.style.use('seaborn-talk')
    print(plt.style.available)
    
    # Test the orthogonality of functions
    x = sympy.Symbol('x')
    f = sympy.sin(x)  # First function
    g = sympy.cos(2*x)  # Second function
    a = 0 # interval lower limit
    b = 2*sympy.pi # interval upper limit
    interval = (0, 2*sympy.pi)  # Integration interval
    inner_product = sympy.integrate(f*g, (x, interval[0], interval[1]))
    
    if sympy.N(inner_product) == 0:
        print("The functions",str(f),"and",str(g),"are orthogonal over the interval [",str(a), ",",str(b),"].")
    else:
        print("The functions",str(f),"and",str(g),"are not orthogonal over the interval [",str(a), ",",str(b),"].")
    
    # Plotting the functions
    x_vals = np.linspace(float(interval[0]), float(interval[1]), 100)
    f_vals = np.sin(x_vals)
    g_vals = np.cos(2*x_vals)
    
    plt.plot(x_vals, f_vals, label=str(f))
    plt.plot(x_vals, g_vals, label=str(g))
    plt.plot(x_vals, f_vals*g_vals, label=str(f)+str(g))
    plt.xlabel('x')
    plt.ylabel('Function values')
    plt.legend()
    plt.title('Plot of functions')
    plt.grid(True)
    plt.show()

    Output

    The functions sin(x) and cos(2*x) are orthogonal over the interval [ 0 , 2*pi ]

    Orthogonality of discrete functions

    To check the orthogonality of discrete functions, you can use the concept of the inner product (same as above). In discrete settings, the inner product can be thought of as the sum of the element-wise products of the function values at corresponding points.

    Here’s an example code snippet in Python that demonstrates how to check the orthogonality of two discrete functions:

    import numpy as np
    
    def inner_product(f, g):
        if len(f) != len(g):
            raise ValueError("Functions must have the same length.")
        return np.sum(f * g)
    
    def are_orthogonal(f, g):
        result = inner_product(f, g)
        return result == 0
    
    # Example functions (discrete)
    f = np.array([1, 0, -1, 0])
    g = np.array([0, 1, 0, -1])
    
    # Check if functions are orthogonal
    if are_orthogonal(f, g):
        print("The functions are orthogonal.")
    else:
        print("The functions are not orthogonal.")

    References

    [1] Smith, J.O. Mathematics of the Discrete Fourier Transform (DFT) with Audio Applications, Second Edition ↗

    Implementing Markov Chain in Python

    Keywords: Markov Chain, Python, probability, data analysis, data science

    Markov Chain

    Markov chain is a probabilistic models that describe a sequence of observations whose occurrence are statistically dependent only on the previous ones. This article is about implementing Markov chain in Python

    Markov chain is described in one of the earlier posts. For better understanding of the concept, review the post before proceeding further.

    We will model a car’s behavior using the same transition matrix and starting probabilities described in the earlier post for modeling the corresponding Markov chain model (refer Figure 1). The matrix defines the probabilities of transitioning between different states, including accelerating, maintaining a constant speed, idling, and braking.

    Figure 1: Modeling a car’s behavior using Markov chain model

    The starting probabilities indicate that the car starts in the break state with probability 1, which means it is already stopped and not moving.

    Python implementation

    Here’s the sample code in Python that implements the above model:

    import random
    
    # Define a transition matrix for the Markov chain
    transition_matrix = {
        'accelerate': {'accelerate': 0.3, 'constant speed': 0.2, 'idling': 0 , 'break': 0.5 },
        'constant speed': {'accelerate': 0.1, 'constant speed': 0.4, 'idling': 0 , 'break': 0.5 },
        'idling': {'accelerate': 0.8, 'constant speed': 0, 'idling': 0.2 , 'break': 0 },
        'break': {'accelerate': 0.4, 'constant speed': 0.05, 'idling': 0.5 , 'break': 0.05 },
    }
    
    # Define starting probabilities for each state
    starting_probabilities = {'accelerate': 0, 'constant speed': 0, 'idling': 0, 'break': 1}
    
    # Choose the starting state randomly based on the starting probabilities
    current_state = random.choices(
        population=list(starting_probabilities.keys()),
        weights=list(starting_probabilities.values())
    )[0]
    
    # Generate a sequence of states using the transition matrix
    num_iterations = 10
    for i in range(num_iterations):
        print(current_state)
        next_state = random.choices(
            population=list(transition_matrix[current_state].keys()),
            weights=list(transition_matrix[current_state].values())
        )[0]
        current_state = next_state
    

    In this example, we use the random.choices() function to choose the starting state randomly based on the starting probabilities. We then generate a sequence of 10 states using the transition matrix, and print out the sequence of states as they are generated. A sample output of the program is given below.

    >>> exec(open('markov_chain.py').read()) #Python 3 syntax
    break
    idling
    accelerate
    break
    accelerate
    break
    accelerate
    constant speed
    break
    accelerate

    Array pattern multiplication of phased array antennas

    Key focus: Array pattern multiplication: total radiation pattern of N identical antennas is product of single-antenna radiation vector and array factor.

    Antenna arrays

    Ferdinand Braun invented the Phased Array Antenna in 1905. He shared a Nobel Prize in physics in recognition of their contributions to the development of wireless telegraphy.

    An antenna array is a collection of numerous linked antenna elements that operate together to broadcast or receive radio waves as if they were a single antenna. Phased array antennas are used to focus the radiated power towards a particular direction. The angular pattern of the phased array depends on the number of antenna elements, their geometrical arrangement in the array, and relative amplitudes and phases of the array elements.

    Phased array antennas can be used to steer the radiated beam towards a particular direction by adjusting the relative phases of the array elements.

    The basic property of antenna arrays is the translational phase-shift.

    Time-shift property of Fourier transform

    Let’s focus for a moment on the time-shifting property of Fourier transform. The timeshifting property implies that a shift in time corresponds to a phase rotation in the frequency domain.

    \[F \left\{x(t−t_0) \right\}=e^{-j \omega t_0}X(\omega) \quad \quad (1)\]

    Translational phase-shift property

    Now, let’s turn our attention to antenna elements translated/shift in space. Figure 1 depicts a single antenna element having current density J(r) placed at the origin is moved in space to a new location that is l0 distant from the original position. The current density of the antenna element at the new position l0 is given by

    \[J_{l_0}(r) = J(r – l_0) \quad \quad (2)\]
    Figure 1: Current density of antenna element shifted in space

    From the discussion on far-field retarded potentials, the radiation vector F(θ,ɸ) of an antenna element is given by the three dimensional spatial Fourier transform of current density J(z).

    \[\mathbf{F} \left(\mathbf{k} \right) =\int_V J(r)e^{j \mathbf{k} \cdot r } d^3 r, \quad \quad \mathbf{k} = k\hat{r} \quad \quad (3) \]

    Therefore, from equations (2) and (3), the radiation vector of the antenna element space-shifted to new position l0 is given by the space shift property (similar to time-shift property of Fourier transform in equation (1))

    \[\begin{aligned} \mathbf{F}_{l_0} \left(\mathbf{k} \right) &=\int_V J_{l_0}(r)e^{j \mathbf{k} \cdot r } d^3 r \\ &= \int_V J(r-l_0)e^{j \mathbf{k} \cdot r } d^3 r \\ &= \int_V J(r)e^{j \mathbf{k} \cdot (r+l_0) } d^3 r \\ &= e^{j \mathbf{k} l_0}\int_V J(r)e^{j \mathbf{k} \cdot r } d^3 r \\ &= e^{j \mathbf{k} l_0} \mathbf{F} \left(\mathbf{k} \right) \end{aligned} \\ \Rightarrow \boxed{ \mathbf{F}_{l_0} \left(\mathbf{k} \right) = e^{j \mathbf{k} l_0} \mathbf{F}\left(\mathbf{k} \right) } \quad \quad (4) \]

    Note: The sign of exponential in the Fourier transform does not matter (it just indicates phase rotation in opposite direction), as long as the same convention is used throughout the analysis.

    From equation (4), we can conclude that the relative location of the antenna elements with regard to one another causes relative phase changes in the radiation vectors, which can then contribute constructively in certain directions or destructively in others.

    Array factor and array pattern multiplication

    Figure 2 depicts a more generic case of identical antenna elements placed in three dimensional space at various radial distances l0, l1, l2, l3, … and the antenna feed coefficients respectively are a0, a1, a2, a3,…

    Figure 2: Current densities of antenna elements shifted in space – contributors to array factor of phased array antenna

    The current densities of the individual antenna elements are

    \[\begin{aligned} J_{l_0}(r) &= a_0 J(r – l_0) \\ J_{l_1}(r) &= a_1 J(r – l_1) \\ J_{l_2}(r) &= a_2 J(r – l_2) \\ & \vdots \end {aligned} \quad \quad (5)\]

    The total current density of the antenna array structure is

    \[J_{total} = a_0 J(r – l_0) + a_1 J(r – l_1) + a_2 J(r – l_2) + \cdots \quad \quad (6)\]

    Applying the translational phase-shift property in equation (4), the total radiation vector of an N element antenna array is given by

    \[ \begin{aligned} \mathbf{F}_{total} \left(\mathbf{k} \right) &=\mathbf{F}_{l_0} \left(\mathbf{k} \right) + \mathbf{F}_{l_1} \left(\mathbf{k} \right) + \mathbf{F}_{l_2} \left(\mathbf{k} \right) + \cdots \\ &= a_0 e^{j \mathbf{k} l_0} \mathbf{F} \left(\mathbf{k} \right) + a_1 e^{j \mathbf{k} l_1} \mathbf{F} \left(\mathbf{k} \right) + a_2 e^{j \mathbf{k} l_2} \mathbf{F} \left(\mathbf{k} \right) + \cdots\\ &= \mathbf{F} \left(\mathbf{k} \right) \sum_{i=0}^{N} a_i e^{j \mathbf{k} l_i} \\ &= \mathbf{F} \left(\mathbf{k} \right) \mathbf{A} \left(\mathbf{k} \right)\end{aligned} \]
    \[ \boxed{\mathbf{F}_{total} \left(\mathbf{k} \right) = \mathbf{F} \left(\mathbf{k} \right) \mathbf{A} \left(\mathbf{k} \right) \quad \quad (\text{array pattern multiplication}) } \quad \quad (7)\]

    The quantity A(k) is called array factor which incorporates the relative translational phase shifts and the relative feed coefficients of the array elements.

    \[\boxed{\mathbf{A} \left(\mathbf{k} \right) = \sum_{i=0}^{N} a_i e^{j \mathbf{k} l_i} \quad \quad (\text{array factor}) }\quad \quad ,\mathbf{k} = k\hat{r} \quad \quad (8)\]

    or equivalently,

    \[\boxed{\mathbf{AF} \left(\theta, \phi \right) = \sum_{i=0}^{N} a_i e^{j k \left( \hat{\theta} \times \hat{\phi} \right) l_i} \quad \quad (\text{array factor}) } \quad \quad (9)\]

    The array pattern multiplication property states that the total radiation pattern of an antenna array constructed with N identical antennas is the product of radiation vector of a single individual antenna element (also called as element factor) and the array factor.

    Effect of array factor on power gain and radiation intensity

    Let U(θ,ɸ) and G(θ,ɸ) denote the radiation intensity and the power gain patterns of an antenna element. The total radiation intensity and the power gain of an antenna array, constructed with such identical antenna elements, will be modified by the array factor as follows.

    \[\boxed{\begin{aligned} U_{total}\left( \theta, \phi \right) &= \mid A(\theta, \phi)\mid ^2 U\left( \theta, \phi \right)\\ G_{total}\left( \theta, \phi \right) &= \mid A(\theta, \phi)\mid ^2 G\left( \theta, \phi \right) \end{aligned}} \quad \quad (10)\]

    The role of array factor is very similar to that of the transfer function of an linear time invariant system. Recall that if a wide sense stationary process x(t) is input to the LTI system defined by the transfer function H(f), then the power spectral density of the output is given by

    \[S_y(f) = \mid H(f) \mid ^2 S_x(f)\]

    Illustration using a linear array of half-wave dipole antennas

    Linear antennas are electrically thin antennas whose conductor diameter is very small compared to the wavelength of the radiation λ.

    A linear antenna oriented along the z-axis has radiation vector (field) whose components are along the directions of the radial distance and the polar angle. That is, the radiation intensity U(θ,ɸ) and the power gain G(θ,ɸ) depend only on the polar angle θ. In other words, the radiation intensity and the power gain are omnidirectional (independent of azimuthal angle ɸ).

    Figure 3, illustrates an antenna array with linear half-wave dipoles placed on the x-axis at equidistant from each other.

    Figure 3: A linear antenna array with half-wave dipole elements

    We are interested in the power gain pattern G(θ,ɸ) of the antenna array shown in Figure 3.

    The normalized power gain pattern of an individual antenna element (half-wave dipole) is given by

    \[G_{dipole}(\theta, \phi) = \frac{cos^2 (\frac{\pi}{2} cos \theta)}{sin^2 \theta} \quad \quad (11)\]

    From the Figure 4 given in this post, the maximum value for the normalized power gain occurs at θ =90°=π/2 radians, i.e, along the xy plane.

    \[G_{dipole}(\theta = \pi/2, \phi) = 1 \quad \quad (12)\]

    The array factor for the arrangement in Figure 3, computed at θ =90°=π/2 radians is given by

    \[A(\theta = \pi/2, \phi) = a_0 + a_1 e^{(j \frac{2 \pi}{\lambda} \; l\; cos \phi)} + a_2 e^{(j \;2\; \frac{2 \pi}{\lambda} \; l \; cos \phi)} \quad \quad (13)\]

    The total normalized power gain, along the xy plane (θ =90°=π/2 radians), of the array of dipole antennas arranged as given in Figure 3, is given by

    \[G_{total}\left( \theta = \pi/2, \phi \right) = \mid A(\theta = \pi/2, \phi)\mid ^2 G_{dipole}\left( \theta = \pi/2, \phi \right) \quad \quad (14)\]

    Dropping the θ for convenience in representation

    \[G_{total}\left( \phi \right) = \mid A(\phi)\mid ^2 G_{dipole}\left( \phi \right) \quad \quad (15)\]

    Simulation

    Figure 4 illustrates equation (15) – the effect of array factor on normalized power gain of an array of half-wave dipole antennas. The plot is generated for separation distance between antenna elements l=λ and the feed coefficients for the antenna elements a = [1, -1, 1].

    Check out my Google colab for the python code. The results are given below.

    Figure 4: Illustrating the effect of array pattern multiplication on normalized power gain of antenna array

    References

    [1] Orfanidis, S.J. (2013) Electromagnetic Waves and Antennas, Rutgers University. https://www.ece.rutgers.edu/~orfanidi/ewa/

    [2] Constantine A. Balanis, Antenna Theory: Analysis and Design, ISBN: 978-1118642061, Wiley; 4th edition (February 1, 2016)

    BPSK bit error rate simulation in Python & Matlab

    Key focus: Simulate bit error rate performance of Binary Phase Shift Keying (BPSK) modulation over AWGN channel using complex baseband equivalent model in Python & Matlab.

    Why complex baseband equivalent model

    The passband model and equivalent baseband model are fundamental models for simulating a communication system. In the passband model, also called as waveform simulation model, the transmitted signal, channel noise and the received signal are all represented by samples of waveforms. Since every detail of the RF carrier gets simulated, it consumes more memory and time.

    In the case of discrete-time equivalent baseband model, only the value of a symbol at the symbol-sampling time instant is considered. Therefore, it consumes less memory and yields results in a very short span of time when compared to the passband models. Such models operate near zero frequency, suppressing the RF carrier and hence the number of samples required for simulation is greatly reduced. They are more suitable for performance analysis simulations. If the behavior of the system is well understood, the model can be simplified further.

    Passband model and converting it to equivalent complex baseband model is discussed in this article.

    Simulation of bit error rate performance of BPSK using passband simulation model is discussed in this article.

    Figure 1: BPSK constellation

    BPSK constellation

    In binary phase shift keying, all the information gets encoded in the phase of the carrier signal. The BPSK modulator accepts a series of information symbols drawn from the set m {0,1}, modulates them and transmits the modulated symbols over a channel.

    The general expression for generating a M-PSK signal set is given by

    Here, M denotes the modulation order and it defines the number of constellation points in the reference constellation. The value of M depends on the parameter k – the number of bits we wish to squeeze in a single M-PSK symbol. For example if we wish to squeeze in 3 bits (k=3) in one transmit symbol, then M = 2k = 23 = 8 and this results in 8-PSK configuration. M=2 gives BPSK (Binary Phase Shift Keying) configuration. The parameter A is the amplitude scaling factor, fc is the carrier frequency and g(t) is the pulse shape that satisfies orthonormal properties of basis functions.

    Using trigonometric identity, equation (1) can be separated into cosine and sine basis functions as follows

    Therefore, the signaling set {si,sq} or the constellation points for M-PSK modulation is given by,

    For BPSK (M=2), the constellation points on the I-Q plane (Figure 1) are given by

    Simulation methodology

    Note: If you are interested in knowing more about BPSK modulation and demodulation, kindly visit this article.

    In this simulation methodology, there is no need to simulate each and every sample of the BPSK waveform as per equation (1). Only the value of a symbol at the symbol-sampling time instant is considered. The steps for simulation of performance of BPSK over AWGN channel is as follows (Figure 2)

    1. Generate a sequence of random bits of ones and zeros of certain length (Nsym typically set in the order of 10000)
    2. Using the constellation points, map the bits to modulated symbols (For example, bit ‘0’ is mapped to amplitude value A, and bit ‘1’ is mapped to amplitude value -A)
    3. Compute the total power in the sequence of modulated symbols and add noise for the given EbN0 (SNR) value (read this article on how to do this). The noise added symbols are the received symbols at the receiver.
    4. Use thresholding technique, to detect the bits in the receiver. Based on the constellation diagram above, the detector at the receiver has to decide whether the receiver bit is above or below the threshold 0.
    5. Compare the detected bits against the transmitted bits and compute the bit error rate (BER).
    6. Plot the simulated BER against the SNR values and compare it with the theoretical BER curve for BPSK over AWGN (expressions for theoretical BER is available in this article)
    Figure 2: Simulation methodology for performance of BPSK modulation over AWGN channel

    Let’s simulate the performance of BPSK over AWGN channel in Python & Matlab.

    Simulation using Python

    Following standalone code simulates the bit error rate performance of BPSK modulation over AWGN using Python version 3. The results are plotted in Figure 3.

    For more such examples refer the book (available as PDF and paperback) Digital Modulations using Python

    #Eb/N0 Vs BER for BPSK over AWGN (complex baseband model)
    # © Author: Mathuranathan Viswanathan (gaussianwaves.com)
    import numpy as np #for numerical computing
    import matplotlib.pyplot as plt #for plotting functions
    from scipy.special import erfc #erfc/Q function
    
    #---------Input Fields------------------------
    nSym = 10**5 # Number of symbols to transmit
    EbN0dBs = np.arange(start=-4,stop = 13, step = 2) # Eb/N0 range in dB for simulation
    BER_sim = np.zeros(len(EbN0dBs)) # simulated Bit error rates
    
    M=2 #Number of points in BPSK constellation
    m = np.arange(0,M) #all possible input symbols
    A = 1; #amplitude
    constellation = A*np.cos(m/M*2*np.pi)  #reference constellation for BPSK
    
    #------------ Transmitter---------------
    inputSyms = np.random.randint(low=0, high = M, size=nSym) #Random 1's and 0's as input to BPSK modulator
    s = constellation[inputSyms] #modulated symbols
    
    fig, ax1 = plt.subplots(nrows=1,ncols = 1)
    ax1.plot(np.real(constellation),np.imag(constellation),'*')
    
    #----------- Channel --------------
    #Compute power in modulatedSyms and add AWGN noise for given SNRs
    for j,EbN0dB in enumerate(EbN0dBs):
        gamma = 10**(EbN0dB/10) #SNRs to linear scale
        P=sum(abs(s)**2)/len(s) #Actual power in the vector
        N0=P/gamma # Find the noise spectral density
        n = np.sqrt(N0/2)*np.random.standard_normal(s.shape) # computed noise vector
        r = s + n # received signal
        
        #-------------- Receiver ------------
        detectedSyms = (r <= 0).astype(int) #thresolding at value 0
        BER_sim[j] = np.sum(detectedSyms != inputSyms)/nSym #calculate BER
    
    BER_theory = 0.5*erfc(np.sqrt(10**(EbN0dBs/10)))
    
    fig, ax = plt.subplots(nrows=1,ncols = 1)
    ax.semilogy(EbN0dBs,BER_sim,color='r',marker='o',linestyle='',label='BPSK Sim')
    ax.semilogy(EbN0dBs,BER_theory,marker='',linestyle='-',label='BPSK Theory')
    ax.set_xlabel('$E_b/N_0(dB)$');ax.set_ylabel('BER ($P_b$)')
    ax.set_title('Probability of Bit Error for BPSK over AWGN channel')
    ax.set_xlim(-5,13);ax.grid(True);
    ax.legend();plt.show()

    Simulation using Matlab

    Following code simulates the bit error rate performance of BPSK modulation over AWGN using basic installation of Matlab. You will need the add_awgn_noise function that was discussed in this article. The results will be same as Figure 3.

    For more such examples refer the book (available as PDF and paperback) Digital Modulations using Matlab: build simulation models from scratch

    %Eb/N0 Vs BER for BPSK over AWGN (complex baseband model)
    % © Author: Mathuranathan Viswanathan (gaussianwaves.com)
    clearvars; clc;
    %---------Input Fields------------------------
    nSym=10^6;%Number of symbols to transmit
    EbN0dB = -4:2:14; % Eb/N0 range in dB for simulation
    
    BER_sim = zeros(1,length(EbN0dB));%simulated Symbol error rates
        
    M=2; %number of constellation points in BPSK
    m = [0,1];%all possible input bits
    A = 1; %amplitude
    constellation = A*cos(m/M*2*pi);%constellation points
    
    d=floor(M.*rand(1,nSym));%uniform random symbols from 1:M
    s=constellation(d+1);%BPSK modulated symbols
        
    for i=1:length(EbN0dB)
        r  = add_awgn_noise(s,EbN0dB(i));%add AWGN noise
        dCap = (r<=0);%threshold detector
        BER_sim(i) = sum((d~=dCap))/nSym;%SER computation
    end
    
    semilogy(EbN0dB,BER_sim,'-*');
    xlabel('Eb/N0(dB)');ylabel('BER (Pb)');
    title(['Probability of Bit Error for BPSK over AWGN']);

    Rate this article: Note: There is a rating embedded within this post, please visit this post to rate it.

    Reference

    [1] Andrea Goldsmith, “Wireless Communications”, ISBN: 978-0521837163, Cambridge University Press; 1 edition, August 8, 2005.↗

    Books by the author


    Wireless Communication Systems in Matlab
    Second Edition(PDF)

    Note: There is a rating embedded within this post, please visit this post to rate it.
    Checkout Added to cart

    Digital Modulations using Python
    (PDF ebook)

    Note: There is a rating embedded within this post, please visit this post to rate it.
    Checkout Added to cart

    Digital Modulations using Matlab
    (PDF ebook)

    Note: There is a rating embedded within this post, please visit this post to rate it.
    Checkout Added to cart
    Hand-picked Best books on Communication Engineering
    Best books on Signal Processing