Mastering Vectors in Machine Learning: "A linear Algebra Approach"

INTRODUCTION

I established the foundation in my earlier piece Linear Algebra Introduction for AI/ML ,by going over key ideas like matrices and vectors and emphasizing their importance in data representation.

Since vectors are essential to comprehending machine learning algorithms, we will be concentrating on them today. The characteristics and functions of vectors will be discussed in this article, along with some real-world machine learning applications. I strongly advise you to read the introduction if you haven't already in order to lay a strong basis. Let's get started and become experts at how vectors are changing machine learning!

What is a Vector?

In simple terms, a vector is a list of numbers. These numbers (or elements) represent a position in space. Formally, a vector can be seen as an ordered tuple of values:

\(\mathbf{v} = \begin{pmatrix} v_1 \\ v_2 \\ v_3 \end{pmatrix} \)

For example, in machine learning, if you have a dataset of housing prices with features such as size, number of rooms, and age of the house, each house (data point) can be represented as a vector:

\(\mathbf{v} = \begin{pmatrix} \text{size} \\ \text{number of rooms} \\ \text{age} \end{pmatrix} \)

Operations on Vectors

1. Vector Addition

Two vectors of the same dimension can be added together by adding their corresponding elements:

\(\mathbf{v} + \mathbf{w} = \begin{pmatrix} v_1 \\ v_2 \\ v_3 \end{pmatrix} + \begin{pmatrix} w_1 \\ w_2 \\ w_3 \end{pmatrix} = \begin{pmatrix} v_1 + w_1 \\ v_2 + w_2 \\ v_3 + w_3 \end{pmatrix} \)

Python code example :

#import numpy library
import numpy as np

# Using 2D vectors here
v = np.array([1,2])
w = np.array([4,-6])
V_Plus_W = v+w

# print out all three vectors
print(v)
print(w)
print(V_Plus_W)

# Note that here we don't need to worry about vector orientation (row vs. column), 
# so for simplicity the vectors are created orientationless.

2. Norm of a Vector

The norm of a vector represents its magnitude (or length). For a vector v the most commonly used norm is the Euclidean norm, which is defined as:

\(\| \mathbf{v} \| = \sqrt{v_1^2 + v_2^2 + v_3^2} \)

The norm is often used in machine learning to regularize models and reduce overfitting (e.g., L2 regularization in linear regression).

Vectors in Machine Learning

1. Representing Features and Data Points

In machine learning, vectors are used to represent both features and data points. A dataset of n data points, each with m features, can be represented as an n×m matrix, where each row is a data point (vector) and each column is a feature.

For example, consider a dataset for house prices with three features: size, number of rooms, and age. If we have five houses, the feature vectors can be represented as:

\(X = \begin{pmatrix} \text{size}_1 & \text{rooms}_1 & \text{age}_1 \\ \text{size}_2 & \text{rooms}_2 & \text{age}_2 \\ \vdots & \vdots & \vdots \\ \text{size}_5 & \text{rooms}_5 & \text{age}_5 \end{pmatrix} \)

2. Gradient Descent

Gradient descent is a fundamental optimization technique in machine learning that mainly depends on vector operations.

\(\theta = \theta - \alpha \nabla J(\theta) \)

Here, θ represents the parameter vector, α is the learning rate, and ∇J(θ) is the gradient vector of the loss function.

Conclusion

Machine learning algorithms can represent, handle, and optimize data in high-dimensional environments thanks to vectors, which are important building blocks in the field. Vectors are utilized in a variety of sophisticated techniques such as PCA and SVMs, as well as feature representation and optimization (gradient descent) that enable models to learn from data.

Anyone looking to go deeper into the mathematical underpinnings of machine learning algorithms must have a solid understanding of vector operations and their function in machine learning. With linear algebra as a major tool, machine learning practitioners can utilize.

For more insights about Vector application , get time and watch this video Mathematics for Machine Learning.