Understanding Linear Regression-Part 1
Introduction
In machine learning, you will usually work with predicting an outcome , and based on what you are predicting ,you can classify an ML problem into two :
Regression : Predicting a Continuous Value.
Classification : Predicting a Discrete Class.
Before we drill down to linear regression in depth, let me just give you a quick overview of what is regression as linear regression is one of a type of Regression algorithm.
What is Regression ?
Regression analysis is a form of predictive modeling technique which investigates the relationship between a dependent and independent variable.
I hope this article finds its way to your bookmarks! For now, let’s dive deep into it!
Table of Contents
What is meant by Linear Regression?
Working for Simple Linear Regression
What is a Loss function?
What is Gradient Descent?
Overview of Multiple Linear Regression
Assumptions of Linear Regression
What is meant by Linear Regression?
Linear means in a particular line and Regression means a measure of the relationship hence Linear Regression is a linear relationship of the data (independent variable) with the output (target variable).
Types of linear Regression
Simple Linear Regression
Multiple Linear Regression
Working for Simple Linear Regression
Simple Linear Regression is used for finding the relationship between two continuous variables i.e. finding the relationship between the independent variable (predictor) and the dependent variable (response).
Finding the line that best fits the data is the main goal of the Simple Linear Regression procedure. The loss function is minimized to achieve this. The loss function is what? will be covered in this blog post later.
What does ‘m’ denote?
If m > 0, then X (predictor) and Y (target) have a positive relationship. This means the value of Y will increase with an increase in the value of X.
If m < 0, then X (predictor) and Y (target) have a negative relationship. This means the value of Y will decrease with an increase in the value of X
What does ‘c’ denote?
It is the value of Y when X=0. Suppose, if we plot a graph in which the X-axis consists of Years of Experience (independent feature) and Y-axis consists of Salary (dependent feature). For Years of Experience = 0 what will be the Salary, this is what is denoted by ‘c’.
Now that you have understood the theory about the regression line, let’s discuss how can we select the best-fit regression line for a particular model using loss functions.
What is a Loss Function ?
Figure : Simple Linear Regression with Loss
A loss function calculates the difference between the algorithm's predicted output and the expected result. It serves as a measure of how well the model is capturing the patterns in the data. Loss functions generally fall into two categories: those for classification tasks, which involve predicting discrete values (like 0, 1, 2...), and those for regression tasks, which involve predicting continuous values.
Example of Loss functions
The terms cost and loss functions almost refer to the same thing. The cost function is calculated as an average of the loss function. The loss function is a value which is calculated at every instance. So, for a single training cycle loss is calculated numerous times, but the cost function is only calculated once.
Note: For algorithms relying on Gradient descent to optimize model parameters, every loss function selected has to be differentiable.
Consider the loss function to be Mean Squared Error (MSE) in our case. Figure below shows the formula of this loss function where n is the number of samples in the dataset, Y is the actual value and Ŷ is the corresponding predicted value for iᵗʰ data point.
Figure : Formula for Mean Squared Error
We will about Gradient Descent in the next article that is “Understanding linear regression -Part 2 !