Day 9: Multiple Linear Regression

If is linearly dependent only on , then we can use the ordinary least square regression line, . However, if shows linear dependency on variables , , , , then we need to find the values of and other constants (). We can then write the regression equation as:

Matrix Form of the Regression Equation

Let's consider that depends on two variables, and . We write the regression relation as . Consider the following matrix operation:

We define two matrices,

and

Now, we rewrite the regression relation as . This transforms the regression relation into matrix form.

Generalized Matrix Form

We will consider that shows a linear relationship with variables, , , , . Let's say that we made observations on different tuples :

Now, we can find the matrices:

Finding the Matrix B

We know that

Note: is the transpose matrix of , is the inverse matrix of , and is the identity matrix.

Finding the Value of

Suppose we want to find the value of for some tuple , then,

Example

Consider shows a linear relationship with and :

Now, we can define the matrices:

Now, find the value of :

So, , which means , , and .

Let's find the value of at

Multiple Regression in R

x1 = c(5, 6, 7, 8, 9)
x2 = c(7, 6, 4, 5, 6)
y = c(10, 20, 60, 40, 50)
m = lm(y ~ x1 + x2)
show(m)

Running the above code produces the following output:

Call:
lm(formula = y ~ x1 + x2)

Coefficients:
(Intercept)           x1           x2  
     51.953        6.651      -11.163

Multiple Regression in Python

from sklearn import linear_model
x = [[5, 7], [6, 6], [7, 4], [8, 5], [9, 6]]
y = [10, 20, 60, 40, 50]
lm = linear_model.LinearRegression()
lm.fit(x, y)
a = lm.intercept_
b = lm.coef_
print a, b[0], b[1]