Ordinary Least Squares Regression: linear regression that has minimum mean squared error(MSE).
Multiple Linear Regression: when we have more than 1 predictor variable
- When we interpret the coefficients from Multiple Linear Regression, we have to say ‘when all else hold’ before interpret any one of the coefficients.
Assumptions for Linear Regression:
- L — Linearity: Relationship between x and y should be approximately linear.
- I — Independence. Observations should not affect one another. (The most common time we’d have to worry about this assumption is when we have time series data, when multiple measurements are made on a subject over time.)
- N — Normality. Residuals should be approximately normally distributed. Normality of residual can be examined using the histography plot of residuals.
- E — Equal variance of errors(homoscedasticity). Residuals should have approximately equal variances for each x. We can visualize this by plot out the residuals using a scatter plot, if there is fanning pattern or the residuals have very wide range of values then it has no equal variance.
- M — No Multicolinearlity ( this one is for Multiple Linear Regression ): multicolinearlity ruins model interpretability and inference, you can’t tell which independent variable should get how much credit for affecting the outcome variable.
What if LINE assumptions are violated?
One quick fix is using log regression.