There are 3 types of errors that a models can have, error due to bias, error due to variance, and irreducible errors due to randomness or natural variability in an existing system. Linear regression model is usually low variance and high bias.
Bias:
- Underfitting
- How bad your model is at predicting y / low accuracy
- Model is too simple to model the signal
- to reduce bias, we can add more features into the X, increase the model complexity
Variance:
- Overfitting
- How bad your model is at generalizing to new data / high accuracy
- Model is too complicated
- modeling the noise — not the data
The goal of modeling is to minimize the error, by using cross validation, we can score our model, score the model on the train sample and test sample, if the train sample score is higher than test sample, then it means the model is overfitting with high variance; if the train sample score is lower than test sample, then it means the model is underfitting with high bias.
source:GA