Robust regression

In this section, we are going to consider two solutions that can be employed when the dataset contains outliers. Unfortunately, a linear regression is very sensitive to them because the coefficients are forced to minimize the squared error and hence, the hyperplane is forced to move closer to the outliers (which yield a higher error). However, in the majority of real-life applications, we expect a good ability to discriminate between points belonging to data-generating processes and outliers. The algorithms presented in this section have been designed to mitigate this very problem.

RANSAC

A common problem with linear regressions is caused by the presence of outliers. An ordinary least- square approach will take them into account and the result (in terms of coefficients) will be therefore biased. In the following graph there's an example of such a behavior:

Example of a dataset containing outliers

The shallower-sloped line represents an acceptable regression that discards the...