Support Vector Regression

SVMs can also be efficiently employed for regression tasks. However, it's necessary to consider a slightly different loss function that can take into account the maximum discrepancy between prediction and the target value. The most common choice is the ε-insensitive loss (which we've already seen in passive-aggressive regression):

In this case, we consider the problem as one of a standard SVM where the separating hyperplane and the (soft) margins are built sequentially to minimize the prediction error. In the following diagram, there's a schema representing this process:

Example of Support Vector Regression; the empty circles represent two support vectors

The goal is to find the optimal parameters so that all predictions lie inside the margins (which are controlled by parameter ε). This condition minimized the ε-insensitive loss and guarantees a tolerable absolute error between target values and predictions. However, as already discussed for classification problems...