Chapter 9. Regression and Regularization
We selected binary logistic regression to introduce the basics of machine learning in the Kicking the tires section of Chapter 1, Getting Started. The purpose was to illustrate the concept of discriminative classification. It is important to keep in mind that some regression algorithms, such as logistic regression, are classification models.
The variety and the number of regression models go well beyond the ubiquitous ordinary least square linear regression and logistic regression [9:1]. Have you heard of isotonic regression?
The purpose of regression is to minimize a loss function, the residual sum of squares (RSS) being one that is commonly used. The Accessing a model section in Chapter 2, Data Pipelines, introduced the thorny challenge of overfitting, which will be partially addressed in this chapter by adding a penalty term to the loss function. The penalty term is an element of the larger concept of regularization.
The chapter starts with a description...
