Logistic regression classifier

Recall in the last chapter, we trained the tree-based models only based on the first 100,000 samples out of 40 million. We did so because training a tree on a large dataset is extremely computationally expensive and time consuming. Since we are now not limited to algorithms directly taking in categorical features thanks to one-hot encoding, we should turn to a new algorithm with high scalability to large datasets. Logistic regression is one of the most scalable classification algorithms.

Getting started with the logistic function

Let's start with introducing the logistic function (which is more commonly called sigmoid function) as the algorithm core before we dive into the algorithm itself. It basically maps an input to an output of values between 0 and 1. And it is defined as follows:

We can visualize it as follows:

First define the logistic function:

>>> import numpy as np
>>> def sigmoid(input):
...     return 1.0 / (1 + np.exp(-input))

Input...