Gradient boosting
The adaptive boosting method can't be applied to the regression problem since it is constructed to address the classification problem. The gradient boosting method can be used for both the classification and regression problems with suitable loss functions. In fact, the use of gradient boosting methods goes beyond these two standard problems. The technique originated from some of Breiman's observations and developed into regression problems by Freidman (2000). We will take the rudimentary code explanation in the next section without even laying out the algorithm. After the setup is clear, we will formally state the boosting algorithm for the squared-error loss function in the following subsection and create a new function implementing the algorithm.
The following diagram is a depiction of the standard sine wave function. It is clearly a nonlinear relationship. Without explicitly using sine transformations, we will see the use of the boosting algorithm to learn this function...
