AutoML uses unsupervised algorithms for performing an automated process of algorithm selection, hyperparameter tuning, iterative modeling, and model assessment. When your dataset doesn't have a target variable, you can use clustering algorithms to explore it, based on different characteristics. These algorithms group examples together, so that each group will have examples as similar as possible to each other, but dissimilar to examples in other groups.
Since you mostly don't have labels when you are performing such analysis, there is a performance metric that you can use to examine the quality of the resulting separation found by the algorithm.
It is called the Silhouette Coefficient. The Silhouette Coefficient will help you to understand two things:
Cohesion: Similarity within clusters
Separation: Dissimilarity among clusters
It will give you a value between 1 and -1, with values close to 1 indicating well-formed clusters.
Clustering algorithms are used to tackle many different tasks such as finding similar users, songs, or images, detecting key trends and changes in patterns, understanding community structures in social networks.
This tutorial deals with using unsupervised machine learning algorithms for creating machine learning pipelines.
The code files for this article are available on Github.
This article is an excerpt from a book written by Sibanjan Das, Umit Mert Cakmak titled Hands-On Automated Machine Learning.
Commonly used clustering algorithms
There are two types of commonly used clustering algorithms: distance-based and probabilistic models. For example, k-means and Density-Based Spatial Clustering of Applications with Noise (DBSCAN) are distance-based algorithms, whereas the Gaussian mixture model is probabilistic.
Distance-based algorithms may use a variety of distance measures where Euclidean distance metrics are usually used.
Probabilistic algorithms will assume that there is a generative process with a mixture of probability distributions with unknown parameters and the goal is to calculate these parameters from the data.
Since there are many clustering algorithms, picking the right one depends on the characteristics of your data. For example, k-means will work with centroids of clusters and this requires clusters in your data to be evenly sized and convexly shaped. This means that k-means will not work well on elongated clusters or irregularly shaped manifolds. When your clusters in your data are not evenly sized or convexly shaped, you many want to use DBSCAN to cluster areas of any shape.
Knowing a thing or two about your data will bring you closer to finding the right algorithms, but what if you don't know much about your data? Many times when you are performing exploratory analysis, it might be hard to get your head around what's happening. If you find yourself in this kind of situation, an automated unsupervised ML pipeline can help you to understand the characteristics of your data better.
Be careful when you perform this kind of analysis, though; the actions you will take later will be driven by the results you will see and this could quickly send you down the wrong path if you are not cautious.
Creating sample datasets with sklearn
In sklearn, there are some useful ways to create sample datasets for testing algorithms:
# Importing necessary libraries for visualization
import matplotlib.pyplot as plt
import seaborn as sns
# Set context helps you to adjust things like label size, lines and various elements
# Try "notebook", "talk" or "paper" instead of "poster" to see how it changes
sns.set_context('poster')
# set_color_codes will affect how colors such as 'r', 'b', 'g' will be interpreted
sns.set_color_codes()
# Plot keyword arguments will allow you to set things like size or line width to be used in charts.
plot_kwargs = {'s': 10, 'linewidths': 0.1}
import numpy as np
import pandas as pd
# Pprint will better output your variables in console for readability
from pprint import pprint
# Creating sample dataset using sklearn samples_generator
from sklearn.datasets.samples_generator import make_blobs
from sklearn.preprocessing import StandardScaler
# Make blobs will generate isotropic Gaussian blobs
# You can play with arguments like center of blobs, cluster standard deviation
centers = [[2, 1], [-1.5, -1], [1, -1], [-2, 2]]
cluster_std = [0.1, 0.1, 0.1, 0.1]
# Sample data will help you to see your algorithms behavior
X, y = make_blobs(n_samples=1000,
centers=centers,
cluster_std=cluster_std,
random_state=53)
# Plot generated sample data
plt.scatter(X[:, 0], X[:, 1], **plot_kwargs)
plt.show()
We get the following plot from the preceding code:
cluster_std will affect the amount of dispersion. Change it to [0.4, 0.5, 0.6, 0.5] and try again:
cluster_std = [0.4, 0.5, 0.6, 0.5]
X, y = make_blobs(n_samples=1000,
centers=centers,
cluster_std=cluster_std,
random_state=53)
plt.scatter(X[:, 0], X[:, 1], **plot_kwargs)
plt.show()
We get the following plot from the preceding code:
Now it looks more realistic!
Let's write a small class with helpful methods to create unsupervised experiments. First, you will use the fit_predict method to apply one or more clustering algorithms on the sample dataset:
class Unsupervised_AutoML:
def __init__(self, estimators=None, transformers=None):
self.estimators = estimators
self.transformers = transformers
pass
Unsupervised_AutoML class will initialize with a set of estimators and transformers. The second class method will be fit_predict:
def fit_predict(self, X, y=None):
"""
fit_predict will train given estimator(s) and predict cluster membership for each sample
"""
# This dictionary will hold predictions for each estimator
predictions = []
performance_metrics = {}
for estimator in self.estimators:
labels = estimator['estimator'](*estimator['args'], **estimator['kwargs']).fit_predict(X)
estimator['estimator'].n_clusters_ = len(np.unique(labels))
metrics = self._get_cluster_metrics(estimator['estimator'].__name__, estimator['estimator'].n_clusters_, X, labels, y)
predictions.append({estimator['estimator'].__name__: labels})
performance_metrics[estimator['estimator'].__name__] = metrics
self.predictions = predictions
self.performance_metrics = performance_metrics
return predictions, performance_metrics
The fit_predict method uses the _get_cluster_metrics method to get the performance metrics, which is defined in the following code block:
# Printing cluster metrics for given arguments
def _get_cluster_metrics(self, name, n_clusters_, X, pred_labels, true_labels=None):
from sklearn.metrics import homogeneity_score,
completeness_score,
v_measure_score,
adjusted_rand_score,
adjusted_mutual_info_score,
silhouette_score
print("""################## %s metrics #####################""" % name)
if len(np.unique(pred_labels)) >= 2:
silh_co = silhouette_score(X, pred_labels)
if true_labels is not None:
h_score = homogeneity_score(true_labels, pred_labels)
c_score = completeness_score(true_labels, pred_labels)
vm_score = v_measure_score(true_labels, pred_labels)
adj_r_score = adjusted_rand_score(true_labels, pred_labels)
adj_mut_info_score = adjusted_mutual_info_score(true_labels, pred_labels)
metrics = {"Silhouette Coefficient": silh_co,
"Estimated number of clusters": n_clusters_,
"Homogeneity": h_score,
"Completeness": c_score,
"V-measure": vm_score,
"Adjusted Rand Index": adj_r_score,
"Adjusted Mutual Information": adj_mut_info_score}
for k, v in metrics.items():
print("t%s: %0.3f" % (k, v))
return metrics
metrics = {"Silhouette Coefficient": silh_co,
"Estimated number of clusters": n_clusters_}
for k, v in metrics.items():
print("t%s: %0.3f" % (k, v))
return metrics
else:
print("t# of predicted labels is {}, can not produce metrics. n".format(np.unique(pred_labels)))
The _get_cluster_metrics method calculates metrics, such as homogeneity_score, completeness_score, v_measure_score, adjusted_rand_score, adjusted_mutual_info_score, and silhouette_score. These metrics will help you to assess how well the clusters are separated and also measure the similarity within and between clusters.
K-means algorithm in action
You can now apply the KMeans algorithm to see how it works:
from sklearn.cluster import KMeans
estimators = [{'estimator': KMeans, 'args':(), 'kwargs':{'n_clusters': 4}}]
unsupervised_learner = Unsupervised_AutoML(estimators)
You can see the estimators:
unsupervised_learner.estimators
These will output the following:
[{'args': (),
'estimator': sklearn.cluster.k_means_.KMeans,
'kwargs': {'n_clusters': 4}}]
You can now invoke fit_predict to obtain predictions and performance_metrics:
predictions, performance_metrics = unsupervised_learner.fit_predict(X, y)
Metrics will be written to the console:
################## KMeans metrics #####################
Silhouette Coefficient: 0.631
Estimated number of clusters: 4.000
Homogeneity: 0.951
Completeness: 0.951
V-measure: 0.951
Adjusted Rand Index: 0.966
Adjusted Mutual Information: 0.950
You can always print metrics later:
pprint(performance_metrics)
This will output the name of the estimator and its metrics:
{'KMeans': {'Silhouette Coefficient': 0.9280431207593165, 'Estimated number of clusters': 4, 'Homogeneity': 1.0, 'Completeness': 1.0, 'V-measure': 1.0, 'Adjusted Rand Index': 1.0, 'Adjusted Mutual Information': 1.0}}
Let's add another class method to plot the clusters of the given estimator and predicted labels:
# plot_clusters will visualize the clusters given predicted labels
def plot_clusters(self, estimator, X, labels, plot_kwargs):
palette = sns.color_palette('deep', np.unique(labels).max() + 1)
colors = [palette[x] if x >= 0 else (0.0, 0.0, 0.0) for x in labels]
plt.scatter(X[:, 0], X[:, 1], c=colors, **plot_kwargs)
plt.title('{} Clusters'.format(str(estimator.__name__)), fontsize=14)
plt.show()
Let's see the usage:
plot_kwargs = {'s': 12, 'linewidths': 0.1}
unsupervised_learner.plot_clusters(KMeans,
X,
unsupervised_learner.predictions[0]['KMeans'],
plot_kwargs)
You get the following plot from the preceding block:
In this example, clusters are evenly sized and clearly separate from each other but, when you are doing this kind of exploratory analysis, you should try different hyperparameters and examine the results.
You will write a wrapper function later in this article to apply a list of clustering algorithms and their hyperparameters to examine the results. For now, let's see one more example with k-means where it does not work well.
When clusters in your dataset have different statistical properties, such as differences in variance, k-means will fail to identify clusters correctly:
X, y = make_blobs(n_samples=2000, centers=5, cluster_std=[1.7, 0.6, 0.8, 1.0, 1.2], random_state=220)
# Plot sample data
plt.scatter(X[:, 0], X[:, 1], **plot_kwargs)
plt.show()
We get the following plot from the preceding code:
Although this sample dataset is generated with five centers, it's not that obvious and there might be four clusters, as well:
from sklearn.cluster import KMeans
estimators = [{'estimator': KMeans, 'args':(), 'kwargs':{'n_clusters': 4}}]
unsupervised_learner = Unsupervised_AutoML(estimators)
predictions, performance_metrics = unsupervised_learner.fit_predict(X, y)
Metrics in the console are as follows:
################## KMeans metrics #####################
Silhouette Coefficient: 0.549
Estimated number of clusters: 4.000
Homogeneity: 0.729
Completeness: 0.873
V-measure: 0.795
Adjusted Rand Index: 0.702
Adjusted Mutual Information: 0.729
KMeans clusters are plotted as follows:
plot_kwargs = {'s': 12, 'linewidths': 0.1}
unsupervised_learner.plot_clusters(KMeans,
X,
unsupervised_learner.predictions[0]['KMeans'],
plot_kwargs)
We get the following plot from the preceding code:
In this example, points between red (dark gray) and bottom-green clusters (light gray) seem to form one big cluster. K-means is calculating the centroid based on the mean value of points surrounding that centroid. Here, you need to have a different approach.
The DBSCAN algorithm in action
DBSCAN is one of the clustering algorithms that can deal with non-flat geometry and uneven cluster sizes. Let's see what it can do:
from sklearn.cluster import DBSCAN
estimators = [{'estimator': DBSCAN, 'args':(), 'kwargs':{'eps': 0.5}}]
unsupervised_learner = Unsupervised_AutoML(estimators)
predictions, performance_metrics = unsupervised_learner.fit_predict(X, y)
Metrics in the console are as follows:
################## DBSCAN metrics #####################
Silhouette Coefficient: 0.231
Estimated number of clusters: 12.000
Homogeneity: 0.794
Completeness: 0.800
V-measure: 0.797
Adjusted Rand Index: 0.737
Adjusted Mutual Information: 0.792
DBSCAN clusters are plotted as follows:
plot_kwargs = {'s': 12, 'linewidths': 0.1}
unsupervised_learner.plot_clusters(DBSCAN,
X,
unsupervised_learner.predictions[0]['DBSCAN'],
plot_kwargs)
We get the following plot from the preceding code:
Conflict between red (dark gray) and bottom-green (light gray) clusters from the k-means case seems to be gone, but what's interesting here is that some small clusters appeared and some points were not assigned to any cluster based on their distance.
DBSCAN has the eps(epsilon) hyperparameter, which is related to proximity for points to be in same neighborhood; you can play with that parameter to see how the algorithm behaves.
When you are doing this kind of exploratory analysis where you don't know much about the data, visual clues are always important, because metrics can mislead you since not every clustering algorithm can be assessed using similar metrics.
To summarize we learned about many different aspects when it comes to choosing a suitable ML pipeline for a given problem. You gained a better understanding of how unsupervised algorithms may suit your needs for a given problem.
To have a clearer understanding of the different aspects of automated Machine Learning, and how to incorporate automation tasks using practical datasets, check out this book Hands-On Automated Machine Learning.
Read more
Google announces Cloud TPUs on the Cloud Machine Learning Engine (ML Engine)
How machine learning as a service is transforming cloud
Selecting Statistical-based Features in Machine Learning application
Read more
United States
Great Britain
India
Germany
France
Canada
Russia
Spain
Brazil
Australia
South Africa
Thailand
Ukraine
Switzerland
Slovakia
Luxembourg
Hungary
Romania
Denmark
Ireland
Estonia
Belgium
Italy
Finland
Cyprus
Lithuania
Latvia
Malta
Netherlands
Portugal
Slovenia
Sweden
Argentina
Colombia
Ecuador
Indonesia
Mexico
New Zealand
Norway
South Korea
Taiwan
Turkey
Czechia
Austria
Greece
Isle of Man
Bulgaria
Japan
Philippines
Poland
Singapore
Egypt
Chile
Malaysia
