Unsupervised Learning

Learning outcomes

After this lesson, you will be able to:

• Explain how unsupervised learning differs from supervised learning.
• Distinguish the feature matrix X from the absence of a predefined target variable y.
• Explain clustering and compare K-means, hierarchical clustering, and DBSCAN.
• Understand the purpose of dimensionality reduction and the role of PCA.
• Describe anomaly detection and the limits of its output.
• Choose initial quality checks and state the limits of discovered structure.

1. What does unsupervised learning mean?

In the previous lesson, we used pairs (X, y): an object's features and a known answer that a model learns to predict. Here, the starting point is usually a feature matrix X without a target variable y to reproduce.

The absence of target labels does not mean the algorithm has no objective. The researcher defines the question: group similar energy-use profiles, compress many measurements into a few components, or find observations that differ sharply from typical behaviour.

Do not confuse this with self-supervised learning. In self-supervised learning, the training signal is created from the data itself: for example, a model may learn to reconstruct a hidden part of an input or predict the next part of a sequence. This approach is widely used for pretraining large models. It is often treated as a separate setting, although terminology varies across the literature. Semi-supervised learning, by contrast, combines a small labeled dataset with a larger unlabeled one.

Imagine operating records from hundreds of buildings: temperature, humidity, energy use, time of day, and equipment mode. An algorithm may discover recurring profiles such as night operation, working days, or peak load. A person assigns meaningful names only after inspecting the groups.

2. Clustering: finding similar observations

The algorithm does not know what “similar” means in your domain. Group buildings by annual energy use and you may get one structure; group them by daily load shape and you may get another. The result depends on selected features, their scale, the distance measure, and algorithm settings.

K-means: groups around centres

K-means requires the number of groups, k, in advance. It places centres, assigns each observation to the nearest centre, recomputes each centre as the mean of its assigned observations, and repeats until assignments stabilise. It is a useful baseline for numeric features, but works best for compact groups of roughly comparable scale.

You must choose k. Different initial centres can produce different solutions, so implementations use multiple initialisations. K-means is sensitive to feature scales and outliers: a feature with large numeric values can dominate the distance calculation.

Hierarchical clustering

Hierarchical methods build a tree of merges or splits. In the agglomerative approach, every observation starts in its own group; nearby groups are then merged step by step. The tree can be inspected at different levels of detail, but the distance metric and linkage rule change the result.

DBSCAN: groups by density

DBSCAN searches for dense regions, can find irregularly shaped groups, and may mark isolated points as noise. It does not require the number of clusters as a direct input, but is sensitive to neighbourhood radius and minimum-neighbour settings. It can struggle when group densities differ substantially.

3. Evaluating clusters without correct labels

There may be no ground-truth labels against which to compare the result directly. Evaluation is therefore more difficult than calculating classification accuracy.

• Internal evaluation. The silhouette coefficient assesses within-cluster compactness and separation from nearby clusters. Values near 1 generally indicate clearer separation, values near 0 suggest overlap, and negative values may indicate poor assignments. It describes geometry, not usefulness.
• Stability. Does the structure recur with a different initialisation, subsample, or small parameter change?
• Domain review. Can group profiles be described using original features? Does the separation make sense for the task? Could it be an artefact of scaling or data errors?
• External evaluation. Independent labels can be used later for evaluation, but should not quietly be passed into training if the result is called fully unsupervised.

4. Dimensionality reduction and PCA

If each object has 80 measurements, it is difficult to see the overall picture. Dimensionality-reduction methods create compact representations for visualisation, noise reduction, or further analysis.

PCA (Principal Component Analysis) creates new axes—principal components—that are linear combinations of the original features. The first component points in the direction of greatest variance; subsequent components explain as much remaining variance as possible while being orthogonal to previous components. PCA does not select the “most important original features” and does not guarantee preservation of rare but important events.

t-SNE and UMAP are often used to visualise complex representations in two dimensions. Distances and group sizes on such maps can depend on parameters and randomness. Do not automatically interpret separated islands as objective classes.

5. Anomaly detection

Examples include an unusual building energy profile, a rare combination of equipment temperature and vibration, or an atypical sequence of user actions. An anomaly is not necessarily an error or a fault: it is a signal to investigate.

Isolation Forest attempts to isolate rare observations through random partitions. Local Outlier Factor compares a point’s local density with the density of its neighbours. Simple threshold rules can also help when physical limits or operating requirements define meaningful boundaries.

Distinguish a data outlier from a meaningful rare event. An unusual value may be a sensor error, an operating-mode change, a real fault, or a new but valid state. Do not automatically delete anomalies; first investigate their origin.

6. Worked example: building operating modes

Suppose we have hourly building records: outdoor temperature, energy use, humidity, occupancy, and time of day. The goal is to understand recurring operating modes—not to predict a predefined class.

1. Frame the question. Find typical hourly or daily profiles.
2. Choose the observation unit. For example, one building-day. Do not mix hourly and daily records without an explicit aggregation rule.
3. Select features. Mean and peak load, night-time consumption, temperature range, and share of operating hours. Each measure needs units and a calculation period.
4. Prepare features. Check missing values, duplicates, time zones, and scales. Scaling is especially important for distance-based methods.
5. Compare methods. Start with K-means, then check whether hierarchical clustering or DBSCAN changes the conclusion.
6. Validate the result. Inspect group profiles, stability, internal metrics, and domain interpretation.
7. State a cautious conclusion. “Three recurring profiles were found in this dataset” is safer than “there are exactly three types of buildings”.

7. Limitations and a review checklist

• An algorithm may find mathematical structure with no useful domain meaning.
• Feature selection, distance, scale, and parameters can change the groups.
• In high dimensions, distances may become less informative; rare structures can be lost during compression.
• A cluster is not necessarily a natural category, and an anomaly is not necessarily an error.
• A two-dimensional visualisation does not prove clusters exist in the original feature space.
• If reliable labels become available and the task becomes predictive, supervised learning may be appropriate—with correct data separation.

8. Python practice: K-means

This example creates synthetic points. The generator also returns teaching labels, but we discard them: the clustering algorithm receives only the feature matrix X.

from sklearn.datasets import make_blobs
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score

# Create synthetic features; discard the generated reference labels.
X, _ = make_blobs(
    n_samples=300,
    centers=3,
    cluster_std=1.2,
    random_state=42
)

X_scaled = StandardScaler().fit_transform(X)
model = KMeans(n_clusters=3, n_init=10, random_state=42)
cluster_ids = model.fit_predict(X_scaled)

# Measure geometric separation, not semantic correctness.
score = silhouette_score(X_scaled, cluster_ids)
print("Silhouette score:", round(score, 3))
print("Cluster IDs:", sorted(set(cluster_ids)))

Explanation: make_blobs creates points; StandardScaler puts features on comparable scales; KMeans searches for three groups; silhouette_score measures geometric separation. Cluster IDs are technical identifiers: cluster 0 is not better or more important than cluster 1.

Environment: Python 3.9+ and a current scikit-learn release. Install with python -m pip install scikit-learn. In real projects, fit transformations only on data permitted for that stage of analysis.