Unlike methods like K-Means where you must specify the number of clusters beforehand, hierarchical clustering builds a tree of clusters without that requirement. The result is not a single set of groups, but a multi-level hierarchy where clusters at one level are merged or split to form clusters at the next level. This gives you a much richer view of your data's structure, allowing you to see relationships at various scales.

This method is particularly useful when you don't have a strong assumption about the number of natural groupings in your data, or when the underlying structure is itself hierarchical. Think of the animal kingdom, with its species, genus, family, and so on. A hierarchical approach is perfect for uncovering such relationships. There are two primary ways to build this hierarchy.

Hierarchical clustering can be approached from two directions: from the bottom up or from the top down.

Agglomerative Clustering is the bottom-up approach. It starts by treating each data point as its own cluster. Then, in each step, it merges the two closest clusters until only one cluster remains, which contains all the data points. This is the more common and conceptually simpler of the two methods.

Divisive Clustering is the top-down approach. It's the reverse of agglomerative clustering. It starts with a single cluster containing all data points. In each step, it splits a cluster into two until every data point is its own cluster. Divisive methods are more computationally complex because they have to consider all possible ways to split a cluster at each step.

The output of a hierarchical clustering algorithm is typically visualized as a tree-like diagram called a dendrogram. This diagram doesn't just show the final clusters; it illustrates the entire merging or splitting process.

In a dendrogram, the individual data points are the leaves of the tree. As you move up from the leaves, branches merge at nodes. The height of the node represents the distance or dissimilarity between the clusters being merged. A taller branch indicates that the two clusters being joined are less similar to each other.

By looking at a dendrogram, you can understand the data's structure at a glance. Cutting the tree at a certain height will give you a specific number of clusters. For example, a horizontal cut that crosses three branches would partition the data into three clusters.

For the algorithm to decide which clusters to merge (or split), it needs rules. Two key components define these rules: a distance metric and a linkage criterion.

A distance metric determines how the similarity or dissimilarity between two individual data points is calculated. You might recall Euclidean distance from geometry, which is a common choice. It's the straight-line distance between two points.

For two points $p = (p_1, p_2, ..., p_n)$ and $q = (q_1, q_2, ..., q_n)$, the Euclidean distance is:

A linkage criterion defines how the distance between clusters (which can contain multiple points) is measured. Since a cluster is a set of points, we need a rule to decide how to calculate the distance from one set to another. Should we measure the distance between the closest points in the two clusters? The farthest points? The average distance? Each of these choices represents a different linkage criterion and can lead to different cluster structures.

Let's check your understanding of these foundational ideas.

Understanding these core components—agglomerative vs. divisive approaches, the role of dendrograms, and the importance of distance and linkage—is the first step to mastering hierarchical clustering.