# K-Means Clustering: How It Works With a Worked Example

K-means clustering is an unsupervised algorithm that splits data into $k$ groups by placing a center (centroid) in each group and assigning every point to its nearest center [1]. It repeats two moves, assign points then recompute centers, until the centers stop moving [1]. This article explains the mechanics, walks through a full worked example on 30 customers, and shows how to read the output.

## Quick Answer

- K-means clustering partitions $n$ data points into $k$ clusters, where you choose $k$ in advance [2].
- Each cluster is defined by a centroid, the mean position of the points assigned to it [2].
- A point belongs to the cluster whose centroid is closest in Euclidean distance [2].
- The algorithm alternates between assigning points and updating centroids until assignments stop changing or a stopping rule fires [1].
- It scales as $O(nk)$, which makes it fast on large datasets compared with hierarchical methods that look at all pairs of points [1].

## What K-Means Clustering Means

In plain terms, k-means clustering is a way to sort unlabeled data into a fixed number of similar groups. You give it a table of features and a number $k$, and it returns $k$ group centers plus a group label for every row [2].

The precise definition: k-means is an unsupervised partitioning method that seeks $k$ centroids and a label $c^{(i)}$ for each data point $x^{(i)}$, minimizing the total squared Euclidean distance between each point and its assigned centroid [2]. Because no labels $y^{(i)}$ are supplied, this is an unsupervised learning problem [2]. The objective it minimizes is called inertia, or the within-cluster sum of squares [3].

## How It Works

The algorithm minimizes this objective:

$$\sum_{i=1}^{n} \lVert x^{(i)} - \mu_{c^{(i)}} \rVert^2$$

Each symbol means the following. $n$ is the number of data points. $x^{(i)}$ is the feature vector for point $i$. $c^{(i)}$ is the cluster assigned to point $i$. $\mu_{c^{(i)}}$ is the centroid of that cluster. The double bars denote Euclidean distance, so the term inside the sum is the squared distance from a point to its own centroid [2].

The procedure runs in these steps:

1. Choose $k$, the number of clusters [1].
2. Pick initial centroids, often by sampling points or by a seeding method such as k-means++ [3].
3. Assign each point to the nearest centroid, producing $k$ initial clusters [1].
4. Recompute each centroid as the mean of the points now assigned to it [1].
5. Repeat steps 3 and 4 until the centroids stop moving or a stopping criterion is met [1].

Because initialization is random, runs can differ. Running the algorithm several times and keeping the best result by a quality metric is the standard remedy [1]. The scikit-learn implementation exposes this through the `n_init` parameter, which controls how many times the algorithm is run with different centroid seeds [4].

## Worked Example

The dataset holds 30 customer records with two features: annual spend in USD and visits per month. We set $k = 3$.

| annual_spend | visits_per_month | annual_spend | visits_per_month | annual_spend | visits_per_month |
|---|---|---|---|---|---|
| 120 | 1 | 340 | 4 | 600 | 8 |
| 145 | 1 | 360 | 5 | 640 | 8 |
| 160 | 2 | 380 | 5 | 680 | 9 |
| 180 | 2 | 400 | 5 | 720 | 9 |
| 210 | 2 | 420 | 6 | 760 | 10 |
| 230 | 3 | 450 | 6 | 810 | 10 |
| 250 | 3 | 480 | 6 | 860 | 11 |
| 270 | 3 | 510 | 7 | 920 | 12 |
| 300 | 4 | 540 | 7 | 980 | 13 |
| 320 | 4 | 570 | 7 | 1050 | 14 |

The run uses deterministic, evenly spaced starting centroids by spend: (120.0, 1.0), (450.0, 6.0), and (1050.0, 14.0). After the assignment and update steps converge, the clusters come out as follows.

| Cluster | Size | Centroid (mean spend, mean visits) |
|---|---|---|
| 0 | 12 | (240.4167, 2.8333) |
| 1 | 11 | (515.4545, 6.7273) |
| 2 | 7 | (871.4286, 11.2857) |

The within-cluster sum of squares, or inertia, is 254834.6353. This is a local minimum. Running `KMeans(n_clusters=3, n_init=10, random_state=0)` on the same data finds a slightly tighter split of 13, 10 and 7 points with inertia 252637.7352. The point labels, in dataset order, are:

```
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2]
```

Here is the code that produces this result.

```python
import pandas as pd
from sklearn.cluster import KMeans

df = pd.DataFrame({'annual_spend': [...], 'visits_per_month': [...]})
km = KMeans(n_clusters=3, init=[[120, 1], [450, 6], [1050, 14]], n_init=1).fit(df)
print('centroids =', km.cluster_centers_.round(4).tolist())
print('labels =', km.labels_.tolist())
print('inertia =', round(km.inertia_, 4))
```

Output:

```
centroids = [[240.4167, 2.8333], [515.4545, 6.7273], [871.4286, 11.2857]]
labels = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2]
inertia = 254834.6353
```

The scatter plot of these 30 customers, colored by cluster with centroids marked as black X's, shows three bands running from low spend and few visits to high spend and many visits.

## How to Interpret It

Read the centroids first. Cluster 0 averages 240.42 in spend and 2.83 visits per month, so it collects the low-engagement customers. Cluster 1 sits at 515.45 and 6.73, a middle group. Cluster 2 averages 871.43 and 11.29, the high-value segment. The labels array tells you which cluster each row belongs to, in the same order as the input rows.

Inertia measures how tight the clusters are. Lower is better for the same $k$, but inertia always falls as $k$ rises, so it cannot be compared across different values of $k$ on its own. Use it to compare runs at the same $k$, for example to check whether a different initialization found a tighter solution [1].

Cluster sizes matter too. Here the split is 12, 11, and 7. Uneven sizes are normal and not a defect, though very small clusters can signal that $k$ is too high or that outliers are pulling centroids away from the main mass of data [5].

## When to Use It (and when not to)

Use k-means when your data forms roughly circular, similar-sized groups and you have a reason to pick $k$ up front [1]. It works well on large tables because it scales as $O(nk)$ [1]. It is a common first step for customer segmentation, document grouping, and image compression.

Avoid it when clusters are elongated, nested, or defined by density instead of distance. K-means effectively treats data as a set of roughly circular distributions, and real data with outliers or density-based clusters may not match that assumption [1]. It also struggles as dimensions grow, because Euclidean distances become inflated and less able to separate examples [6][5]. If you need a method that handles arbitrary shapes, look at density-based or spectral approaches [5].

## K-Means vs K-Nearest Neighbors

These two names sound alike but solve different problems. K-means is unsupervised and finds groups. K-nearest neighbors is supervised and predicts a label for a new point by looking at its closest labeled neighbors. If you want the supervised counterpart explained, see [K-Nearest Neighbors (KNN): Algorithm and Examples](/blog/data-analysis/k-nearest-neighbors).

| Aspect | K-Means | K-Nearest Neighbors |
|---|---|---|
| Learning type | Unsupervised [2] | Supervised |
| Input labels | None needed [2] | Required |
| Meaning of $k$ | Number of clusters [1] | Number of neighbors to vote |
| Output | Centroids and cluster labels [2] | Predicted class or value |
| Main use | Grouping unlabeled data [2] | Classification and regression |

## Common Mistakes

- Choosing $k$ by habit instead of evidence. Fix: try several values and compare a quality metric such as inertia or silhouette analysis, which scikit-learn demonstrates for KMeans [4].
- Running the algorithm once and trusting the result. Fix: run multiple initializations and keep the best outcome, since random initialization can produce varying results [1].
- Forgetting to scale features. Fix: standardize columns so a feature measured in large units, like annual spend here, does not dominate the distance calculation.
- Leaving outliers in place. Fix: remove or clip outliers before fitting, because centroids can be dragged by outliers or outliers may form their own cluster [5].
- Comparing inertia across different values of $k$. Fix: only compare inertia at the same $k$, and use another criterion when choosing $k$.
- Assuming clusters are real groups. Fix: treat clusters as a summary of distance structure and validate them against domain knowledge, since k-means assumes roughly circular distributions [1].

## Limitations

K-means cannot tell you the right number of clusters. You supply $k$, and the algorithm will happily return that many groups even if the data has no natural grouping [1]. It also gives no probability or confidence for an assignment, so a point near a boundary is treated the same as one at a centroid.

The method degrades in high dimensions. As the number of dimensions rises, pairwise distances become more similar and k-means loses the ability to distinguish examples, a problem known as the curse of dimensionality [5]. Reducing dimensions with PCA before clustering can help [5]. K-means also assumes clusters are isotropic with similar variance, and its speed advantage shrinks if you must restart it many times to avoid a poor local minimum [6].

## Frequently Asked Questions

### How do I choose the number of clusters k?

There is no single rule. Fit the model at several values of $k$ and compare a quality metric, then check whether the resulting groups make sense for your problem. Scikit-learn documents silhouette analysis as one way to select the number of clusters for KMeans [4]. Domain knowledge should have the final say.

### Does k-means always find the same clusters?

No. Random initialization means different runs can land in different local minima and produce different results [1]. Running the algorithm multiple times and selecting the best outcome by a quality metric is the recommended practice [1]. Setting a fixed random state makes a run reproducible.

### What does inertia actually measure?

Inertia is the within-cluster sum of squares, the total squared distance from each point to its assigned centroid [3]. Smaller values mean tighter clusters. It always decreases as $k$ increases, so compare it only across runs that use the same $k$.

### Can k-means handle categorical data?

Not directly, because it relies on Euclidean distance between numeric feature vectors [2]. You would need to encode categories numerically or choose a different algorithm suited to mixed data types. Distance-based methods generally assume a meaningful numeric distance between points.

### What is k-means++ and why does it matter?

K-means++ is a seeding method that picks better initial centroids than plain random selection [3]. Better seeds reduce the chance of a poor local minimum and often mean fewer restarts are needed [5]. It can also be called independently to select seeds for other clustering algorithms [3].

If you are new to the wider toolkit, [Python for Machine Learning: A Beginner's Guide](/blog/data-analysis/python-for-machine-learning-guide) covers the setup used in the example above, and [Introduction To Statistical Learning](/blog/guides/introduction-to-statistical-learning) places clustering alongside supervised methods. For a broader view of grouping techniques, see [Multivariate Analysis: Definition, Methods and Examples](/blog/data-analysis/multivariate-analysis) and [Clustering Algorithms for Single-Cell Proteomics: A Comparison of PhenoGraph, FlowSOM, and k-Means](/knowledge/bioinformatics/clustering-algorithms-for-single-cell-proteomics-a-comparison-of-phenograph-flowsom-and-k-means).

## References

1. [What is k-means clustering? | Machine Learning | Google for Developers](https://developers.google.com/machine-learning/clustering/kmeans/overview)
2. [CS221](https://stanford.edu/~cpiech/cs221/handouts/kmeans.html)
3. [2.3. Clustering, scikit-learn 1.9.1 documentation](https://scikit-learn.org/stable/modules/clustering.html)
4. [KMeans, scikit-learn 1.9.1 documentation](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html)
5. [Advantages and disadvantages of k-means | Machine Learning | Google for Developers](https://developers.google.com/machine-learning/clustering/kmeans/advantages-disadvantages)
6. [Demonstration of k-means assumptions, scikit-learn 1.9.1 documentation](https://scikit-learn.org/stable/auto_examples/cluster/plot_kmeans_assumptions.html)

## Further Reading

- [K-Means Cluster Analysis | Columbia Public Health | Columbia University Mailman School of Public Health](https://www.publichealth.columbia.edu/research/population-health-methods/k-means-cluster-analysis)

## Related Articles

- [K-Nearest Neighbors (KNN): Algorithm and Examples](/blog/data-analysis/k-nearest-neighbors)
- [Python for Machine Learning: A Beginner's Guide](/blog/data-analysis/python-for-machine-learning-guide)
- [Support Vector Machines (SVM): Definition and Examples](/blog/data-analysis/support-vector-machines-svm)
- [Bivariate Data: Definition, Examples and Analysis](/blog/data-analysis/bivariate-data-definition-examples)
- [Multivariate Analysis: Definition, Methods and Examples](/blog/data-analysis/multivariate-analysis)
- [Introduction To Statistical Learning](/blog/guides/introduction-to-statistical-learning)
- [Clustering Algorithms for Single-Cell Proteomics: A Comparison of PhenoGraph, FlowSOM, and k-Means](/knowledge/bioinformatics/clustering-algorithms-for-single-cell-proteomics-a-comparison-of-phenograph-flowsom-and-k-means)
- [Fundamental Statistics: Core Concepts Explained](/blog/research-skills/fundamental-statistics-core-concepts-explained)