Kernel PCA, Sparse PCA and NMF

Lesson 5 hunted anomalies inside a time series by first stripping out its daily and weekly rhythm, so that a quiet Tuesday could finally look quiet. Now we come back to the idea from Lesson 4: score a point by how badly a model rebuilds it. A linear autoencoder, you saw, is just plain old PCA: squeeze each customer down to a few numbers, rebuild them, and flag whoever rebuilds worst. It works, but the subspace PCA hands you is a blunt instrument. It only draws straight axes, it spreads every feature across every component, and it happily uses negative weights.

Meet Bean Box, a coffee-subscription startup. It wants to understand its customers and flag the odd account, and this lesson upgrades PCA three ways to help it: bend the axes into curves (kernel PCA), force each component to name only a few features (sparse PCA), and rebuild data from additive parts that can never go negative (NMF).

By the end of this lesson you will be able to:

• Say why plain PCA misses curved structure, and build kernel PCA from an RBF kernel matrix
• Turn PCA's dense loadings into a handful you can actually name, with a sparsity penalty
• Decompose a table of counts into additive, parts-based components with NMF
• Pick the right one of the three for a given data shape and anomaly-detection goal

Prerequisites: you can read base R (indexing, matrices), and you have done Lesson 4: Autoencoders (scoring a point by how badly it rebuilds, and that a linear autoencoder is PCA) and Lesson 1: What Is an Anomaly?. Plain PCA, its loadings and variance explained are recapped in the next step.

Plain PCA, and three places it falls short

Principal Component Analysis finds new axes, the principal components, ordered so the first captures the most spread in the data, the second the most of what is left, and so on. The recipe for the first component is: find the direction \(w\) (a weight per feature) that maximises the variance of the projected data \(\operatorname{Var}(Xw)\), subject to \(\lVert w \rVert = 1\) so the weights cannot just blow up. Those weights \(w\) are the loadings: they tell you how much each feature contributes to that component.

Let us give Bean Box six flavour scores for 200 customers, built so one hidden taste, how dark-roast a customer likes, drives bitter, roast and body together, while acidity, fruity and sweet mostly do their own thing. Run this first; it is the data the plain-PCA and sparse-PCA steps both use.

That single component already tells a story: bitter, roast and body carry the big weights, so PC1 is essentially the dark-roast axis. But look closely, and three cracks appear, one for each tool in this lesson.

We take them in that order.