Autoencoders for Anomaly Detection

Every detector so far scored a point by asking a version of the same question: where does this point sit? The isolation forest asked how few cuts fence it off; LOF asked how sparse its neighbourhood is; the one-class SVM asked which side of a boundary it lands on. This lesson asks something genuinely different. Not "where is the point?" but "can a model that has only ever seen normal data even rebuild it?"

That is the idea behind an autoencoder: squeeze each point through a narrow bottleneck and rebuild it. Trained on normal charges, it rebuilds normal charges almost perfectly, but it cannot rebuild a point that does not fit the pattern. The size of the rebuilding mistake, the reconstruction error, becomes the anomaly score. Right through the lesson we follow one card, belonging to a shopper named Maya, whose charges track a simple rule of about \$8.50 per item. A \$78 charge for just 5 items looks ordinary on each number alone, yet it is impossible for her, and reconstruction error is exactly what catches it. And in the simplest (linear) case this turns out to be exactly PCA, so you can build a working detector in a few lines of base R.

By the end of this lesson you will be able to:

• Describe the autoencoder pipeline: encode to a bottleneck, decode, and score by reconstruction error
• Explain why reconstruction error catches a charge that is normal on every single feature but breaks the relationship between features
• Use the linear-autoencoder = PCA fact: reconstruction error is the distance to the subspace normal data lives on
• Compute reconstruction error in R with prcomp and turn it into an anomaly flag
• Name the assumptions that make it work: clean-ish training data, a bottleneck narrower than the input, and feature scaling

Prerequisites: you can run R and read base R (scale, prcomp, rowSums, indexing), and you have done Lesson 1: What Is an Anomaly? (global vs local outliers, the base-rate trap) and Lesson 3: LOF and One-Class SVM (novelty detection: learn normal once, judge new points). No PCA background is needed; the one fact we use is built up here.

A compressor that only knows normal

Picture Maya's normal spending as a habit. Her charges follow a pattern: a bigger basket costs more, roughly a fixed price per item. Because of that pattern, you barely need two numbers to describe a normal charge. Tell me the basket size and I can guess the amount; the pattern fills in the rest. A normal charge is almost one number of real information dressed up as two.

An autoencoder turns that observation into a machine, in three parts:

1. The encoder squeezes each charge down to a tiny summary, its bottleneck code (here, a single number).
2. The bottleneck is deliberately too narrow to store everything. It can only keep the dominant pattern, not the fine detail or the noise.
3. The decoder rebuilds the full charge from that summary alone.

You train the whole thing on normal charges only, tuning the encoder and decoder so the rebuilt charge comes back as close to the original as possible. Because the bottleneck is a squeeze, the network is forced to spend its tiny budget on the pattern that normal charges share. The gap between the original charge and its rebuild is the reconstruction error, and that gap is the whole game.