Navigate
Courses Tutorials Practice Pricing Tools Upgrade
Lesson 7 of 7

Self-Supervised and Contrastive Learning

Aurora runs 50,000 home-security cameras. Every night, each camera turns its motion clip into a short numeric fingerprint: total motion, the number of moving blobs (people, pets, headlights, swaying branches), the active seconds, and the brightness. Last night one camera logged clip #201: motion 70, one moving blob, 23 seconds of activity, brightness 54. Was it a family member fetching a parcel, or an intruder? Nobody knows, because nobody has labeled which of the millions of nightly clips are break-ins. Hand-labeling them all is impossible.

Lesson 6 upgraded PCA three ways to reshape the coordinates you score a point against. But every method so far, PCA included, is still hand-built: you pick the transform. This finale asks the machine to learn the representation itself, straight from the unlabeled clips, using a trick with a wonderful name: contrastive learning. Two slightly different views of the same clip should end up close together; two different clips should end up far apart. From that one rule, and no labels at all, a representation emerges in which a genuine intruder finally stands out.

By the end of this lesson you will be able to:

  • Say why anomaly detection is so often label-free, and what a "representation" is and why a good one matters
  • Describe a pretext task: how hiding part of the data and predicting it turns unlabeled data into a learning signal
  • Explain the contrastive idea, positive pairs pulled together and negatives pushed apart, and read the InfoNCE loss
  • Explain representational collapse, and why a good representation must be well-spread and decorrelated
  • Show, in a few lines of base R, that a decorrelated representation makes a relationship-breaking anomaly stand out

Prerequisites: you can read base R (indexing, matrices, scale, cov, lm), and you have done Lesson 4: Autoencoders (scoring a point by how badly it rebuilds, and that a linear autoencoder is PCA), Lesson 6: Kernel PCA, Sparse PCA and NMF (PCA, loadings, variance explained), and Lesson 1: What Is an Anomaly? (the base-rate trap). No deep-learning background is needed; every idea is built here from scratch.

The problem

No labels, and single features lie

Start with what makes this hard. A representation is just a re-description of each clip as a list of numbers, chosen so that distance between two lists means "how different are these two clips." The raw fingerprint is one representation, and a poor one, as you are about to see. We have no labels, so we cannot train a classifier of normal-versus-intruder. We can only learn from the shape of the data itself.

Let us build Aurora's clips and plant one intruder, then check whether any single feature gives it away. Normal clips share a simple habit: a busier night means more motion, more moving blobs, and longer activity, all rising together. The intruder (clip #201) has busy motion and long activity but only one moving blob, a lone figure where a busy clip would show many small blobs.

RInteractive R
set.seed(1) n <- 200 a <- runif(n, 0, 1) # latent "how busy" level of each clip motion <- round(20 + 60*a + rnorm(n, 0, 3), 1) # busier clip -> more total motion blobs <- round( 1 + 8*a + rnorm(n, 0, 0.5)) # busier clip -> more moving blobs active_sec <- round( 4 + 22*a + rnorm(n, 0, 1.5), 1) # busier clip -> longer active time brightness <- round(55 + rnorm(n, 0, 6), 1) # ambient light, unrelated to how busy it is clips <- data.frame(motion, blobs, active_sec, brightness) clips <- rbind(clips, data.frame(motion = 70, blobs = 1, active_sec = 23, brightness = 54)) kind <- c(rep("normal", n), "INTRUDER") # kept only to CHECK our work, never to train tail(clips, 2) #> motion blobs active_sec brightness #> 200 66.1 7 20.8 51.8 #> 201 70.0 1 23.0 54.0 # how extreme is the intruder on each single feature? (z-score against the normal clips) nm <- kind == "normal" z <- sapply(names(clips), function(v) (clips[nrow(clips), v] - mean(clips[nm, v])) / sd(clips[nm, v])) round(z, 2) #> motion blobs active_sec brightness #> 1.16 -1.85 1.26 -0.16

  

  





Look at those z-scores: 1.16, -1.85, 1.26, -0.16. Not one of them is past 2 standard deviations. Every single-feature rule, an amount fence, an IQR whisker, a per-column z-score, waves this clip through as unremarkable. Yet busy motion with a single blob is not how a normal night looks. The tell is in the relationship between the features, exactly the kind of anomaly Lesson 4 caught with reconstruction error. Here we will catch it a different way: by learning a representation in which that broken relationship becomes a long distance.