Maximum likelihood, from the curve up
Today let's work out how to find the one value of an unknown number, hidden somewhere in a process, that explains the data you already collected better than any other value could.
Mia has a coin she suspects is weighted, and she wants to find out by how much. She flips it 40 times and counts the heads: 12 of them. That's 30% heads, 70% tails.
The coin has one unknown number attached to it, the true chance it lands heads, the kind of number statisticians call a parameter. Call it p. Mia doesn't know p. All she has is this one set of 40 flips and the 12 heads inside it.
So here's the question this lesson answers: given that data, 12 heads in 40 flips, which value of p explains it best? Not guessed, not assumed. Found.
It only takes three moves to get there.
That's the whole shape of it. Everything from here works through those three moves on Mia's coin, slowly enough to see why each one works.
Scoring one candidate value against the data
Let's try something simple first. Pick a value for p and ask: if that value were true, how probable would Mia's actual data, 12 heads out of 40 flips, be?
R already has a function for exactly this. dbinom(k, n, p) returns the probability of getting exactly k successes in n independent trials, each with success probability p. Mia's data is k = 12 heads out of n = 40 flips.
Let's score three candidate values of p: 0.1, 0.3 and 0.6.
Look at those three numbers. At p = 0.1, the data you actually saw, 12 heads, would be a near-impossible outcome: a score of 0.000292. At p = 0.6 it's even less likely: 0.000088. But at p = 0.3, the exact same data scores 0.136574: hundreds of times higher than 0.1's score, and well over a thousand times higher than 0.6's.
So 0.3 explains Mia's 40 flips far better than 0.1 or 0.6 does. That score, dbinom(12, 40, p) worked out for one fixed value of p with the data held fixed, is called the likelihood of that value of p. It's a score for how well a candidate explains data you already have. It is not a probability that the candidate is the true value, and that distinction is worth holding onto.