The frequency domain and the periodogram

Today let's understand the frequency domain and the periodogram, using ten weeks of a real restaurant's lunch covers as the running example.

Northside Diner is a lunch restaurant. Every day for ten weeks, seventy days in all, it logged how many covers it served, covers being the restaurant term for parties served a meal. Here is the whole seventy days, in the order they happened.

The line rises and falls, and it does that roughly once a week. That rise and fall is what this lesson is about: not just noticing that a series repeats, but naming the frequency it repeats at, and finding that frequency directly from the data with a tool called a periodogram.

Meet Northside Diner's ten weeks of lunch covers

Before anything else, build that series and look at its numbers directly, since every step from here on works with this same data.

RInteractive R
# Build ten weeks of Northside Diner's daily lunch covers set.seed(2024) day <- 1:70 baseline <- 120 fundamental <- 30 * cos(2 * pi * (day - 5) / 7) harmonic <- 12 * cos(4 * pi * (day - 5) / 7) noise <- rnorm(70, 0, 10) northside_covers <- round(baseline + fundamental + harmonic + noise) range(northside_covers) round(mean(northside_covers), 1) round(sd(northside_covers), 1) #> [1] 68 178 #> [1] 119.2 #> [1] 25.4

  

The covers range from 68 on the slowest day to 178 on the busiest, averaging 119.2 a day with a standard deviation of 25.4. That's a lot of day-to-day swing for a restaurant that plans its staff and its prep roughly the same way every week.

Group those seventy days by weekday and average each group, and the swing turns out not to be random at all.

RInteractive R
# Average covers by day of week, Monday through Sunday library(ggplot2) day_names <- c("Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun") weekday <- factor(day_names[((day - 1) %% 7) + 1], levels = day_names) dow_means <- round(tapply(northside_covers, weekday, mean), 1) dow_means #> Mon Tue Wed Thu Fri Sat Sun #> 100.1 103.6 97.1 131.6 162.1 137.7 102.2 dow_df <- data.frame(weekday = factor(day_names, levels = day_names), covers = as.numeric(dow_means)) ggplot(dow_df, aes(weekday, covers)) + geom_col(fill = "#1f7a55") + labs(x = "day of week", y = "average covers")

  

Wednesday is the slowest day, 97.1 covers on average. Friday is the busiest by far, 162.1, with Saturday close behind at 137.7. That's a real, repeating weekly pattern, the same shape week after week, not just a few Fridays that happened to be busy.

Saying "there's a weekly pattern" is a start, but it isn't precise. The next step gives that pattern a name that can actually be measured.

A repeating wave has a frequency, a period, and an amplitude

A repeating pattern like Northside Diner's weekly rise and fall can be written as a wave: a curve that goes up, comes back down, and repeats the same shape over and over. Three numbers describe any such wave completely.

The period is how many days pass before the wave repeats itself exactly. The frequency is how many times it repeats in one day, which is just 1 divided by the period. And the amplitude is how far the wave swings above, and below, its centre, not the level it sits at.

Pull the 7-day wave out of Northside Diner's series on its own, with no noise and no second wave mixed in, and those three numbers become concrete.

RInteractive R
# Isolate the 7-day wave alone, with no noise and no second wave wave_7day <- 30 * cos(2 * pi * (day - 5) / 7) round(head(wave_7day, 7), 2) #> [1] -27.03 -27.03 -6.68 18.70 30.00 18.70 -6.68 wave_df <- data.frame(day = day, wave = wave_7day) wave_peak <- data.frame(day = 5, wave = 30) ggplot(wave_df, aes(day, wave)) + geom_line(color = "#1f7a55", linewidth = 1) + geom_point(data = wave_peak, aes(day, wave), color = "#b04a52", size = 3) + geom_hline(yintercept = c(30, -30), linetype = "dashed", color = "grey60") + labs(x = "day", y = "covers above or below centre")

  

This wave takes 7 days to repeat, so its period is 7 days. Its frequency is 1/7, about 0.142857 cycles a day. And its amplitude is 30: the red point marks day 5, where the wave sits exactly 30 covers above its centre, and by the same symmetry it swings 30 covers below centre exactly half a cycle later, the dashed lines marking that ceiling and floor.

This one wave, on its own, is already most of the shape behind Northside Diner's Friday peak. But it isn't the whole shape. The next step adds a second wave on top of it.

Building the real shape by summing two waves

Northside Diner's real Friday peak is sharper than one 7-day wave alone can produce, and its early-week dip is flatter. That's because a second, smaller wave rides on top of the first one, a wave that repeats every 3.5 days instead of every 7.

Toggle between line and point below to see all three curves that build Northside Diner's real shape: the 7-day wave, the 3.5-day wave, and their sum.

The 3.5-day wave has amplitude 12, a little under half the 7-day wave's 30, and it completes two full cycles inside every seven days, twice the 7-day wave's frequency. That makes it what's called a harmonic of the 7-day wave. Added together, the two waves peak on the same day, Friday: 30 from the 7-day wave plus 12 from the 3.5-day wave, 42 in total, a sharper peak than the 7-day wave gives on its own.

This is the whole idea behind splitting a series into waves: any repeating shape, however complicated, can be built by adding together enough sine and cosine waves at the right frequencies and amplitudes. Northside Diner's real series needs only two.

Quick check: frequency, period and amplitude

The widget above also carries the 3.5-day wave, amplitude 12, completing one full cycle every 3.5 days.

Right. Frequency is always 1 divided by the period: 1 / 3.5 = 2/7, about 0.286 cycles a day, double the 7-day wave's 1/7. And amplitude is the swing away from centre in each direction, never a level the series sits at.
Amplitude is always the swing above and below centre, never an average: this wave pushes covers 12 higher at its peak and 12 lower at its trough. And frequency is 1 divided by the period, not the period itself: a 3.5-day period gives 1 / 3.5 = 2/7, about 0.286 cycles a day, double the 7-day wave's 1/7.

spec.pgram() turns a noisy series into a list of frequencies

Northside Diner's real series isn't two clean waves. It's those two waves plus random noise, day-to-day variation with no pattern at all. Picking frequencies out of a series like that by eye, the way the last two steps did, doesn't work once noise is mixed in. What's needed is a tool that checks every candidate frequency at once and reports how much of the series' variation each one explains.

That tool is called a periodogram, and base R builds one with a function called spec.pgram(). Feed it Northside Diner's actual covers, noise included, and see what it finds.

RInteractive R
# Turn the noisy covers series into a periodogram: one ordinate per candidate frequency pg <- spec.pgram(ts(northside_covers), taper = 0, detrend = FALSE, demean = TRUE, fast = FALSE, plot = FALSE) length(pg$freq) #> [1] 35 top6 <- order(pg$spec, decreasing = TRUE)[1:6] data.frame( frequency = round(pg$freq[top6], 6), period = round(1 / pg$freq[top6], 3), ordinate = round(pg$spec[top6], 1) ) #> frequency period ordinate #> 1 0.142857 7.000 14997.4 #> 2 0.285714 3.500 3576.7 #> 3 0.442857 2.258 345.8 #> 4 0.071429 14.000 334.1 #> 5 0.271429 3.684 307.6 #> 6 0.185714 5.385 269.2

  

spec.pgram() checked 35 candidate frequencies, one for every k/70 from 1/70 up to 0.5. A series sampled once a day can never show a cycle faster than once every two days, which is why the frequencies stop at 0.5. The arguments above turn off every adjustment spec.pgram() knows how to make: taper = 0 and detrend = FALSE skip its usual tapering and trend removal, demean = TRUE only centres the series, and fast = FALSE keeps the frequency grid exactly at k/70. What comes back is a raw periodogram, untouched by any smoothing. For each frequency it returns a number called an ordinate: how strongly the series moves in step with a wave of exactly that frequency. The taller the ordinate, the more of the series' variation that frequency explains.

Two ordinates tower over the rest. Frequency 0.142857, which is 1/7, the exact frequency of the 7-day wave built into this series, has ordinate 14,997.4. Frequency 0.285714, 2/7, the exact frequency of the 3.5-day wave, has ordinate 3,576.7, a good deal smaller but still far ahead of anything else. Every other ordinate in the table sits under 350.

That's the periodogram working as intended: both real frequencies, 1/7 and 2/7, come straight out of a noisy series, with no need to know in advance where to look.

Away from the two real peaks, the periodogram is jagged

Those two tall ordinates are only 2 of the 35 the periodogram computed. Read the other 33 on their own, away from frequencies 1/7 and 2/7, and a different picture shows up: no smooth curve, no gentle slope down from the peaks, just a scatter of small numbers jumping around with no obvious pattern.

The two real peaks are left out of the chart below on purpose, since they already had their moment in the last step: the taller of the two alone stands over forty times higher than the tallest point on this floor, so on the same chart it would flatten the floor down to a line hugging zero. Toggle between line and point to see the floor's raggedness both ways.

That's genuinely what a raw periodogram looks like away from a real cycle: no smooth decline, no story, just noise. And this isn't a sign that anything went wrong. Northside Diner's series was built from exactly two waves plus noise, nothing else, and the periodogram still came back this jagged everywhere except at those two frequencies. The next step explains why that jaggedness never goes away, no matter how much data you add.

Why a raw ordinate does not get quieter with more data

A natural guess: an ordinate looks noisy with 70 days of data, so collecting more data, say 700 days instead of 70, should calm it down, the way a sample mean gets more precise with a bigger sample. Test that guess directly. Rebuild Northside Diner's series 400 times over, with fresh random noise each time, at 70 days and then again at 700 days, and watch what happens to one ordinate with no real cycle behind it and one that has a real cycle behind it.

RInteractive R
# Rebuild the same diner series many times, and see whether a periodogram ordinate calms down simulate_ordinates <- function(n, reps, seed, floor_freq = 0.30, peak_freq = 1 / 7) { set.seed(seed) d <- 1:n base_f <- 30 * cos(2 * pi * (d - 5) / 7) base_h <- 12 * cos(4 * pi * (d - 5) / 7) floor_vals <- numeric(reps) peak_vals <- numeric(reps) for (i in 1:reps) { series <- round(120 + base_f + base_h + rnorm(n, 0, 10)) p <- spec.pgram(ts(series), taper = 0, detrend = FALSE, demean = TRUE, fast = FALSE, plot = FALSE) floor_vals[i] <- p$spec[which.min(abs(p$freq - floor_freq))] peak_vals[i] <- p$spec[which.min(abs(p$freq - peak_freq))] } c(floor_mean = mean(floor_vals), floor_sd = sd(floor_vals), peak_mean = mean(peak_vals), peak_sd = sd(peak_vals)) } r70 <- simulate_ordinates(70, 400, 11) r700 <- simulate_ordinates(700, 400, 11) round(r70, 1) round(r700, 1) #> floor_mean floor_sd peak_mean peak_sd #> 99.0 95.0 15792.8 1821.0 #> floor_mean floor_sd peak_mean peak_sd #> 94.7 91.6 157602.8 5622.0

  

Look at the floor ordinate first, the one near frequency 0.30 where nothing real repeats. At 70 days, across the 400 reruns, it averages 99.0 with a standard deviation of 95.0, almost as large as the mean itself, about 96%. At 700 days, ten times the data, it averages 94.7 with a standard deviation of 91.6, still about 97% of the mean. Ten times the data bought this ordinate nothing: it's exactly as unreliable at 700 days as it was at 70.

Now look at the real peak, the ordinate at frequency 1/7. At 70 days it averages 15,792.8 with a standard deviation of 1,821.0, about 11.5% of the mean. At 700 days it averages 157,602.8 with a standard deviation of 5,622.0, about 3.6% of the mean. This one did settle down. More data made it far more reliable.

So the same fix, more data, works for a real cycle and does nothing for noise. That's because of how spec.pgram() works: more days don't hand any single ordinate more information about itself, they just add more ordinates to the list, one for every new k/n. A real cycle's ordinate keeps concentrating the same total power onto fewer and fewer neighbouring frequencies as n grows, and that concentration is what settles it down. A floor ordinate has no such concentration to fall back on. It's just noise, computed the same jittery way no matter how long the series runs.

Key Insight
A raw periodogram ordinate away from a real cycle does not get more reliable with more data. Only an ordinate that sits on a genuine cycle does.

Closing quiz: reading and trusting a periodogram

A different series' periodogram comes back with one tall ordinate at frequency 0.25, and every other ordinate low and jagged, the same way Northside Diner's floor looked two steps back.

Right. A tall ordinate marks a real frequency, and its period is always 1 divided by that frequency: 1 / 0.25 = 4. The ragged floor around it is exactly what a raw periodogram looks like away from a real cycle, proven two steps back, not a symptom to fix with more data.
Frequency and period are reciprocals, never the same number. Frequency 0.25 means the cycle repeats every 1 / 0.25 = 4 time units, not every 0.25 of one. And a jagged floor isn't a data shortage: the earlier replicate simulation showed ten times the data left a floor ordinate exactly as noisy, so a ragged floor next to one tall spike is the periodogram working correctly, not a warning sign.

Your turn: read a period off a periodogram peak

pg, from a few steps back, still holds all 35 frequencies and their ordinates, in pg$freq and pg$spec. Find the biggest ordinate with which.max(), read off its frequency, then turn that frequency into a period.

Right: frequency 0.142857, period 7, the same 7-day cycle the periodogram found back in the spec.pgram() step.Index pg$freq at which.max(pg$spec) to get the frequency, then divide 1 by that same expression to get the period: pg$freq[which.max(pg$spec)] and 1 / pg$freq[which.max(pg$spec)].
Show answer
# Find the frequency of the biggest ordinate, then its period
which.max(pg$spec)
pg$freq[which.max(pg$spec)]
1 / pg$freq[which.max(pg$spec)]
#> [1] 10
#> [1] 0.1428571
#> [1] 7

References

Quick recap

You can now look at a repeating series and name what's really driving the repetition, not just eyeball it. To summarize:

  • Any repeating series can be written as a sum of waves, each with a frequency (cycles per day), a period (days per cycle) and an amplitude (the swing above and below centre).
  • spec.pgram() turns a series into one ordinate per candidate frequency, and the tallest ordinates point straight at the real cycles: 1/7 and 2/7 here.
  • Away from a real cycle, an ordinate's relative noise does not shrink with more data. It stayed at about 96% to 97% of the mean whether the series ran 70 days or 700.

Next, you'll smooth the periodogram on purpose, trading away some of that noise for a steadier picture of the spectrum.