Interaction and Spline Features
In Lesson 3 you reshaped a single feature's distribution, taming skew so a model could use it. This lesson reshapes something different: the model's flexibility. We hand a plain linear model the power to bend.
Meet our running example. A small ice-cream cart on a seaside promenade writes down, for each day, the high temperature, whether it was a weekend, and how many cups it sold. Sales climb as the day warms up, peak around a perfect beach afternoon near 28C, then fall when it gets too hot to stand in line. That shape is a curve, and a straight line cannot follow it.
Drag the line in the chart below. Each dot is a day: warmer to the right, more cups sold higher up. However you tilt the line, it misses. The red squares (each one a squared error) never shrink away, because the relationship bends and a line does not.
By the end of this lesson you will be able to:
- Build an interaction so one feature's effect can depend on another
- Use polynomials and splines to fit a curved relationship with
lm() - Add just enough flexibility to capture the shape without overfitting
Prerequisites: you can fit lm() and read a coefficient (OLS Regression from Scratch), and you have met dummy-coding a factor (Encoding Categorical Variables).
Why a linear model is "straight"
Suppose we just hand the cart's two features to lm(). The fitted equation looks like this:
\( \widehat{\text{sales}} = \beta_0 + \beta_1\,\text{temp} + \beta_2\,\text{weekend} \)
Reading the symbols: \(\widehat{\text{sales}}\) is the predicted number of cups; \(\text{temp}\) is the day's temperature; \(\text{weekend}\) is a 0/1 dummy (1 on weekends, 0 otherwise); \(\beta_0\) is the intercept; \(\beta_1\) is cups per extra degree; \(\beta_2\) is the flat weekend bump. "Linear" means the prediction is a weighted sum of the columns, that is, linear in the coefficients \(\beta\).
That innocent-looking equation makes two rigid promises:
- One straight slope. \(\beta_1\) is a single number, so every extra degree adds the same cups whether it is 14C or 34C. No hump allowed.
- A parallel shift. Weekend only adds a flat \(\beta_2\); the temperature slope is identical on weekdays and weekends, so the two groups can never differ in how strongly heat drives sales.
Real life breaks both promises, and there is a clean fix for each. Fix 1, interactions: let one feature's slope depend on another. Fix 2, basis expansion: let a single feature's effect curve. The rest of the lesson is those two moves.