Tidy temporal data with tsibble

Today let's understand tsibble, the data structure R uses for time series, with a real example.

Riverside Bike Share runs two rental kiosks, one at Riverside and one Downtown. Here is each kiosk's daily rentals for June and July 2024, 61 days, one dot per day.

Riverside's dots run all the way across, one for every day. Downtown's row of dots breaks three times, on June 11, 18 and 25. That break is the whole problem this lesson solves: a plain data frame cannot tell you it has a gap, and a tsibble can.

A data frame that happens to have a time column

Before tsibble can help, look at what an ordinary data frame gives you for free, and what it does not.

Riverside Bike Share logs one row per kiosk per day: which kiosk, which date, how many rentals. Build that log for June and July 2024, 61 days at each kiosk, then look at its shape.

That's 119 rows: 61 for Riverside and 58 for Downtown. Downtown is short because its payment terminal failed to restart on three Tuesdays and logged nothing those days.

But bike_long is a plain data frame. R treats date as just another column, exactly like site or rentals. Nothing about it says the rows are in time order, or that the 61 Riverside rows and 58 Downtown rows form two separate ongoing series. You could shuffle every row in bike_long and R would not object, even though shuffling a time series destroys the one thing that makes it a time series: the order the values arrived in.

The gap is easy to miss too. Nothing in bike_long flags that Downtown's log is short compared to Riverside's, or which dates are missing. You would have to go looking for it by hand. That is exactly what tsibble is built to catch.

The tsibble: an index and a key, and the trip back

A tsibble, short for tidy time series table, is a data frame that knows two extra things: which column is time, and which column separates one series from another.

Convert bike_long into a tsibble with as_tsibble(), telling it that date is the index, the time column, and site is the key, the column that separates Riverside's rows from Downtown's.

The header now reads 119 x 3 [1D], 119 rows and 3 columns, same as before, but [1D] is new: tsibble worked out that every kiosk-day is exactly 1 day apart, and it will use that fact from here on. Key: site [2] says the 119 rows split into 2 separate series by site.

Underneath, bike_tsbl is still an ordinary tibble; as_tsibble() only adds the index and key on top of it. You can always take them back off with as_tibble().

That is exactly the class of a plain tibble: no index, no key, no interval. Whatever you do to a tsibble mid-analysis, the trip back to an ordinary data frame is one function call away.

Regular vs irregular time intervals

The [1D] you just saw is called the interval, and tsibble computes it straight from the index column: the gap between one kiosk-day and the next. Every kiosk-day in bike_tsbl sits exactly 1 day after the last one for that site, so the interval is a fixed 1 day, and tsibble calls that a regular tsibble.

Not every time-stamped log works that way. Riverside Bike Share also keeps a repair log: one row every time a technician actually visits a kiosk, whenever something breaks. There is no fixed schedule to it.

The header shows [!] instead of a day count. That ! is tsibble's way of saying there is no fixed interval to check: one visit on June 5 at 09:15, another the same day at 14:40, then nothing again until June 19. regular = FALSE tells as_tsibble() not to even try computing one.

That distinction matters for what comes next. A regular tsibble like bike_tsbl has a fixed interval to check every row against, so tsibble can tell you exactly which dates are missing. An irregular tsibble like maint_tsbl has no fixed interval to check against, so the idea of a missing date does not even apply to it.

Implicit gaps: a missing row, not a missing value

Downtown's 3 outage days never became rows in bike_long at all. Nobody wrote NA for June 11; the row for June 11 simply does not exist. tsibble calls that an implicit gap: a gap the regular interval implies but the data does not show.

Because bike_tsbl is regular, with a fixed 1-day interval, tsibble can check every site's rows against that interval and tell you exactly where a day is missing. has_gaps() does the check, one row per key.

Downtown's .gaps reads TRUE, Riverside's reads FALSE. count_gaps() goes further and lists the missing dates themselves: three separate 1-day gaps, all Downtown, all in June, the 11th, the 18th and the 25th. Those are exactly the three Tuesdays its payment terminal failed to restart.

Plot both kiosks' daily rentals, one dot per day and colored by site, and the three gaps count_gaps() just listed show up as three missing dots in Downtown's row.

Riverside's dots run unbroken across all 61 days. Downtown's stop exactly at June 11, 18 and 25, the same three dates count_gaps() found.

fill_gaps(): making the missing dates explicit

An implicit gap is easy to miss and awkward to work with: you cannot filter, plot or model a row that is not there. fill_gaps() fixes that by inserting a real row for every date count_gaps() found, with NA in every other column, turning the implicit gap into an explicit NA.

bike_filled has 122 rows now: the original 119 plus the 3 Downtown rows fill_gaps() just added, one for June 11, 18 and 25. Their rentals value is NA, not 0.

That distinction matters. The terminal outage means Downtown's rental count for those three days is unknown, not zero. Riverside Bike Share may well have handed out bikes at Downtown that day; the terminal just never recorded them. Writing 0 would claim something you do not know to be true, and it would drag down any total or average that includes those days. NA says, correctly, that the value is unknown, and every function you run on rentals from here on has to decide, on purpose, what to do with it.

Downtown's row count before and after fill_gaps() shows the fix in one picture.

fill_gaps() brought Downtown level with Riverside: both kiosks now have exactly 61 rows, one per day, June 1 through July 31.

Quick check: gap or NA?

Before fill_gaps() runs, Downtown's June 11 row is missing from bike_tsbl entirely, and count_gaps() lists it as a gap. What does fill_gaps() do to fix it, and what value does the new row get?

index_by(): collapsing daily rentals into monthly totals

With every date accounted for, roll the daily rows up into a coarser total. index_by() does for the time index what group_by() does for an ordinary column: it groups rows by whatever time expression you give it, ready for summarise(). yearmonth(date) turns each date into its calendar month.

bike_filled had 122 daily rows across 2 kiosks and 2 months; bike_monthly collapses that down to 4 rows, one per kiosk per month.

na.rm = TRUE inside sum() is doing real work here. Without it, Downtown's June total would itself come back NA, because 3 of June's 30 rows for Downtown are the NAs fill_gaps() added. na.rm = TRUE tells sum() to add up the days it does know and skip the ones it does not.

Here is that same 4-row result as a report-ready table.

Why fable needs a complete, regular tsibble

tsibble's gap-checking is not just tidiness. fable, the forecasting package that fits models directly on a tsibble, refuses to fit a model until every implied date is present.

Try fitting an ETS model, a common forecasting model, on Downtown's series before its gaps are filled.

model() prints its result as a mable, short for model table: one row per key, with a model column holding the fit. Here fable will not guess past a hole in the series, so it refuses outright and names the exact problem, implicit gaps, and the exact fix, fill_gaps(). The model column shows <NULL model>: no model was fit.

Fit the identical model on the filled series instead.

Same model, same call, no error this time: ETS(A,N,N), a model fable was able to fit.

The reason is that fable places every forecast date by counting forward from the tsibble's own interval, day 62, day 63 and so on. If a day inside the series is silently missing, that count is off from the very first forecast date onward. fill_gaps() is not a formality before modelling; it is what keeps that day count accurate, from the very first forecast date onward.

Quick check: the tsibble pipeline

Parkside Library logs daily visitor counts, with some days missing because the front desk scanner malfunctioned. You want monthly visitor totals with the gaps handled properly. Which order gets you there?

Your turn: fill the gaps and aggregate

bike_tsbl still holds the 119-row tsibble, before its gaps are filled. Insert Downtown's missing dates with fill_gaps(), calling the result bike_filled. Then roll bike_filled up to one total per kiosk per month with index_by() and summarise(), remembering na.rm = TRUE.

# Close the gaps, then roll the filled series up to monthly totals
bike_filled <- fill_gaps(bike_tsbl)

bike_monthly <- bike_filled %>%
  index_by(month = yearmonth(date)) %>%
  group_by(site) %>%
  summarise(total_rentals = sum(rentals, na.rm = TRUE), .groups = "drop")

bike_monthly
#> # A tsibble: 4 x 3 [1M]
#> # Key:       site [2]
#>   site         month total_rentals
#>   <chr>        <mth>         <dbl>
#> 1 Downtown  2024 Jun          2696
#> 2 Downtown  2024 Jul          3072
#> 3 Riverside 2024 Jun          4306
#> 4 Riverside 2024 Jul          4475

References

• tsibble objects, Forecasting: Principles and Practice - Hyndman and Athanasopoulos (2021), 3rd edition. The index/key structure this lesson builds by hand.
• tsibble package reference - as_tsibble(), fill_gaps(), index_by() documentation.
• Wang, E., Cook, D., and Hyndman, R.J. (2020), "A New Tidy Data Structure to Support Exploration and Modeling of Temporal Data," Journal of Computational and Graphical Statistics. The design paper behind index and key.
• fable package reference - model()'s gap-checking behavior this lesson demonstrates.

Quick recap

as_tsibble(index =, key =) marks which column is time and which column separates one series from another. A regular interval turns a missing date into a gap tsibble can actually check for, with has_gaps() and count_gaps(). fill_gaps() turns that gap into an explicit NA, never a guessed number. index_by() and summarise() roll the daily rows up to a coarser total once na.rm = TRUE is told what to do with those NAs. And fable's model() only fits once every implied day is present, because that is how it places each forecast date correctly.

Next, you will read what a time series actually says before any model touches it: its level, trend, seasonal pattern, cycles and noise, off three real series.