AI-Assisted Analysis

In Lesson 2, Priya, the analyst at the Inkwell Books chain, learned to lead with the answer. This Monday she is stuck on a different problem. A new feedback form has produced about 4,000 free-text customer reviews across the six stores, things like "Great books, painfully slow checkout." There is no star rating, just sentences. She cannot read 4,000 of them before the morning meeting.

A large language model (LLM), a program trained on huge amounts of text that, given an instruction, writes a sensible text response, can read every review in about a minute. It can summarize the themes and tag each review as positive, neutral, or negative. The question this whole lesson answers is the hard one: where is that genuinely useful, and where will the model quietly make something up?

By the end of this lesson you will be able to:

• Use an LLM from R to summarize free text and label each row, with the ellmer package
• Force structured output so the model hands back a tidy column, not a paragraph
• Ground the model in your real data with tools, so it stops guessing numbers
• Decide, task by task, when to trust an LLM and when to compute the answer in R instead

Prerequisites: you can run R and load a package with library(), you know the dplyr verbs filter(), count() and summarise() from The dplyr Verbs, and you have a reproducible report from Lesson 1. Every new term is defined as it appears.

The widget below is where we are heading: an LLM reasoning over Priya's store data, one step at a time, to answer a real question. Press Step to watch. We spend the lesson earning the right to trust it.

An LLM is a text-to-text function

Strip away the hype and an LLM is one thing: a function whose input is text and whose output is text. You hand it an instruction, called a prompt, and it returns words. Under the hood it works in tokens (short chunks of text, roughly a few characters each) and predicts the next likely token over and over. You do not need that machinery to use it; you need to know what it is good at.

For a data analyst, two jobs fit it almost perfectly, and both turn messy text into something you can use:

• Summarize: many rows of text in, one short paragraph out. "Here are 4,000 reviews; give me the three most common complaints."
• Label (classify): one row of text in, one category out. "Is this single review positive, neutral, or negative?" Run it over every row and you have a new column.

Notice what these have in common. Both take language, the thing spreadsheets and mean() cannot touch, and hand back something tidy: a paragraph you can read, or a category you can count. That is the whole reason an LLM earns a place next to dplyr. Keep one thing in mind from the start, though: it is reading and judging text, not doing arithmetic. We will lean on that distinction for the rest of the lesson.