Model Cards and Documenting a Model

Lessons 1 to 5 taught you to interrogate a model: which features moved a prediction (SHAP), the shape of each effect (partial dependence), and whether the model treats groups equally (fairness). You now know a great deal about one model. But everything you learned lives in your head and your notebook. The moment someone else picks up your model, that knowledge is gone.

This last lesson fixes that. A model card is a short, standard document that travels with a model and answers the questions every future user will have: what is this model for, what was it trained on, how well does it work (and for whom), and where does it break?

We will build one for a real, running example: Willow Creek Clinic trained a model that flags adult inpatients at risk of being readmitted to hospital within 30 days, so nurses can prioritise follow-up calls. The card below is what you will produce. Toggle between Source, the plain text you write, and Rendered, the finished card.

By the end of this lesson you will be able to:

• Name the six sections of a model card and say what each one documents
• Explain why an undocumented model gets misused, and what a card carries across the handoff
• Report performance the honest way: measured, and broken down by subgroup
• Document the factors a model leans on and where its training data may not represent its users
• Write a card whose numbers are computed, not typed, and state its known limits plainly

Prerequisites: you can fit and use a model in R and read a predict() output (for example a random forest), and you have done Lesson 5: Fairness Basics, where you computed per-group rates like recall.

A model with no label is a loaded tool

Think of a bottle of pills. Before anyone takes one, the label tells them the dose, who it is for, what it treats, and the side effects to watch for. Ship the same pills in a blank bottle and someone will take the wrong dose for the wrong reason. A trained model is exactly this: powerful, and dangerous without a label.

Here is how the danger plays out. The Willow Creek team that built the readmission model knows things that never made it into the model file: it was trained on adult inpatients at this hospital over the last two years. Six months later a colleague finds the model, sees it predicts readmission, and runs it on the emergency-room walk-ins at a different site. It returns confident numbers. They are quietly wrong, and nobody gets an error message.