The worksheet
Use it with your team
The meeting guide, with room to write in every box, followed by a page on the research behind it. Fill it in on screen or print it.
Download the three questions on one printable page (PDF) to keep on the meeting table.
Five steps
Select a step to see what the team does there.
Step 1
Name the decision this work has to inform.
Who makes it, and when? If no decision is waiting on this work, write that down too.
Step 2
What matters most to know right now?
Write the one question the team most needs answered before that decision. If you have several, keep the one whose answer would change the decision.
Coming soon A clear question before the data, a workbook that walks your team through this step. Email me to hear when it’s ready.
Step 3
What do we need to keep learning over time?
List the measures you will repeat, collected the same way each time. Note when you will record the baseline, before anything changes.
Step 4
Which data are requirement-driven rather than value-driven?
List what you already collect or report. Mark why you collect it, and whether it helps answer the question in step 2. For each required measure that doesn’t help, name one thing you could show next to it.
Step 5
Agree what you will do with the answer.
Before anyone opens the data, write what the team will do if the answer points one way, and what it will do if it points the other. Then name who looks first and when the team meets again.
What the research says
Open a line to see the research behind it.
Start with the decision the work has to inform.
Patton puts the focus of utilization-focused evaluation “on intended use by intended users.” Naming the decision, who makes it and when tells the team who will use the work and for what.
- Patton, M. Q. (2012). Essentials of utilization-focused evaluation (p. 4). SAGE. Further reading
A clear question comes before the data.
A RAND synthesis of its studies of data use in schools notes that some describe educators as “drowning” in data, and recommends training on how to formulate questions before working with results. Coburn and Turner’s review of research on data use opens with the lesson that data “are only as good as how they are used.”
- Marsh, J. A., Pane, J. F., & Hamilton, L. S. (2006). Making sense of data-driven decision making in education: Evidence from recent RAND research. RAND Corporation. Source
- Coburn, C. E., & Turner, E. O. (2011). Research on data use: A framework and analysis. Measurement: Interdisciplinary Research and Perspectives, 9(4), 173–206. Source
Measurement for improvement starts with a baseline.
Solberg, Mosser, and McDonald separate measurement for improvement from measurement for accountability and for research. Measurement for improvement includes establishing a baseline and checking newly changed processes, which is why the sheet asks for the baseline before anything changes.
- Solberg, L. I., Mosser, G., & McDonald, S. (1997). The three faces of performance measurement: Improvement, accountability, and research. Joint Commission Journal on Quality Improvement, 23(3), 135–147. Source
Required data often can’t answer improvement questions.
Solberg and colleagues write that data for accountability “do not usually illuminate how the outcomes were achieved or how processes might be changed to improve them.” In the RAND studies, state test results generally arrived too late to adjust the current school year, and more than 80 percent of superintendents found local assessments more useful for decisions than state tests.
- Solberg, L. I., Mosser, G., & McDonald, S. (1997). The three faces of performance measurement: Improvement, accountability, and research. Joint Commission Journal on Quality Improvement, 23(3), 135–147. Source
- Marsh, J. A., Pane, J. F., & Hamilton, L. S. (2006). Making sense of data-driven decision making in education: Evidence from recent RAND research. RAND Corporation. Source
Deciding in advance protects against hindsight.
Nosek and colleagues argue for defining the question and analysis plan before seeing results, a practice called preregistration, because ordinary biases such as hindsight bias make it hard to judge results fairly afterward. Step 5 adapts that practice to a team meeting.
- Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600–2606. Source
This tool can also run as a full-day session with your team. See ways to work together.
If you want support planning before your team opens the data, I would love to hear about it.
Send a short note about the program or question and the decision you are facing.
I read every note myself and reply within the week.