PMR2

About the Measures

Our work on the development of practical measures, routines, and representations is inspired by the Carnegie Foundation for the Advancement of Teaching and, in particular, Tony Bryk and colleagues’ use of improvement science to address persistent educational problems (Bryk, 2009). Improvement science consists of a set of principles, tools, and methodologies aimed at supporting educators to use scientific inquiry to develop solutions to practical problems that impact their own educational settings.  Such solutions can then be adapted to support improvement efforts in other contexts.

A leading principle of improvement science is that “we cannot improve at scale what we cannot measure” (Bryk, Gomez, Grunow, & LeMahieu, 2015). As such, a key tool of improvement science concerns what Carnegie has called “practical measures,” or “measures for improvement” (Yeager, Bryk, Muhich, Hausman, & Morales, 2013).  Practical measures are designed to provide practitioners with frequent, rapid feedback that enables them to assess and improve their practices.

Characteristics of practical measures include the following (adapted from Bryk et al., 2015; Yeager et al., 2013):

  • The focus of the measure is specific to an improvement goal.
  • The measure uses language that is relevant and meaningful to practitioners.
  • Data collection and analysis are undemanding and can be easily embedded in practitioner routines, thus making it feasible to use the measure on a monthly, weekly, or even daily basis.
  • The measure is sensitive to change.
  • The data produced by the measure are relevant to practitioners and have implications for action.

We are currently developing a suite of practical measures that assess the quality of middle-grades mathematics instruction and the quality of supports for teachers to improve their classroom practices. We are designing these measures so that they can be used to inform instructional improvement efforts.

We have developed a measure of the quality of small group discussion and a measure of the quality of whole class discussion, and are currently in the process of developing a measure of the rigor of mathematical tasks that the teacher has selected for a lesson. We have focused on assessing the quality of discourse and the quality of mathematics tasks for two reasons: 1) district leaders and school-based coaches in our partner districts identified both as key areas for improvement; and 2) mathematics education research suggests the rigor, or cognitive demand, of a task (Stein, Grover, & Henningsen, 1996) and the quality of discussion (Franke, Kazemi, & Battey, 2007) matter greatly for students’ development of deep mathematical understandings and productive mathematical dispositions.

An assumption of our work is that administering measures is, by itself, unlikely to support instructional improvement. Instead, the use of measures needs to be embedded in ongoing professional learning. Future work will focus on developing routines for implementing the measures as part of ongoing professional learning, and on developing data representations that are useful for a range of users including teachers, instructional coaches, and district mathematics leaders.

These measures are works in progress, and we’re hoping that others will help us improve the measures by trying them and adapting them to their specific improvement initiatives and organizational contexts. If you are interested in trying out the measures, you can learn more the focus and development of the measures in our White Paper and access the measures here.