Overview

This glossary organizes a selection of key terms used in the book, Learning to Improve: How America’s Schools Can Get Better at Getting Better, that have formal meaning. In many cases our definitions align with those established in prior writing on improvement research and networked communities. In some cases they have been modified slightly for the education context. In a few cases they are original to this work.

The terms are grouped together according to the following topics:

• About Networked Improvement Communities
• About Improvement Science and Quality Improvement
• About Standard Work and Processes
• About Systems
• About Driver Diagrams and Working Theories of Practice Improvement
• About Doing Improvement Research
• About Measurement
• About Evidence for Improvement

About Networked Improvement Communities

Colleagueship of Expertise
A community of academic, technical and clinical experts deliberately assembled to address a specific improvement problem. All involved are improvers seeking to generate strong evidence about how to achieve better outcomes more reliably.

Networked Improvement Communities
An intentionally designed social organization with a distinctive problem-solving focus; roles, responsibilities and norms for membership; and the maintenance of narratives that detail what they are about and why it is important to affiliate with them. A NIC is marked by four essential characteristics. It is:

• focused on a well specified common aim;
• guided by a deep understanding of the problem, the system that produces it, and a shared working theory to improve it,
• disciplined by the methods of improvement research to develop, test and refine interventions, and
• organized to accelerate their diffusion out into the field, and effective integration into varied educational contexts.

Network Hubs
A core group formed either as single organization or distributed across network members that carry out critical functions necessary for the support and effective operations of a networked improvement community. These functions include, but are not limited to: improvement science expertise, analytics, knowledge management, convenings, communications, and technological support.

Network Initiation Team
A team that accepts responsibility for the formation of a networked improvement community. It leads a set of processes that articulates of the problem to be solved, analyzes the system that produces current undesirable outcomes, and develops the aim statement and an initial working theory of practice improvement. The initiation team also takes the lead in securing the necessary supports for the network (both political as well as material), recruiting initial members into the community, and engaging the academic and technical expertise relevant to the specific problem to be solved.

NIC as a Scientific Community
It is organized around a shared theory and shared measures for its aim and primary drivers. Participants engage in discipline inquiries using established inquiry methods such as PDSA cycles. Promising results are subject to replication across the network to warrant claims that changes are improvements.

NIC Charter
A document that provides sustaining guidance to the distributed efforts of NIC members. It is composed of the Network’s Aim, Causal System Analysis, (often consisting of both fishbone diagrams and system improvement map), and the Working Theory of Improvement (typically represented in a driver diagram).

About Improvement Science and Quality Improvement

Continuous Improvement
Improvement research that involves multiple iterative cycles of activity over extended time periods.

Improvement Research
These are particular acts of inquiry, or projects, that aim for quality improvement.

Improvement Science
The methodology that disciplines inquiries to improve practice. Undergirding it is an epistemology of what we need to know to improve practice and how we may come to know it.

Quality Improvement
An effort to increase the capacity of an organization to produce successful outcomes reliably for different sub-groups of students, being educated by different teachers and in varied organizational contexts.

Solutionitis
The tendency to jump quickly on a solution before fully understanding the actual problem to be solved. It results in incomplete analysis of the problem to be addressed and fuller consideration of potential problem-solving alternatives. It is silo-ed reasoning—seeing complex matters through a narrow angle lens—that can lure leaders into unproductive strategies.

About Standard Work and Processes

High-Leverage Process
A process that has the following properties: 1) it consumes substantial resources, especially teacher or student time; 2) its execution and outcomes vary considerably; and 3) there are reasons to believe that changes to it might improve resource efficiency and effectiveness.

Macro-Process
A process for which the execution typically entails a sequence of more discrete micro-processes.

Micro-Process
An elemental activity or segment of work taken to achieve a particular end.

Standard Work
Regularly occurring processes that are amenable to formulation as best practice routines within a networked improvement community. The purpose of these routines is to assist educators in carry out their work by reducing the cognitive load associated with the performance of complex tasks. Such processes reduce undesirable variation in performance and free educators to better focus their attention thereby advancing quality outcomes more reliably. High leverage processes are attractive candidates for standard work.

About Systems

Causal System Analysis
An analysis that directs attention to the question, “Why do we get the outcomes that we currently do?” In working through this analysis, participants develop a shared understanding of the specific problem(s) they are actually trying to solve. The process also provides a first test as to whether a team seeking to initiate a NIC can engage productively together as a focused improvement community.

Fishbone Diagram
A tool that visually represents a group’s causal systems analysis (sometimes known as a cause-and-effect diagram or an Ishikawa diagram).

System
An organization characterized by a set of interactions among the people who work there, the tools and materials they have at their disposal, and the processes through which these people and resources join together to accomplish its work.

System Improvement Map
An analytic tool that represents what we learn through the causal system analysis about the different organizational levels (e.g., classrooms, schools and districts) and key organizational sub-systems (e.g., human resources, finance, instruction) relevant to solving the identified problem.

About Driver Diagrams and Working Theories of Practice Improvement

Change Idea
An alteration to a system or process that is to be tested through a PDSA cycle to examine its efficacy in improving some driver(s) in working theory of improvement.

Concept Framework
It provides conceptual detail and relevant research findings that form design principles for key drivers and change ideas. It also provide a conceptual basis for the development of practical measures. (Also may be referred to as design principles.)

Driver Diagram
A tool that visually represents a group’s working theory of practice improvement. The Driver Diagram creates a common language and coordinates the effort among the many different individuals joined together in solving a shared problem.

Improvement Aim
A goal for an improvement effort that answers the question What are we trying to accomplish? Improvement aims should clearly specify how much, for whom, and by when? They sit at the far left end of a Driver Diagram.

Pareto Principle
A principle anchored in a long history of organizational studies that 80% of the variability in organizational performance is often associated with only 20% of the possible causes. It aids in the selection of primary, and when needed, secondary drivers as well.

Primary Drivers
Representation of a communities’ hypothesis about the main areas of influence necessary to advance the improvement aim.

Secondary Drivers
A small set of system components that are hypothesized to activate each primary driver.

Working Theory of Practice Improvement
A small inter-related set of hypotheses about key drivers necessary for achieving an improvement aim and specific changes associated with each driver. It requires a creative blending of observations arising from the causal system analysis with relevant research that bears on this problem together with wise judgments from expert educators.

About Doing Improvement Research

Adaptive Integration
Learning how to integrate a change package into a new setting, adapting it as necessary to local conditions, while preserving fidelity to its undergirding principles and assuring local efficacy.

PDSAs (Plan-Do-Study-Act Cycles)
A pragmatic scientific method for iterative testing of changes in complex systems. Each cycle is essentially a mini-experiment where observed outcomes are compared to predictions and discrepancies between the two become a major source of learning.

Process Map
A tool for visualizing the steps in a process that can assist an improvement team in identifying gaps, strengths, and opportunities for improvement.

Run Chart
A graphical display of a some measured characteristic over time. Data from PDSA are often displayed in run charts.

Solution Systems
An intervention that consists of multiple inter-related components that must mesh well together. Formally, such interventions have a systems character. For positive effects to occur reliably coordinated improvements need to occur across all of the drivers that compose the solution system. A material weakness in any one driver can undermine the efficacy of the overall solution.

About Measurement

Balancing Measures
These measures help improvers to keep an eye on the other parts of the system that are not currently the target of improvement, but nevertheless may be affected by the changes being pursued.

Lagging Outcome Measures
Measures that are only available well after an intervention has been initiated.

Leading Outcome Measures
Measures that predict the ultimate outcomes of interest but are available on a more immediate basis.

Measurement for Accountability
Broad, general measures that aim to sort individual or organizational units into performance categories. This use introduces increased formality into the measurement process and places primacy on the reliability of individual scores. Often used as measures for improvement aims.

Measurement for Improvement
Measures that directly link to the specific drivers and work processes that are the object of change. They provide evidence for testing changes and examining hypothesized causal connections in the working theory of improvement.

Measurement for Research
Detailed measures developed by researchers to represent particular theoretical constructs. Used in academic research to test relational propositions among key constructs that form a theory. Often useful as a basis for developing practical measures.

Outcome Measure
Measures that operationalize the aim statement in the driver diagram. These data provide a way of assessing whether progress is being made on the specific problem to be solved. Accountability measures are often used here.

Practical Measurement
Data to inform improvement that is embedded in regular work. Since the intent is to inform continuous improvement, practical measures are collected frequently to assess whether positive changes are in fact occurring. Since the focus is on specific populations and contexts, the measures are framed in a language natural and comprehensible to those asked to answer them.

Primary Driver Measures
Measures associated with primary drivers. Since these drivers are intermediate outcomes in the working theory of improvement (i.e., intermediaries between process changes and leading and lagging outcomes) they play a key role in the testing of a working theory of improvement.

Process Measures
Measures that feed back valuable information about how specific processes being tested are performing under different conditions.

About Evidence for Improvement

A, B, C Level Learning
Three inter-related levels of learning, developed by Douglas Engelbart, that together form a schema for individual and organizational learning. “Level-A” learning represents the knowledge acquired by front line workers as they engage in their practice. “Level-B” learning occurs across individuals within an organization. “Level-C” learning is orchestrated by a network hub and coupled with appropriate technologies to support rapid communications across distributed sites. This confluence is what enables the network to accelerate how it learns to improve.

Evidence-Based Practice
Tools, materials, or sets of routines, typically grounded in theoretical principles, that has been subject to rigorous empirical study. Their use is warranted by results from a rigorous field trial that demonstrated that the intervention can work because it has somewhere.

Know-How
The detailed practical knowledge necessary to get good ideas actually to work in classrooms, schools, and districts.

Practice-Based Evidence
Evidence that grows from practice and can be used to improve it. This evidence, emerging from improvement research, demonstrates that some process, tool, or modified staff roles and relationships can be made to work effectively under a variety of conditions and that quality outcomes will more reliably ensue. It is the evidence of know how.

The UChicago Network for College Success (NCS) began in 2006 out of a growing need for research-based education reform in Chicago. Today, NCS partners with 17 schools and 300 educators that serve over 16,000 students—approximately 16% of the district’s high school student population. NCS brings deep expertise to help schools develop effective change strategies to support and sustain higher student achievement.

The UChicago Network for College Success (NCS) is being spotlighted this year because of its use of continuous improvement to dramatically increase the number of 9th-grade students, especially Black and Latinx children and children from low-income families, on track to high school graduation (i.e., have earned enough credits for on time sophomore year promotion). This focus is based on the fact that students who fail a course or whose attendance falls to unacceptably low levels in 9th grade have greatly diminished chances of making it over the high school finish line. Indeed, 9th graders who end the year on track are four times more likely to graduate. On track is a stronger indicator than race, ethnicity, poverty, or test scores.

NCS achieved its results by focusing annually on 15 to 20 Chicago schools comprised of more than 300 educators and 15,000 students. These schools saw their overall on-track rates rise by nearly 20 percentage points from 72% in 2010 to 91% in 2019. The success of students of color during this period was also noteworthy. For example, the on-track outcomes for Black males grew from 58% in 2010 to 84% in 2019, and those of Latinx males increased from 66% to 84% in the same period. For their longest standing high school cohorts in which NCS has graduation data, being on track in 9th grade translated into an average graduation rate that grew from 71% in 2013 to 84% in 2019.

In this webinar, leaders of the UChicago Network for College Success and practitioners on the ground discuss how they dramatically increased the number of 9th-grade students that were on track to high school graduation.
Download presentation slides (PDF)

Read the full profile of the Network for College Success for more details about the work.

DOWNLOAD PROFILE

---

Spotlight at the Summit

UChicago Network for College Success (NCS) will discuss their work at a featured session at the 2021 Summit on Improvement in Education.

Improving Educational Outcomes in Under-Resourced and Isolated Areas

In Chile, the nonprofit organization Fundación Educacional Oportunidad is partnering with the Ministry of Education, Harvard University, and 60 very under-resourced majority rural schools to improve the language skills of economically disadvantaged preschoolers. Its work builds on over a decade of using improvement science to tackle some of Chile’s most vexing social problems. While the work is just under two years old, it’s noteworthy for blending insights from scholarship, quality improvement, and on-the-ground expertise in support of children’s learning.

Based on research, Fundación understood that for young children to achieve necessary levels of language and literacy development, schools had to maximize instructional time, improve student attendance, and promote effective early literacy instruction. A multidisciplinary team of coaches and experts on improvement and early education worked with 15 districts, 118 school leaders, and 148 teachers and teachers’ aides and determined that these primary drivers for improving language development resonated with practitioners in the field. Of the participating schools, 60% are located in rural areas.

BY ENGAGING PARENTS, TEACHERS, AND SCHOOL LEADERS TO BECOME ACTIVE AGENTS FOR IMPROVING IN THEIR OWN CONTEXT, FUNDACIÓN EDUCACIONAL OPORTUNIDAD IS SHOWING RESULTS AND MAKING PROGRESS ALONG EACH OF THE CORE PROGRAMS.

Together, the team identified a set of drivers aimed at having 80% of children in pre-kindergarten and 90% of children in kindergarten reach the advanced category on a Spanish language evaluation by the end of 2017. The schools then worked with Fundación to implement strategies that included providing teachers with help on time management in the classroom and integrated lesson planning, to maximize instructional time; coaching and video feedback to promote effective literacy instruction; and attendance committees to provide one-on-one support for improving student attendance.

VIDEO: A presentation by Un Buen Comienzo Improvement Network, Fundación Educacional Oportunidad at the Spotlight on Quality in Continuous Improvement Symposium on November 15, 2018 in Washington, DC.

Fundación also brought together parents, school leaders, teachers, teachers’ aides, and local education agencies three times a year to teach them how to use quality improvement tools, such as identifying the root causes of problems and using plan-do-study- act cycles to test whether their approaches to improvement were helping schools make progress in the target areas. During these meetings, the members of the networked improvement community also shared what was working and identified new challenges to be collectively addressed.

Fundación also supported teams of teachers and school leaders in visiting each other’s schools to observe innovations and share data. At the classroom level, teachers proposed innovations and used rapid iterative cycles of experimentation to test whether the innovations were producing improvements and then adjust accordingly.

By engaging parents, teachers, and school leaders to become active agents for improving in their own context, Fundación Educacional Oportunidad is showing results and making progress along each of the core programs. Children’s language scores are improving—although they have not yet reached the 2017 targets. And the schools have seen progress in the amount of time spent on instruction. This work demonstrates that the science of improvement can be used and adapted to spur positive change in the most under-resourced and isolated settings and can be applied internationally.

---

Network Demographics*

• 20 schools
• 24 preschool teachers
• 21 teachers’ aides
• 20 school principals
• 20 school pedagogical leaders