Research & Evaluation Services

Rigorous Evaluation

Increasingly, funders such as the U.S. Department of Education are calling for rigorous evaluation designs. The Education Alliance conducts experimental evaluations using methodologies such as random assignment with control groups, growth modeling, and hierarchical and linear modeling. Our evaluations use multiple measures to assess fidelity of implementation and establish causal links between programs and impacts.

Alliance staff are currently fielding two randomized control trial evaluation studies. Each study is part of a federally funded national initiative to investigate interventions that improve literacy. These studies build on the Alliance's work in adolescent literacy and high school redesign as well as more recent initiatives in early childhood literacy. Alliance evaluators work together with school staff and program providers to clarify program specifications and monitor program implementation to assure the integrity of evaluation data collection and analysis activities. Valid and reliable measures are obtained through extant standardized assessments or collected directly by trained Alliance staff. Evaluators use these measures to respond directly to evaluation questions about whether program interventions work or have an effect on students.

When experimental methodologies are not feasible, Alliance evaluators engage quasi-experimental designs to examine program impact and implementation. Again, collaboration is key to making the right fit between evaluation design and local context and data configurations. Primary data collection using surveys developed and field tested by the Alliance for validity and reliability or extant surveys, interview and observation protocols accessible through established research domains are combined with secondary data sources to provide the outcome and context measures necessary for rigorous evaluation studies. The Alliance has applied matched samples, interrupted time series and other quasi-experimental design options in examining school, teacher or learner effects in evaluations of magnet schools, smaller learning communities, bilingual and ESL programs, after-school and summer program initiatives, comprehensive school reform, Early Reading First, and professional development programs such as Teaching American History, Math and Science Partnerships, and NIH and NSF science, technology, engineering and math (STEM) initiatives.