Using a Text-as-Data Approach to Understand Reform Processes: A Deep Exploration of School Improvement Strategies
- Resource Type
- Authors
- Junmeng Zhu; Zachary LeClair; Jing Liu; Min Sun
- Source
- Educational Evaluation and Policy Analysis. 41:510-536
- Subject
- Program evaluation
Strategic planning
Medical education
05 social sciences
Attendance
050301 education
Academic achievement
Education
Intervention (counseling)
0502 economics and business
050207 economics
Computational linguistics
Psychology
Educational planning
0503 education
- Language
- ISSN
- 1935-1062
0162-3737
Although program evaluations using rigorous quasi-experimental or experimental designs can inform decisions about whether to continue or terminate a given program, they often have limited ability to reveal the mechanisms by which complex interventions achieve their effects. To illuminate these mechanisms, this article analyzes novel text data from thousands of school improvement planning and implementation reports from Washington State, deploying computer-assisted techniques to extract measures of school improvement processes. Our analysis identified 15 coherent reform strategies that varied greatly across schools and over time. The prevalence of identified reform strategies was largely consistent with school leaders’ own perceptions of reform priorities via interviews. Several reform strategy measures were significantly associated with reductions in student chronic absenteeism and improvements in student achievement. We finally discuss the opportunities and pitfalls of using novel text data to study reform processes.