On-time graduation is an important metric for public high schools across the United States. Large “graduation gaps” point to the inequities and shortcomings of the American education system, and federal law requires states and districts to report high school graduation rates and intervene in schools with low rates. Improving drop-out rates is difficult, however, as school counselors are tasked with large caseloads and identifying at-risk students requires context and time. Many state and local agencies have adopted data-driven modeling tools to address these challenges, including the Kentucky Department of Education’s Early Warning System.
The Kentucky Department of Education (KDE) developed the Early Warning System, an automated, machine learning based tool, in collaboration with Infinite Campus, a software company that hosts the state’s student data entry system. Based on this continually updating source of student information, the Early Warning System uses machine learning to measure how risk factors (such as attendance, academics, and home stability) predict graduation. The system automatically scores each student’s likelihood of graduating on a scale from 50 to 150, which indicates high, medium or low drop-out risk (the lower the number, the higher the risk). The Early Warning System’s interactive interface allows educators to view, filter and search these risk assessments in real time to ensure each student receives the necessary support. A visual dashboard allows users to view overall score distributions at various levels to help district and school personnel better understand what policies yield the greatest impacts on graduation.
Though the Early Warning System began in KDE, Infinite Campus serves over 2,000 school districts across 45 states, and it made its Early Warning System available in additional states beginning in 2019. Michigan, Montana and Sheridan County School District #2 in Wyoming are among the other state and local agencies using this system. Other government organizations have adopted similar approaches: as early as 2013, 26 jurisdictions used early warning reports to identify students at risk of dropping out.
Like other information technology software, these tools present privacy concerns. The student data used in these assessments contain personally identifiable information that is covered by privacy laws, such as the federal Family Educational Rights and Privacy Act. Though Infinite Campus’ Early Warning System doesn’t store individual student information, confidential student data have been surreptitiously used in the past. A 2020 Tampa Bay Times investigation, for example, uncovered that a sheriff’s office used school district data to label children as potential future criminals. Furthermore, the Early Warning System learns risk factors based on population-level trends, which could actually result in biases against some demographic groups. A student’s stability rating (a subsection of the overall risk assessment) takes into account information about race/ethnicity and gender, for example.
Additionally, even if these efforts effectively mitigate dropouts, they do not fully address general criticisms about legislation based on graduation rates. The national graduation rate has increased since 2002, reaching approximately 88% in the 2017-2018 academic year. However, critics question whether this may reflect regulatory policy incentivizing graduation over quality of education. Reports have also shown how using graduation rate as a metric overlooks schools that struggle to advance students through high school and how low-income and students of color are disproportionately affected by this.
As education agencies throughout the country have already adopted various drop-out risk assessment systems, journalists can begin by reviewing news coverage and studies of existing algorithms, such as public evaluations of the interventions that individual agencies implement for different risk ratings. Many organizations provide resources for educators to navigate these systems, which could also be useful to journalists. Similarly, state education department websites post state-specific plans related to federal education laws and information on student privacy protections. In addition to monitoring other clients that adopt Infinite Campus’ Early Warning System, journalists can watch for new drop-out risk assessment systems. Furthermore, while Kentucky has one of the highest average high school graduation rates in the country — the state had a 4-year graduation rate of 91.1% in 2020 — journalists might consider researching early warning systems in low-performing high schools.