Kentucky Public Schools as Educational Bright Spots
BY MICHAEL CHILDRESS, CENTER FOR BUSINESS AND ECONOMIC RESEARCH, GATTON COLLEGE OF BUSINESS AND ECO-NOMICS, UNIVERSITY OF KENTUCKY
Each academic year a select group of Kentucky’s public schools perform better than expected on measures of educational achievement. These measures include things like the percentage of elementary students who achieve proficiency or distinguished in reading, or the proportion of less-advantaged middle school students who show a similar level of competency on the math assessment. Understanding the reasons for better-than-expected performance is fundamentally important. While our analysis does not fully address the question of why students perform better than expected, our results can be used to inform further inquiry on that question. Our work is best viewed as a statistical sieve designed to narrow the list of candidate schools worthy of closer examination. By subjecting a school to closer scrutiny, one can gain a sense of confidence about identifying the constellation of factors facilitating exceptional performance—and hopefully facilitate the adoption of these practices to other schools.
Organized within 173 school districts, there are wide differences in the learning environments, sizes, finances, and student outcomes among and within Kentucky’s 1,466 schools. Since schools are likely to reflect the underlying economic conditions of their surrounding communities, the socioeconomic characteristics of Kentucky’s schools are as diverse as the state itself.
Student outcomes, of course, are the bottom lines for schools and districts, and there is a wide distribution of outcomes across the state’s public schools. From a broad range of student outcomes, family and community backgrounds, and school characteristics, we identify schools that have performed better than expected—which we refer to as “bright spots.” For example, Knox County Middle School and South Laurel Middle School in Laurel County performed similarly on the 2018-2019 K-PREP middle school mathematics assessment, demonstrated by 50.9 and 51.1 percent of their students scoring proficient or distinguished, respectively. Yet, once we consider student, school, district, and community factors, only one of these schools performs “better than expect”—Knox County Middle School. While South Laurel Middle School performs at a level we expect, Knox County Middle School performs much better than we expect; in fact, it performs 20 percentage points higher than we expect.
METHOD
Using a school-level database that includes, but is not limited to, data from the Kentucky Department of Education (KDE), the Kentucky Center for Statistics (KYstats), and the U.S. Census Bureau, we analyze data covering eight academic years—2011-12 to 2018-19. We estimate an expected level of school-level performance using district-level fixed effects panel regression analysis—a statistical method for estimating, expressing, and understanding the relationships between variables—and then compare it to the actual performance. The difference between actual performance and model-based expected performance is the residual. If the size of the residual is sufficiently large and positive, we consider it a “bright spot candidate.” The development and creation of our statistical models is informed by Prichard Committee personnel, the scholarly literature on factors affecting student outcomes, data availability, and technical considerations regarding variable selection and model construction.
Our 35 educational outcome variables include K-PREP reading and mathematics proficiency scores at the elementary and middle school levels, ACT scores for 11th graders, and college going rates for graduating seniors. There are two conditions that a school must meet in order to satisfy our definition as a “bright spot.” First, we evaluate all students on an outcome measure, such as K-PREP elementary mathematics outcomes, to assess whether a school exhibits better-than-expected performance at least once from 2011 to 2018; in other words, we are looking for significant positive residuals. Second, while focusing on the same educational outcome measure, but for at-risk students (e.g., low-income or disabled students), we analyze the model residuals to assess whether a school exhibits a significant improvement in performance relative to expectations over the time period; in this case, we regress the residuals on year, and if year is positive and statistically significant, then it is improving relative to expectations over the time period. Any school that satisfies both of these conditions on an educational outcome is deemed a “bright spot.”
Bright Spots Results
The information provided below in Table 1 shows the 47 “bright spot” schools meeting our two conditions. There are 28 elementary schools, 4 middle schools, and 15 high schools that can be viewed as bright spots by virtue of student performance on K-PREP or ACT assessments, or successfully transitioning to college; and some schools qualify as bright spots in more than one category. Since we do not have data on college going for at-risk students—only the total graduating class—we used different criteria. For these two outcome measures, we assess the change over time for the total group instead of at-risk groups.
From left to right, the columns in Table 1 show the district and school identifier assigned to a school by the Kentucky Department of Education (Sch_Cd), the school district where the school is located, the school name, the educational outcome category, and the number of years from 2011 to 2018 where all students performed better than expected. The three columns on the right indicate whether groups of students exhibited significant improvement relative to expectations over the time period, either all students (TST), those qualifying for the National School Lunch Program (LUP), and those with an individualized educational plan (ACD); the numbers shown are the t-values of the bivariate regression slope, where residuals are regressed on year. For example, the first row shows Oakview Elementary School is a bright spot for 3rd grade reading. It demonstrated better-than-expected performance for all students in one year, and students participating in the NSLP (LUP residual) evidenced significant improvement during the time period with a t-value of 3.3; IEP students (ACD residual) did not show significant positive improvement, which is reflected by the “not sig.”
Conclusions
The 47 “bright spot” schools that performed better than expected from 2011 to 2018 are located in all regions of the state and 30 different counties, as illustrated in the county-level map below; these are diverse settings—urban-rural, east-west, distressed areas as well as prosperous ones.
Our analysis confirms what research has long revealed—that less-advantaged and minority students can face difficult obstacles in the pursuit of academic success. Of the 35 educational outcome models we tested, the predictor variables of less-advantaged students (i.e., % NSLP participants) and minority students (i.e., % nonwhite) were statistically significant and negative in 34 and 30 models, respectively. Additionally, teacher experience—the average number of years teaching—was statistically significant and positive in 20 of the 35 models; the impact of experienced teachers was mostly concentrated in the elementary level KPREP reading, 8th grade KREP reading, and in each of the nine ACT models.
Understanding the reasons for better-than-expected performance is fundamentally important. These results of this analysis can be combined with other pieces of information, if desired, to identify educational bright spots worthy of closer examination. With closer qualitative examination, it is possible to identify the critical factors leading to better-than-expected educational outcomes. Given the wide geographic distribution of educational bright spots, there are many candidates available across the Commonwealth for further study and examination.
