In Washington DC, there are 66 early childhood public charter school programs that serve more than 15,000 students from age 3 to second grade. Collectively, they use more than 30 different assessments to measure reading and math skills and evaluate their programs’ academic performance… and that’s just the tip of the iceberg. Standardized testing kicks in at the 3rd grade, but accountability is just as important for schools that serve children at a younger age.
How can authorizers track progress in the absence of commonly accepted measures?
The Public Charter School Board (PCSB) in Washington, DC measures school quality using its Performance Management Framework (PMF) for grades 3 -12 and adult education programs. In thinking about how to evaluate early childhood programs, we first noticed that some of the reading assessments measured several different literacy skills while others only evaluate one, such as vocabulary identification. This makes comparisons and identification of quality programs problematic for our board, as well as for parents and other stakeholders. Another challenge we have found was that the PCSB’s enabling statute states that it cannot mandate all EC schools to adopt the same assessment. Charter schools have the freedom to choose assessments that align with their mission, while DC’s public schools use the same assessments to measure the quality of all of their early childhood programs. An additional hurdle is determining how quality indicators are typically defined for EC programs. Early experts tend to look at inputs as indicative of quality EC program, such as the ratio between the number of students to teachers or the quantity of teacher certifications – but, again, that very Washington, DC law does not allow authorizers to hold schools accountable for these inputs. We determined that in order to assess school quality, and thus the academic performance of its programs, we would need to measure student outcomes.
Developing an Early Childhood Performance Management Framework
Beginning in 2013 we met monthly with the PMF Task Force, comprised of representatives from charter schools and charter advocacy groups. This group was tasked with creating a common accountability framework for early childhood programs and considering the pros and cons of implementing different approaches. Task force members focused especially on student progress and/or achievement measures. To fully understand what the measures implied we investigated the assessments schools had already been using, and researched the grade level expectations determined by each publisher. We also identified a cohort of students whose second-grade achievement on the assessment and third-grade proficiency rate on the state assessment could be reviewed. This step helped us determine if the grade level expectations were reasonable yet rigorous enough to help students succeed by third grade and beyond.
After several task force meetings we determined that the early childhood PMF would give each school an overall numerical score and locate it within a tier, the same general approach we have used in our other PMF, based on:
- Student progress for those in pre-kindergarten (ages 3 and 4)
- Student achievement or progress for grades K-2
- CLASS, spell out and identify, scores for pre-kindergarten classrooms
- Attendance
- Parent satisfaction as indicated by the number of families re-enrolling their child
Schools are rated tier 1, 2 or 3, with 1 being the best. Tier 1 schools are high performing scoring between 100-65; tier 2 or mid performing schools score 64.9-35; and tier 3 or low performing schools score 34.9 to 0.
Strengthening the Framework
As with any plan, report or assessment tool, our work had just begun. Last year PCSB released the first Early Childhood Performance Management Framework (EC PMF) for the 2013-14 school year. Because this new framework was drastically different from previous early childhood accountability plans the task force asked us, PCSB, not to publish the overall score and tier. As we plan to release the second EC PMF for the 2014-15 school year we realize that the framework needs to be strengthened in order to show the difference between high and low quality programs. We are focusing more on ensuring that this PMF highlights schools which are truly preparing students to achieve proficiency by the third grade. This means extending the dialogue with the PMF Task Force and schools that continue to use multiple assessments. Our number-one priority is to provide a PMF that measures real rigor and positive outcomes for students.
We know that designing a framework to measure early childhood program quality is a work in progress. As we work to strengthen the early childhood performance framework, our door remains open for comments, suggestions and ideas. If your organization is also struggling to develop an early childhood performance management framework, please drop a comment below.
Erin Kupferberg is a manager in the school quality and accountability department at the DC Public Charter School Board. The board is responsible for academic achievement for the 112 public charter schools in Washington, DC.
