Engaging School Leaders
DC PCSB continues to explore new ways to engage school leaders. This begins with prioritizing the voices of school leaders that educate large percentages of students in historically marginalized groups. This work began when we conducted multiple small engagement sessions with a handful of public charter school leaders. These sessions allowed leaders to strengthen relationships, be heard, and to speak candidly about the barriers and challenges of student outcomes.
These sessions paved the way for creating two advisory groups, one for PK-12 schools, and another for adult schools. Each group was limited to one representative from each public charter school network or charter LEA. This approach ensured that all ideas, opinions, and beliefs were heard. Before the meeting, a list of guiding questions was shared with participants to allow for a rich discussion and encourage full participation. More than 25 small group meetings were held and a survey was disseminated.
Here is a timeline* of activities related to the development:
*As of August 2023, this timeline has shifted. More information will be shared soon.
What School Leaders Had to Say: PK-12 + Adult PCS Discuss Accountability
In the spring of 2021, DC PCSB held monthly small engagement sessions to discuss high-level conceptual changes and priorities to the accountability framework. Each session included up to one representative per charter LEA. The sessions are an opportunity to receive feedback about potential changes to the current Performance Management Framework (PMF). Read more about the PMF here.
The information gathered from these engagement sessions assisted us in the curation of our next phase of engagement, which will focus on the technical aspect of collecting and reporting these potential data points.
These sessions are part of our ongoing engagement that will continue as we revise the accountability framework.
This summary represents the high-level takeaways heard during the sessions:
PK-12 PCS Sessions:
- Public Charter LEAs have concerns around the implications of altering the purpose of the assessments from an internal tool used to inform instruction to one that informs DC PCSB’s accountability decisions, adding new measures to identify growth, and increasing testing in order to collect measures
- Limiting a public charter school’s flexibility by using standardized test results
- Public Charter LEAs would like to find new approaches to understand growth in student populations
Adult PCS Sessions:
- Adult Public Charter LEAs discussed concerns with reaching students (e.g., students’ phone numbers constantly changing)
- Attendance metrics fail to capture the engagement of the adult learner due to the structure of course offerings for various adult programs; adult LEAs focus on outcomes not in-seat attendance
- Finding ways to better capture synchronous and asynchronous learning
- Bringing more awareness to completion of certifications that are not just industry certifications, emphasizing the importance of the micro credentials necessary for the outcome (e.g., state-level certifications like cosmetology, barber, etc.)
Based on the takeaways from PK-12 and Adult PCS leaders, DC PCSB will include the information in the revised accountability framework:
- Provide an in-depth explanation of each growth measure and what it will achieve
- Celebrate the milestones of adult students in the adult education framework
- Revise the measurement for the attendance indicators
- Include school selected nationally normed growth assessments in the business rules
Using Various Metrics to Inform Approach, Discussion and Decisions
On July 18, 2022, the DC PCSB Board approved the Data Collection Summary for the Revised Accountability Framework. The Data Collection Summary is an inventory of the data that will be collected to model potential approaches to inform the changes that will be made to the accountability framework. It is important to state that not every data point may be used in the Revised Accountability Framework. Data analyses and business rule modeling will inform discussions with stakeholders to make decisions on floors, targets, and weights for the new framework.
DC PCSB is expanding existing measures and exploring alternative ways of capturing existing data. Key changes are highlighted in yellow.
The only measure we have removed is attendance for adult learners. Adult learners are non-compulsory and are not required by law to attend. We will continue to capture adult learner persistence and outcomes.
Description of Data Collection Points Collected to Model Potential Approaches for Framework Revisions
Median Growth Percentile (MGP) and School Selected Nationally Normed Growth Assessments
Historically, DC PCSB has used the MGP on the PARCC as the growth measure which assesses the relative year-to-year progress made by individual students at a school. School selected nationally normed growth assessments include assessments such as i-Ready or NWEA MAP where charter LEAs will be able to choose between these nationally normed options for this growth measure during the pilot. DC PCSB will explore both options and share with charter LEAs the findings of each to inform conversations around growth measures for accountability.
College and Career Acceptance
Traditionally, DC PCSB has collected college acceptance data from charter LEAs. Expanding this measure to capture career post-secondary pathways allows students to be recognized for several successful pathways available to them. Career pathway guidance is still being developed; however, this is intended to include acceptance into the military, the police academy, fire academy, etc. This would likely not be a new measure, but become part of the college acceptance measure, with students either having a college or career program acceptance.
Adult Learner Certifications
Adult LEAs provide certification pathways including Carpentry Level 1, National Construction Career Test, Child Development Associate (CDA), Certified Nursing Assistant (CAN), Leadership in Energy and Environmental Design (LEED), NAFTrack, and NAFTrack Certified Hiring. DC PCSB is exploring criteria for additional certifications to be included in this measure and will share guidance with charter LEAs once these criteria are developed.
In-Seat Attendance
Historically, DC PCSB has captured in-seat attendance, which captures the percentage of school days students are “in their seats” across the entire school year.
DC PCSB is exploring chronic absenteeism as a possible alternative attendance measure. Chronic absenteeism captures the number of students who are absent (excused or unexcused) for 10 percent or more of the school's days on which the student was enrolled across the entire school year.
We will explore this data to identify which attendance measure best captures attendance for public charter schools.
-----
Lastly, some measures will be displayed and not included in the overall calculation of an accountability score. These measures, where available, may be included in the framework to provide additional context for community members to make decisions on where they’d like to attend or enroll their student.
DC PCSB also continues to explore the following measures: discipline by student group, teacher tenure and demographics, Qualitative Site Reviews, and mission-specific goals. PK performance is already displayed in the accountability framework and includes data around student performance using the charter LEAs chosen PK assessment.
Update: Our Working Model on the Revised Accountability Framework
DC PCSB continues to work on revising the accountability framework. This work supports DC PCSB’s overarching vision that Every DC student receives a quality education that makes them feel valued and prepares them for lifelong learning, fulfilling careers, and economic security.
We fundamentally believe that schools should produce equitable outcomes for all students, ensuring that students from historically marginalized groups receive a quality education.
In order to facilitate this expectation, the revised accountability framework will include:
- More visibility on students who are furthest from opportunity
- Report disaggregated data by race and ethnicity
- Offer more distinction between the levels of performance across schools
- Increase the bar for proficiency
- Have a greater emphasis on growth in response to the impact of COVID-19
This effort seeks to lead to a community of public charter schools that achieve greater outcomes to meet the needs of students each school educates.
Our revised accountability framework will emphasize schools’ contributions to student success, rather than focus simply on what scholars individually achieve. It is our hope that this will allow us to better and more fairly evaluate schools.
Schools will continue to be evaluated on some of the categories present in the Performance Management Framework or PMF. The existing categories that will remain are Student Progress, School Performance, and School Environment. We will also continue to use state-mandated metrics like PARCC to evaluate schools, though this will not be the only metric included.
The following are broad categories detailing how schools will be evaluated.
As stated in our previous blog about our accountability framework, we will consider student’s growth in academic performance via school-selected and nationally normed assessments. We will also consider students’ achievements in careers, not just college, and focus on chronic absenteeism.
In giving schools the freedom to decide some of their own metrics and goals, we will be able to better evaluate each individual school’s performance, take into account their student population and unique programs.
Another important consideration in this framework is how different student populations perform. Different from the Performance Management Framework or PMF, DC PCSB will consider school progress based on student demographics, keeping in mind our goal to help students from historically marginalized groups thrive. Student success among historically marginalized groups will impact how well a school is scored.
Finally, to better evaluate Adult Programs, we will remove our previous attendance measure. School attendance is optional for adults, and thus not relevant to a school’s performance. We seek to focus on expanding our measures for adult learner performance, including certifications.
From Three Tiers to Five Levels
Taking into account the nuanced scores public charter schools can achieve and our metrics, the revised accountability framework will shift to a five level system. This allows us and school leaders to better understand how different conditions can affect schools’ performance, and leads to more accurate evaluations.
Beforehand, schools with drastically different levels of student performance tended to be grouped into the same middle rating or Tier 2, for example. A five-tier system will allow us to highlight achievement gaps and also better showcase which schools are doing well for their historically marginalized students.
The numerical ranges will be similar to the Performance Management Framework or PMF, where 5 indicates a lower level, and 1 indicates a higher score.
Questions and Concerns
As always, DC PCSB is committed to working with school leaders and members of the community to ensure our accountability framework can best serve DC public charter school students.
DC PCSB will continue to offer engagement opportunities. And invite the public to submit a comment on our website.
As we work to meet our Strategic Priority to fine-tune this framework, we will continue to provide regular updates at our monthly board meetings and here on this blog.