The N.C. Department of Public Instruction (DPI) has convened an advisory group to redesign school performance grades ahead of the next legislative session, and the meeting held yesterday focused on data quality in accountability models, an early analysis of EdNC’s survey results, and small group discussions about the public’s input.

“If we make these priorities, schools will make these priorities, and that’s exciting,” said Leah Carper, the 2022 Burroughs Wellcome Fund North Carolina Teacher of the Year.

Here is DPI’s landing page for the working group including the members, and here is the timeline:

Data quality

Diana Lys, assistant dean for educator preparation and accreditation at UNC’s School of Education and also on the working group, talked about data quality in school accountability models. She reviewed six factors that are important when it comes to data quality: is the data reliable, is it valid, is it differentiated, is it representative, is it cumulative, and is it actionable?

“The team at DPI is really invested in making sure that we have all of these pieces for quality data to move this work forward,” said Lys.

“There are a lot of folks out there in the field in education and parents that have lost trust with the system and what those school ratings are currently,” she said. “So ‘actionable’ should be something that can drive change.”

To do that, she said, the data needs to be trustworthy.

An early analysis of the data from EdNC’s survey

Dr. Jeni Corn, DPI’s director of research and evaluation, presented her analysis of the data from EdNC’s survey on school performance grades to the working group. Of 26,262 survey participants, her team focused on the 19,160 who answered at least one question other than sharing their role and zip code as well as 4,884 of the 5,980 comments.

Across roles, Corn found the highest agreement with these survey items:

• The North Carolina legislature needs to reform school performance grades.
• K-12 schools should have different measures of success for elementary, middle, and high schools.
• Some level of standardized testing is necessary to understand how students are doing.
• School performance grades should include measures beyond test scores and student growth.

For indicators of school quality beyond test scores, Corn found the highest agreement with these items across roles:

• Courses offered outside of core academics (i.e., arts, world languages, career and technical education)
• Durable skills (i.e., communication, collaboration, critical thinking, creativity, persistence)
• High school graduation rate
• School climate (overall social and learning climate)
• School safety (i.e., incidents of school crime, violence, school resource officer)

The least agreement was found for these items across roles:

• Discipline (suspensions and expulsions)
• Entrance exams (College: ACT, PSAT, SAT; Career: WorkKeys; Military: ASVAB)
• Health of students (physical fitness, mental health, social-emotional health)
• Per pupil expenditures

In an early look at the first 2,000 comments, Corn’s qualitative analysis found these words mentioned most often by participants:

• Growth. Some suggested 100% growth or flipping the formula to be 20% achievement and 80% growth.
• Equity
• Students demographics
• Teacher effectiveness/retention/job satisfaction

The working group participated in small group breakouts to review the survey responses, grouped by academic indicators, non-traditional academic indicators, and school quality indicators.

Corn’s next steps include an analysis by region and finishing the qualitative review of all of the comments.

All of the data from EdNC’s survey is available to the public, and we invite you to analyze it too. What do you see?

What happens next?

“I’m grateful this is a project that doesn’t have an already made solution,” said Superintendent Don Phipps of the Caldwell County Schools, “and that there really is the opportunity to provide insight and feedback and hopefully that will be represented in the final product.”

In the weeks between now and the next meeting on Nov. 7, 2022, members of the working group will use a conversation guide created by DPI to bring more people into the process.

There will also be a feedback room at DPI’s upcoming AIM Conference for educators.

Resources for your consideration

We want you to continue to weigh in on the conversation about school performance grades.

Here is the current law, N.C. General Statute section 115C-83.15.

Here is EdNC’s survey data.

Here are EdNC’s maps that allow you to see how the school grades change across North Carolina as you change the current formula.

If you want to think about the purpose of school performance grades before thinking about what a new model might look like, this brief from EPIC is a good starting point.

If you want to think about how other states approach school performance grades, this 50-state resource from Education Commission of the States is worth a look.

And — from the way, way back — on May 2, 2017, a group of superintendents — including Freddie Williamson, Rodney Shotwell, Robert Taylor, Lory Morrow, and Ron Phipps — presented this proposed model for school accountability to the N.C. State Board of Education. It proposes different models for elementary, middle, and high schools.