In June, the Hunt Institute released a report titled “Connecting the Continuum: Longitudinal Data Systems in North Carolina” that examines the history, purpose, and future of a statewide longitudinal data system (SLDS) in North Carolina and highlights lessons learned from other states. While the report finds that significant progress has been made in creating longitudinal data systems in North Carolina, it also says:
“North Carolina continues to struggle to connect disparate data sources to one another and ensure consistency across systems, keeping the state from fully leveraging data to regularly inform policy and practice in a meaningful way.”
In North Carolina, numerous state agencies — including the Department of Commerce, the community college system, the Department of Health and Human Services, Independent Colleges and Universities, the Department of Public Instruction, and the UNC System — have worked together for years to make progress on North Carolina’s statewide longitudinal data system. By linking education and workforce data over time, formerly-siloed information about students is joined in a seamless manner that tracks how they’re doing from the moment they enter the early childhood system through their workforce outcomes.
Longitudinal data systems allow stakeholders to answer policy questions like: What industries employ the most high school and college graduates? Are ACT scores correlated to workforce salaries? Which students from which schools need remediation classes in higher education? Is participation in NC Pre-K correlated to postsecondary attainment?
I recently sat down with Allison Goff, a policy analyst with the Hunt Institute, and Patrick Sims, director of policy and research for the Hunt Institute, to discuss the report. The following Q&A has been edited for clarity and length.
Who is the intended audience of the report?
We really wanted to focus on policymakers. There are a lot of big questions in the education space in North Carolina, and having a connected, functional statewide longitudinal data system really offers the ability to help policymakers make more informed data-driven decisions when it comes to education policy across the education continuum.
Policymakers are our key audience across all of our work. But we certainly also wanted to be supportive for helping build communication across all the agencies as well, where we could. I think there’s value to other practitioners who are interested in this, but given that our audience is policymakers, we wanted to make sure that people understood the value of data and what it could do to help improve education in North Carolina. So in as much as we want policymakers to get that sense of urgency, we’d also like for practitioners and the public to understand the value of what data systems can do in North Carolina.
Why did you choose to produce the report, and what is its purpose?
We saw that there’s a lack of understanding and information on the many data systems that exist in North Carolina. We also wanted to help clarify the roles that each of those systems play and understand the greater impact that could be felt if they were to all work together and communicate with one another. So we see all these disparate systems that don’t talk to one another and data that’s collected in different ways and in different silos. A lot of our work went into breaking down the silos, and we wanted to make sure we understood what’s happening at this point and think about what the long term positive gains could be, as well as some of the concerns that come along with it, like ensuring privacy, among others.
We also wanted to really get a clear understanding of where the system is, what its current status is, and spell that out in a way to ensure that all the partners involved were on the same page, but also to ensure that the public and the policymakers had a good handle on: What is the current status of the system? What are the immediate next steps we need to be thinking about for how to move this work forward in bringing the systems together and having data that can be used?
How did you set out to answer these big questions, and what did that research process look like?
The first step was identifying who the leaders were at the agencies and the education systems who have been leading this work for over 10 years. This isn’t new work in North Carolina. I think identifying all those people, and then reaching out and connecting with them and having a lot of conversations was the first step. When you have a system this big, and a number of individuals representing their agencies, it’s really important to get all of those voices at the table together to get a really clear understanding of what’s working and also what the challenges are for the system. So we spent a good amount of time this spring engaging with those agency leaders and system leaders who have been working in this space.
Part of that engagement that we found really helpful and interesting was engaging people at different levels within systems. So looking more from a policy perspective, as well as looking from a technical perspective, and just getting both of those perspectives in order to help build a fuller picture of what the system is doing and can do.
We also knew that we wanted to highlight a couple of other states who were doing really good work in the data space, so we did some background research to identify who some of those states might be and chose to focus on Kentucky and Washington. Then we just completed research based on what was available and conducted interviews with the system leaders of both of those longitudinal data systems in those respective states. We really dug into their systems from the lens of what we’d found out about North Carolina. That definitely helped guide our questions, like: This may have been something that was challenging here, was it challenging for you? And finding some of those common threads across systems.
We also conducted more secondary research on the broader history of data systems across the country and what has incentivized them to exist today, what the history of grant funding at the national level has been, and why there’s been a greater emphasis nationally as well.
Can you discuss where we’ve been with longitudinal data in the state, where we are now, and what the next steps moving into the future are?
The goal of a true statewide longitudinal data system would be to connect data that starts in early childhood all the way through K-12, postsecondary, and some workforce and employment data. For many years now, North Carolina has had the Common Follow-up System, which provides that connection at the end between postsecondary, community colleges, the UNC System, and the Department of Commerce. And so that is a chunk of the longitudinal sector that has been there, but to really connect all of them is the goal and is also the challenge.
Currently, early childhood has their own system, which connects a number of different data sources. And that’s run through the North Carolina Department of Health and Human Services — ECIDS is the name of that system. K-12 and DPI have had a really strong system through CEDARS that has been really reliable and has served an important role in creating and encouraging the use of the UID system, which from a matching standpoint is going to be critical for a longitudinal system to work. The UNC System has also had their own system for a long time, and the North Carolina Independent Colleges and Universities are working to establish a system.
NCSchoolWorks was the initial vision through some of the federal grant funding from the U.S. Department of Education, which connected K-12 with postsecondary education and commerce. And now we’re looking at an even bigger system to bring in that early childhood data, and also support the gathering of the community colleges data through the North Carolina Education Longitudinal Data System.
I think it’s important to emphasize how much great work has gone into all these different systems, including NCSchoolWorks which has done a lot to connect the systems. The basis in North Carolina is really strong to build off of because of all the great work of our partners on the ground. One other important dimension about NCSchoolWorks is that it is a federated system rather than a centralized system, which means that each of the data sources exist separately and independently of one another. And they can be connected upon approved requests, but they aren’t centrally in one place with one entity who is regularly conducting analyses or cleaning and merging the data. So there are benefits and drawbacks to federated and centralized systems. But it is an important distinction.
Why should folks care? What should students, parents, or teachers know about the system and its future implications?
There’s a great opportunity for some really impactful and important data to come out of a longitudinal data system that would serve each of these groups. One thing I do want to clarify up front is that this is not going to provide individualized data. It will track data on student cohorts, but you won’t be able to say, for example, this specific eighth grader at this middle school — were they able to complete college in four years? It would talk about that whole class. So I think that’s important for parents and for students also to know.
There are a lot of really big questions that we don’t have answers to right now. The value of pre-K, for example — if you are in pre-K, what does that mean about your likelihood of being a proficient reader in third grade? We need a connected longitudinal data system in order to answer that question specific to North Carolina. So as a parent, it would be really helpful to know, how big of a priority should pre-K be? How big of a priority are AP classes in order for my students to graduate in four years from a college or university? Having the system will help us answer questions and set policy that can better serve students across all ages in North Carolina.
I think it’s also important to think about how information from that system is presented to different audiences, and about how that will need to differ depending on who needs to access that data. So if we talked about a district versus parents, they need different levels of information, different granularity, and different subsets. It’ll be important as we move forward to think about what public-facing information should look like for each of these different audiences.
Right now, the vision for the system is that the data will be used by the partners themselves who are contributing data and also by researchers, so it would be really dense, high level data sets. And that’s going to just provide a very different type of information than what I’m sure a district would need. Looking ahead, I think they’re thinking about how to ensure that this is publicly accessible.
How did you select the other states to study? Were you looking for systems that were similar or different from North Carolina? What did you learn from exploring those two states?
We were really thinking about which states have been successful in getting systems up and running and functionally used. We talked to partners nationally and said: Who would you recommend doing good work and leveraging these longitudinal data systems to improve policy? And we repeatedly heard both Washington and Kentucky. We actually had connected with the former leader of Kentucky’s system through one of our events that we hosted which kind of spurred our interest in what they were doing there.
The systems in both Kentucky and Washington are centralized systems. So that is inherently different than the North Carolina system. We also found that both of those systems have full-time, dedicated staff who run those systems, and that is their job. And I think that is really important when you think about their ability to produce data that is accurate and that can be used in a meaningful way.
There were some differences in the governance structure, like how each system is overseen and how they’re run. And in some instances, removing them from a traditional agency overhead and letting them sit kind of on their own really helped negate some of the feelings of overreach and the feelings of defensiveness and ownership over individual data if it’s owned by this kind of non-education based team. I was most excited to see both systems really being used to inform policy and drive policy change. We have some great examples of those in the report.
The director of Washington’s system said that in between legislative sessions they work with the nonpartisan legislative staff and with legislators to outline questions they think will come from the legislators in the coming session. And they go ahead and pull those data and have them organized so that when they come back into session, they can just drop those in and help them move ahead quickly. That’s a great vision to have about ways to ensure that policymakers have access to data and that they know that it’s there.
It was also interesting to see that both systems struggle with some of the same challenges we see a lot of systems struggling with. From a funding standpoint, there has been so much federal grant support, both from the U.S. Department of Education but also from the Department of Commerce for workforce data. And I think that has been really critical to getting these systems up and running. But as can be the challenge, a lot of the time was spent at the state level to really prove their value to get sufficient funding to ensure that the systems can continue running. And I think that could be a challenge here.
What stood out to you or what was most surprising to you throughout the research process?
I don’t know if it was necessarily a surprise, but I guess it was a welcome reinforcement: the wealth of information that we have in North Carolina. It stands unique among many of its peers in that way. And then, similarly unsurprising, reinforcing the challenges that go into trying to connect all these data and all these systems that have their own rules and ownership.
When we think about early childhood all the way through workforce, that’s a lot of agencies. Even within each of those agencies, the data are coming from a lot of places. So when you think overall about how much work needs to go into really building a fully operational educational longitudinal data system, it could be intimidating for some. It just reinforces how this is going to be a long process, and it’s not going to be simple. It’s going to take a lot of work. But as we look at these other states and see the successes they’ve had, we can draw some inspiration as to what we can continue to work towards here in North Carolina.
I was really pleased to see that, even though this has been a long and complicated journey, all the partners are still deeply committed to this work. They’re deeply committed to maintaining their presence at the table. Because I do think they see the value of the system and appreciate the value not only for their respective system, but overall for the state to move forward. And that’s so important. Maintaining that level of trust and collaboration and shared hope and goals for the system is going to be really important to keep this moving forward, because it is going to be a long journey.
What should readers be expecting or looking for in terms of next steps?
One thing that has been a really important thing that has occurred in the last year or so is the Governor’s Education Cabinet has really backed the system. And I think having those senior agencies and leaders at the table and continuing to drive this work is going to be critical to moving forward. So I think continuing to see the role that the education cabinet plays.
The partners are having conversations about how we think about the kind of the next iteration of the system, like what could a public facing dashboard look like? What are the challenges that we continue to face? How can we ensure that we have analysts there who are able to really analyze the data who have a strong background in analyzing education data to ensure that it’s interpreted accurately in a way that’s meaningful?
These agencies are working with one another actively, and they’re committed to figure out how to better serve policy in North Carolina and improve their own practice as well.
For more on longitudinal data systems in North Carolina, read the two-pager below, or click here to read the full report.#EdData in NC The Hunt Institute