Updated at 10:04 a.m. ET with Louisiana study

It is the education debate of the Trump era. With the president and Education Secretary Betsy DeVos using policy and the bully pulpit to champion private school vouchers, supporters and critics have tangled over the question:

Do low-income, public school students perform better when they're given a voucher to attend a private school?

For years, the answer from researchers has been a muddle, while a handful of recent studies have clearly shown voucher students backsliding academically. Today, much-anticipated reviews of not one but two of the nation's largest voucher programs add some depth and a few twists to the voucher narrative.

First, Indiana. No one has studied the largest, single statewide program in the nation ... until now. More than 34,000 students are enrolled in Indiana's Choice Scholarship Program. That's 3 percent of students statewide.

In a recent investigation of the program, NPR found some private schools turning away children with disabilities and LGBTQ students, but it was impossible to say, at the time, whether those students who are using vouchers are any better off academically.

Researchers Mark Berends of the University of Notre Dame and R. Joseph Waddington of the University of Kentucky have spent years studying this question, and they've given NPR an early look at their findings.

The backslide

In their unpublished research, which is now being peer-reviewed, Waddington and Berends studied the standardized test scores of low-income, public school students (grades 3-8) who qualified for free or reduced-price lunch and who used a voucher to switch to a private school.

When comparing these students' achievement after the switch to their test scores the previous year, the researchers found:

  • Voucher students experienced "modest annual achievement losses" in math, especially in the first two years after leaving public school.
  • In English/language arts (ELA), voucher students showed no benefits.
  • These results echo what other researchers have found: that voucher students often backslide academically after switching to private school.

When students stick around, they improve

The researchers studied student data for the program's first four years and noticed an interesting pattern. If students stayed in their voucher schools long enough, the backslide stopped and their performance began to improve.

"The longer that a student is enrolled in a private school receiving a voucher, their achievement begins to turn positive in magnitude — to the degree that they're making up ground that they initially lost in their first couple of years in private school," Waddington tells NPR. "It's like they're getting back to where they started" before they enrolled in a private school.

New voucher students fell statistically significantly behind their public school peers in math after switching. On average, those losses continued for two years in private school before students began making up ground. In the fourth year, those who were still enrolled in a voucher school appeared to catch up.

In ELA, voucher students also lost ground but, ultimately, surpassed their public school peers by the fourth year.

This pattern may give new hope to voucher supporters, but it comes with an important caveat: Many students did not stay in the system long enough to see this improvement, instead bouncing back to public schools, especially the lowest-achieving voucher students.

This isn't a study of all voucher students ...

It's a study of roughly 3,300 low-income students in grades 3-8 who used a voucher to switch from their public school to a private school and were then matched by the researchers to similar students who remained in public schools. That's another important caveat because, in 2017, more than half of Indiana's voucher students had never attended a public school.

In other words, these results may not be representative of the program as a whole.

Private schools aren't cream-skimming

It's a common refrain among voucher critics: Private schools that are allowed to pick and choose voucher students often accept high-achievers and turn away low-achievers. Not so, say Waddington and Berends, at least not among the students they studied in Indiana.

The state's low-income voucher students "appear to be lower-achieving students from the public sector," they write, "and enter private schools substantially behind their private school peers."

The researchers conducted dozens of interviews with principals, teachers, parents and students in 13 voucher-accepting private schools and found a common theme: Children who came from public schools often struggled to meet new homework expectations and found the first year in a private school particularly challenging. The fact that voucher students' academic performance gradually improved over multiple years suggests that they had to adjust to those demands and that their private schools had to adjust as well.

Waddington believes many private schools struggled to adapt quickly to Indiana's voucher program and that teachers and administrators had to scramble to meet the needs of students who were "largely low-achieving and more demographically diverse than the students already enrolled in these private schools ... It's important to be able to have the time and systematically think about how these students can be best served," Waddington says.

Next steps

"Our hope is that this research can inform." Berends says. "That voucher programs can be more nuanced. There might be some losses but then, over time, there may not be losses forever."

Waddington and Berends are the first to admit that their work, which is still being revised, will not end the debate over vouchers' effectiveness, but it does offer new clarity for Indiana policymakers.

Earlier this year, Jennifer McCormick, the state superintendent, told NPR she was eager for such clarity.

"You know, we're spending roughly $146 million on a program and not really reviewing it," McCormick said. "That is irresponsible."

McCormick's office declined to comment on the research results until after the peer review process is complete.

Louisiana

As those Indiana findings were sinking in this morning, another new voucher study, this one from Louisiana, landed on our desks. And there are parallels aplenty. Like the Indiana results, this isn't a slam dunk for vouchers, but it, too, shows students struggling early, then rebounding over time.

The Louisiana Scholarship Program paid for almost 7,000 low-income children to attend private schools in the most recent school year. Unlike Indiana, Louisiana requires applicants to either be entering kindergarten or coming from a low-rated public school. Another important difference: Seats are allocated by lottery. Most of the private schools are low-income, low-tuition and Catholic; nearly 9 out of 10 students in the program are African-American.

Previous evaluations found that Louisiana's vouchers had a large, negative effect on student learning. Achievement in math, science, social studies and reading all declined in the first year that students attended voucher schools; that's compared with students who struck out in the lottery and stayed at their low-performing public schools. Those negative voucher effects continued for at least two years.

This new Louisiana study was released Monday morning by Jonathan N. Mills and Patrick J. Wolf for the Education Research Alliance at Tulane University and the School Choice Demonstration Project of the University of Arkansas. It brings somewhat better news for voucher proponents:

  • After three years, voucher students did about the same on state English and math tests, compared with similar students at public schools. In other words, as was the case in Indiana, that initial backslide in test scores eventually reversed.
  • A subset of students who were lower-achieving to begin with showed gains in English over their public school counterparts.
  • However, students who left for private schools in lower grades still showed learning losses in math.

Mills, one of the co-authors, tells NPR that the study does not explain why scores dropped, then improved. Switching schools, in itself, can be "particularly disruptive" for students — as Waddington and Berends found in Indiana. The private schools may have also needed time to adjust to the new students, Mills says, and to prepare them for state tests, which they hadn't all done before.
The lower performance in math vs. reading, however, is consistent with many other voucher studies, including Indiana, Washington, D.C., Milwaukee, Florida, and Ohio. One reason for that consistency, Mills says, may be math's dependence on sequencing, which can make changing schools particularly disruptive.

Copyright 2017 NPR. To see more, visit http://www.npr.org/.