The Boston Public Schools has awarded three consecutive contracts worth nearly $4 million to a testing firm, even though it hasn’t met the district’s own requirement that its test be free of racial and ethnic bias, according to documents reviewed by WGBH News.
That test, the Independent School Entrance Exam, or ISEE, is used as the entrance exam for Boston Latin and the city's two other exam schools.
Since 2012, three contracts have gone to the nonprofit behind the ISEE, the Educational Records Bureau, or ERB. They were for $1.6 million in 2012, $1.75 million in 2015 and $558,900 in 2018. ERB was the sole bidder each time.
The district's requirements for the last three bids included either “evidence that the assessments are appropriate for a diverse student population” or “evidence that test instruments are bias-free and equitable assessments for a diverse student population.”
The ERB’s 2017 study lacked sufficient sample sizes of students of color to make conclusions about how a high score on its entrance test corresponded to their performance in high school. In what's called a predictive validation study, the only sample large enough to make that judgment was composed of white students. When contacted by WGBH News, ERB did not identify any studies done before 2017 showing that the ISEE accurately predicts the high school performance of students of color.
A spokesperson for Boston Public Schools said the district was “aware of the study, and cognizant of its limitations.” Enrollment in Boston's public schools is more than 80 percent students of color.
The district said the past three superintendents, Tommy Chang, John McDonough and Carol Johnson, deemed the test maker’s bids appropriate.
“The Educational Records Bureau (ERB) was the only bid submitted for the past three invitations/requests, and their bid was accepted for the past three contracts,” a district spokesman said. “For the most recent contract, the superintendent at the time selected ERB as the ‘lowest responsible and responsive bidder for the [evaluation criteria] indicated.’”
Bidding documents for the three contracts also note the city’s right to “accept or reject any and all bids.”
“We are always leery of any time there’s only one vendor repeatedly,” said Matthew Cahill, executive director of the Boston Finance Commission, an independent watchdog of the city's government. “There has to be some sort of work done to find out why, and to see if there’s a way to increase that pool of vendors.”
Cahill said it's fairly common for city departments to ask potential vendors why they didn’t submit bids.
“If they’ve been the sole bidder for a few years, then somebody along the way should be looking at it, saying ‘Why are they the sole bidder?,’ and actually contacting other testing companies and saying, ‘Did you see this [bid]?'” said Cahill.
The district's use of the ISEE as the entrance test for the Boston exam schools has faced criticism over the last few weeks after a Harvard Rappaport Institute report showed basing exam school admissions on the MCAS would increase black and Hispanic enrollment at Boston Latin School by 50 percent.
Boston isn’t the only large urban public school district whose exam school admissions process has come under scrutiny. New York City Mayor Bill de Blasio has proposed eliminating the city’s entrance test for its specialized high schools to improve diversity, and Chicago revised its selective admissions criteria in 2009 to include socioeconomic status.
New York uses the Specialized High School Admissions Test, made by Pearson Education. Chicago uses the Measures of Academic Progress, produced by the Northwest Evaluation Association.
Besides Boston's exam schools, area independent schools like Beaver Country Day School, Boston College High and Milton Academy also use the ISEE.
Related: Entrance Exam Used By Boston Latin May Not Be Up To The Test
The most recent contract with the ERB lasts through 2019. This weekend the district is scheduled to administer the ISEE to students interested in attending Boston Latin, Boston Latin Academy or the John D. O'Bryant School. Next year, the district plans to administer the ISEE in sixth-grade classrooms throughout the district in an effort to get more black and Hispanic students to apply to those schools. Admission is based on a student's GPA and ISEE score.
The test maker has been providing the exam to the city since the mid-1990s, though the contract for administering the Boston exam school entrance test has only been publicly bid and awarded since 2000, according to a city spokesman.
“Twenty years of one vendor — that’s unusual around here,” Cahill said. “When there’s only one [bidder], and certainly more vendors around the country that do this type of work, it’s important that they start finding out why.”
Heather Hoerle, the executive director and CEO of the Enrollment Management Association, which creates another independent school entrance exam, said the district did not reach out about bidding for its entrance test.
"Our organization has not been contacted by the Boston Public School system regarding an opportunity to become the admission assessment for the Boston Exam Schools," Hoerle said in an email. "We look forward to an opportunity to bid and work with the Boston Public Schools in the future."
Northwest Evaluation Association, which makes Chicago's exam, indicated it is not interested in bidding to provide Boston's exam.
"We actively monitor and respond to procurement opportunities with Boston Public Schools," spokeswoman Jessica Schwartz Hahn said in an email. "We focus our search on opportunities most closely aligned with our assessment products and services. Entrance exams like the ISEE are not typically a focus of that search."
Pearson Education, which makes New York City's test, could not be reached for comment.
As of 2012, exam school students made up about one percent of public high school students in the country, according to a study published in the journal EducationNext.
