In the ongoing hearing on which tool North Carolina’s Department of Public Instruction (DPI) will use to measure K-3 student reading, the CEO of one vendor said Thursday that it was never notified of changes in DPI’s procurement requirements when going into final negotiations.
Amplify CEO Larry Berger took the witness stand Thursday to share the perspective of his company, which held the state contract in recent years to measure early-childhood literacy but lost the most recent bidding process to Istation. Berger said he thought the state purposefully changed the criteria, and their order of importance, to put more emphasis on areas in which Amplify scored poorly.
“I remember thinking, not just a random change happened and we were disadvantaged by it, but a change designed to disadvantage us had happened,” Berger testified.
DPI and Superintendent of Public Instruction Mark Johnson have stuck by their process and decision, saying Istation offered the best solution for the state.
Andrew Brown, an attorney for Istation, filed a motion for a directed verdict at the end of Thursday’s arguments, claiming that Amplify had failed to present evidence that showed “substantial prejudice” in the agency’s decision.
“There’s just simply no way to know that Amplify would have won,” Brown said, arguing that the evidence presented so far does not show that Amplify would have been the only vendor to enter into final negotiations and would have won the contract if not for procurement errors. “If the only thing you can do is speculate, then you can’t show substantial prejudice under case law,” he said.
Department of Information Technology (DIT) General Counsel Jonathan Shaw may respond to the request for a directed verdict Friday morning.
DPI’s procurement process set out to choose a vendor to satisfy two statutes — Read to Achieve legislation, aimed at improving reading results; and a state law requiring districts to screen students for risk of dyslexia.
DPI canceled that process for a second time in March 2019 and went into direct negotiations with Amplify and Istation. The four evaluation criteria listed on the final award recommendation are different from the six criteria in DPI’s earlier request for proposals (RFP). Vendors’ responses to that RFP were extended to remain valid throughout the negotiation process, despite cancellation of the procurement process.
After Istation was awarded the contract in June 2019, Amplify asked DIT to issue a stay on the Istation contract until the process could be reviewed. In the meantime, Istation provided its services for free and trained teachers on its tool, ISIP, to administer reading tests through the end of the school year. Below is a timeline of events since last summer.
Berger was one of several key players in the procurement process who testified this week, highlighting central questions in the case.
Mitch Armbruster, an attorney representing Amplify, has said the DIT — the agency presiding over the case — should cancel DPI’s contract with Istation and immediately award the contract to Amplify.
Armbruster’s argument has three main tenets: that DPI changed the evaluation criteria between the second procurement process and the negotiation stage without telling vendors; that DPI cancelled the second procurement process improperly; and that Istation does not meet a dyslexia screening requirement that was part of DPI’s requirements of vendors.
Change in criteria
The criteria in DPI’s request for proposal were as listed, in order of importance:
- Substantial conformity to specifications.
- RFP desired specifications.
- Proof of concept/demonstration.
- Vendor cost proposal.
- Vendor relevant experience and reference checks.
- Vendor financial stability.
The criteria that the final evaluation team used during the negotiations phase were as listed below, according to the final award recommendation document.
- Cost.
- Financial stability.
- Formative/diagnostic assessment.
- Personalized learning.
Special Assistant General Attorney Tiffany Lucas, representing DPI, pointed Thursday to a letter DPI sent to vendors notifying them of cancellation of the procurement process and the start of a negotiation phase. Lucas highlighted that the letter includes requirements listed for the new negotiation phase, which are different from the original RFP requirements. These items had cost listed first but included other items besides the four final criteria in the award document.
“Didn’t Amplify understand from reading this letter that those are the requirements that would inform the negotiations that it was going to engage in with DPI?” Lucas asked Berger during DPI’s cross-examination.
Berger said he took those items to be discussion topics for the negotiation meeting, rather than a new set of requirements.
Lucas asked him whether Amplify did not understand that certain topics laid out in the letter, such as cost and personalized learning, had “become increasingly important to DPI.”
Berger said he understood DPI wanted more information on those topics but that he thought the original RFP requirements were still being used for evaluation.
In response to questioning from Kieran Shanahan, Istation’s attorney, Berger said almost every page of Amplify’s proposal would have been different if the company had been aware of a change in criteria.
DIT procurement officers who oversaw the procurement process also testified this week about the ways they guided DPI staff in handling the procurement process, cancellation, and negotiation phase. Both Andrea Pacyna, a DIT deputy state procurement officer, and Patti Bowers, DIT’s chief IT procurement officer, said they did not notice the change in evaluation criteria going into negotiations.
“I don’t know how or when these changed,” Bowers said. She said she was unclear whether the shortened criteria encompassed the longer original criteria.
Procurement cancellation
According to March 2019 meeting minutes, evaluation team members were told the procurement had been canceled because of a breach in confidentiality by a team member.
In public records later released by DPI, that breach of confidentiality was explained as a text message chain about a January evaluation team meeting and vote.
Armbruster, representing Amplify, argued this week that DPI did not tell DIT, the agency that had to approve the cancellation, the full story.
In a document approved by DIT, the cancellation was justified by the large difference in cost between the proposals from Istation and Amplify.
Bowers and Pacyna, both from DIT, testified earlier this week that they were told by DPI staff that there were multiple reasons for the cancellation: a potential breach in confidentiality, the cost disparity, and a lack of consensus on the evaluation team. They also said they didn’t know the evaluation team had reviewed the costs of the vendors.
“DIT was not told that the bids had all been reviewed on cost, so there’s clear basis in the evidence that the cancellation was not proper on basis of cost and that DIT was not told everything,” Armbruster said Thursday.
The definition of “consensus” was raised multiple times in the hearing this week. Armbruster argued that the evaluation team had reached consensus to move Amplify forward before the procurement was canceled.
Bowers testified that DIT did not have a formal definition of consensus but that her understanding of the word is a unanimous agreement.
Brown argued Thursday that neither the evaluation team’s ranking of the vendors in November 2018 nor the team’s vote in January 2019 constituted a consensus.
In answering questions from Lucas, Bowers said that she believes the contract award to Istation followed proper procurement procedures.
Dyslexia screening
Brown, representing Istation, argued that the statute included in the RFP that requires students to be screened for dyslexia was not a mandatory requirement to win the contract, given that the law does not mention DPI.
House Bill 149 requires local districts to screen children for risk of dyslexia with support and oversight of the State Board of Education.
Because the bill does not require dyslexia screening from DPI, Brown argued Thursday, the RFP specification was not mandatory.
“Patti Bowers told us clearly that any specification not required by law is optional,” Brown said. “It’s effectively a desired specification.”
Armbruster, representing Amplify, argued Istation’s tool does not reflect “best practices” as it should according to the RFP specification. Armbruster also pointed to the early evaluation team’s work where all 11 members either voted “no” or “maybe” to whether Istation’s tool met the specification.
Earlier in the week, Susan Laney, a DPI data consultant and evaluation team member, said she does not believe Istation’s tool reflects best practices in screening children for dyslexia. She said it was concerning that Istation’s tool did not require students to produce oral sounds but instead relied on clicking or touching a screen.
Laney did, however, agree that Istation’s tool met basic psychometrics to screen students for reading difficulty.
Brown argued Thursday that best practices are not clearly defined in the field.
“The state of the art right now is that it’s very difficult to distinguish reading difficulty from dyslexia in … the kind of assessment that we’re looking at here,” Brown said.
The hearing will continue at least through Friday. Shaw will recommend a decision to DIT Secretary Eric Boyette who will make the final decision in the hearing.
