Skip to content

Johnson sticks by Istation pick, explains controversial evaluation process

Superintendent of Public Instruction Mark Johnson stood by his choice to switch to a new tool from a company called Istation to measure K-3 reading proficiency Friday, denying a protest filed by Amplify, the company the state has contracted with in recent years to assess early literacy. That protest questioned the ability of Istation’s platform to meet the state’s needs and raised concerns about the procurement process that led to Johnson landing on Istation in June. 

“Istation is the best diagnostic tool for the state of North Carolina,” Johnson wrote in a response to Amplify’s protest, which can be found in full below. “Istation received the contract award through a fair and objective process conducted by an evaluation committee that adhered to all laws, rules, and policies. The evaluation committee recommended Istation to me and to the State Board of Education (the ‘State Board’). I approved that recommendation, and the State Board unanimously approved that recommendation.”

Istation President Ossa Fisher released a statement Friday saying the company was awarded the contract legally and fairly.

“We are grateful, but not surprised, to learn that the Department of Instruction upheld its decision to select Istation as the state’s K-3 assessment tool and has finalized the selection process surrounding this contract. Istation was legally and appropriately awarded the contract confirming that Istation’s tools meet the requirements of the state’s Read to Achieve law while also addressing all of the areas of Universal Screening, progress monitoring, rates of improvement and gap analyses. Istation’s programs also meet North Carolina’s dyslexia screening laws, according to the initial screening criteria set by the state.”

Amplify CEO Larry Berger said the document above still shows issues in the procurement process.

“The more evidence that comes out, including what we see in the DPI letter today, the more it seems to us that there were serious flaws in the procurement. We will take the next steps in the protest process and remain open to constructive solutions that work for North Carolina educators and students.”

Documents formerly released by DPI showed the evaluation team Johnson assembled to choose a new vendor recommending Amplify as its first choice and Istation as its second. Activists and Democratic legislators have called for investigations into why that recommendation was not followed.

However, Johnson and DPI have insisted that the team did not reach a consensus or make a top recommendation. In Johnson’s response above from Friday, he repeats that claim: “The only vote in RFP-2 occurred in January 2019 – it was a vote on which vendor(s) to continue negotiating with – and did not result in consensus, so there was no recommendation.”

January 2019 meeting notes explain a voting process where Johnson left the room and evaluation team members wrote their choices on sticky notes. According to the document, six voting members chose Amplify, three voting members chose Istation, and one voting member chose to continue negotiations with both vendors. 

Johnson also provided explanations around why the RFP (request for proposal) process was canceled twice, once in March 2018 and once in March 2019. DPI spokesperson Graham Wilson previously said the first cancellation was because of a conflict of interest.

Friday’s response said the first cancellation was “due to a lack of competition and errors in specifications.”

The document goes on to read: “Even if RFP-1 had not been canceled for this reason, the Department would have had to cancel RFP-1 due to an undisclosed conflict of interest.”

The second RFP was canceled in March 2019, according to Johnson’s response, because of both a breach of confidentiality and evaluation strategies that made the process unfair.

“The Department discovered improper and unfair procedures were used during the evaluation and that a voting member of the evaluation committee breached the Confidentiality Agreement by disclosing confidential details about the procurement process.”

The response goes on to list the reasons for the second cancellation: 

  1. breach of confidentiality agreement;
  2. unfair and improper ranking procedures;
  3. misstatements of fact leading to improper rankings;
  4. improper weights leading to improper rankings; and
  5. lack of consensus and unclear basis for comparison
 

Evaluation team members signed non-disclosure agreements at the beginning of the process, according to Friday’s document. The breach of confidentiality is explained in a chain of text messages between two former DPI employees, where one individual said he or she had talked to an evaluation team member about the RFP process and members’ votes. 

In those texts from January, the same individual also says Johnson came to an evaluation team meeting and tried to influence members’ votes. “However the vote did not go his way so it will be interesting to see how he gets his way on this,” the text message goes on to read. The same individual says, after that meeting, “… several people said what just happened?” Read the full text message exchange below.

 
 

During the second evaluation process, the response says the ranking process of the vendors turned members’ “maybe” votes into “no’s” without informing those members. Some of those “maybe’s” and “no’s,” the document says, were also based on “misstatements of fact.”

The document goes on to say that evaluation team members in the second process let their own opinions of platforms get in the way of providing unbiased advice.

“The voting members of the evaluation committee for RFP-2 followed guidance from whomever they considered to be a given topic’s subject matter expert on the evaluation committee. Unfortunately, in multiple cases, the opinion voiced by a subject matter expert was not grounded in fact. Members of the evaluation committee, whether intentionally or not, were putting forth untrue statements about vendor products, but these falsehoods were treated as facts by the rest of the evaluation committee. Consequently, RFP-2 voting and rankings were based on misstatements of fact and incorrect data.”

The document also says the evaluation process used weights in scoring the vendors, going against the selection method that the solicitation document required.

“The RFP-2 evaluation committee improperly added weights to the scoring that prevented a best value procurement process. Because of this violation combined with the unfair and improper procedures described above, even a worthy product that was offered at no cost could have scored lower than Amplify.”

Friday’s document also defends Istation’s platform that will be implemented in the upcoming school year. Teacher training has already begun, but formal data collection for the state’s EVAAS growth measure will not be collected until the middle of the year. The document responds to claims from Amplify that the tool is developmentally inappropriate and lacks appropriate dyslexia screening measures, among others.

“Throughout this protest for Amplify, you make incorrect claims about Istation to argue in favor of Amplify’s request. Istation has already been used successfully in other states and more than satisfies the requirements to be the reading diagnostic tool for North Carolina. Contrary to your incorrect claims:

  • Istation satisfied the requirements of the procurement,
  • Istation meets the requirements of law,
  • Istation predicts students who are at risk of a reading difficulty,
  • Istation screens for dyslexia,
  • Istation is developmentally appropriate,
  • Istation assesses required measures,
  • Istation is effective at measuring a student’s progress,
  • Istation allows for students to interact with and read aloud to teachers,
  • Istation provides sufficient data to allow determinations of short-term student progress,
  • Istation accurately and reliably determines which students will be reading on grade level by the end of the school year, and
  • Istation is already on track for successful implementation in North Carolina.”
 
Liz Bell

Liz Bell is the early childhood reporter for EducationNC.