The timeline: Records show the immediate aftermath of the cyber attack on Rochester Public Schools
Although the district initially shut everything on its network down, it soon after restored email access to the district's top administration.
ROCHESTER — It was 4:30 a.m. on Thursday, April 6, 2023, when an IT worker first flagged the cyberattack on Rochester Public Schools.
That was the beginning of a damage-control process that would continue on for more than a month and which is still, technically, underway.
The situation that RPS eventually confirmed to be a ransomware attack forced the district to cut off its virtual networks to mitigate the damage and restore its operations. That, in turn, caused the district to announce it was canceling classes for a day to allow the staff to deal with the situation.
The series of events affected every person educated or employed by the school district, altering both its operations, how teachers taught and how students learned their lessons.
In response to a data request from the Rochester Post Bulletin, RPS released a series of communications between school leaders that took place in the wake of the attack. Although the district initially shut its network down, it soon thereafter restored email access to the district's top administration.
The following is a timeline of the situation:
4:30 a.m., Thursday, April 6: An IT worker notices the security breach.
9:29 a.m. Thursday, April 6: RPS officials communicate briefly with each other about the district’s cyber liability insurance policy. It lists the following sections:
Incident response costs, legal and regulatory costs, IT security and forensic costs, crisis and communication costs, privacy breach management costs, system damage and rectification costs, income loss and extra expense, dependent business interruption, network security liability, privacy liability, management liability, regulatory fines, PCI fines, penalties and assessments, defamation, intellectual property rights infringement, and court attendance costs.
Almost all the sections list an “aggregate limit of liability” of $2 million and a deductible of $75,000 for “each and every claim.” For the section of court attendance costs, the aggregate limit of liability is listed as $1 million.
10:23 a.m., Thursday, April 6: RPS Chief Administrative Officer John Carlson notifies members of the Rochester School Board about the situation, explaining how the attack came through a compromised vendor account.
“A technology employee logged in to the network from home to do some work on servers outside of normal business hours,” Carlson wrote in his message to the school board. “The employee noticed someone with a vendor account (not an employee of RPS) with higher level access in the system shutting things down inappropriately. We believe the vendor who had the username and password was compromised.”
Carlson went on to say that the district created an “incident response team."
11:35: a.m., Thursday, April 6: Carlson updated the board again, saying that the district had contacted the FBI and filed a report with the Rochester Police Department.
“We are holding on bringing anything back online until the cyber liability insurance incident response team gives us their supports and directions so we don’t make things worse and put us down longer,” Carlson’s update said.
The update also notified the board that IT workers found an electronic ransom note, which said the bad actors’ demand would depend on the school district’s response.
The ransom note reads:
“!!! THE ENTIRE NETWORK IS ENCRYPTED !!!
YOUR BUSINESS IS LOSING MONEY
All documents, databases, backups and other critical data were encrypted and leaked. The program uses a secure AES algorithm, which makes decryption impossible without contacting us. If you refuse to negotiate, the data will be auctioned off.
The price depends on how soon you will contact us.”
1:38 p.m., Thursday, April 6: Peter Alsis, a representative from Minnesota IT Services, emails RPS IT Director Mike Johnson.
"Please let us know if there's anything the state can do to provide support. We can establish threat intel searches retroactively and moving forward," Alsis wrote.
Alsis wrote that message after receiving a notification about the cyberattack from an organization called MS-ISAC, or the Multi-State Information Sharing and Analysis Center.
11:25 p.m., Thursday, April 6: Rochester Public Schools Communications Director Mamisoa Knutson emails Superintendent Kent Pekel a draft message they were preparing to send to families in the district. Knutson and Pekel were reviewing edits to the draft made by FleishmanHillard, a global public relations firm.
Among other changes, the draft message included a notable cross-out edit recommended by the PR company:
Cyberattack (note from them: 'Cyberattack' is severe language that we prefer to avoid when possible)."
6 a.m., Friday, April 7: Rochester Public Schools notifies families in the district about the situation, saying it had detected “irregular activity on its network.”
4:48 p.m. Saturday, April 8: Rochester Public Schools cancels classes for the following Monday in order to allow staff time to address the developing situation.
"Because it would be very difficult to provide students with instruction and school services without access to the Internet and core systems, we are going to ask students not to report to school on Monday, April 10," the notice said. "We will use that day to plan on how to operate school with no or reduced access to technology systems starting on Tuesday, April 11."
7 a.m., Monday, April 10: The district administration provides a list of talking points to school principals. The document exemplifies just how many of the district’s systems can be affected by a cyberattack, which staff were instructed to refer to as a “cyber event.” Door buzzers and fobs would work. Copiers wouldn’t. It was unknown how some systems, like payroll and the thermostats, would function.
“We expect temperatures to at least heat up to 60 and cool down to 80 which are the holiday-mode settings they were in when this happened,” the talking points document said. “We may not be able to change temperatures.”
5:30 p.m., Tuesday, April 11: The Rochester School Board recognizes students who competed in the Minnesota State Science and Engineering Fair. At the end of the presentation, the superintendent made a light-hearted job pitch to the students.
“I want to know if any of you are interested in cybersecurity?” Pekel asked, prompting laughs from the audience.
Various times and dates: RPS leaders get a variety of community feedback, ranging from the irritated, to the sympathetic, to the helpful:
The irritated: “Can I get clarification as to why Rochester Public Schools needs a day off to … figure out how to teach without technology? Haven’t teachers been teaching without technology for decades? Centuries?
Please, explain to me, what Monday will accomplish. Other than establishing YET AGAIN that RPS has yet to put the children — their wellbeing, and their education first.”
- The sympathetic: “Wishing RPS all the best in getting the technology sorted out soon and hope you’re all still able to enjoy the Easter weekend.”
- The helpful: “If you need additional highly trained assistance I am here to voluntarily help. Responding to cybersecurity incidents is what I do for a living.”
2:05 p.m., Thursday, May 4: RPS sends an update to families.
“We can now confirm that this was a ransomware event,” the update said. “We have alerted the FBI, and we did not pay a ransom. We could not disclose the ransomware until now so as to protect the integrity of our investigation.”