Kaseya announced it had obtained a universal decryption key for ransom-ware victims
By notorious Russia-linked ransom-ware group REvil in July 2021, the attack on US-based software provider Kaseya is estimated to have affected up to 2,000 global organizations. To launch the ransom-ware attack, REvil targeted vulnerability (CVE-2021-30116) in a Kaseya remote computer management tool, with the fallout lasting for weeks as more and more information on the incident came to light.
The event served as a reminder of the threats posed by software supply chains and sophisticated ransom-ware groups.
Kaseya’s incident response team, on 2 July detects a potential security incident involving its remote computer management tool Kaseya VSA
The company, with an investigation underway, advised all on-premises customers to shut down their VSA servers until further notice, while also shutting down its SaaS servers as a precautionary measure. Alerting enforcement and government cyber-security agencies, including the FBI and CISA, Kaseya’s internal team, alongside security experts, worked to determine the cause of the issue. Kaseya said early indicators suggested that only a small number of on-premises Kaseya customers (40) were affected and that they had identified the vulnerability source. A patch was being prepared as of 10PM EDT.
Kaseya, on 3 July confirms that it was the victim of a cyber-attack
Kaseya continued to strongly recommend its on-premises customers to keep VSA servers offline until it released a patch. It also advised any customers that were experiencing a ransom-ware attack and had received communication from the attackers to avoid clicking on any links. The company announced it was making a compromise detection tool available to VSA customers to help them assess the status of their systems. Kaseya continued to contact impacted users and stated that CEO Fred Voccola would be interviewed on the incident on Good Morning America the following day.
Kaseya announces on 4 July delay in bringing data centres back online, releases compromise detection tool
Kaseya’s executive committee met and determined that, to best minimize customer risk, more time was needed before bringing data centres back online. In an interview on Good Morning America, Voccola said, “We are confident we know how it happened and we are remediating it.” The compromise detection tool was made publicly available via download, while the FBI and CISA issued their own joint guidance for MSPs and their customers impacted by the attack, urging them to take action such as ensuring backups are up to date and stored in an easily retrievable location that is air-gapped from the organizational network, reverting to a manual patch management process and implementing multi-factor authentication. REvil operators boasted on the group’s “Happy Blog” that more than a million individual devices were infected, and that they would provide a universal decryption key to Kaseya for $70 million in Bitcoin.
Kaseya, on 5 July claims fewer than 60 customers compromised, patch being tested
Kaseya promised that the patch for on-premises users was being tested and would be made available within 24 hours. Amid widespread media reports of the attack, the company estimated that it would be able to bring its SaaS severs back online between 4PM and 7PM EDT on 6 July.
Kaseya, on 6 July adds security layers to SaaS infrastructure
Kaseya began configuring an additional layer of security to its SaaS infrastructure to change the underlying IP address of its VSA servers, allowing them to gradually come back online. However, upon rollout, an issue was discovered, delaying the release. Operations teams worked through the night to fix the issue with an update due the following morning. An update on the on-premises patch stated that 24 hours or less remained the estimated timescale. Across the pond, the UK’s National Cyber Security Centre said the impact of the attack on UK organizations appeared to be “limited”, though it advised customers to follow Kaseya guidance as a precaution.
Kaseya, on 7 July apologizes for SaaS and on-premises fix delays
Kaseya published a guide for on-premises customers to prepare for the patch launch and stated that a new update from Voccola was to be emailed to users clarifying the current situation. The company apologized for on-going delays with SaaS and on-premises fix deployment.
US government, on 8 July, tells Russia it will be held accountable; Kaseya pushes back patch release
Jen Psaki, White House press secretary said that a “high level” of US national security had contacted top Russian officials about the Kaseya attack to make clear its intentions to hold Russia responsible for criminal actions taking place within its borders. She also said that another ransom-ware-focused meeting between the two countries was scheduled for the following week.
Meanwhile, Kaseya set a new estimate of Sunday 11 July for the launch of the on-premises patch, while it was starting deployment to its SaaS infrastructure. Kaseya released two update videos, one from Voccola and another from CTO Dan Timpson, addressing the situation, progress, and next steps. The company also warned of spammers exploiting the incident by sending phishing emails with fake notifications containing malicious links and attachments. It stated that it would not send any email updates containing links or attachments.
Kaseya, on 9 July updates VSA hardening advice
Kaseya updated its VSA On-Premise Hardening and Practice Guide while executive vice president Mike Sanders spoke of the team’s continued work towards getting customers back up and running. He also raised awareness of on-going, suspicious communications coming from outside Kaseya.
Report says, on 10 July Kaseya was warned of security flaw
Kaseya said it remained on course to release the on-premises patch and have its SaaS infrastructure online by Sunday 11 July at 4PM EDT. The latest video update from Sanders outlined steps companies could take to prepare for the launch. Meanwhile, a Bloomberg article reported that, according to ex-employees of the company, executives at Kaseya were warned of critical security flaws in its software on several occasions between 2017 and 2020, which they failed to address.
Kaseya release patch on 11 July, begins SaaS restoration
Kaseya launched the on-premises patch and began restoring its SaaS infrastructure ahead of the 4PM target. As of 10PM EDT, it claimed to have 60 per cent of SaaS customers live and servers due online for the rest of its customers in the coming hours. Support teams were working with any on-premises customers requiring assistance with the patch.
On 12 July, SaaS restoration completed
As of 3:30AM EDT, the restoration of Kaseya’s SaaS infrastructure was complete. However, it was forced to carry out unplanned maintenance due to performance issues, causing a short downtime. It continued to support on-premises users with patch assistance.
On 13 July, REvil websites disappear
Leaving security experts to speculate potential action by US or Russian governments, all REvil ransom-ware gang websites suddenly went offline. This left some victims unable to negotiate with REvil to recover data through a decryption key to unlock encrypted networks. At Kaseya, advisors prompted users to continue to review its various customer guides to dealing with the incident and getting back online.
Kaseya, on 14 July, issues patch install check advice to customers
“When running the Kinstall patch on your VSA, if you chose to reinstall VSA and either unchecked the default option to install the latest patch, or reran the Reinstall VSA process a second time without the ‘install patch’ option selected, it’s possible your patch was not re-applied,” the company wrote. “While these are rare edge cases, we recommend that you verify that the latest patch was installed properly. We have made a tool that enables you to ensure the patch is properly installed.”
On 16 July, victims struggle with decryption tool, Kaseya releases non-security patch
Despite having paid for the decryption tool but with no way of contacting REvil for support as REvil’s websites was still offline, some victims struggled to unlock files and systems. Kaseya announced it was releasing a non-security-related patch (9.5.7.3011) to fix functionality issues caused by enhanced security measures and other bugs. Deployments were estimated to begin on 17 July (SaaS) and 19 July (on-premises).
First updated SaaS patch deployments go live on 17 July
Remainder of updated SaaS patch deployments go live on 19 July
New functionality patches released on 20 July
Kaseya provided further patch updates (9.5.7.3015) to fix functionality issues and bugs, and made the updated on-premises patch available.
SaaS functionality updated on 21 July
Kaseya again updated SaaS instances to remediate functionality issues and provide minor bug fixes. This resulted in a brief interruption (2 to 10 minutes) as services were restarted.
Kaseya acquires universal decryption key on 22 July
Kaseya announced it had obtained a universal decryption key for ransom-ware victims. “We can confirm that Kaseya obtained the tool from a third party and have teams actively helping customers affected by the ransom-ware to restore their environments, with no reports of any problem or issues associated with the decryptor,” the company wrote. “Kaseya is working with Emsisoft to support our customer engagement efforts, and Emsisoft has confirmed the key is effective at unlocking victims. Customers who have been impacted by the ransom-ware will be contacted by Kaseya representatives.” Across the industry, mass speculation arose as to exactly how Kaseya accessed the decryption tool and whether a ransom payment was involved.
On 23 July, another functional patch and SaaS update released, Kaseya reportedly requests non-disclosure for decryptor
Details of how it became available remained unclear as news of the decryption key made global headlines. Meanwhile, for on-premises customers to resolve three non-security issues, Kaseya released a quick fix patch 9.5.7b (9.5.7.3015). All SaaS instances were also updated. According to a CNN report, Kaseya was requesting the signing of a non-disclosure agreement for customer access to the decryptor.
Kaseya, on 24 July, declines to comment on ransom payment
With Kaseya declining to comment on whether it had paid a ransom, security sources and outlets continued to speculate as to the details of how the decryption key was obtained.
Kaseya, on 26 July says decryption tool “100 per cent effective,” no ransom paid
Kaseya released the following statement on the decryption key: “Throughout this past weekend, Kaseya’s incident response team and Emsisoft partners continued their work assisting our customers and others with the restoration of their encrypted data. We continue to provide the decryptor to customers that request it, and we encourage all our customers whose data may have been encrypted during the attack to reach out to your contacts at Kaseya. The decryption tool has proven 100 per cent effective at decrypting files that were fully encrypted in the attack”.
Despite claims that Kaseya’s silence over whether it had paid attackers a ransom could encourage additional ransom-ware attacks, the company argued that nothing was further from its goal. “While each company must make its own decision on whether to pay the ransom, Kaseya decided after consultation with experts to not negotiate with the criminals who perpetrated this attack and we have not wavered from that commitment. As such, we are confirming in no uncertain terms that Kaseya did not pay a ransom either directly or indirectly through a third party to obtain the decryptor.
Also read: CIO News interviews Shri Wangki Lowang, Minister (IT) of Arunachal Pradesh
Do Follow: CIO News LinkedIn Account | CIO News Facebook | CIO News Youtube | CIO News Twitter
