Salt Vulnerabilities Exploited with Targeted Cryptomining Attack on DigiCert
Skybox Blog Team May 20, 2020
The SSL/TLS certificate distributor DigiCert recently revealed that it has fallen foul of attackers who took advantage of two Salt vulnerabilities to disrupt its exposed infrastructure with cryptocurrency miner software. For the uninitiated, Salt is a configuration management and orchestration system that is used to monitor servers and maintain their configurations; it works with a master that publishes commands in queues that can then be subscribed to by slave machines that it calls “minions”.
The attack could have been far worse: the attackers could have gained access to one of DigiCert’s Certificate Transparency (CT) server’s signing key. But they appeared to be too distracted by making sure that their cryptominers were operational. A close shave for DigiCert, with some lessons that can be learned.
How do the Salt Vulnerabilities Work?
The two vulnerabilities (SBV-11651 AND SBV-11652) work in a chain. Essentially, when exploited, attackers are able to request server port on master open to root RCE. This function is an integral part of Salt’s pub/sub structure that has been set up to “send results to the Salt master, and to securely request files and minion-specific data values”.
While the vulnerabilities work collaboratively, they have two separate functions: the first focuses on authentication bypass where functionality is unwittingly exposed to unauthenticated network clients, and the second on directory traversal where untrusted input has not been properly vetted.
When investigating these flaws, cybersecurity firm F-Secure discovered that they also accept insertions to the publish queue that are then able to be executed on any or all minions as root. When this happens, it leaves a key unprotected that can then be used to open the door to local root commands on the master. So when data come in through a door meant for slaves and gets injected in the master, the master then bounces it back to other slaves to do the bidding of the sender – conjuring up an interesting reflective image. The scope of the commands made is arbitrary due to the directory traversal in a function that accepts paths as filenames.
How were the Salt Vulnerabilities Discovered?
When F-Secure discovered the chain of vulnerabilities, the company considered them to be too trivial for a motivated hacker to get up and running. As a result, it did not release a PoC when it first came across them on April 30th, 2020. Although F-Secure were not keen to start the PoC ball rolling, that didn’t stop a number becoming publicly available soon after from several different sources.
This doesn’t mean that these vulnerabilities are brand new: the bug was five years old before it was first privately disclosed on March 16th, 2020, with a patch created on April 29th,2020. F-Secure estimates that over 6,000 instances of the Salt service have been exposed to the public internet, including some critical machines like those in data centers. Although those managing such sensitive machines may be hesitant to apply the patch, they should do so now if they haven’t already – DigiCert’s close shave should be seen as a clear and present warning signal.
The DigiCert Exploit
Perhaps the most surprising aspect of this story is the fact that the attackers decided to exploit the vulnerabilities to deploy cryptocurrency miners. By the time they made their move on May 2nd, 2020, the malware had long since passed the time when it could have been considered a bête noire. In the face of declining cryptocurrency values and decreasing yields, it appeared that many attackers were giving up on the malware in favor of higher yield tactics like ransomware. Indeed, the Skybox Security Vulnerability and Threat Trends Report 2020 found that the creation of new cryptomining samples decreased by 48 percent over 2019 – a fall from its 2018 high as the most popular malware family.
This attack, however, shows that security teams still need to be vigilant against cryptomining attacks. The malware can still be profitable if attackers can scoop up a lot at once, particularly if it’s used within large networks or distributed log servers. Consider DigiCert’s position as the second-largest company in the website signing field, issuing SSL certificates to 600,000 domains, and it’s clear that the criminals saw this attack as an opportunity that was too good to pass up.
DigiCert was also a good target for attack because its services need to be accessed by sites on the internet at all times, thus increasing its exposure to the vulnerabilities. This heightened exposure level wasn’t helped by their admission that they were working with weak resources that had outlived their security, with DigiCert’s EVP of Product Jeremy Rowley sharing that, “it’s a very old log (and tangentially one of the reasons we’ve been trying to shut it down). This means that the Salt-master had to be exposed to the public internet as that was the way salt worked. When the attacker gained access to the event bus, they could have requested the master to deliver the CT server’s signing key.”
This isn’t the only time that these flaws have been exploited, with subsequent successful attacks made on LineageOS, Ghost and Algolia, the latter of which admitted that the attack on 500 of its servers and its configuration management system was made possible because it had lapsed in its security management, admitting that, “the impacted component was an older tool, and has been running fine, but we were planning to rework it in the following two quarters.”
What Lessons Can Be Learned?
It’s easy to sense the palpable relief shared by Rowley after the attack was made public, stating that “the attacker doesn’t seem to realize that they gained access to the keys and were running other services on the infrastructure.” But what started out as a minor issue has since developed. The malware has the ability to tear-down anything that’s CPU-intensive (like confluence, databases, containers, java applications, etc.), to achieve its cryptomining goals. Additional backdoors are being installed and a large number of private keys are now being stolen.
Anticipating the release of a patch, attackers established a workaround that hid behind a false payload and has been able to continue operations in the background. The best way to stop these attackers in their tracks is probably not feasible for most organizations – to wipe and restore all systems. If that isn’t possible, it might not be enough to merely update your Salt-master, but to create an entirely new Salt-master instead.
By their own admissions, the larger-scale attacks on DigiCert and Algolia could have been preventable if they had more robust security management processes in place. These attacks need to be taken as reinforcement that criminals do not rest in the pursuit of profit and that organizations need to have a united ecosystem that is set-up to understand which of their vulnerabilities are most exposed; and are in a position to apply rapid patches.