What could go wrong when your employees commit internal information to public GitHub repos?
While we were the first to report on the SolarWinds security vulnerability that possibly could have exposed their Downloads FTP server credentials letting attackers to push malicious binaries and attack the US government and SolarWinds' other high profile clients, some more information has surfaced regarding the SolarWinds security vulnerability since then, that gives more insight into what possibly was exposed and whether it could have led to this massive breach of the US government. While majority of security researchers are of the opinion that this wasn't the main reason of the breach, and that there was a complex and sophisticated supply chain attack targeting SolarWinds, we believe these small security lapses could have given the attackers a larger attack surface to carry out their attacks and eventually might have helped strengthen their foothold into the SolarWinds infrastructure, to perform reconnaissance and evade detection.
Important – Please note that we are not claiming this is how SolarWinds got hacked. This post covers the insecure and lax security practices of SolarWinds which might have contributed to the security breach. But we are not saying this is how, it happened!
Plain old FTP to the blame?
As per the screenshot posted by Vinoth, which we wrote about in our previous post, SolarWinds were possibly using unencrypted plain FTP server for their Downloads server in the age of global CDN technologies. However, not a direct attack vector its very likely that the FTP server had more vulnerabilities and unencrypted communication can always be intercepted, and modified. But we don't believe this maybe something as concerning as the FTP password leak.
SolarWinds Credentials were possibly leaking since 2018
Security researcher Vinoth Kumar, told us that "SolarWinds had been possibly exposing the FTP credentials to the Download server since at least 2018". To corroborate his claim, Vinoth shared the the following link to the Configuration file exposed that was exposed in the mib-importer GitHub repo possibly belonging to a SolarWinds employee, https://github.com/xkozus00/mib-importer/blob/master/Src/Lib/PurgeApp/PurgeApp.exe.config and he further added that, upon supplying the repo base url to the Web Archive, it shows Web Archive had first archived the page back in June 2018, and that was the last time the page was archived. So we concluded that the credentials to the FTP server and other potentially sensitive information in that exposed repository possibly existed for more than 1 year in the public domain until Vinoth reported it to the SolarWinds PSIRT.
Exposed Information might have allowed Attackers to gain foothold over SolarWinds Infrastructure
This shows that SolarWinds might have been exposing their sensitive internal credentials since a fairly long time before it was brought to their notice, which in turn might have given its attackers an opportunity to steal certificates and other valuable internal information about SolarWinds to carry out the large scale attack against US government and other top organizations using the backdoored SolarWinds Orion software.
The mib-importer GitHub repository
A "mib-importer" public GitHub repository, possibly belonging to a SolarWinds employee with secrets (like FTP username and password) exposed, was found on GitHub by the security researcher in November 2019, which is said to have existed from around June 2018
Upon analyzing, the SaveBreach team found out that SolarWinds Orion lets users import MIB files into it. MIB files are used for monitoring network devices. Apparently, the mib-importer tool was developed by SolarWinds to import MIB files into Orion. We found the following data from the SolarWinds documentation pages regarding importing MIB files (Reference – 1, 2, 3)
Management Information Base (MIB) is a structure that describes all objects a device can report on, such as CPU, fan, or temperature. MIB contains the name, datatype, and the object identifier (OID). MIB is a hierarchical structure, displayed as a navigation tree. Every entry in the MIB tree is a value for a specific component on a specific device.
SolarWinds maintains a MIB database that serves as a repository for the OIDs used to monitor a wide variety of network devices. The MIB database is updated regularly.
Q&A with cybersecurity researcher Vinoth
From our most recent conversation with Vinoth, it appears that the credentials and possibly more sensitive data about SolarWinds was lying in public domain for a long time before finally being taken down. Vinoth doubts that the data might have also included certificates and not just FTP credentials, which was alone sufficient to sign the malicious binaries and upload them to the FTP server while passing off as legitimate software.
Q: Since when do you think the GitHub repo might have been exposed? Do you think the attackers could have gained persistence into their infrastructure for almost 3 years to carry out the attack?
Vinoth: I’m not sure, even on June 2018, 40 commits were there (in that repo). There was only one page available on archive couldn’t find anything else.
Q: The Attackers had put signed binaries on the Download server. What was the process the attackers might have followed after getting access to the file upload server to sign the binaries?
Vinoth: Not sure, but I could have missed checking the repo which could have had the certificate in it.
Vinoth had tweeted that the GitHub repo was open to public since 17th June, 2018
Certificates used to sign malicious binaries exposed through GitHub repo?
This raises many questions. Were the certificates used to sign the binaries obtained from that public GitHub repository or, from any other information leaked publicly? Exposed certificate could have allowed hackers to sign their malicious SolarWinds Orion binaries and pass them off as legitimate software developed by SolarWinds, subsequently uploading them to the Downloads server with the previously found leaked FTP credentials.
Was this How SolarWinds got hacked?
We can come to a partial conclusion that the internal information exposed on GitHub was there for a sufficiently long time for the attackers to have already exploited them to gain their initial foothold. Although unclear at this point, as there maybe a more sophisticated and complex attack chain with evasion techniques as being claimed by FireEye and security researchers, but we do feel this might have been a precursor to the SolarWinds breach and the widespread cyber attack against the US Government.