SolarWinds — how can we protect the software development?
External attackers of the SolarWinds Orion platform used for infrastructure monitoring and management introduced a command & control backdoor in the source code. It resided in a digitally signed dll. Product was widely used and a lot of clients became victims (government agencies including nuclear, Microsoft, FireEye, Intel, Cisco).
Due to bugs in VMware identity and access management the attackers gained full access to on-premise systems, stole keys used in SAML Single Sign On and achieved access to cloud resources. Modifying the source code of Orion platform was easy.
A new C# class named OrionImprovementBusinessLayer was added. Every change in the codebase should be labeled with the Jira ticket number and created in the dedicated feature branch. This branch should be integrated into main line using pull request and code review. If the hacker possessed authentication token then was able to amend feature branch commit and merge commit. The development process looked legit — a feature branch, a code review and a merge. With a standard Scrum it was not possible to detect this breach.
How can we protect? We need another role in a development process — a gatekeeper who would verify the codebase between releases if the increment is align with the Sprint goals and there are no injected commits not related to the business requests. It’s that simple, but time consuming. It looks like that’s the only possible countermeasure now. We can also automate the process of detecting amended git commits using reflog (see: https://www.atlassian.com/git/tutorials/rewriting-history). Still contact of the gatekeeper and developers for explanations is needed (and time for this).
