The Danger Hiding in Your SSH User Keys
Sometimes a problem lies lurking in the background, silently growing in scope and severity until it erupts in devastation. After the fact, people go back and try to put the pieces together so it doesn’t happen again. This has happened in international politics, family dynamics, the stock market, business and is now happening in your network.
Specifically, the problem hidden in your network is that of poor SSH protocol and user key management. Surprisingly, this challenge has strikingly similar parallels to the 2008 housing crash. The comparison may seem strange at first, but there are three parallels. The first is the understanding of the problem, or lack thereof. Second is the challenge related to oversight of the problem. And the third is how significant the impact or consequences are of not addressing the problem in our enterprises.
Ignorance, Not Bliss
The housing crash was precipitated by the subprime loan market, which very few people understand. In the same way, the SSH protocol is something that very few understand in sufficient detail, but it is the underlying critical access it is providing to our most important infrastructure. In this case however, ignorance is not bliss.
Administrators and developers need an effective means of gaining encrypted access to critical infrastructure, and more than 95 percent of the world’s enterprises rely on SSH to provide it. This includes access to operating systems, applications, payment processing systems, databases, human resource and financial systems, routers, switches, firewalls and other network devices. It is a lifeline of traffic flow within our data centers, our cloud environments and how our third-party vendors and supply chain access our environments. It has done its job quietly and efficiently over the last two decades. Unfortunately, the access that SSH has been providing, in particular the access SSH user keys provide, has gone largely unmanaged – to an epic degree.
As an example, a typical financial enterprise with 20,000 Unix/Linux servers is likely to have up to 4 million SSH user keys providing interactive and machine-to-machine-based access. In many cases, we will see that 10 to 20 percent of these keys provide root-level access and cannot be associated to an owner within the enterprise. Root-level access is the highest level of privilege at an operating system level. It is not just a compliance and risk issue. It is an issue of resilience that has the opportunity to impact the potential downtime of critical services within our operations.
Oversight Has Been Overlooked
This is, indeed, a problem of epic proportions. How, then, has it escaped notice for so long? Primarily because SSH has long been seen as an encryption protocol rather than a means of access and, as a result, has not been considered as a part of our access governance processes and frameworks. In fact, up until October 2015, there were no NIST guidelines related to the best practices associated with SSH user key-based access.
Access controls are mentioned in regulatory guidelines like HIPAA, PCI and SOX—things like least privilege and segregation of duties—but none of them specifically address SSH user keys as a form of access that needs to be controlled. Is this because of the lack of understanding of SSH or an unwillingness to open Pandora’s Box?
In terms of the lack of oversight and governance of SSH user key-based access, three technical issues must be addressed. First, SSH user keys are the only form of access a user can provision themselves without oversight or control. Unlike passwords, an administrator, developer or third-party vendor can provision themselves access to your critical infrastructure or automate a process or file transfer. This process of provisioning, de-provisioning and recertification of SSH user key-based access is infrequently if never addressed in the IAM frameworks of enterprises.
The second technical issue is that SSH user keys—unlike certificates—don’t have expiration dates. This means that SSH user key-based access continues to exist, even after an application is decommissioned or a user leaves the company.
The third issue is that an SSH user key does not need to be associated to a user account. This means a key will not necessarily establish the identity of the user associated with it. When an SSH user key is generated, the identity that is associated is not connected.
What these technical issues mean in aggregate is that a user can provision SSH user key-based access themselves without oversight to my most critical infrastructure. This key-based access does not expire. Does that mean it is not clear which identity it is associated with? And there are millions of these across my enterprise, which we don’t have an inventory of, and they are providing access to my most critical systems? Yes.
Hidden Potential – For Disaster
At the same time that there is a dearth of understanding of the technical aspects of the SSH protocol, there is also a lack of regulatory oversight and governance of SSH user key-based access. But wait – there’s more. We have an additional concern.
Cyber criminals, be they hostile insiders or a hackers, prefer SSH as their tool of choice. It is the primary means by which they move laterally within an enterprise to gain access to new assets and further elevate privilege, as well as how they exfiltrate data and assets from an organization. In fact, LightCyber’s Cyberweapons 2016 Report indicates that in over 50 percent of the cases, the SSH protocol is the tool of choice to gain this lateral movement across the assets of an enterprise.
But wait – there’s still more. Cyber criminals know that SSH user keys are not being managed and, as a result, they go after private keys to gain access to assets. From there, they will often create new key pairs, which will generate them access to the outside directly or move those assets automatically to servers in the cloud. This is all achieved using something called “port forwarding” via “SSH tunneling,” which would allow the malicious actor to extract this data via the encryption itself, rendering firewalls ineffective.
Because we don’t know who owns the keys to attain that encrypted access and because we are not able to look inside the encryption of those sessions, this is all happening under our noses. The potential impact is clear. We lose our data – or worse, our customer’s data. We lose our intellectual property. We lose our reputation and, in turn, we lose revenue and shareholder value.
Operational resilience is the other primary consideration. The potential downtime in critical systems, should SSH user key-based access to those systems be compromised, is a concern we need to consider seriously. Our disaster recovery systems, which are failovers to our production systems, often share identical SSH user keys that ensure the processes of those systems fail over in an identical manner. This means that any compromised keys in production environments can also equally affect the failover to disaster recovery systems.
Don’t Let It End Like This
Let’s revisit the comparison of the SSH protocol situation with the 2008 housing crash.
We have a lack of market understanding around the power of the SSH protocol and the criticality of the access it provides to our infrastructures. We have a gap in regulatory and governing oversight of how this SSH user key-based access should be monitored, provisioned, de-provisioned and recertified. We have an encrypted protocol, which malicious actors use as their tool of choice to extend their access and reach within our enterprises, create backdoors and exfiltrate data. As a result, we have a potential financial, operational and brand impact that can conservatively be described as significant to our businesses.
Like the subprime mortgage debacle, the SSH story doesn’t have to end in tragedy. What’s needed is careful attention across the board—from identity access management, cryptography, infrastructure and security teams—and immediate action. No one wants to explain to the C-suite why a preventable disaster wasn’t prevented.
About the author:
Matthew McKenna brings over 15 years of high technology sales, marketing and management experience to SSH Communications Security and drives strategy, key account sales and evangelism. His expertise in strategically delivering technology solutions that anticipate the marketplace has helped the company become a market leader.
Prior to joining the company, Matthew served as a member of the executive management team of ADP Dealer Services Nordic and Automaster Oy, where he was responsible for international channel operations and manufacturer relations. In addition, he was responsible for key accounts including Mercedes Benz, General Motors, and Scania CV. Before this, Matthew played professional soccer in Germany and Finland.
Matthew holds a Bachelor of Arts degree in German from the University of South Carolina and an MBA from the Helsinki School of Economics and Business Administration.
Edited by Alicia Young