SSH Key Management
The SSH protocol is the global gold standard for remote system administration and secure file transfers. SSH is used in every data center and in every major enterprise. One of the features behind the immense popularity of the protocol is the strong authentication using SSH keys.
- SSH keys are a critical access management problem
- Risks of unmanaged SSH keys
- Our solution to SSH key management
- Request more information
SSH keys are a critical access management problem
SSH keys provide the same access as user names and passwords. Furthermore, they often grant access to privileged accounts on the operating system level, giving a command line. Yet, in many cases, SSH keys have been completely overlooked in identity and access management planning, implementation, and audits. Users have been able to create and install keys without oversight and controls. This has led to violations of corporate access policies and dangerous backdoors.
Over the last few years, it has turned out that most large organizations have massive numbers of SSH keys in their environment. These keys are like passwords. They grant access to resources - production servers, databases, routers, firewalls, disaster recovery systems, financial data, payment systems, intellectual property, and patient information.
Information security starts from controlling who is given access to systems and data. If there is no control over access, there is no security, no confidentiality, no integrity, and no guarantees of continued operation.
Insights from real customer cases
We have worked with many companies, including several global top-10 banks, leading retailers, and other large Fortune 500 companies. Based on our findings, most organizations:
- Have extremely large numbers of SSH keys - even several million - and their use is grossly underestimated
- Have no provisioning and termination processes in place for key based access
- Have no records of who provisioned each key and for what purpose
- Allow their system administrators to self-provision permanent key-based access - without policies, processes, or oversight.
In the case of one representative customer, we went through a quarter of their IT environment as part of a major SSH key management project. They had five million daily logins using SSH, most of them using SSH keys for automation. We analyzed 500 business applications, 15000 servers, and found three million SSH keys that granted access to live production servers. Of those, 90% were no longer used. Root access was granted by 10% of the keys.
NIST issues guidance on SSH key management
US National Instute of Standards and Technology (NIST) as issued guidance on SSH key management as NIST IR 7966. It is a good starting point for understanding how to manage access using SSH. We wrote most of the NIST guidelines, and have expanded upon them in our internal processes. We also invented SSH (Secure Shell). We are the best subject matter experts in the field.
Regulatory compliance requires SSH key management
Typical requirements for compliance include:
- Managing identities and credentials - SSH keys are access credentials
- Provisioning and termination process for access - including access based on SSH keys
- Segregation of duties - elimination of key-based access from test and development systems into production
- Disaster recovery - limiting attack spread from primary systems to disaster recovery sites and backup systems
- Privileged access controls - SSH keys are often used to bypass jump servers
- Boundary definition and documentation of connections for payment systems, financial data environments, patient data environments, or between government information systems
- Incident response and recovery - being able to change compromised SSH keys.
Analysts are addressing SSH keys
- A Gaping Hole in Your IAM Strategy (IDC)
- SSH Governance is Needed to Reduce Risk & Bridge the Trusted Access Gap (IDC)
- Smartly Manage Secure Shell Keys (IDC)
- Executive View: Universal SSH Key Manager (KuppingerCole)
Risks of unmanaged SSH keys
Unmanaged access exposes organizations to significant risks that could in the worst case bring down critical information systems for months. Unmanaged keys risk systemic failure of critical infrastructure, especially in a cyberwarefare scenarios.
Likelihood of stolen and misused keys is high
There are way more SSH keys around than anyone seems to believe. In one financial sector customer case we encountered as 3 million keys (750,000 distinct keypairs) from 15,000 servers. In another case the customer found 4.5 million authorized keys from their 100,000 servers. Typical numbers for Fortune 500 companies range from hundreds of thousands to millions - many times more than they have employees or system administrators with command line operating access.
Many keys means that the likelihood of keys being breached, stolen, and used for attack spread is high. A single key is enough to gain access.
Impact of compromise is very high
We have found that in several customer cases about 10% of the discovered keys grant root access. An attacker getting root means they can do anything on the server - including inject fraudulent data, subvert encryption software, install persistent malware, or outright destroy the system. Confidentiality, integrity, and continuity of operations are all compromised. Even if the key gives non-root access, local privilege escalation vulnerabilities can often result in the attacker gaining root access.
There is substantial risk that SSH keys can be used to spread an attack to a majority of all production servers in the organization. This could take down a Fortune 500 enterprise for months and cause billions of dollars of damage to shareholders. Many companies use SSH keys to push data to disaster recovery sites and backup systems, exposing them to attacks as well.
When keys are used for file transfers between business partners, they may be used to spread the attack between organizations. Improperly configured SFTP file transfer connections may be used to log into other organizations using stolen keys. The keys may be used for stealing other authentication credentials or installing data collection software for furthering the attack in other ways.
For the modern society, cyberwarfare is a relevant risk, and a coordinated attack across critical infrastructure with the intention to destroy and confuse is a real possibility. Recently leaked CIA hacking tools were collecting SSH keys.
Our solution to SSH key management
The role of SSH Communications Security in these projects is typically to provide the software and help structure and manage the project and define SSH-related policies. We typicall provide 1-2 subject matter experts for larger projects to work with the customer's engineers or outsourcing partners.
SSH Risk Assessment™
We usually start with an assessment of SSH and SSH key usage, even before any formal project starts. The assessment helps understand the severity of the issue and evaluate risk and priority. The fast and low-cost SSH Risk Assessment service is designed for this.
Universal SSH Key Manager®
Universal SSH Key Manager is our flagship product for managing SSH keys. It has been used by numerous large and mid-sized organizations for solving their key management problems. It handles the entire life cycle for key-based access, and integrates to leading identity and access management systems, privileged access and privilege elevation systems, as well as SIEM (Security Information and Event Management) and configuration management.
Together with consulting services, Universal SSH Key Manager makes it easy for customers to solve their key management issues. We also can run the whole project for you, working with your application teams and identity & access management, cryptography, security engineering, operations, and/or IT transformation groups as needed, globally. We've run SSH key management projects in the US, UK, Germany, and Singapore, among others.
PrivX On-Demand Access Manager™
We also recommend looking at PrivX On-Demand Access Manager. It is a privileged access management solution that helps eliminate both passwords and permanent SSH keys from servers entirely.