Key Based Trust from a Process-Driven Goalkeeper's Perspective
Like for any goalkeeper, the worse thing - other than a torn ACL - is getting scored on. During my playing days, I was obsessed with the concept of how to organize my defense in a way to minimize goals against as well as minimize opportunities of my opponents. My teammates used to joke and wonder how I played at the level I did. I was not particularly fast or strong, did not have particularly great hands and was not super athletic in any way. But I was quite good at programming my defense and midfield to run a repeatable process to make it very difficult for opponents to penetrate. Unlike soccer, where you are most likely going to get scored on at some point, businesses must keep a zero goals against average for their entire careers.
My first in depth business-side experiences with operations outside of the goalpost came from the world of automotive IT. In the automotive space, the obsession around process is continuous. As a result of this indoctrination, I still have a tendency to look at things from the glasses of process regardless of what I am looking at. I regularly discuss, on the one side, with our customers about their processes and, on the other side, with our product managers, developers and delivery and services team about ours. Of course, always looking at the things from the perspective as to how we do something more efficiently and better.
The matter of access is no different. We have a tendency to focus on the security aspects and features that our solutions provide and too infrequently connect this to a desired process to be achieved. In most enterprises, there is currently a concerning lack of defined process as to how public key authentication may be used to gain access to systems.
Public key authentication has historically been used to simplify and quicken the authentication process of interactive as well as automated connections. It is not the culprit, but rather it has never been controlled from a process point of view. In most organizations, administrators, application owners can quite freely setup their own keys and create new trust relationships either interactive or automated in nature. This multiplied across 10 years, hundreds or even thousands of newly setup applications…times thousands of key setups per year equates to a number of trust relationships I don’t want to think about, a lack of control of the overall access process and of course a significant gap of concern in the overall security posture of an enterprise.
Our deployments show that up to 95% of key based trust relationships are actually obsolete, up to 20% of keys deployed are unknown with root level access and so on. The statistics any way we slice and dice them are concerning.
So from a process point of view, the starting point of any discussion to get this topic under control is first to ask the question do I have a policy which outlines the use cases for interactive and automated access utilizing public key authentication, what is permitted and what is not? Once that is defined, what are my major concerns of policy that we would like to remediate against? One of the largest concerns we see organizations addressing today is how to block unauthorized access from non production to production environments. How do we stop the transient capabilities of key based access penetrating deep into our infrastructures? How do we see if individuals or applications are sharing private keys? How do we lock down production to production trust relationships to ensure that what commands a key can run are forced and that the trust relationship in question is locked down directly between those two machines?
Once we have an idea of what we are up against from our starting point (our process today) to our desired point (our desired policy and process) we need to somehow gain visibility and map the current risk in terms of key based trust relationships which go against policy that exist in our estate, secondly we need to understand and monitor how key based authentication is being utilized to determine what trusts in the environment may or may not be obsolete and thirdly we must find a way to lockdown the ability to create new unauthorized trust relationships. Only after addressing the first three points successfully do we have a chance to stop the bleeding.
I often get the question and request, well why can’t we just go discover the trust relationships and then remove the violations that go against our policy? I call this rotating the problem away. It unfortunately is just putting a band-aid over the bloody wound, since we are actually not solving the initial problem of how key based trust relationships may be created in the environment. And even if we remove or rotate away root keys with access to production environments, it does not inhibit the creation of exactly that same policy violation once again.
The challenges of key based authentication for interactive and automated access are deep and far reaching, and the process to remediating the legacy situation and process challenges surrounding it will be the main indicator or criteria for a better security posture in the future. I would conjecture that 90% of the work related to getting the process and access issue under control is related to mapping of legacy trust relationships, monitoring of the legacy trust relationships and locking down the environment to block unauthorized new setup of keys. Only after these steps are solved in line with a policy for SSH key based access, is it sensible to begin the discussion about how to automate the process of key distributions, removals, rotations, etc…
I will certainly address the process components of automation in my next blog, but for now, let’s consider how we find the required policies around SSH key based access and necessary processes to get us in the direction of the zero goals against average.