When Prospect Theory Meets Chaos Engineering
There is a connection between my research into behavioral models of information security and my Black Hat USA talk with Dr. Forsgren from last week regarding DevOps and the future of information security that might not be immediately obvious. However, it is a connection I believe is worth illuminating.
Adopting the “chaotic infosec” philosophy (the application of chaos engineering to information security) can influence those who build and secure systems to operate more like attackers – that is, to be more risk averse and to more quickly update their mental models given new inputs. This post will explore what exactly this means, and why it represents a valuable mindshift for organizational security.
How does Prospect Theory apply to security?
While I encourage you to read my prior post on applying Prospect Theory to information security, I recognize it is a bit long. I will thus summarize its relevant points here.
Humans make decisions by setting a reference point (their view of the status quo) against which they measure the decisions’ potential outcomes. Empirical evidence shows that people generally prefer options that provide a small but certain gain rather than options that provide a larger but uncertain gain. They also generally prefer options that provide a miniscule chance of losing nothing (with a larger chance of losing a lot) rather than options that provide a certain chance of losing less.
These results, which are modelled by Prospect Theory, suggest that people are risk averse when making decisions with positive potential outcomes (when they are in the “Gain Domain”), and risk seeking when making decisions with negative potential outcomes (when they are in the “Loss Domain”). The further people get away from their reference point (again, their perceived status quo), these trends exaggerate. People who experience heavy losses become even more risk seeking in an attempt to jump out of the figurative hole, and people who experience substantial gains become even more risk averse to preserve their winnings.
My hypothesis was that defenders set their reference point based on a status quo where their organization is perceived to be secure and uncompromised (to simplify a bit). The nature of enterprise defense means that defenders will operate in the Loss Domain as a result – compromises are inevitable, as are failures to maintain a level of security posture that defenders find acceptable. Attackers, in contrast, operate in the Gain Domain – because their reference point is their current level of compromise, and, generally, they are likely to positively further their position of pwnage.
As a result, defenders are more risk seeking, leading them to adopt more speculative solutions to attempt to reach their reference point of perfect prevention instead of adopting more basic solutions with larger probabilities of success but a perception of being less likely to stop a big, sexy attack. Attackers, on the other hand, are more risk averse, moving more cautiously to achieve their goals, opting first for inexpensive methods before moving to expensive methods.
How does chaos transform reference points?
Applying the principles of chaos engineering to information security, as outlined in my Black Hat USA 2019 talk, changes the reference point for defenders. If you extend the chaos engineering philosophy of “things will fail” to “things will be pwned,” then the status quo for security teams becomes the assumption of compromise.
For traditional enterprise security programs, mapping Prospect Theory to security shows a state of being that is fragile to reality. This state anchors those involved in enterprise defense to the Loss Domain, as shown in the graph below, encouraging risk seeking and reactive behaviors.
In contrast, applying chaos engineering to security transforms security’s Prospect Theory graph into being resilient to reality, encouraging risk averse and strategic behaviors. A dearth of significant compromises at your organization now feels like a gain, as shown in the graph below, incentivizing the preservation of that gain – which can be achieved in part through some of the strategies and testing I enumerated in the Black Hat talk (see the “A Phoenix Rises” section).
As demonstrated in these graphs, by assuming “things will be pwned,” there is really only upside – so those who architect and secure their organizations’ systems can begin feeling less defeated and instead feel proud of their efforts. Implementing security “basics” can feel unrewarding, but successfully increasing the cost of attack deserves a sense of accomplishment. While I use the term “stopping” on the Gain Domain side of the graphs for brevity, I do not literally mean preventing attacks by skiddies or APTs from happening. Rather, I mean stopping these attacks from negatively impacting your organization, which is ultimately what matters.
Just as failure-based metrics, such as Time Between Failure, inhibit innovation and make systems more brittle, allowing an organization’s security contributors to languish in the Loss Domain similarly hinders resilience and, ultimately, organizational performance. Instead, a success-oriented metric like Time to Recovery promotes innovation and resilience – and operating in the Gain Domain incentivizes similar behaviors. Therefore, switching your reference point from “maintaining secure systems” to “things will be pwned” – despite seeming like trivial semantics – has the potential to subtly transform how your teams design systems.
The most effective way to engender cultural change is by changing what people do, not what they think. By altering the security mental model through this path of chaos, you change behavioral vectors – harnessing people’s embedded wetware – to promote the type of strategic, innovative, and perceptive decision-making that we so desperately need to build and maintain secure systems. Adopting the mantle of chaos through the philosophy of “things will be pwned” can reorient team perception to operate in a more strategic, less frenetic fashion by re-architecting their relationship between risk and reward.
InfoSecBehavioral InfoSecSecurity Chaos Engineering
2019-08-12 08:00 -0400