We live in a noisy world…one where the amount of information that crosses our desk, overloads our inbox, or distracts our attention from more meaningful activity is overwhelming. For those of us who work in IT and cybersecurity, our world is especially noisy and the signal to noise ratio is overwhelmingly noise. Just look at the log and event ID detail we work with.
As a case in point, Cisco’s ASA reference material includes over 500 pages of syslog detail (this is just one platform). Combine firewalls with routers, switches, servers, operating systems, applications, VPNs, LDAP or AD, and facility systems from multiple vendors and you get the picture. We are literally blinded by information.
As CISOs, we are simply overloaded by the amount of information that we are expected to absorb and respond to in a timely and technically-accurate manner. The tools we have to simplify and order this noise are also challenged. The basic legacy signatures and rules-based approaches to securing our infrastructure cannot keep pace with the talents of those looking to compromise our organizations.
This is the reason why so many attacks are successful. The bad guys know how overwhelmed traditional security and IT departments are and can craft exploits that take advantage of this signal to noise ratio. They can simply send a well-crafted e-mail with a weaponized URL link or attachment. Advanced Persistent Threats (APTs) are essentially below the proverbial radar…overlooked in this noisy environment. We need to be more efficient in reducing the noise associated with our security operations.
The Value of New Approaches, Techniques, and Technologies
There are ways to improve our security operations and enhance our capabilities to find threats to, and within, our environments. On the technical front, there have been fantastic enhancements to automating security analysis, including tools to automate the collection and surfacing of specific event IDs that warrant attention – essentially indicators of compromise (IOCs). Complementing and extending these Security Incident and Event Management (SIEM) tools are newer approaches that leverage network and user behavioral analytics to triage anomalistic behavior. Anomaly detection and reporting offers an innovative and effective approach to focusing on what puts our systems and organizations at risk. The value of these systems is that, when engineered correctly, they leverage machine learning that mitigates the requirement for extensive rules writing and manual intervention.
Apart from the technical improvements we are seeing in the realm of anomaly detection, there is also an increasing maturity in security operations related to agreed-upon security controls and metrics. As discussed previously in this book, the FTC’s enforcement of Section 5 of the Federal Trade Commission Act – focused on unfair and deceptive trade practices – has had the effect of creating a minimum baseline standard for security practices, at least within organizations that have a consumer focus.
There is also new guidance from states attorneys general, such as California’s Kamala D. Harris, recommending the adoption, at a minimum, of the Center for Internet Security’s Critical Security Controls. Essentially, there are now widely-agreed-upon frameworks – including the recent NIST Cybersecurity Framework – that set the minimum bar for security operations and can be used to evaluate and baseline your organization’s security practices.
Security metrics validate the effectiveness of our security operations and controls and provide actionable detail on where organizational improvements are required. Similar to logs, event IDs, and other data points, not all security metrics are created equal. The goal is to have a tailored set of key security metrics that are appropriate to your organization’s size and complexity as well as commensurate with the regulatory environment in which your organization operates. Effectively, as a CISO you want to focus on the return on security metrics employed.
To that end, I strongly recommend grouping metrics into functional areas and focusing only on those that are truly important to the organization and your security operations. Too many metrics can feel like a log environment without an SIEM… too many distractions and nothing is acted upon. Too few metrics and key performance indicators could be overlooked. A balanced and thoughtful approach to security metrics is required to ensure that the signal to noise ratio is aligned with your organization’s risk tolerance.
I recommend grouping metrics into functional areas. There should be metrics that provide insight on administrative functions such as training, policy review and approval, and non-technical indices. Other metrics should focus on the operational and technical side of security. The development of your organization’s metrics dashboard should involve colleagues from the business units and executive management. Their insights and requirements will inform the types of metrics that are ultimately created, implemented, and reviewed. This should be consistent with the core view that the CISO role is transforming into a lead risk management role – evaluating information risk across the entire organization.
Here are some important metrics that I think…
 California Data Breach Report, February, 2016 – https://oag.ca.gov/sites/all/files/agweb/pdfs/dbr/2016-data-breach-report.pdf?
Copyright © 2016 CISO DRG JV – All Rights Reserved.