Introduction

In Chapter 5 we look at how to create a metrics program that will help you measure the performance of your entire organization and determine what to report to your management and your board of directors. Each of the authors has a bias toward objective measurements and sees that as key to fulfilling the role of the trusted authority on your organization’s risk posture. They collectively emphasize the value of using widely adopted security frameworks to create a comparable baseline from which to measure improvement and extoll the virtues of being disciplined in the performance of preventive and periodic controls.

Bill begins with a brief historical review of tying measurement to business objectives and briefly discusses the evolution of control coverage to measuring the impact on service delivery. He provides several recommendations for frameworks you can use to establish your baseline. To conclude his section on measuring process effectiveness, he offers a helpful set of principles for deciding the metrics reported and how to maximize the impact of the reports. Bill then pivots to a discussion on the CISO’s role in risk management and how to measure the effectiveness of this strategic function.

Matt points out that there is no shortage of things to measure and helps the reader understand how detrimental an unchecked onslaught of raw data can be. He skillfully guides the reader through an analysis of key categories of risk and the relevant measurements to capture and report. Some of the categories he covers include legal, financial, human resources, vendor management, software, data, and system hygiene.

Gary focuses on how to effectively frame information for management and the board of directors to, in his words, “tell a story.” After outlining the criteria for developing the set of metrics the CISO will collect and share, including sample metrics and a formula for creating a useful metric, Gary pivots to organizing the information for consumption and action. He brings all of this home for the reader by sharing lessons learned, including the types of reports and dashboards to share (and with whom), establishing relationships with the recipients of the dashboards, and putting the information into context before they even see the report.

Some of the questions the authors used to frame their thoughts for this chapter include:

♦  What are metrics? Why are metrics important? What steps should the CISO and security team take to create valid metrics for their program?

♦  What are some examples of dashboards you can develop as strategic assets?

♦  What types of reports should a CISO create to educate executive management and sponsor a more resilient, cyber-aware corporate culture?

Cybersecurity Metrics – Stamper

We live in a noisy world, one where the amount of information that crosses our desk, overloads our inbox, or distracts our attention from more meaningful activity is overwhelming. For those of us who work in IT and cybersecurity, our world is exceptionally noisy, and the signal to noise ratio is overwhelmingly noise. Just look at the log and event ID detail with which we work.

As a case in point, Cisco’s ASA reference material includes over 500 pages of Syslog detail (this is just one platform). Combine firewalls with routers, switches, servers, operating systems, applications, VPNs, LDAP or AD, and facility systems from multiple vendors, and you get the picture. Information blinds us.

As CISOs, we are simply overloaded by the amount of information that we are expected to absorb and respond to in a timely and technically-accurate manner. The tools we have to simplify and order this noise are also challenged. The basic legacy signatures and rules-based approaches to securing our infrastructure cannot keep pace with the talents of those looking to compromise our organizations.

This information overload is the reason why so many attacks are successful. The bad guys know how overwhelmed traditional security and IT departments are and can craft exploits that take advantage of this signal to noise ratio. They can simply send a well-crafted e-mail with a weaponized URL link or attachment. Advanced Persistent Threats (APTs) are mainly below the radar, overlooked in this noisy environment. We need to be more efficient in reducing the noise associated with our security operations.

The Value of New Approaches, Techniques, and Technologies

There are ways to improve our security operations and enhance our capabilities to find threats to, and within, our environments. On the technical front, there have been fantastic enhancements to automating security analysis, including tools to automate the collection and surfacing of specific event IDs that warrant attention – essentially indicators of compromise (IOCs). Complementing and extending Security Incident and Event Management (SIEM) tools are newer approaches that leverage network and user behavioral analytics to triage anomalous behavior. Anomaly detection and reporting offers an innovative and practical approach to focusing on what puts our systems and organizations at risk. The value of these systems is that, when engineered correctly, they leverage machine learning that mitigates the requirement for extensive rules writing and manual intervention.

Apart from the technical improvements we see in the realm of anomaly detection, there is also an increasing maturity in security operations related to agreed-upon security controls and metrics. As discussed previously in this book, the FTC’s enforcement of Section 5 of the Federal Trade Commission Act – focused on unfair and deceptive trade practices – has had the effect of creating a minimum baseline standard for security practices, at least within organizations that have a consumer focus.

There is also  precedent from states attorneys general, including Kamala D. Harris (former Attorney General for California and now U.S. Senator), recommending the adoption, at a minimum, of the Center for Internet Security’s Critical Security Controls. Essentially, there are now widely-agreed-upon frameworks – including the recent NIST Cybersecurity Framework – that set the minimum bar for security operations and can be used to evaluate and baseline your organization’s security practices.

Security metrics validate the effectiveness of our security operations and controls and provide actionable detail on where organizational improvements are required. Similar to logs, event IDs, and other data points, not all security metrics are created equal. The goal is to have a tailored set of crucial security metrics that are appropriate to your organization’s size and complexity as well as commensurate with the regulatory environment in which your organization operates. Effectively, as a CISO you want to focus on the return on security metrics employed.

To that end, I strongly recommend grouping metrics into functional areas and focusing only on those that are truly important to the organization and your security operations. Too many metrics can feel like a logging environment without a SIEM… too many distractions and nothing upon which you can act. Too few metrics and you overlook key performance and risk indicators. A balanced and thoughtful approach to security metrics is required to ensure that you align the signal to noise ratio with your organization’s risk tolerance.

I recommend grouping metrics into functional areas. There should be metrics that provide insight into administrative functions such as training, policy review and approval, and non-technical indices. Other metrics should focus on the operational and technical side of security. The development of your organization’s metrics dashboard should involve colleagues from business units and executive management. Their insights and requirements will inform the types of metrics you ultimately create, implement, and review. These metrics should be consistent with the core view that the CISO role is transforming into a lead risk management role – evaluating information risk across the entire organization.

Introduction

In Chapter 6 we turn to our interactions, as CISOs, with our management and our board of directors. As we note, there is a heightened awareness of cybersecurity within both the senior management team (what we often refer to in this book as the “C-suite”) and the board of directors. This heightened awareness comes from the ever-increasing profile of cybercrime and the concomitant increase in scrutiny from regulatory bodies, whether to protect our critical infrastructure or protect the victims of breaches and leaks. While this heightened scrutiny is both expected and, in many ways, needed, our higher calling is to be the best partner we can be to our peers within our organization.

Bill brings three points front-and-center: your role as the CISO within your organization, the roles of the individuals with whom you are communicating, and the outcomes you wish to achieve from these encounters. To Bill, the key results are to inform, collaborate, and take action. Bill also asks the reader to consider the natural filters as well as the differing duties that each member of their audience brings to the conversations. As the CISO, he reminds us, you will need to supply the narrative, so others don’t do it for you.

Matt implores us to take our duty to the board of directors and our management team seriously and realize that how we communicate the status of our security program and our risk posture matters significantly. He provides the point of view of a member of the board as a unique and informed way to clearly describe what a board member is concerned about, how they expect to be informed, and what they will do with the information you provide. Through his narrative, he helps CISOs to be more effective in advocating for their requirements.

Gary articulates one of the new fears that members of the board harbor when it comes to cybercrime: “… if their company will be next.” Gary also emphasizes how important it is to form relationships within the organization to keep constant tabs on the competing business objectives, both to inform the CISO about the needs of the organization and to tailor briefings to enable better outcomes. Gary provides a treasure trove of “been there, done that” advice for new and aspiring CISOs on how to make the most out of the extraordinary opportunities that CISOs now have to participate with senior leadership and influence the board of the modern company.

Some of the questions the authors used to frame their thoughts for this chapter include:

♦  If the CISO were a board member, what data would he/she would most want to see? What would he dashboard look like?

♦  What does the CISO want from the board in support of their information security responsibilities?

♦  What are recommended practices for reporting cybersecurity requirements to the board?

♦  How should the information be presented?

♦  What important aspects of cybersecurity and risk should the CISO ensure are conveyed to the board?

Management and the Board of Directors – Hayslip

In today’s uncertain business environment, the board of directors is becoming more security aware. They watch the news and read articles on the latest cyber incidents and wonder to themselves if their company will be next. Many of them also wonder what their competitors are doing to reduce their cybersecurity risk.

As the CISO, you will be the organization’s expert for this evolving uncertainty. It will be incumbent upon you to report to your organization’s executive management on issues relating to risk exposure, cybercrime, compliance issues, and newly evolving threats. To do this effectively, you will need to establish an executive-sponsored cybersecurity program. This program will enable you to provide “Cybersecurity as a Service” (CaaS) to your organization and its business units. Periodically report these cyber services, their impact on the organization, and any resulting risk exposure to executive management. It is this process of presenting to the board and executive management that we will cover in the discussion that follows.

As I mentioned in the previous chapter, reporting to management and the board of directors is a unique experience. The way you prepare your reports, how you present your data, and the preparation required to ensure you are effective are skills you must learn as CISO if you expect to grow your cybersecurity program and be seen by the company as a business enabler.

Are You Board Ready?

To begin, let’s assume you have a mature security program in place and you are collecting metrics that you will use to measure the maturity and growth of its value to the organization. To analyze this data and use it to implement change, you have created dashboards to display this information to support your organization’s business units. Now as CISO, you are excited about the trends you are seeing in the information you have collected, and you communicate this news to upper management. Then one afternoon you get “the email,” that’s right the email that comes from your organization’s executive assistant for the board of directors. The board is requesting that you present to them the information you have on your cybersecurity program and the current risks the organization faces. At first, if you have never done an executive presentation, you may be apprehensive. However, recognize that this is an incredible opportunity.

You, in your role as the CISO, have the chance to educate the board and executive management on how cybersecurity is providing value to the organization. So, let’s discuss how you can approach this opportunity and not lose your job with the following questions: “What are recommended practices for reporting cybersecurity requirements to the board? How should the information be presented? What important aspects of cybersecurity and risk should the CISO ensure are conveyed to the board?”

Boards of directors are tasked with protecting their organizations from significant risk. Their duties generally fall within six areas:

1.  Governance

2.  Strategy

3.  Risk

4.  Talent

5.  Compliance

6.  Culture

To corporate boards, cybersecurity risk is as significant to the business as risks posed by strategic, operational, financial or compliance operations. For the board, providing effective oversight of cybersecurity risk means the difference between learning about cybersecurity after a breach with significant damages and having a mature cybersecurity program in place that can mitigate the costs of a breach with minimal exposure to the company. In today’s fast-moving business environment, boards can’t claim lack of awareness as a defense against allegations of improper oversight. Boards of directors and executive management must educate themselves about cybersecurity and its risk exposure to their organizations. This knowledge is crucial; it enables board members to make strategic decisions with the full understanding of how cyber risk impacts their business plans. With this strategic view in mind, let’s discuss how the CISO, the security program, and security teams can assist the board with its mission of providing proper strategic oversight.

At the executive management level, the CEO is ultimately responsible to the board of directors for the business’ cybersecurity risk strategy. However, the CEO will typically look to an executive, (CIO, CTO, CRO, etc.) who has governance responsibilities over information technology or risk management to execute this strategy. This executive will be expected to interface with the board and be held accountable to the CEO for this strategy’s implementation and overall management.

As I mentioned in Chapter 1, it’s my opinion that the CISO should report to another C-level executive who understands the importance of the CISO position and how cybersecurity can be used as a valuable asset to support the organization’s strategic objectives. This senior executive is critical to the CISO. Business tends to try to decentralize itself to be nimble and competitive while cybersecurity programs tend to try to centralize the business to be more effective in managing risk. It’s evident that these conflicting views will be in a constant state of opposition unless there is a senior executive to provide context and mentorship to the CISO. It’s this partnership between the senior executive and CISO that enables the CISO to see cybersecurity and risk from a more strategic viewpoint and understand its impact on the business.

So back to our plight. Your presence is requested to report to the board of directors on the state of your cybersecurity program and the company’s current level of exposure to cybersecurity risk. Your relationship with the senior executive you report to is critical. He/she will be able to assist you in articulating the value of cybersecurity in business terms and demonstrating how the program provides clear business value.

Ideas for painting this picture on business value

♦  Approach this opportunity as if presenting a financial report on a budget.

♦  Provide a balanced cost-benefit analysis on cybersecurity projects based on expected results.

♦  Describe a reduction in risk based on the use of specific cybersecurity controls or work processes (it is good to have metrics here to back up this picture).

♦  Demonstrate some quantifiable financial returns. Show how an increase in a specific cyber metric allows a more specific service or reduces risk to a critical business process. Describe how a mature cybersecurity risk management program increases productivity or allows for a reduction in cost – how the automation of controls or processes reduces time required to touch equipment or rewrite code.

♦  Discuss how the cybersecurity program enables corporate competitiveness. The company can leverage new technologies to be more competitive, reduce operations costs, and provide superior service to its customers. Describe how your security program enhances revenue by reducing risk to business operations.

Management has the responsibility to develop and implement the cybersecurity strategy; however, the board must fully understand the company’s risk exposure to cyber-related issues. Boards, due to their positions and breadth of governance, tend to look at issues from a broader macro level of operations while management operates at a more tactical level within their specific departments or divisions. Your job when you present to the board is to tell a story, a story that is concise, simple, and connects the organization’s business goals to your cybersecurity program’s risk management objectives. As you can see, this is very similar to the process you implemented when you created security metrics for your program and architected dashboard views to manage them. When you address the board, your story needs to have a beginning, middle, and end. It also needs to be interesting and should have a goal:

1.  Inform and Educate – you wish to tell the board that leveraging a new technology provides opportunities, however it also provides new risks that must be addressed.

2.  Influence a Decision – make the case for why a specific action should be taken, for example the cybersecurity program should be moved out of the IT department to address “segregation of duties” issues.

3.  Change Behavior – show how a current organizational process, behavior, standard, etc. is opening the organization up to substantial risk. Demonstrate workable alternatives that will reduce risk exposurewith minimal impact to business operations.

Since you are in effect telling a story, it is crucial to know how you want your audience to feel. To ensure that you are constructing the correct message, test it on one or more business executives to get their opinion on the information you present and whether it seems valid. Ask them to review your terminology and provide suggestions. You want to be sure that your story is demonstrating how cybersecurity is providing value to the business.

To assist in preparing for your board presentation, ask senior management for a board-level sponsor. This sponsor will be your sounding board as you create your presentation and can help you convey your message and answer the dreaded question, “What do you need from us?” There are multiple strategies to assist you in formulating your narrative. One that I would suggest you start with is to increase your business operations knowledge. You need to review the organization’s strategic plans and annual reports and interview executives within your company. The information you get will give you more insight into the business drivers that are critical to the board. They are also essential for you – you must ensure that your metrics and presentation are aligned to support them. Another strategy I would suggest is to compare/contrast with your peers if possible or use a framework such as NIST CSF or ISO 27001. Risk posture is difficult to measure.