Modernize your Cybersecurity Measurements

Published on 20th October 2022 Author: Rob Black

Among our various Black Family traditions, one of my favorites is reading to my kids at bedtime. It’s a great way to power down after a busy day and enjoy some screen-free, quality time together.

Recently, my 11-year-old son and I have been reading Alexandre Dumas’s The Count of Monte Cristo. First published in 1844, it has stood the test of time for good reason (let’s hear it for old school revenge!). It does, however, contain many words and historical references that require an explanation.

For example, last night, my son asked, “Dad, what is a ‘league?’”

“Hmm… let’s see. Well, it’s a measure of distance, but I’m not sure how far. A mile? But then there’s ‘20,000 Leagues Under the Sea,’ which would make the Earth humongous.”

So we checked Wikipedia.

Apparently, a league is a “nonstandard measurement,” equal to 1.4, or 1.5, or sometimes, 3 miles. For a time, in France, it was the equivalent of “2,000 body lengths,” a unit of measurement that I assume was abandoned once cars were invented and fuel efficiency calculations became overly cumbersome.

Cybersecurity Has Lots of Nonstandard Measurements, Too

If you ever find yourself at a party filled with cybersecurity professionals — and before you realize the enormity of your mistake and head for the door — pull a few of these folks aside and ask them to precisely explain the difference between a risk, a threat, and a vulnerability. I assure you, you will be in for a long and amusing discussion.

This is not the case in most “technical” professions. Scientists don’t disagree on whether or not force equals mass times acceleration (thank goodness, or we’d never get a plane off the ground). Financial professionals don’t debate the merits of tracking revenue, expense, and profit.

But in the world of cybersecurity, there is little consensus about what matters most and how best to measure it.

That’s problematic. Absent meaningful metrics for your business – Key Performance Indicators (KPI) – you have no reliable way of knowing whether or not your organization is on track and out of harm’s way.

For example, here are some questions worth considering…

How quickly do you patch “important vulnerabilities?” For that matter, what do you consider an “important vulnerability?”
What percent of your employees have multi-factor authentication, antivirus, disk encryption, mobile device management? What should those numbers be?
What metrics do you track regarding incidents (e.g., time to detect, resolve)?
How long do cybersecurity or other key technical positions go unfilled?
How long does it take to off-board an employee? What is an acceptable error-rate for mistakes that occur within this process?
What percent of your employees fail phishing tests the first time? How much should that decrease with additional training?
What percent of your vendors have been evaluated for security?

This is by no means an exhaustive list, but you get the idea.

Define and Track Your Key Performance Indicators

Because there are few industry-accepted standards, it’s up to you to define the metrics that are meaningful to your organization given your particular circumstances.

For example, what kinds of information do you collect? A bank, a bakery, and a hospital all maintain customer records, but they operate within radically different environments and threat levels.

How large is your business? A company with 10,000 customers is a much more attractive target to evil doers than one with just 10.

Whatever your specifics, you need a plan:

What will you measure?
What are your standards of success or failure?
How often and in what format will measurements occur and how will that information be shared internally?
Who is responsible for ensuring that your measurements are done accurately and on time?

Any Plan is Better Than None

Seat-of-the-pants cybersecurity management is no longer an option. The threat landscape has changed dramatically in just the past five years, as the ease of attacking and complexity of systems have both increased significantly.

Your measurements need not be comprehensive nor perfect (they won’t be). However, you do need a system. Beginning with a few metrics that are consistently measured is a good starting point; over time, you can layer on others. Make sure that senior management finds these measurements meaningful and treat them with the same urgency and importance that you do your other critical business management tools.

Now if you’ll excuse me, I need to drive to my daughter’s basketball game and according to my GPS, it is nearly 28,000 body lengths away.

Want to get great cybersecurity content delivered to your inbox? Click here to sign up for our monthly newsletter, Tales from the Click.

Rob Black, CISSP

Rob founded Fractional CISO in 2017 and has helped dozens of companies improve their security posture as a vCISO. He consults, speaks, and writes on IoT and security. Rob has held product security and corporate security leadership positions at PTC ThingWorx, Axeda and RSA Security. He received his MBA from the Kellogg School of Management and holds two Bachelor of Science degrees from Washington University in St. Louis in Computer Science and System Science and Engineering. He is also a Certified Information Systems Security Professional (CISSP).