Mastering Cybersecurity Measurement

WHY MEASUREMENT MATTERS

1.1: Our Current Cybersecurity Landscape

Organizations around the world are dealing with big challenges today: migrating to the cloud, resource constraints, and — to top it all off — an ever-expanding attack surface.

According to Gartner, enterprises’ digital transformation initiatives increasingly grow their attack surface – and thereby, expose them to threat actors. As cloud and SaaS adoption continue to skyrocket, so too does the number of misconfigurations, the leading cause for data breaches and data loss in the cloud.

So how can IT and security teams demonstrate their value in light of these challenges? By measuring and reporting against the cybersecurity KPIs that explain the progress and impact of their work.

THE KEY TO SHOWING THE VALUE OF YOUR CYBERSECURITY EFFORTS IS BY COMMUNICATING IN A WAY THAT RESONATES WITH THE AUDIENCE YOU'RE PRESENTING TO.

SELECTING METRICS

2.1: Our Current Cybersecurity Landscape

When it comes to selecting security metrics, start by identifying the gaps in your current measurement capabilities. Prioritize metrics that require minimal effort to track, provide valuable insights, and align closely with your organization's goals and objectives. By initially tackling the low hanging fruit, you can quickly demonstrate the value of your cybersecurity program.

For example, let’s say you’re trying to signal high risk. By pulling metrics that identify the number of cloud instances and accounts that don’t adhere to industry frameworks and benchmarks, like CIS and Cloud Benchmarks, you can accurately indicate weaknesses in your organization.

2.2: Map Your Metrics to Business Objectives

Ensure your metrics have a meaningful impact by mapping each metric to a specific business objective. For instance, you might want to look at the total number of devices running unsanctioned software to help the business cut down on software spend. This connection to business goals will give your metrics purpose and relevance. As you identify areas to measure, ask yourself, "Why does this metric matter? How does it contribute to the organization's success?"

When reporting metrics to executives and board members, avoid those without a clear purpose or that are too technical. Instead, focus on metrics that align with their priorities. Provide insights they can use to make strategic and informed decisions.

REDUCED BUSINESS RISK:

Look for metrics that demonstrate improvements in security posture and a reduction in overall risk. This could include mean time to detection or mean time to remediation. Beyond that, consider metrics that tie to employee productivity, highlighting how your security initiatives help employees work faster and more efficiently. For example, maybe you recently moved your organization away from passwords to single sign-on (SSO). SSO drives productivity by eliminating the need for employees to manage several different logins and passwords, making it easier and faster for them to access the systems they need to do their jobs.

BETTER CUSTOMER EXPERIENCE:

Identify metrics that indicate improvements in customer experience while maintaining a strong security posture. For example, maybe you’ve noticed an increase in customer satisfaction or dwell time due to improved login processes. Measuring these types of metrics can signal how your security strategies impact customer experience.

MARKETING GROWTH AND EXPANSION:

Connect your security initiatives to business growth objectives by selecting metrics that highlight the role of cybersecurity in market growth and expansion. For instance, you can measure the successful launch of new services in specific regions or track the security requirements that help secure partnerships and contracts.

CUSTOMER RETENTION:

Whether your organization sells directly to consumers or to other companies, security has become an essential part of the business procurement process. Highlight metrics that link your security posture to customer retention. For example, consider how you can use demonstrable risk reduction as a competitive advantage.

BEWARE OF MEASUREMENT CHALLENGES

3.1: Incomplete or Missing Data

One of the main challenges in measuring cybersecurity is managing incomplete or missing data. When data is incomplete, it hinders your ability to obtain an accurate measurement and perform reliable analysis. To ensure the integrity of your metrics, strive for as close to 100% data completeness as possible. This means identifying the sources of incomplete data and taking steps to address the gaps — whether it's improving data collection processes or implementing tools that provide more comprehensive data. Remember, if you don’t know your denominator, you can’t measure percentages.

3.2: Biased Data

Another challenge is relying on a single data source for your metrics. This can introduce bias into your measurements and potentially skew your insights. For example, how do you reconcile the difference in vulnerabilities reported from your vulnerability scanner versus vulnerabilities observed by other platforms like endpoint protection? Ask yourself these types of questions to scrutinize your data sources.

To mitigate the risk of biased data, it's crucial to derive metrics from multiple data sources and perspectives. Consider where your data is coming from. How can it be reconciled? By incorporating diverse and reconciled data sets, you can strengthen the validity of your metrics and get a more accurate representation of your cybersecurity landscape. This approach will help you easily identify patterns, commonalities, and discrepancies — leading to more meaningful, reliable metrics.

3.3: Outdated Data

Security landscapes are constantly changing and evolving, meaning accurate and reliable data is more important than ever — and must be ongoing. However, getting up-to-date data remains a persistent challenge. For example, aggregate data like those on devices, users, and other organizational assets may be outdated because asset inventories are typically created and maintained manually through massive spreadsheets with pivot tables. Further, point-in-time data scans provide little value, especially when providing context on ephemeral environments or assets like BYOD laptops or phones. This is not only tedious and time-consuming, but results in stale data that may no longer reflect the current state of security. By automating data collection from an integrated environment which allows for aggregation and correlation, we not only simplify a complex challenge, but also ensure that metrics are based on the most recent and relevant data available to help you accurately assess your current security environment.

3.4: Ever-Expanding Attack Surface

As IT environments grow and become increasingly complex, so too does cybersecurity measurement.

The proliferation of SaaS applications and ephemeral devices has only caused these types of questions to increase. If these questions sound familiar, the good news is there are solutions to help answer them.

Cyber Asset Attack Surface Management (CAASM) and SaaS Security Posture Management (SSPM) solutions increase visibility for improved security. When used together, they provide a system of record for all digital infrastructure, which is critical to securing your organization. These tools enable security and IT professionals to confidently mitigate threats, navigate risk, decrease incident response time, automate action, and inform business-level strategy – all while eliminating manual, repetitive tasks.

CAASM: CAASM solutions provide a consolidated view of all assets by detecting and identifying all software, hardware, cloud assets, network assets, and network infrastructure, as well as vulnerabilities within those assets. Relying on API integrations with an organization’s existing tools, CAASM solutions aggregate and correlate information across different technologies to answer questions and help identify and remediate vulnerabilities. CAASM solutions also remove the manual processes from data collection and correlation, which streamline and automate audits and reporting, whether for regulatory compliance or executive briefings.
SSPM: SSPMs are specific to SaaS applications and, similar to CAASM solutions, connect to assets via APIs. However, SSPMs leverage APIs to monitor various risks like misconfigurations, unnecessary user accounts, excessive permissions, and standards compliance within them. These tools identify and manage data that exists outside of an organization’s boundaries, like when it’s stored within SaaS applications, and give IT and security teams a better understanding of the interconnectivity of SaaS tools and capabilities.

Together, CAASM and SSPM tools connect the dots to help IT and security professionals understand how different entities and assets relate to one another. Beyond comprehensive asset visibility and inventory, CAASM and SSPM address key business use cases like device-to-SaaS correlation, which help security practitioners protect sensitive data, incident impact with better clarity and contextual awareness for those experiencing a cyber incident, Zero Trust reconciliation, policy management and enforcement, and even device ownership.

OUTDATED DATA:

By automating the tracking and management of asset data, CAASM and SSPM solutions ensure information is up to date. This eliminates the need to rely on manual processes (which often result in stale data).

BIASED DATA:

These solutions aggregate data from various sources, creating a more comprehensive and diverse data set. By leveraging multiple sources, CAASM and SSPM tools reduce the risk of bias and provide a more accurate representation of the security landscape.

INCOMPLETE DATA:

CAASM and SSPM solutions help address gaps in data completeness by pulling in asset information from different sources. This gives organizations a more holistic view of their assets, helping them make informed decisions based on complete data.

DYNAMIC DATA:

With the ever-changing nature of cybersecurity, real-time or near-real-time data is crucial. CAASM and SSPM solutions offer automation, ensuring that organizations stay updated with the latest information.

SECURITY POSTURE MANAGEMENT:

Measure metrics that evaluate and enhance your organization’s overall security stance. Assessing factors like framework adherence, mean dwell time, number of external facing assets, number of admin accounts, unsanctioned software, and more provides valuable insights into security maturity and can identify areas for improvement to strengthen security measures.

SECURITY SOLUTION COVERAGE:

Assess incident response efficiency by tracking metrics like the number of unmanaged devices, analyst time per alert, mean time to recovery, and intrusion attempts. These metrics will shed light on potential vulnerabilities, resource allocation, and incident resolution, while optimizing incident response processes.

SECURITY OPERATIONS AND INCIDENT RESPONSE:

Consider metrics like the amount of anti-malware protected systems in your organization, the percentage of assets scanned for vulnerabilities, use of secure authentication methods, and the average number of usage activities per IT asset. These will help evaluate the extent of your security coverage, identify potential vulnerabilities, and highlight where to enhance security across your environment.

VULNERABILITY MANAGEMENT:

What do tracking metrics like the number of identified vulnerabilities, patch rate, average vulnerability age, vulnerability debt, and patch response time have in common? They provide organizations comprehensive visibility into the state of their vulnerabilities. These metrics can help you allocate resources efficiently, reduce overall vulnerability exposure, and improve vulnerability management processes.

CLOUD SECURITY:

Metrics like the number of employees with admin privileges, violations of cloud security policies, and the percentage of misconfigured assets highlight potential risks and vulnerabilities within cloud environments. By tracking these metrics, organizations can proactively address security gaps, strengthen access controls, and maintain compliance with cloud security best practices.

SaaS SECURITY:

Metrics like misconfigured SaaS application settings, shadow SaaS applications, and third-party extensions with access to sensitive data help organizations manage the risks associated with SaaS usage. By monitoring these metrics, you can ensure proper security configurations, prevent unauthorized application usage, and mitigate potential vulnerabilities.

EXECUTIVE LEVEL REPORTING:

Focus on the “why” and “who” when it comes time to share security performance like incident costs, loss-to-value ratio, risk quantification, and audit certification non-conformities, which can help communicate the value of cybersecurity initiatives to executive stakeholders. Tracking these types of metrics can demonstrate the financial impact of security incidents, help prioritize security investments, and maintain compliance with industry regulations and standards.

Conclusion

Managing your digital infrastructure is only going to get more complex with time. As cybersecurity and IT teams undergo budget and workforce constraints amid a growing attack surface, understanding how to measure and track cybersecurity KPIs is essential to securing executive confidence in your cybersecurity program.

Though determining the right metrics to measure will be unique to every organization's business objectives, keep in mind three main goals as you measure and track security metrics:

Strive for accuracy. Identify metrics that are driven from numerous correlated data sources to remove bias.
Utilizing a cyber asset management solution like Axonius lets you track and automate the data sources you need to keep up with the pace of an ever-evolving threat landscape.
Focus on outcomes. The right solutions will help you effectively communicate the "why" behind every metric you measure so you can convey the organizational value of a mature cybersecurity program.

Measurement matters now more than ever in cybersecurity — but diving into the world of metrics can feel overwhelming. To learn how to navigate common metrics challenges, watch our on-demand workshop, MEASURING CYBERSECURITY: THE KPIS THAT MATTER.

Axonius gives customers the confidence to control complexity by providing a system of record for all digital infrastructure. With a comprehensive understanding of all assets including devices, identities, software, SaaS applications, vulnerabilities, security controls, and the context between all assets, customers are able to mitigate threats, navigate risk, decrease incident response time, automate action, and inform business-level strategy — all while eliminating manual, repetitive tasks. Recognized as creators of the Cyber Asset Attack Surface Management (CAASM) category and innovators in SaaS Management Platform (SMP) and SaaS Security Posture Management (SSPM), Axonius is deployed in minutes and integrates with hundreds of data sources to provide a comprehensive asset inventory, uncover gaps, and automatically enforce policies and automate action. Cited as one of the fastest-growing cybersecurity companies in history, with accolades from Deloitte, CNBC, Forbes, and Fortune, the Axonius Platform covers millions of assets for customers around the world.

Resource Center

Case Studies

Blog

Glossary

MASTERING CYBERSECURITY MEASUREMENT:

How to Show Metrics that Actually Matter

INTRODUCTION