Agentic Security Academy

Threat Landscape

Alteryx

Why Vulnerability Counts Keep Rising (and Why That Is the Wrong Metric)

7 min read

Steph Newman

Steph Newman

Takeaways

  • CVE publication rates keep climbing: Over 28,000 CVEs were published in 2023, up from 14,000 in 2017, and the pace continues to accelerate.

  • Rising counts do not mean worsening security: Increased scanning coverage, higher CVE publication rates, and IT environment growth all inflate counts independent of actual security posture.

  • Raw counts conflate signals: They do not distinguish between risk levels, account for compensating controls, or reflect program effectiveness.

  • Risk reduction metrics are more meaningful: Aggregate risk score trends, MTTR, SLA compliance, and exploitable exposure provide actionable insight that counts do not.

  • Context prevents misinterpretation: Framing count increases alongside coverage expansion and performance improvements prevents misleading conclusions.

The Rising Tide of CVE Publications

The number of CVEs published annually has increased dramatically over the past decade. In 2017, approximately 14,000 CVEs were published. By 2023, that number exceeded 28,000. The pace has continued to accelerate, driven by the expansion of the CNA program (more organizations authorized to assign CVEs), increased attention to software security across the industry, the proliferation of open source components with their own vulnerability disclosure processes, and the growing complexity and interconnection of software ecosystems.

This rising count creates a perception problem for vulnerability management programs. When a CISO reports to the board that the number of vulnerabilities detected in the environment increased by 30% year over year, the natural conclusion is that security is deteriorating. In reality, the increase may reflect expanded scanning coverage (the program now sees more of the environment), the rising CVE publication rate (more vulnerabilities are disclosed across all software), or growth in the IT environment (more assets means more potential findings). The count went up, but the organization's security posture may have improved, stayed the same, or deteriorated, and the raw count alone cannot distinguish between these outcomes.

Why Are Raw Vulnerability Counts a Misleading Metric?

Raw vulnerability counts conflate several distinct signals into a single number that obscures more than it reveals. An organization that expanded its scanning coverage from 70% to 95% of assets will see vulnerability counts increase substantially, even if the per-asset vulnerability rate remained constant or improved. The increase reflects better visibility, not worse security. An organization that upgraded from monthly to weekly scanning will detect vulnerabilities sooner, increasing the number of open findings at any point in time even if remediation speed is unchanged.

Counts Ignore Risk Differentiation

Counts do not distinguish between risk levels. An organization with 1,000 open vulnerabilities, 5 of which are critical and actively exploited on internet-facing systems, is in a different risk posture than an organization with 500 open vulnerabilities, 50 of which are critical and actively exploited. The first organization has more findings but less actual risk. Counts cannot capture this distinction.

Counts do not account for compensating controls. An organization that has applied compensating controls (network segmentation, WAF rules, enhanced monitoring) to 200 critical findings has reduced the effective risk of those findings substantially, but they still count as 200 open vulnerabilities in a raw count. The count suggests the same risk as 200 critical findings with no compensating controls, which is misleading.

Counts Are Gameable

Counts are gameable. Teams can reduce vulnerability counts by narrowing scan scope (scanning fewer assets produces fewer findings), adjusting scan policies (disabling checks reduces findings), or suppressing findings (marking them as false positives or accepted risks removes them from the count without reducing actual risk). Count-based metrics incentivize these behaviors, which reduce the number without reducing the risk.

What to Measure Instead

Effective vulnerability management metrics measure risk reduction, operational performance, and coverage rather than raw finding counts. Risk reduction over time tracks whether the organization's aggregate risk score (weighted by severity, exploitability, and asset criticality) is decreasing. A declining risk score demonstrates genuine security improvement regardless of whether the absolute finding count goes up or down.

Operational Performance Metrics

MTTR by severity measures how quickly the organization addresses findings at each severity level. Decreasing MTTR demonstrates improving operational performance. SLA compliance rate measures whether the organization meets its own remediation commitments. Consistent high compliance demonstrates reliable execution.

Scan coverage percentage reveals how much of the environment the program actually assesses. Coverage that increases from 75% to 95% may increase vulnerability counts but represents a genuine improvement in the program's visibility and effectiveness. Presenting the coverage increase alongside the count increase provides context that prevents the misleading interpretation.

Exploitable Exposure Metrics

Exploitable exposure metrics focus on the subset of findings that represent actionable risk: actively exploited vulnerabilities (KEV matches), findings with high exploitation probability (EPSS above threshold), and critical findings on internet-facing or high-value assets. Tracking this subset over time provides a focused measure of the risk that matters most, separate from the noise of low-risk findings that inflate the total count.

Communicating Effectively About Vulnerability Metrics

Security leaders should frame vulnerability metrics in terms of risk and operational performance rather than raw counts. Instead of reporting "we found 15,000 vulnerabilities this quarter," report "we reduced our exploitable exposure on internet-facing systems by 35% and improved our critical MTTR from 22 days to 14 days." The first statement invites concern. The second demonstrates progress.

When counts do need to be reported, provide context. "Total vulnerability findings increased 20% this quarter, driven primarily by our expansion of scanning coverage to the cloud environment. Of the new findings, 3% are critical, and our MTTR for critical findings improved to 12 days." This contextualizes the count increase as a positive development (expanded coverage) and focuses attention on the performance metrics that reflect actual program effectiveness.

Educating leadership about why counts rise helps set appropriate expectations. Boards and executives who understand that rising CVE publication rates, expanding IT environments, and improved scanning coverage all contribute to higher counts are less likely to interpret count increases as security failures. This education is an ongoing responsibility for security leaders and should be built into regular reporting narratives rather than delivered as a one-time explanation.

Reframing the Narrative for Leadership

Security leaders have a responsibility to educate their leadership peers about why vulnerability counts are misleading and what metrics provide genuine insight into security posture. This education is not a one-time conversation; it requires consistent messaging across reporting cycles that reinforces the relationship between metrics and risk. Every quarterly report should frame vulnerability data in risk terms rather than volume terms, building a shared understanding over time that the right question is "are we safer?" not "how many vulnerabilities do we have?"

Peer benchmarking can contextualize metrics for leadership audiences. If industry data shows that comparable organizations have similar vulnerability finding rates, the message shifts from "we have a lot of vulnerabilities" to "our finding rates are consistent with industry norms, and our remediation performance is improving." This contextualization prevents the perception that the organization is uniquely vulnerable and focuses attention on the operational metrics that differentiate effective programs from struggling ones.

Connecting vulnerability metrics to business outcomes makes the data relevant to non-technical stakeholders. "Our 40% reduction in internet-facing critical vulnerability exposure correlates with zero externally sourced incidents this quarter" links vulnerability management performance to a business outcome (no incidents) that leadership cares about. "Our 12-day critical MTTR means we close exploitation windows before most attackers can target us" translates operational performance into risk language. These connections make vulnerability metrics meaningful to audiences who do not understand CVSS scores or CVE counts but do understand business risk and operational performance.

Metrics That Drive Action

The best vulnerability management metrics are those that drive specific actions when they change. MTTR increasing for critical findings should trigger an investigation into remediation pipeline bottlenecks. SLA compliance dropping below threshold should trigger a review of SLA realism and remediation capacity. Scan coverage decreasing should trigger an investigation into which assets fell out of scope and why. KEV findings remaining open past SLA should trigger an executive escalation. Each metric is tied to a response, making it operationally valuable rather than informational.

Metrics without associated actions are noise. If a metric appears on the dashboard but nobody changes behavior when it moves in the wrong direction, it serves no purpose and should be removed to reduce dashboard clutter. A focused set of 4-6 metrics with defined response thresholds and assigned owners provides more operational value than a comprehensive dashboard with 30 metrics that nobody acts on. The discipline of linking every metric to a response ensures that measurement drives improvement rather than generating reports for their own sake.

Vulnerability count trends at the industry level have implications for how the security community thinks about software security maturity. The increasing count could be interpreted pessimistically as evidence that software is getting less secure despite industry investment in secure development. Alternatively, it could be interpreted as evidence that the security community is getting better at finding and disclosing vulnerabilities, which is actually a positive development for defenders. The truth likely combines both interpretations: software complexity continues to introduce new vulnerability classes, while improved discovery, disclosure, and scanning capabilities reveal flaws that previously would have remained hidden.

For vulnerability management programs, the practical implication of perpetually rising counts is that the program must be designed to scale. Processes that depend on manual analysis of each finding, manual ticket creation, manual prioritization, and manual reporting cannot scale with growing finding volumes. Automation at every stage, from scan execution through prioritization, ticketing, remediation tracking, and reporting, is not optional for programs operating in environments where finding counts grow year over year. Programs that invest in automation early maintain effectiveness as volumes grow; programs that defer automation eventually drown in manual workload.

Organizations that successfully manage the narrative around vulnerability metrics build stronger relationships with their leadership teams. When executives trust that the security team is measuring the right things, interpreting data honestly, and making sound prioritization decisions, they are more willing to fund program improvements and support the operational changes that effective vulnerability management requires. This trust is built through consistent, contextual reporting that connects metrics to business outcomes rather than presenting raw numbers that invite alarm without providing insight.

BBoIoYkI  aO  dYe9m7oW

See Cogent In Action

Schedule a personalized demo today to learn how Cogent can supercharge your vulnerability management program.

Book a demo

Book a demo

Free risk assessment

Free risk assessment

BQoEo4kT  a5  dEeDmXo9

See Cogent In Action

Schedule a personalized demo today to learn how Cogent can supercharge your vulnerability management program.

Book a demo

Book a demo

Free risk assessment

Free risk assessment

B%o2o3kN  aW  dCeOmXoU

See Cogent In Action

Schedule a personalized demo today to learn how Cogent can supercharge your vulnerability management program.

Book a demo

Book a demo

Free risk assessment

Free risk assessment