When security leaps from "subsidiary KPI" to a hard metric of business resilience, measurement and observability are no longer optional. The final chapter of the "Six Pillars of DevSecOps" published by the Cloud Security Alliance (CSA) Greater China - "Measure, Monitor, Report, and Act" (Pillar 6) provides a set of "visible, measurable, and improvable" closed-loop methodologies for those engaged in information security, information technology management, and business functions.
Original article link: https://mp.weixin.qq.com/s/0iUhV2DU69D_wh59L5Lu7A
Why Focus on Pillar 6?#
Validate Investment: Without quantification, it is impossible to prove security ROI or synchronize with the business.
Unified Language: Use a unified metric system to bring development, operations, security, and management onto the same "dashboard."
Drive Improvement: Monitor ➜ Visual Report ➜ Precise Action, achieving a "Discover-Fix-Retrospect" pipeline.
"One Diagram to Understand" Pillar 6 Framework#
- Measure the "security health" using a unified metric pool;
- Monitor by integrating these metrics into four data pipelines: logs, metrics, tracing, and user experience, forming a real-time "digital twin";
- Report by translating monitoring data into insights that both management and frontline can understand, following the four principles of "Visible-Focused-Iterative-Collaborative";
- Finally, action will automatically assign report-driven improvement tasks to development, operations, and security toolchains, closing the loop.
This sequence from measurement → observability → decision-making → closure breaks down the abstract goals of DevSecOps into concrete, executable, and reviewable daily workflows, making security both "visible" and "responsive."
12 Core Metrics#
| Dimension | Metric Examples | Value |
|---|---|---|
| Vulnerabilities | MTTI, MTTR, unresolved burn-down rate, development speed ratio | Transforming "issue backlog" into "risk curve" |
| Architecture Security | Control reuse rate, depth defense score, security model adoption rate, threat-risk mapping | Quantifying the extent of "design as security" implementation |
| Incident Response | MTTD, MTTC, MTTRN, post-review closure rate | Ensuring "alert → recovery" is reviewable and optimizable |
Before formally listing the "core metrics," let's first answer a commonly asked question by executives: Why 12 metrics, and not more or fewer?
Pillar 6 divides the entire lifecycle of DevSecOps into three main branches: "Vulnerability Management - Security Architecture - Incident Response," with each branch selecting four metrics that are most explanatory for the business and easy to automate. This approach covers the three dimensions of left-shift security (vulnerability exposure in design and coding phases), systemic security (depth defense and model implementation), and right-shift security (alert closure and recovery resilience), while avoiding the overload of metrics that could hinder implementation.
[!TIP]
It is recommended to first apply 1-2 "North Star Metrics" in a pilot line, such as MTTI + MTTR or MTTD + MTTRN, to validate data collection and dashboard visualization over 2-3 iterative sprints; once the metric data stabilizes, gradually complete the remaining 8-10 metrics. During this process, be sure to document sources, criteria, and thresholds to facilitate horizontal team benchmarking and avoid communication costs caused by "different interpretations of the same metrics."
Maturity Self-Check: Alpha → Beta → Charlie#
| Team Quadrant | Current Status | Best Next Step |
|---|---|---|
| Alpha | Missing metrics, delayed patches, event backlog | Establish minimum metric set + weekly scans |
| Beta | Dispersed metrics, fragmented processes | Automated controls + cross-functional dashboards |
| Charlie | Closed-loop metrics, data-driven decisions | Refined cost-benefit & cultural solidification |
In practice, the transition from Alpha → Beta → Charlie is not a linear "level-up" process but more like a mirror reflecting the organization's security culture and engineering capabilities in real-time. The correct approach is to first use core metrics for a health check to identify which quadrant you are in; then, refer to the metric pool and improvement manual provided by Pillar 6 to clarify the shortcomings to be addressed in the next phase. This "Measure - Gap Analysis - Iteration" closed loop can avoid the dilemma of blindly adopting tools and piling up processes in pursuit of high scores.
It is worth noting that many teams encounter "cross-functional collaboration bottlenecks" during the transition from Beta to Charlie—technical debt decreases, but organizational collaboration bottlenecks emerge. Experience from Pillar 6 indicates that when hard metrics like MTTI and MTTR stabilize but security still struggles to advance, it is often because security measurement results are not included in performance and budget allocation. At this point, the four reporting principles (Visibility, Focus, Iteration, Collaboration) should be embedded in OKRs and quarterly reviews, using data to drive resource allocation and incentives to truly break the deadlock.
Finally, whether you are currently Alpha, Beta, or Charlie, you should maintain a "observability-first" mindset: turning every defect, every alert, and every improvement into data points. When the team can tell the "security value story" using the same set of metrics, the maturity curve will naturally rise, and security ROI will become easier to quantify and gain executive buy-in—this is the long-term goal that Pillar 6 aims to help you achieve.
Five-Step Implementation Roadmap#
The first step is to set 3-5 "North Star Metrics" that are most closely aligned with business SLAs (such as MTTI, MTTR, MTTD, etc.) for the team, transforming security from an abstract concept into a measurable, aligned common goal. Only when development, operations, security, and management see the same set of numbers on the same dashboard will subsequent investments and collaborations have a clear direction.
The second step is to conduct a proof of concept (PoC) using a single product line or golden pipeline. Validate the complete closed loop of data collection, dashboard visualization, and alert linkage within the smallest viable scope, quickly proving that "security observability can indeed enhance delivery reliability," thereby gaining organizational trust and resources for expansion.
The third step, once the PoC is successful, is to enter the vertical expansion phase: along the three main branches of "Vulnerability → Security Architecture → Incident Response," gradually complete the 12 core metrics and integrate logs, metrics, tracing, and UX data into a unified observability platform. This will create a full-link closed loop from left to right while avoiding data silos caused by fragmented tools.
The fourth step is to initiate horizontal promotion once the data pathways are stable. Copy the unified metric criteria, thresholds, and dashboard templates to more R&D/operations teams, allowing different functions to discuss risks and improvements using the same language. This corresponds to the rhythm of "first deepening vertically, then expanding horizontally across multiple projects" in the report, ensuring that successful experiences are quickly scaled rather than falling into the vortex of reinventing the wheel.
The final step is continuous feedback and cultural solidification. Through monthly or quarterly "Measure - Report - Action" cycles, embed metric results into OKRs, performance, and budget decisions; when resource allocation and incentive distribution are driven by data, DevSecOps measurement will evolve from pilot tools to organizational habits, truly achieving the long-term goal advocated by the report of "managing enterprise-level risks with security observability."