We recently had a conversation with Matthew Hurewitz, Director of Platforms and Application Security at Best Buy, on remediating cloud vulnerabilities and misconfigurations. Here are the 5 key takeaways from our chat:
1. Teams experience ‘5 Stages of Grief’ in Risk Remediation
When remediating cloud vulnerabilities and misconfigurations, teams often experience a process akin to the five stages of grief. This analogy, as discussed in the webinar, reflects the emotional journey security and DevOps teams undergo when addressing risks on a daily basis, particularly in large enterprises with complex environments, a high volume of incoming risks and frequent organizational changes.
While today’s tools have gotten really good at identifying cloud risks, there are still a lot of manual steps that need to happen in order to actually do something about them. First, analysts need to understand the nature, severity, and potential business impact of the identified risk or vulnerability. This requires a lot of organizational context, that in many cases, is not readily available. From there, there is a lot of coordination that needs to happen with other internal teams – such as DevOps and engineering (the teams that can actually apply the fix). Further, depending on the finding’s potential impact, it also needs to be escalated up the chain to ensure it gets prioritized. Sometimes, if relationships are well-established, the fix might be implemented quickly, but the process varies widely depending on the context and complexity of the situation and is generally time consuming and difficult to navigate.
2. Achieving a Meeting of the Minds is a top Remediation Challenge
When discussing the major blockers when it comes to remediating risks efficiently, Matt highlighted the biggest challenge as achieving a “meeting of the minds,” closely followed by the difficulty of finding the best path to resolution.
- A meeting of the minds: One of the biggest challenges in getting risks prioritized for resolution within an organization is the complex communication chain. It often requires extensive teaching, listening, and navigating through different levels of authority. A senior engineer may direct you to their manager, but even after gaining their support, you may need to escalate to higher levels of leadership. This can make it difficult to find the right decision maker, especially in large organizations. Ensuring alignment, or a “meeting of the minds,” is crucial but is often a very time-consuming and challenging task.
- Finding the best path to resolution: Finding the right solution for a vulnerability can be tricky, especially when the recommended fix isn’t always feasible in your specific environment. For example, while vendors may provide straightforward guidance, implementing it might not always be possible due to unique system constraints. The challenge then shifts to identifying whether to apply a mitigating control or a permanent fix that suits your environment. This requires collaboration with different teams and stakeholders, each responsible for implementing different solutions to either fix or mitigate the problem.
3. The Cost of Remediation = Time, Resources and Missed Opportunities
Quantifying the cost of remediation and tracking it over time can be incredibly complex due to the many variables involved. However, doing so presents a compelling business case for implementing changes that drive efficiency. When breaking down the cost of remediation, we discussed both the direct and opportunity costs associated:
- Direct costs: This includes budget and resources allocated to addressing vulnerabilities. These costs grow when inefficiencies arise – like the need for multiple meetings to align stakeholders and research solutions. Matt gave an example in the webinar of what this might look like. If it takes 3-4 meetings, with each meeting involving 3 to 5 attendees, just to gain alignment on a single issue, that could amount to 20 person-hours dedicated solely to this effort. If you calculate this based on an average rate of $150 per hour, that results in approximately $3,000 for just one vulnerability. Considering organizations often face hundreds or even thousands of vulnerabilities, the cumulative costs can be a lot. While not every vulnerability requires such a time-intensive and expensive approach, many do.
- Opportunity costs: When understanding the cost of remediation, you also have to think about opportunity costs. The question becomes: What opportunities are being missed while teams are focused on resolving vulnerabilities? When teams are forced to prioritize remediation efforts over other strategic or revenue-generating initiatives, it can become a problem for the business, especially as organizations face tighter budgets and workforce reductions.
4. Threats, Market Pressures, and Cost Savings Drive Change
When it comes to driving change and making various investments that drive more efficient risk remediation programs (e.g. implementing new processes or technologies), news-worthy threats, market pressures and cost reduction were discussed as key driving factors.
- External market pressures and high-profile threats: While not the most proactive, the urgency to address something is often driven by vulnerabilities in the wild, security threats that dominate the news, and emerging market trends, such as the rapid adoption of new technologies such as AI and LLMs.
- Operational cost reduction: Given that risk remediation places significant demands on teams outside of security, such as DevOps and engineering, showing concrete metrics like hours saved or cost reductions can justify and drive investments. In this case, more efficient remediation can drastically reduce operational costs and free up valuable time for engineers to concentrate on other projects.
5. Efficient Risk Remediation Programs Require Alignment and Consistent Tracking
When asked about two key changes Matt has implemented to enhance the efficiency of risk remediation programs, he highlighted the importance of ensuring alignment and clear roles and responsibilities, as well as the necessity of consistent tracking to accurately identify where improvements and investments should be made.
- Ensuring alignment and ownership: Being aligned with engineering counterparts requires mutual accountability and clear roles. A culture must be fostered where engineers understand their responsibility in securing the business (whether that’s preventing vulnerabilities from reaching production and remediating those that do). For example, by implementing release blocking practices, organizations can encourage engineers to prioritize secure coding and recognize the impact of their work on overall security.
- Consistent tracking: Consistently measuring risk remediation processes reveals surprising insights about time and resource allocation. Many organizations operate on assumptions, but quantifying efforts provides clarity on how often vulnerabilities occur and the actual costs involved. By tracking the process, even for just a week, you can start to uncover the extent of the issue and develop a compelling business case for prioritizing remediation efforts.
If achieving a new standard for cloud risk and vulnerability remediation is something you’re working towards, learn more by visiting our website or request a demo to see the ZEST platform in action.
