DevOps is a high priority project for almost 80% of enterprises. How are organizations measuring their return on investment in DevOps?
According to Gartner, DevOps represents a change in IT culture, focusing on rapid IT service delivery through the adoption of agile, lean practices in the context of a system-oriented approach. DevOps emphasizes people (and culture), and seeks to improve collaboration between operations and development teams.
The ultimate goal of DevOps is to release software and applications more quickly at higher quality. Google, Amazon and Netflix now provide concrete proof points of fast-movers applying DevOps practices to disrupt entire industries with rapid-fire, digital-innovation based on cloud, big data and mobile technologies.
While DevOps practices are transformational, they also involve an investment in changing behaviors, processes and tools over a long period of time. Discussions with companies who have matured their DevOps practices such as Netflix, Amazon and others, suggest that this process can take over 5 years. If organizations are going to be involved in a change that will take 5 years or longer to be successful, the return must be significant to justify the cost and disruption.
Expected Returns from DevOps Investments
While not all benefits are easily translated into financial measures, the following metrics have been reported:
Lower IT costs and improved employee productivity – reduction in unnecessary app development, testing, deployment and operations results in a cost savings of 25% per year.
Faster/increased revenue growth – companies expect that speed to market improvements will result in 15-20% faster revenue growth.
Higher service availability – IT reported data show that eliminating unplanned downtime results in savings of $100K/hr of reduced downtime for infrastructure and $500K-1M /hr of reduced application failure time).
The benefits that are most difficult to measure are also perhaps the most important. These include competitive advantage through a great customer experience, faster time to market and better product quality, as well as reduced security and compliance risk through better and more comprehensive testing and auditable processes.
Expected Costs of DevOps Implementation
There are associated costs with achieving these outcomes and these will become significant over the 5 years that it takes to mature a DevOps practice.
Change management was listed as the biggest risk to successful DevOps adoption by over 50% of companies surveyed by the International Data Corporation (IDC) in its report “DevOps and the Cost of Downtime: Fortune 1000 Best Practice Metrics Quantified”. Change management costs include such things as the cost of training time, materials and cost of classes for all employees, the cost of acquiring new tools that conform to the new methodology, as well as the cost of losing employees who are not accepting of the change and replacing them with similarly skilled employees. Finally, once change is implemented, productivity inevitably goes down until the team gets comfortable and skilled at executing under the new processes.
Once DevOps processes are adopted, they must be applied to all applications. Applications moving to DevOps are said to be onboarded (where the original build, deploy, test and release automation is created). To successfully onboard an application requires education of the sponsoring executives, training of the impacted developers and testers, and coordination with the change and release managers. In addition, there is a pure people cost for onboarding applications to the DevOps process. For example, a typical DevOps engineer can onboard five applications a week. Onboarding a large number of application components can consume many engineers over an extended period of time.
In addition, DevOps fosters continuous improvement and as a result, application development methods continuously evolve and may adopt new technology components. Collection of meta data about the behavior of application components is no small task, requiring frequent follow-ups and action items to reach completion. This data collection and analysis may result in significant changes that require revisiting the DevOps automation to adapt to these ongoing revisions and updates.
Bi-Modal Infrastructure Costs
The costs are lower for developing new applications that can be designed specifically for a more agile DevOps friendly environment. For existing applications, the cost to move into a DevOps process can be significantly higher. In addition, the cost of working with existing infrastructure is very high. Infrastructure cost for new applications is often cloud based (whether public or private) and the cost is incurred “on-demand”. Infrastructure cost for pre-existing applications is often datacenter-based and fixed. Existing applications must be re-factored over time to fully benefit from DevOps and this application re-factoring cost can be very high.
Steps for Finance Executives
As your organization is embarking on its DevOps journey, there are a number of critical steps that you can take as the finance executive to help ensure the best results with the least wasted costs:
- Ensure executive support. This is a top factor in the success of DevOps initiatives
- Ask for metrics. DevOps is based on LEAN six sigma quality practices and emphasizes measurement of key performance indicators followed by continuous improvement. Getting the teams to focus on metrics at the beginning is important to enable this process.
- Ensure that security testing is integral to DevOps. Often security testing is run by a separate group for compliance reasons. But, this doesn’t mean that it should be performed outside of the DevOps process. In fact, incorporating security and compliance requirements into the DevOps processes can generate a big return in terms of security and compliance outcomes.
- Establish a roll-out process that minimizes risk and maximizes return. DevOps takes time and the early projects will have a big influence on the long term success of the transformation to DevOps.
- Avoid bi-modal solutions. It is not desirable to implement DevOps practices that are different for different projects. It is also much more expensive. Forcing some centralization of tools and processes can save costs.
- Provide legal and regulatory oversight of continuous deployment. The ultimate goal of DevOps is to deploy automatically into production on a continuous basis. For many organizations that are subject to legal and business compliance requirements, this may never be truly possible. It is still critical that compliance processes be implemented as part of DevOps.
- Align incentives for all groups collaborating in a DevOps process – from development to IT operations. Traditionally these groups have been measured by different metrics. In order for DevOps to work, these groups need to have common incentives (if not become part of a single organization).
DevOps is one of the major digital transformation enablers. This practice and methodology is a must for any organization that develops software as part of its business operations. It is unlikely that companies will be able to compete in 5 years if they have not developed mature DevOps processes, which means that they must start now. Yet this transformation is difficult and has a high risk and a high cost. Organizations that apply return on investment criteria to DevOps will be in a better position to manage the costs and risks while working to maximize their return from this IT transformation.
Joan Wrabetz is the CTO of Quali.•