CHAOS REPORT 2015 - standishgroup.com

1 The CHAOS REPORT 2015 is a model for future CHAOS Reports. There have only been two previous CHAOS Reports, the original in 1994 and the 21st edition of 2014. This new type of CHAOS REPORT focuses on presenting the data in different forms with many charts. Most of the charts come from the new CHAOS database from the fiscal years 2011 to 2015 . The CHAOS fiscal year starts March 1 and runs until the end of February. A few of the charts are from the new SURF database to highlight certain information. The purpose of this REPORT is to present the data in the purest form without much analysis and little thought leadership. Analysis and thought leadership are offered in the CHAOS Manifesto series of reports. Another major change is how we define success. We have multiple definitions, including our newest.

2 We coded the new CHAOS database with six individual attributes of success: OnTime, OnBudget, OnTarget, OnGoal, Value, and Satisfaction. Our Traditional definition is OnTime, OnBudget, and OnTarget. This means the project was resolved within a reasonable estimated time, stayed within budget, and contained a good number of the estimated features and functions. Our new Modern definition is OnTime, OnBudget, with a satisfactory result. This means the project was resolved within a reasonable estimated time, stayed within budget, and delivered customer and user satisfaction regardless of the original scope. We have the flexibility to present the results for one to six of these attributes in any combination. Copyright 2015 The Standish Group International, REPORT 2015 TRADITIONAL RESOLUTION FOR ALL PROJECTSONBUDGETONTIMEONTARGETThe Traditional resolution of all software projects from FY2011 2015 within the new CHAOS percentage of projects that were OnBudget from FY2011 2015 within the new CHAOS percentage of projects that were OnTime from FY2011 2015 within the new CHAOS percentage of projects that were OnTarget from FY2011 2015 within the new CHAOS database.

3 *All data , unless otherwise noted, represents results from FY2011- 2015 . The total number of software projects is 25,000-plus, with an average of 5,000 per yearly 56%NO 60%NO 44%YES 44%YES 40%YES 56%20112012201320142015 SUCCESSFUL39%37%41%36%36%CHALLENGED39%46 %40%47%45%FAILED22%17%19%17%19%Copyright 2015 The Standish Group International, Modern Resolution definition is OnTime, OnBudget, with a satisfactory result. This definition encompasses both a success rate for the project management of a project and for the project itself. The Traditional Resolution of OnTime, OnBudget, and OnTarget clearly supports the goals of project management, but not the customer or user of the product or project. The reason we consider this to be the best definition is that it combines the project management process and the end results of a project.

4 We have seen many projects that meet the triple constraints of OnTime, OnBudget, and OnTarget, but the customer was not satisfied with the outcome. This is evident in the data , which shows a 7% decrease in the success rate and a 7% increase in the challenged changing from the OnTarget constraint to satisfactory we avoid penalizing a project for having an evolving target, which all projects have, even the very small ones. Customers have a clear opinion on the satisfaction level whether or not all the features and functions that they asked for in the beginning of the project are realized. In our research we found that both satisfaction and value are greater when the features and functions delivered are much less than originally specified and only meet obvious needs.

5 In other research we found that most features and functions of software are not used. These additional features increase cost, risk, and quality but do not necessarily provide value. MODERN RESOLUTION FOR ALL PROJECTSVALUABLEONGOALSATISFACTORYThe Modern Resolution (OnTime, OnBudget, with a satisfactory result) of all software projects from FY2011 2015 within the new CHAOS database. Please note that for the rest of this REPORT CHAOS Resolution will refer to the Modern Resolution definition not the Traditional Resolution definition. The percentage of projects considered valuable from FY2011 2015 within the new CHAOS percentage of projects that were OnGoal from FY2011 2015 within the new CHAOS percentage of projects considered satisfactory from FY2011 2015 within the new CHAOS 38%NO 44%NO 41%YES59%YES 62%YES 56%Copyright 2015 The Standish Group International, size has always been a major element in the CHAOS research.

6 It was clear from the very beginning of the CHAOS research that size was the single most important factor in the resolution of project outcome. On this page we show two tables: resolution of all software projects by size; and size of the software projects by resolution. These tables clearly show the impact of size on the results of Ontime, OnBudget, with a satisfactory result. It is also clear that the larger the project, the less valuable the return rate. In many cases larger projects never return value to an organization. The faster the projects go into production the quicker the payback starts to accumulate. One of the major services of our Value Portfolio Optimization and Management Service is to break up large software projects into multiple small projects, with early delivery for success, quicker return on value, and greater customer and user satisfaction.

7 We have found that most software projects only require a small team for a short duration in order to deliver value to the organization; only in very rare cases do projects need to be larger and longer. Most, if not all, large, complex, multi-year projects are unnecessary. This is especially true for standard infrastructure software such as middleware, databases, and system management. PROJECT SIZE BY CHAOS RESOLUTIONCHAOS RESOLUTION BY PROJECT SIZESUCCESSFUL CHALLENGEDFAILEDTOTALG rand6%51%43%100%Large11%59%30%100%Medium 12%62%26%100%Moderate24%64%12%100%Small6 1%32%7%100%SUCCESSFUL CHALLENGEDFAILEDG rand2%7%17%Large6%17%24%Medium9%26%31%Mo derate21%32%17%Small62%16%11%TOTAL100%10 0%100%VALUE FOR LARGE PROJECTSn Very High Value 4%n High Value 14%n Average 23%n Low 16%n Very Low 43%The return of value for large projects from FY2011 to 2015 within the new CHAOS FOR SMALL PROJECTSn Very High Value 17%n High Value 27%n Average 36%n Low 9%n Very Low 11%The return of value for small projects from FY2011 2015 within the new CHAOS size of software projects by the Modern Resolution definition from FY2011 2015 within the new CHAOS resolution of

8 All software projects by size from FY2011 2015 within the new CHAOS 2015 The Standish Group International, at project resolution by industry provides another view of the CHAOS database. The table on this page shows the resolution of all software projects by industry from FY2011 2015 within the new CHAOS database. The results show that retail projects had the highest success rate at 35% using the Modern definition of success. The results also show that government projects had the highest failure rate at 24%, and financial and government projects had the highest challenged rate at 56%. The comparison of satisfaction level for banking versus retail shows that retail also has better results. Many of our Benchmark clients like to compare their results to other organizations in the same industry and we do this as matter of course.

9 However, we found that industry is not the most accurate or important metric of comparison. The most accurate is to consider industry as a minor filter, with project type, size, skills, and methodology as primary filters. Other minor filters would include organizational size and geography. In our Resolution Benchmark Membership we use this technique to benchmark project portfolios. CHAOS RESOLUTION BY INDUSTRYBANKING PROJECTSn Very Satisfied 9%n Satisfied 21%n Somewhat Satisfied 28%n Not Satisfied 19%n Disappointed 23%The satisfaction level for banking projects from FY2011 2015 within the new CHAOS PROJECTSn Very Satisfied 12%n Satisfied 27%n Somewhat Satisfied 32%n Not Satisfied 18%n Disappointed 11%The satisfaction level for retail projects from FY2011 2015 within the new CHAOS CHALLENGEDFAILEDB anking30%55%15%Financial29%56%15%Governm ent21%55%24%Healthcare29%53%18%Manufactu ring28%53%19%Retail35%49%16%Services29%5 2%19%Telecom24%53%23%Other29%48%23%The resolution of all software projects by industry from FY2011 2015 within the new CHAOS database.

10 Copyright 2015 The Standish Group International, SELECTION PROCESSn Only the boss/senior executive 27%n There is a formal process to do it 50%n Each sector defines its own priorities 21%n It is very varied or I do not know 3%We asked the 37% of SURF respondents who said that the key executive sponsor sets the budget with the project team, In general, who participates in project selection/approval in your organization? This is based on 111 responses in the SURF our CHAOS REPORT 2014 we showed that project resolution differed slightly by most of the areas of the world. In that REPORT we used the traditional success metrics of OnTime, OnBudget, and OnTarget. In this REPORT we used the Modern definition of success of Ontime, OnBudget, with a satisfactory result.