Defeating Benefits Fraud Project Challenge Sept 25 th

Defeating ‘Benefits Fraud’ Project Challenge Sept 25 th 2008 Stephen Jenner stephen. jenner@cjit. gsi. gov. uk

Major change programmes - the rationale

But the track record is not good… “Implementation of IT systems has resulted in delay, confusion and inconvenience to the citizen and, in many cases poor value for money to the taxpayer. ” Public Accounts Committee of the House of Commons, 2000 “Deficiencies in benefits capture bedevils nearly 50% of government projects” OGC Gateway News, 2003 “ 30 -40% of systems to support business change delivery no benefits whatsoever. ” OGC Successful Delivery toolkit, 2005 “the committee has no confidence that the amounts being assessed have any relationship to the benefits anticipated to be returned. ” US Senate Appropriations Committee, 2007

And ‘horror’ stories abound… • The Dome visitors down by 5 -6 million • Channel Tunnel – Eurostar passenger forecast in yr 1 of full operations = 15. 9 m, actual = 2. 9 m • Bangkok skytrain – demand 2. 5 times over-estimated.

And research indicates this is a widespread problem… • 2002 Mott Mac. Donald study for HM Treasury: • 2% benefits optimism bias BUT and • it’s a BIG BUT – most did not either quantify the benefits and/or evaluate the benefits realised. • Flyvbjerg et al (2005) Demand forecasting in road and rail projects found: • The average overestimation in rail projects was 106% • Road projects - around half were inaccurate by +/-20% • No improvement in accuracy over the last 30 years.

And it’s concluded… “There is a demonstrated, systemic tendency for project appraisers to be overly optimistic. This is a worldwide phenomenon that affects both the private and public sectors…appraisers tend to overstate benefits, and underestimate timings and costs. ” HM Treasury Green Book “Delusional optimism: we overemphasise projects’ potential benefits and underestimate likely costs, spinning success scenarios while ignoring the possibility of mistakes. ” Lovallo and Kahneman “forecasters generally do a poor job of estimating the demand for transport infrastructure projects”. Flyvbjerg et al

The explanations… 1. The technical explanation - our investment appraisal tools are inadequate in relation to future uncertainty? 2. The psychological explanation – Optimism bias, derived from cognitive biases? 3. The economic explanation – approval creates work for project managers, suppliers etc 4. The political explanation – “Strategic misrepresentation”?

The explanations - Strategic Misrepresentation? “the planned, systematic, deliberate misstatement of costs and benefits to get projects approved. ” In short, “that is lying”. And that’s because, “Lying pays off, or at least economic agents believe it does. ” Flyvbjerg et al “Figures don’t lie, but liars can figure” Sharpe and Keelin 38% of respondents in one survey openly admitted to overstating benefits to get funding with the traditional investment appraisal process being, “seen as a ritual that must be overcome before any project can begin” Ward

The result - Business Cases contain ‘assumptions masquerading as facts’

Examples of ‘Benefits Fraud’ or being, ‘economical with the actualité’ • Double counting • Inconsistent valuation • Over-estimation of impact • Under/over valuation of impact • Efficiency ‘savings’ or buying vouchers

Key messages – 3 challenges… 1. Ensuring benefits claimed are robust and realisable 2. Capturing all forms of value created 3. Realising benefits and creating value.

1. 1 Ensuring Benefits forecasts are robust and realisable – the Benefits Eligibility Framework • The set of rules about what benefits can and can’t be claimed and how they should be valued • Ensures a methodologically sound approach to measuring and valuing benefits • Allows a consistent approach across the Portfolio – a level playing field and so • Enables comparisons over time and between projects “Use a benefits framework to define business benefits” Source: Gartner, ‘Show Me the Money: Advanced Practices in Benefits Realisation’

1. 2 Independent review – a ‘fool’s errand’ • Ayers suggests an, “’Advocatus Diaboli’…whose job it is to poke holes in pet projects. These professional “No” men could be an antidote to overconfidence bias. ” • Davidson Frame proposes the use of “murder boards” – a cross departmental panel charged with pulling a proposal apart to, “make sure that arguments in support of project ideas do not have built into them the seeds of their own destruction. ” • Steve Jenner - I suggest a ‘fool’ to ask the questions others dare to ask and identify those assumptions that masquerade as facts.

1. 3 Triangulation • Use more than one appraisal method, or value lens: • combine financial metrics with multi -criteria analysis and decision conferencing, and • examine ‘Attractiveness’ in the context of ‘Achievability’.

1. 4 Validation in the CJS IT portfolio… Project View of Benefits Contributor Business Case Development CJIT VMO Measuring & Tracking: Initial Baselining 1 -Page Summary Benefits Report If funding approved Benefits Eligibility Framework Benefits Evidence & Revised Forecast Check alignment with BEF e. g. double counting, cost avoidance Benefits Recipients Efficiency Planners Strategic Planners Project Live Service Project Benefits Lead and Benefits Realisation Lead agree Efficiency & Effectiveness benefits Benefits Validation with Recipients Benefits Validation with Efficiency Planners Benefit Realisation Plan: Police Benefit Realisation Plan: CPS Benefit Realisation Plan: Courts Benefit Realisation Plan: Corrections Benefits Validation with Strategic Planners Measuring & Tracking: Benefits Realisation CPS Efficiency Plan DCA Efficiency Plan HO Efficiency Plan Benefit Realisation Leads and Efficiency Planners agree Efficiency Benefits that contribute to Departmental Efficiency Plans Confidence Victims & Witnesses Offences Brought To Justice (OBTJ) Enforcement Strategic Planners agree Efficiency & Effectiveness Benefits that contribute to Strategic Targets & priorities

Value Management 2 - Capturing all Forms of Value Political Efficiency Foundation Effectiveness Social Value Categories of Value

Understanding the logic that underpins an investment Initiative Hypothesis (IT Functionality) Root Cause (Level 2) Root Cause (Level 1) Consequences Targets Cost and measurement data underpins consequence

3. Realising benefits and creating value • Leadership – re-setting the standards of expected behaviour • Who writes the business case? • Transparency & Accountability - Regular stage gates with formal recommitment to the cost/benefit case • REAL post implementation reviews - Track results and use them to provide an evidence base for future forecasts • Managing benefits from an enterprise-perspective and asking - Is that the best we can do? And…

‘Clear line of Sight’ from Strategic Intent to Value creation

Thank you – Questions? Stephen. Jenner@cjit. gsi. gov. uk