Скачать презентацию Measuring criminal court efficiency using DEA Charlie Lee

7d047de33ca0dce0d0bf089a3cd13076.ppt

• Количество слайдов: 26

Measuring criminal court efficiency using DEA Charlie Lee Forecasting and Model Development Unit (FMDU) CJ Sig Event March 2013

Agenda • Background - Background information about the project and why we are doing it. • Overview of the DEA methodology - Overview of the main points in the Data Envelopment Analysis methodology. • Developing the DEA model for the Crown courts - A walkthrough of things to consider when developing the DEA modelling. • Next steps 2

Background • HMCTS currently record vast quantities of management information related to the court process. • Regular performance reports and charts are produced and updated to keep track of various measures and targets related to court performance. • However, because there is so much data, it can be difficult to determine which courts are efficient and where to focus resources on. Project aim To explore a new analytical method of comparing the court level performance between each court (for the Crown Courts and magistrates’ courts) using a combined range of indicators. • Identification of most efficient courts for best practice • Identification of least efficient courts for improvement. • Identification of courts that are most similar, and why they differ in efficiency. To provide HMCTS decision makers with useful information and give a new perspective on efficiency performance to enable better use of time and resources 3

What is Efficiency? Efficiency can be defined as a measure of how well inputs are converted into outputs: Inputs Process Outputs In its simplest form, efficiency can be described as: . This can be applied to any input and output, for example: Papers per Author = Goals Per Game = Sandwiches = Per Hour A unit (e. g. a university, a football team, a cafe) can then measure their relative performance in two ways: 1) Against other units operating in the same field, where the higher their score, the higher their efficiency. 2) Against a theoretical benchmark, where the nearer to the benchmark they get, the higher their efficiency. 4

Measuring Efficiency with Multiple Inputs and Outputs In most cases, combining multiple single efficiency ratios can be problematic as they can include contradictory views. For example, who is more efficient, Café A or Café B? The most simple way around this problem is ratio analysis, which works by creating a weighted single score. Although simple, this method requires an agreement on the weighting of scores which can lead to disagreements between units under investigation. More advanced techniques such Data Envelopment Analysis can avoid this problem. 5

Data Envelopment Analysis (DEA), also known as frontier analysis, is a linear programming methodology to measure the efficiency of multiple Decision Making Units (DMUs) when the production process presents a structure of multiple inputs and outputs. • Can handle multiple inputs and outputs • Relative peer to peer measure • Non Parametric approach Results • Gives a single efficiency score, although multiple units are likely to be 100% ‘efficient’. • Shows target input production and/or output consumption required to achieve efficiency. • Shows potential role models (efficient peers) for comparison. 6

How DEA works Stage 1 - Apply weights (ui, vi) to each of the input (xi) and outputs (yi) for the first unit. DEA selects the weights that maximises the units’ efficiency, which eliminates the need to create an agreed set of weighting between units. Outputs Inputs (v 1) * Sandwiches sold (y 1) (u 1) * Number of Staff (x 1) (v 2) * Tea sold (y 2) Café X Stage 2 – Apply the same weights to all other units and see if they can achieve a better rating. This can be a single unit or a selection of units to make a theoretical achievable unit point. Café A Theoretical Cafe Café B Stage 3 – Efficiency scores are created based on the difference between the efficient unit and the unit under investigation given the weighting Stage 4 – Repeat from stage 1 with next unit 7

Graphical Illustration Given an example with two outputs and a single input, an efficiency frontier can be created by drawing straight lines between units that are 100% efficient. In the example below this is Café A, Café B and Café C. It is then assumed that all points on this line are achievable and 100% efficient. Theoretical Achievable Point Café A The efficiency of Café X is therefore the ratio of the distance between 0 and Café X over the distance between 0 and theoretical achievable point. Café B Café X When evaluating a units’ efficiency that does not lie on the frontier, a straight line can be drawn to the origin point. In the example, this is represented by the combination of the solid and dashed line between Café X and (0, 0). Café C 0 8

DEA Formulation The model appeared first in ratio form, but can be converted to its linear form, making it easier to solve. It works by maximising the objective by varying the weights, but ensuring that applying those same weight to every other units will not lead to a score greater than 1. Ratio Formulation Max Subject To Linear Formulation Max Subject To 9

Orientation of DEA Input Orientation Output Orientation Attempts to minimise the inputs given the outputs Attempts to maximise the outputs given the inputs e. g. Use fewer staff to serve the café’s regular customers who always order the same sandwiches and teas. e. g. Keep the café’s staff numbers the same, but sell more sandwiches and teas to the regulars or attract new customers. 10

Economies of Scale Constant Returns to Scale (CCR) Variable Returns to Scale (BCC) Assumes that economies of scale have no impact and everyone operates under the conditions that creating 100 units of outputs i requires exactly 100 times the effort as creating a single unit of output i. Assumes that economies of scale have an impact on units, therefore acknowledging that smaller and larger units outputs may benefit or be affected by the units size. This will always lead to more units becoming 100% efficient. e. g. Decide if size affects how a café operates. A larger café may be more difficult to manage due to its size, but can buy its goods can in bulk. Small cafés may be too small to operate at optimal efficiency, but maybe better at retaining customer loyalty. 11

Developing DEA modelling for the Crown courts 12

Overview of the Criminal Courts in England Wales Recorded Crime Out of Court Disposals Offence Detected CPS discontinue Magistrates’ court Committed for Trial Mags court proceedings Not guilty/ acquitted Guilty Sent for Trial Committed for Sentencing Crown court Mags court Sentencing Appeal 13

Inputs and Outputs Selection Possible Input measures Input measure Court sitting days used Description A measure of the number of days spent to deal with the workload. A courts budget is allocated based on the estimated sitting days needed to deal with its workload. Court sitting hours used Similar measure to sitting days, some courts may sit a different number of hours in a day. This could be used as a proxy for the amount of effort spent in a court. Number of Staff An indication of the resources available to the court. Financial Operating The actual costs involved in running the court, care must be taken to Costs examine in detail what is included, as some types of costs may not be relevant for comparison. 14

Inputs and Outputs Selection Possible Output measures Output measure Disposals Early Guilty Pleas Vacated Trials Ineffective Trials Description A measure of throughput for the court, these are the volumes of cases dealt with. Where the defendant pleads guilty before the trial start. Where a case listed for trial is cancelled before the trial start date. Where the trial is cancelled on the day of the trial. Trial Waiting Time A timeliness measure, the time from lodging the hearing/trial to the start of the hearing/trial. Juror Utilisation Not all jurors selected for Jury are utilised, a low juror utilisation rate means a wastage of resources and time. Witness Waiting time A timeliness measure for witnesses called to give evidence. 15

Inputs and Outputs Selection Issue • The DEA method, in common with all linear programming are extreme point techniques. • The results are therefore sensitive to all inputs and outputs selected, as all variables can be seen as important as each other. • The more inputs and outputs selected, the more 100% efficient units there are. Solution • Careful selection of inputs and outputs. • Work closely with customers to determine priorities and include only a few of the most relevant inputs and outputs. • Check the data availability and quality is feasible for the analysis. 16

Investigating the case mix effect • The different types of cases tried in court will vary in their complexity as indicated by their average court duration time. • Due to regional variations, some courts may deal with more of a particular type of case, their case mix will vary. e. g. Some courts may only deal with serious cases such as murders, whilst others may deal with more burglary and handling stolen goods. • Therefore, we need a method to fairly compare courts given their case mix. The average court duration times shown are for illustration only 17

Investigating the case mix effect • To deal with the case mix effect, we have created a weighting system by sub offence level based on the average court duration time for a particular sub offence compared to the average case in England Wales. • For example, if the average court case is given a weighting of 1 point, then a Murder is worth 13 points, as the average court duration for a murder case may take 13 times longer than an average case. The figures shown above are for illustration only • Using this weighting system enables the courts to be fairly compared against each other when we consider their volumes of disposals as an output. 18

Crown Court Investigation To illustrate the sensitivities when selecting the parameters for DEA modelling, the following simplified models will demonstrate graphically how the results can vary. The models are run using an output orientation on the assumption that inputs are fixed by HMCTS and given these inputs they should be producing a certain amount of outputs. Model 1: One input and one output (Constant returns to scale) Inputs Outputs Sitting Hours Disposals Model 2: One input and one output (Variable returns to scale) Inputs Outputs Sitting Hours Disposals Model 3: One input and two outputs (Variable returns to scale) Inputs Sitting Hours Outputs Disposals Early Guilty Pleas 19

Model 1: one input and one output Efficiency Line DMU 4 DMU 3 DMU 2 DMU 1 The points on the chart represent the 76 Crown courts (DMU’s), disposals as an output are plotted against an input of sitting hours used. This is a standard ratio analysis model to illustrate the differences in results when scale is not considered. 20

Model 1: one input and one output DMU 4 is far away from the efficiency line and relatively not very efficient in this model. DMU 3 DMU 2 DMU 1 We can see the most efficient unit is DMU 1, and those furthest away from the efficiency frontier are considered to be the least efficient. 21

Model 2: One input and one output (Variable returns to scale) DMU 4 DMU 3 DMU 2 DMU 4 now lies on the efficiency frontier and is relatively 100% efficient compared to peers in this model. DMU 1 Using the same data points, if we choose a variable returns to scale approach, the efficiency frontier changes as more units are now captured as being most efficient. The efficiency score for the other units also change as efficiency is relative to the efficiency frontier. 22

Model 3: One input and two outputs (Variable returns to scale) DMU 4 DMU 3 DMU 2 DMU 7 DMU 1 DMU 6 These courts (DMU 5, 6, 7) are not on the frontier for disposals but are considered 100% relative to peers. DMU 5 Using the same data points as before, we include an extra output measure (early guilty pleas) into the model. The same efficient units from the previous model are still efficient, but we now have a few more efficient units (DMU 5, 6, 7), the next chart shows why they are included. 23

Model 3: One input and two outputs (Variable returns to scale) DMU 7 DMU 6 DMU 5 DMU 2 DMU 3 DMU 1 DMU 4 performs relatively poor for EGP, but is considered 100% efficient relative to peers, when both outputs (disposals and EGP) are considered. The new efficient units (DMU 5, 6, 7) are efficient because they lie on the efficiency frontier for early guilty plea. This is an example of DEA using extreme points to consider both outputs simultaneously to give an overall result. 24

In summary… • These basic models have illustrated that careful consideration must be given to select the appropriate parameters, input and output measures to include in model development. • Although the modelling is still developing the initial reaction from our customers have been very positive, they have been engaged in our meetings and very interested in the information this methodology can provide. • The resulting modelling will hopefully help our customers gain greater insight into their business and provide an alternative perspective on court performance, neither of which have previously been possible with the existing MI tools and performance reports 25

Next steps… • Continue to develop and refine the Crown court DEA modelling • Explore and evaluate the results from the selection of inputs and outputs • Drill down to the court level to find out what is driving the efficiency scores • Explore using weight restrictions for some outputs and evaluate the results • Apply a similar methodology to the magistrates’ court. • Magistrates’ court has over 220 courts across 7 regions in England Wales • Possibility to model at regional level • Slightly different priorities and measures Thank you! 26