Methodological and statistical consulting to policy makers in

Methodological and statistical consulting to policy makers in health care finance Jules Ellis Radboud University Nijmegen

My style l Listen l l Determine who advise and who decides l l l People often start in abstract statistical terms (E. g. is X correlated to Y -> gender is related to aggression) Change the question l l l Compare advice to students Ask the substantive context l l E. g. Stork manager To something that can be done (toolbox) To something that you think will be interesting for the client (E. g. is group program correlated with well-being -> does a group program influence well-being) Offer alternatives l E. g. PCA versus ADF

l Be pragmatic l l l Time, money Prior concepts (e. g. clusters vs. factors groups vs scores) Prejudice l l There must be 7 clusters Dichotomization is good / bad Nobody/ everybody is doing this Client must understand it within available time l l Explain at right level (common sense) Write l l l After consult (compare physician) After analysis Educate

Organizational hierarchy Department (VWS) Ass. Nursing Homes (Arcares) Ass. Health Insurance Companies (ZN) Accountancy (PWC) VLP CCC I I Prismant

Tasks in the hierarchy Cost Model Pay for care Care for clients Costs; Efficiency; Best Practice Organizations Client Need Caring Time Quality of Care

Old cost model l Depends on location of client l Nursing homes: € x per client / day l Assisted Living Facilities (Caring homes): € y per client / day l Home Health Care (Extramural care) € z per client / day l Adult Day Care Services

Limitations of old cost model Ignores client differences in need l Ignores overlap in client populations l Emphasis on somatic care l Supply driven, not demand driven l

Time line Pilot: 10 homes (2000 clients) l Test: 100 homes, automating l Benchmark 1: 1000 homes l Benchmark 2: 1000 homes l

Assessment of client needs Two central caring employees observe client during two weeks l Both fill in one questionnaire of 24 items about client l Two sum scores are computed: l l Sumlic: Somatic need for care l Sumpsy: Psycho-social problems

Items for client needs 1 Is in dressing / undressing 2 Is in movements 3 Is in eating or drinking 4 Is in the bathroom … 10 Is incontinent in urine or faeces 11 Is irritated 12 Has difficulties to remember 13 Shows restless behavior 14 Is lonely … 24 Is aware about what happens around him/ her Not / A little / Partially / Very much / Totally dependent Never / Rarely / Sometimes / Often / Always

Assessment of caring time l l l Pilot: Observers make one round every 20 minutes (-> chaos) Later: Employees receive a handheld computer Beeps every 20 minutes, at random moment, whole week 16 hour / day Employee records own behavior in one of 32 actions, + client Total caring time in various categories per client per day is estimated from this

Items for caring time l Direct Client Related l Individual Client Related l l l l General daily life support (eat, wash, …) Assistence in preparing food and/ or drink Individual treatment Communication with family Housekeeping for individual Preserved actions l l Collective medication Collective treatment House keeping for group Individual Client Related l l l Coordination for individual Organization Related l l l Coordination for individual Collective Client Related l l Indirect Client Related Waiting Break Education Travelling Employee related l l Missing Personal time

Time categories Categories: l Client related l Direct Individual l Collective l l Indirect Individual l Collective l l l Organization related Employee related Functions: l Housekeeping services l Personal care l Nursing l Supportive assistance l Activating assistance l Treatment

Other data l Clients l 24 services l On which days present l Membership of collective time groups l Employees l 35 professions l contracts l On which days present

Analysis scheme Best Practice Efficiency Output Client needs Quality Costs Caring time

Question 1: Cluster the clients on basis of needs 7 or 8 clusters l Homogenous in time and cost l Include psychological needs l Applicable in Nursing Homes and Assisted Living Facilities. l “Recognizable” l

Answer to question 1 l l l Select items on inter observer agreement, and variance > 1 Factor analysis on the items -> 2 factors Define 2 subscales: somatic and psycho-social. Reliability analysis (internal consistency) Divide the scores of each subscale in 4 quartiles (Light, Medium, Serious, Very serious) Result: 4 x 4 = 16 groups

Psychosocial problems Very serious (32 -56) Light (0 -6) Medium (7 -18) Serious (19 -31) I II IV Medium (4 -13) V VI VIII Serious (1431) IX X XI XII Very serious (32 -40) XIII XIV XV XVI Somatic care Light (0 -3) need

Reasons l l l l Cluster analysis -> only somatic factor We have already 7 groups for nursing homes alone, which were formed as 3 x 3 groups and combining some of these. The partition on the somatic axis should contain at least these 3 groups (to satisfy the nursing homes) + another one to accomodate the Assisted Living Facilities. Similarly for the psychosocial axis. So we need 4 x 4 groups to begin with -> 16 types Without time & cost data we cannot know how to combine these groups and respect homogeity. But perhaps we can combine some groups later.

First evaluation Initially accepted with reluctance l As of 2006: There are still 16 client types l It took years to accept that l

Question 2: Compute reference times Compute mean individual client time (ICT) for each client type l The mean can be used in the cost model, e. g. to compute the total ICT that the home should deliver. l How reliable is this? l

First answer to Question 2 l l l Using client types is an unnecessary loss of information Conduct a multiple regression analysis, with dependent variable = ICT, independent variables = somatic need, psychosocial need Compute the predicted ICT score for each client Take the mean of the predicted scores for each home. This was not accepted: Too difficult (3 parameters? !); there had to be groups.

Second answer to Question 2 Use answer 1 but change the presentation a little l Compute the predicted ICT score for each client l Compute the mean of the predicted scores for each client type in each home l Compute the mean in each home l

example All homes This home (y’ = 2 x) Type Somatic ICT ICT’ I 5 8 10 6 8 12 II 10 22 20 9 22 18 III 15 30 30 16 30 32 IV 20 42 40 22 42 44 V 25 48 50 27 48 54 Mean 15 30 30 16 30 32

First evaluation l l l Sceptical first, embraced the idea later Reliability of estimates More consistent output l l l Employees in nursing homes typically argue that their client type X is a little more serious than in other homes Reply: Indeed, and therefore we didn’t use the mean time of other homes but corrected it. Negative beta-weight of psychosocial problems Assistance needed in further research Assistance needed automation

Question 3: How reliable are the time measurements? = 153 min 100111 x = 6, n = 10, p = 0. 6 t = 6 * 20 min

Existing answer l l l Why this formula? (the method was bought) Sometimes extreme large n needed. Why? How many weeks observation are needed?

Answer to question 3 l l l Make a distinction between absolute unreliability (AU) and relative unreliablity (RU) Require only that AU is small. For example, if t = 1 0. 5 minute, then RU = 50% but AU = 30 sec (who cares)? The average time can be reliable even though the composing times are unreliable. Distinguish also other forms of unreliability (sampling of clients, homes, weeks)

(Details) Note that t = p. T, where T is the total observation period (fixed). l AU(p) = 0. 5 length of confidence interval for p l RU(p) = AU/p l AU(t) = AU(p)*T l RU(t) = RU(t) l

First evaluation Relevant people understood and accepted the idea immediately (why? ) l One week of observation would be enough if every week is the same l Helped to convince nursing homes l

The aftermath Elections l Sector wide model (including handicapped, psychatric patients, etc. ) l Association of nursing homes refused to make the data available to others l PWC -> KPMG, VLP -> ? , CCC -> l 16 Client types -> 9 Care Heaviness Profiles l Reference times -> Indication times l

Conclusions Prior concepts and prejudices are persistent l Large deviations from this will be rejected l Small deviations are accepted if they are explained at common sense level l Write it down with relevant examples l “A good method is a method that makes my life easier”? l

Methodological and statistical consulting to policy makers in health care finance Comments by Marieke Timmerman, University of Groningen, The Netherlands