A Beano Guide to extracting outcomes data from

A Beano Guide to extracting outcomes data from your information system Alastair Macdonald Clinical advisor South London & Maudsley NHS Foundation Trust

Extract from the Beano glossary: “Easy Peasy” adj. Hopelessly impossible and doomed to fail

Things to discuss a. What you want out b. When you want it out c. What you will do with it when (if) you get it out d. How do you set about getting a) and b) e. How you achieve c) Focus here is on aggregate data rather than for individual service users

Things to discuss a. What you want out Start here b. When you want it out c. What you will do with it when (if) you get it out d. How do you set about getting a) and b) e. How you achieve c) Focus here is on aggregate data rather than for individual service users

mainly What you/want to do with the data • Feed beasts – MHMDS – Commissioners – Managers (e. g. For appraisal or dealing with deficits) • Encourage reflective practice – Teams – Individual consultants

mainly What you/want to do with the data • Feed beasts – MHMDS – Commissioners – Managers (e. g. For appraisal or dealing with deficits) • Encourage reflective practice – Teams Rest of this talk – Individual clinicians

What you want out Outcomes data!

What you want out Outcomes data!. . . er?

Definitions of outcome • The attributable effect of an intervention (or its lack) on a previous health state. UK Department of Health (1994) • a change in the health of an individual, group of people or population which is attributable to an intervention or series of interventions. NSW Health Department (1992)

3 dimensions of clinical outcomes (Broadbent, 2001) All necessary: none sufficient Intervention Change in Health Status over a period of time Condition/ Context

3 dimensions of clinical outcomes (Broadbent, 2001) All necessary: none sufficient Intervention period of time Change in Health Status over a period of time Condition/ Context

3 dimensions of clinical outcomes (Broadbent, 2001) All necessary: none sufficient Intervention Condition Change in Health Status over a period of time Condition/ Context

3 dimensions of clinical outcomes (Broadbent, 2001) All necessary: none sufficient Intervention Context Change in Health Status over a period of time Condition/ Context

3 dimensions of clinical outcomes (Broadbent, 2001) All necessary: none sufficient Intervention Change in Health Status over a period of time Condition/ Context

Choosing a period of time of interest • Arbitrary e. g. Financial year • Period bounded by formally defined events such CPA reviews or cluster reviews • “Episode” of care with a particular team or unit • “Treatment episode” e. g. of psychological therapy • “Inpatient Episode” of continuous care of the same type e. g. Inpatient admission involving several wards • “Combisode”- episode of care by specifically linked teams (e. g. Inpatient ward and HTT) • “Spell” of continuous contact with any Trust team, unit or service

Choosing a period of time of interest • Arbitrary e. g. Financial year • Period bounded by formally defined events such CPA reviews or cluster reviews • “Episode” of care with a particular team or unit • “Treatment episode” e. g. Of psychological therapy • “Inpatient Episode” of continuous care of the same type e. g. Inpatient admission involving several wards • “Combisode”- episode of care by specifically linked teams (e. g. Inpatient ward and HTT) • “Spell” of continuous contact with any Trust team, unit or service Rest of this talk

Period of interest=“Scenario”

Scenario data items • Overall period start date • Overall period end date (blank if still active) • Description of secnario e. g. “Team/ward episode”, “Inpatient episode” • Identifiers of contributing teams • (if only two teams (“Combisode”), include the start date and end date for each team so that the proportional contribution of each team to outcome can be calculated)

“Combisodes” can be used to deal with service redisorganisation. . . • Community team 1 is closed and replaced by two new teams (a and b) • Some continuing cases from 1 go to a, some to b. • Create new episodes where overall start date is with 1 and end date is with a, and new episodes where overall start date is with 1 and end date is with b

Condition • i. e. What’s wrong with the patient • Problems with diagnosis – Two different meanings • shorthand for a hypothesis that attempts to explain the difficulties that the individual patient is experiencing. This hypothesis includes the likely course of the condition, and whether or not it might respond to interventions. Any clinician carrying out an intervention must have a hypothesis so must be capable of this sort of diagnosis! • A rule-based administrative label applied to groups of patients with similar requirements from the organisation. Not all clinicians are comfortable with these rules and labels.

• So a patient with a long-term disorder such as schizophrenia (perhaps the most useful label for administration) may, if presenting with new affective symptoms, be given a diagnosis of a depressive disorder (which will be the most useful for the patient as it will determine the focus of treatment).

Which diagnosis? • For outcomes we use the latest diagnosis within the period of interest (diagnosis type 1) AND • The latest-ever diagnosis (found to be closest to validated diagnosis type 2)

Context • • Age Gender Ethnicity Deprivation index of home address Employment 1 Accommodation 1 Marital status 1 1 Not accurate in our data

Change in health status (Outcome) % of spell

Change in health status (Outcome) • Measurement at start and end (CROM or PROM) of period of interest

Change in health status (Outcome) • Measurement at start and end (CROM or PROM) of period of interest What if the measurements are not timely? RULES

Our current rules for attaching outcomes measurements to simple ward/team episodes • Ratings for simple team or ward episodes must come from the same team or ward. • The first rating is the first available between start of episode and 14 days after the end of the episode. If two such ratings are carried out on the same day then that with the highest cn_doc_id (i. e. the latest entered) is chosen. • The last rating is the last between 14 days after the end of episode and the first rating. – If two ratings straddle the end of the episode then the closest to the end is chosen, irrespective of whether it is before or after the end. If both are equidistant from the end of the episode then the one inside the episode is chosen. • First and last ratings cannot be on the same day, even if rated by different people. If there are only two ratings in an episode and they are on the same day the one with the highest cn_doc_id (i. e. the latest entered) is chosen as the first rating and the last rating is null.

Intervention • We have little reliable structured intervention data • Even frequency of face-to-face contact is poorly recorded • The only way of telling that contacts or interventions have occurred is study of text items (progress notes, letters and discharge summaries) • This is being automated using natural language processing (the other NLP).

Our current position • Since 2011 we have had a SQL extract of all simple ward/team episodes from teams that use Ho. NOS or Ho. NOS 65+ • This does not allow analysis of inpatient episodes, trust spells or “combisodes”, nor of teams using any other measure • For 2015/16 commissioners set a CQUIN which included reporting on % episodes with first and last Ho. NOS-LD • This inspired (ha!) the Trust to fund a new SQL extract (as described here) that would be as future-proof as possible • Work is in progress.

Simple team/ward row? First and last ratings from same team Y Get context data for team and ward episodes N Trust Spell row? First and last ratings from any team Y Add Scenario rows Inpatient episodes Combisodes Trust spells Add context data from team and ward episodes to scenario rows Full context table with data N Inpatient episode row? First and last ratings from any team Y N Nominated combisode row? Y First and last rating from either team The new extracts for different purposes Derive team and ward episode table from referrals and discharges Allocation of any outcomes ratings

Each month the full context Table will be created in SQL Each month certain key outcomes ratings (first and last) will be allocated by SQL procedures to designated rows in the full context table, creating the full outcomes table Each month SQL queries will be run on the full outcomes table and results loaded into SPSS tables- one for each main outcome measure Each month SPSS syntax is run on these tables to generate variables for analysis (e. g. Categorical change) Whenever required SPSS syntax is run on these tables to generate reports and graphics for presentation to specific clinical teams or other audience

What you need (just to get data out) • A SQL analyst dedicated full-time for the duration* • SQL analyst available to maintain system once it is built and tested • An experienced IT person who is able to project manage • Access to reliable IT and database management support* • An experienced clinician interested in outcomes to make up rules (and, if there is time, to consult on these) and carry out analysis of results* • Accommodation* • Coffee (lots)** • SPSS or other data analysis program* * We have these- the other items are sorely missed at present

How long and what does it cost? • We are doing this for £ 40 k in 4 months • To do it properly I now think £ 100 k and 8 months. Ho Hum