Quality Enhancement Research Initiative Integrating Research and Practice

Quality Enhancement Research Initiative Integrating Research and Practice to Bridge the Researcher/Practitioner Divide: Three Examples involving Diabetes Shared Medical Appointments – 3/24/09 Susan R. Kirsh, MD and David C. Aron, MD, MS Louis Stokes Cleveland DVAMC 14(W), 10701 East Blvd. , Cleveland, OH 441061 Case Western Reserve University School of Medicine, Cleveland, OH with Special Thanks to Wen-Chih Wu, MD, and Tracey H. Taveira, Pharm. D (Providence VAMC) and David Edelman, MD (Durham VAMC)

CIPRS Background and Mission QUERI and HSR&D contributions to improved performance and quality of care in the VA have been considerable over past 10 years, but significant challenges remain CIPRS was established to support and strengthen: – VA implementation research (Goal 1) – VA implementation policy and practice (Goal 2) – and to support QUERI program and leadership (Goal 3)

Disclaimer The opinions expressed in the following presentation are solely those of the presenters, and do not represent those of any agency or organization Diogenes: “Of what use is being a philosopher if you don’t annoy anyone?

The Researcher-Practitioner Divide Many suppliers and users of social research are dissatisfied, the former because they are not listened to, the latter because they do not hear much they want to listen to (Lindblom & Cohen, 1979). • • Research-practice gap Science-practice gap Knowledge-practice gap Academic-Management gap

How the research-practice gap is usually depicted: a knowledge transfer (translation) gap.

Original research 18% Negative results variable Dickersin, 1987 Submission 46% Negative results Kumar, 1992 0. 6 year Koren, 1989 0. 5 year Kumar, 1992 Acceptance Publication 35% Lack of numbers Balas, 1995 17: 14 0. 3 year Poyer, 1982 Expert opinion Bibliographic databases 50% Poynard, 1985 Inconsistent indexing 6. 0 - 13. 0 years Antman, 1992 Reviews, guidelines, textbook 9. 3 years Implementation It takes 17 years to turn 14 per cent of original research to the benefit of patient care

R The Pipeline Fallacy of Producing & Vetting Research to Get Evidence-Based Practice* The 17 -year odyssey Publication Peer Review Priorities & Of Grants Peer Review Research Synthesis Priorities for Research Funding Academic appointments, promotion, & tenure criteria Guidelines for Evidence-Based Practice Evidence-based Medicine movement Practice - Funding; patient needs, demands; local practice circumstances; professional discretion; credibility & fit of the evidence. Blame the practitioner or blame dissemination *Based on Green, L. W. From research to “best practices” in other settings and populations. Am J Health Behavior 25: 165 -178, April-May 2001. Full text: www. ajhb. org/25 -3. htm

This is Not a New Problem: The Case of Scurvy • 1593 - Sir Richard Hawkins recommended the following treatment for scurvy: "That which I have seen most fruitfull for this sicknesse, is sower [sour] oranges and lemmons. " • 1601 - Lancaster shows that lemon juice supplement eliminates scurvy among sailors (non-randomized controlled trial)

This is Not a New Problem: The Case of Scurvy • 1747 - Lind shows that citrus juice supplement eliminates scurvy • 1795 - (194 years after Level 2 evidence) British Navy implements citrus juice supplement

NIH Roadmap Initiative --translating discoveries into health* The roadmap less traveled? ** “The Roadmap identifies the most compelling opportunities in three arenas: new pathways to discovery, research teams of the future, and reengineering the clinical research enterprise” (Zerhouni, p. 63). * *Zerhouni E. Science 2003, Oct 3; 302(5642): 63 -72. **Green LW. Am J Prev Med. , 2007; 33(2): 137 -38, after K. Grumbach.

Lawrence W. Green, UCSF "Blue Highways" on the NIH Roadmap* Program Evaluation, CQI, Policy Analysis, TA in EBP adaptation. Systems research. *Westfall, J. M. et al. JAMA 2007; 297: 403 -406.

3 Conceptualizations of the Gap Between Research & Practice • Knowledge Transfer Gap: Practitioners need to receive the lessons of research and put them into practice. • Research and practice are entirely separate disciplines and each must develop their own answers to their own problems • Knowledge Production Gap - Research and practice have complementary perspectives and skills that need to be used together to address the real need, collaborative knowledge production. Van De Ven A, Johnson P. Knowledge for theory and practice. Academy of Management Review. 2006; 31(4).

Differing World Views Practice Research Notions of evidence Colloquial (Contextual) Anything that seems reasonable p<0. 5? Practice relevant Scientific’ (Context free) Proven empirically-rigorously p<0. 05? Theoretically driven Time frames for results Timely As long as it takes Languages for Clear Message communication Practitioner jargon Caveats and qualifications Research jargon Work environment Focus on strict adherence to research rules that give objective validity to results and publication of research findings Influenced by academic achievement, international research reputation, sources of funding • Focus on service delivery Influenced by the need to respond to the immediate reality of human need Based on Davies et al. (2000 b); Pyra (2003); Shonkoff (2000).

Differing World Views • Rigor vs. relevance • Internal vs. external validity • Isolation of a phenomenon from context (so that it can be more rigorously studied) when context matters • Our methods, theories, world view don’t match the problems or apparent solutions T. Greenhalgh Project is fixed Context must adapt Research Context is fixed Project must adapt Practice

Rigor vs Relevance Precise and pointless.

The two kinds of research? • Quantitative / positivist / deductive – Hard and spiky • Qualitative / phenomenological / social constructivist / interpretivist / etc – Soft and cuddly Often suggested that researchers need to choose one or the other. Michael Wood http: //userweb. port. ac. uk/~woodm/

Which side are you on? • To hard and spiky people, soft and cuddly research is lacking in rigour • To soft and cuddly people, hard and spiky research is superficial and lacking in richness and relevance … but is this a genuine dichotomy? Michael Wood http: //userweb. port. ac. uk/~woodm/rm/

Two Models of Rigor vs Relevance ‘As the complexity of a system increases our ability to make a precise and yet significant statement about its behaviour diminishes until a threshold is reached beyond which precision and significance (or relevance) become almost mutually exclusive characteristics. ’ (Zadeh, 1965)

Importance of rigor and relevance Rigour of method Pedantic Puerile Pragmatic Populist: insufficient rigour/incorrect, ill conceived/conducted studies, populist books, inadequate refereeing Puerile: low practical relevance, insufficient rigour/ incorrect, ill conceived/ conducted studies Pedantic: fastidious design and sophisticated analysis but relevance limited (myopic) Pragmatic: rigor in method and relevant to organisations Relevance to practice Modified from MNK Saunders, Oxford Brooks Univ. Developed from: Anderson et al. (2001)

Two separate disciplines? • views knowledge of theory and practice as distinct kinds of knowledge (not necessarily in opposition to each other; complementary). – Each reflects a different ontology (truth claim) – Each reflects a different epistemology (method) for addressing different questions. • Sophia and phronesis

Phronesis • Aristotle distinguishes between two intellectual virtues: sophia and phronesis. Sophia (usually translated "wisdom") is the ability to think well about the nature of the world, and is used in our attempts to discover why the world is the way it is (this is sometimes equated with science); sophia involves deliberation concerning universal truths • Phronesis is the ability to think about how and why we should act in order to change things, and especially to change our lives for the better • Phronesis is about practical wisdom/judgement routed in experience René Kemp

• Phronesis is concerned with particulars, because it is concerned with how to act in particular situations. One can learn the principles of action, but applying them in the real world, in situations which one could not have foreseen, requires experience of the world • The importance of phronesis to contemporary thought has been spelled out by Bent Flyvbjerg in his book Making Social Science Matter: Why Social Inquiry Fails and How it Can Succeed Again; Flyvbjerg argues that instead of trying to emulate the natural sciences, social science should be practiced as phronesis. René Kemp

Viewing the gap as… a knowledge production problem If the duty of the intellectual in society is to make a difference, the [academic] research community has a long way to go to realize its potential. The action steps to resolve the old dichotomy of theory and practice were often portrayed with the minimalist request for researchers to engage with practitioners through more accessible dissemination. But dissemination is too late if the wrong questions have been asked. A wider and deeper form of engagement between researchers and practitioners would entail experimentation with the co-funding, coproduction, and co-dissemination of knowledge. Andrew Pettigrew, “Management Research After Modernism, ” Brit. J Mgmt, 2001, vol. 12, iss. SPI/1, pp. S 61 -S 70

• We contend that: – researchers and practitioners must inform one another in order to optimize both research and care delivery. – links between research and practice can occur at various points in the process, allowing clinical managers and researchers to become better aligned to develop new and potentially generalizable knowledge having significant impact on the improvement of healthcare. – Understanding how best to facilitate linkages and collaboration, can expedite translation of research knowledge into practice and allow more evidence based improvements to be implemented into practice as well as influence research agendas.

• We describe three brief case studies in which attempts were made with varying success to bridge the researcherpractitioner gap. These three examples, all involving Shared Medical Appointments (SMAs) for diabetes will illustrate both fixed and evolving interventions and the degree to which research and clinical operations interfaced.

First, what is an SMA? • Personnel • staff-: 1 MD (general internist), 1 NP/CDE, 1 RN, 1 Pharm D, 1 Psychologist (multi-disciplinary team) • 10 -20 patients/session Sessions • 90 minutes length; Return visit interval: 4 -8 weeks until goals achieved • Group activities • Education; Review of labs; Discussion http: //www. queri. research. va. gov/to ols/diabetes/shared-med-appt. pdf • Individual activities • Medication management • Referrals • Individualized plan of care outline

Case 1. Based upon a research need related to the evidence both published and unpublished, an intervention was designed and implemented. Following implementation, there was minimal alteration of the intervention.

The linear logical model… Identify the problem Commission research Analyse the results Choose the best option Establish the policy Implement the policy Evaluation

Case 2. Intervention Dose Adjustments. Based upon a clinical need, an intervention was designed and implemented. Following implementation the intervention remained fixed, although the dose varied.

Complexity Dose Adjustments TA DA A S A P D S P A P S D *** Danger *** The D journey up the ramp of complexity is NOT linear. A P S D Time

Challenges S D A P P D P P S D AP D S A S Complexity D P Opportunities Time Tomolo, Lawrence, and Aron Legend: P=Plan D= Do = Barrier = Direct flow of impact S=Study A=Act = Lingering background impact Arrowhead = Feedback or feedforward Different Sizes of letters and cycles and bolding of letters = denotes differences in importance/impact

Case 3. Action research. Based on a clinical need and review of evidence both published and unpublished, an intervention was designed and implemented. The implementation process involved multiple plan do study act cycles. These cycles included redesigns of the intervention.

Continuum of Quality Improvement and Research: Rigor vs. Relevance Operations “Relevant” Context-Dependent Problem Solving Quantitative >, <, or = Qualitative Pre-test post-test or Quasi-experimental designs Tends to be NON-LINEAR Potential Synergy Research “Rigorous” Identify generalizable knowledge, i. e. , Eliminate Context Publishable Quantitative>Qualitative RCTs Rule Tends to be LINEAR • Continuum not a dichotomy • Goal is relevance moving as close to rigor as one can

• Initial design: pre-test post-test with 44 patients • Reviewer comments - 3 times – Lack of recognition that this was a quality improvement project • Put ‘QI’ in the title – Organization/Format issues • Inserted SQUIRE as signposts and follow a familiar format

Reviewer #2 comments re: first version • “The major problem is the lack of a control group. The participants were selected based on high levels of particular measures. ” • “The second problem is that the people are treated in clusters, and that therefore the independence assumption of the statistical tests is violated” • “An appropriate method of analysis should be used– such as multilevel models, generalised estimating equations or Huber-White sandwich estimators. ” – (Note that subsequent reviews from this reviewer

Response – Changed design to quasiexperimental: Added a control group – matched, concurrent, but not randomized. • Use as much rigor as possible: Consider sources of bias and address wherever possible eg. , regression to the mean – Required 3 revisions overall • Persistence pays off • It was a much better paper as a result of the reviewers’ comments. Leave your ego out of it.

• Qualitative work adds another dimension that makes quantitative data more meaningful • Used framework of Grol model to add structure and rigor • Triangulation

Models of Researcher-Practitioner Collaboration • Separate and not even talking (~unidisciplinary) • Separate, but at least talking (~multi-disciplinary) • Co-creation of knowledge (~interdisciplinary and maybe even transdisciplinary) – Action Research – Engaged Scholarship – Mode 2 Knowledge Generation On the Impossibility of Collaborative Research – and on the Usefulness of Researchers and Practitioners Irritating each Other. A. Kieser and L. Leizer Paper submitted to Special Issue on “Organization Studies as Applied Science: The Generation and Use of Academic Knowledge about Organizationa, ” Guest Eds. P. Jarzabowski, S. Mohrman, and A. G. Scherer.

Action Research • a form of systematic • inquiry intended to produce both change (action) and understanding (research) it involves ‘stakeholders’ in the research in both shaping and carrying out the research. *Green, George, Daniel, et al. , Participatory Research…Ottawa: R Society of Canada, 1997. www. lgreen. net/guidelines. html

Linking Research & Operations Knowledge Generation • • Mode 1 – Linear and often unidisciplinary – Developed separate from context – Application takes place late in the process of generating knowledge Mode 2 – Iterative, non-linear, and interdisciplinary – Developed in context – Application is incorporated early – Includes a range of stakeholders – Involves alliances among different knowledge communities Basic Research Applied Research Application Reference: Gibbons, Michael; Camille Limoges, Helga Nowotny, Simon Schwartzman, Peter Scott, & Martin Trow (1994). The new production of knowledge: the dynamics of science and research in contemporary societies. London: Sage. Development Technology

Engaged Scholarship (A. Van de Ven) • A collaborative form of inquiry where investigators leverage their different perspectives to learn about a complex problem in reality. “Intellectual Arbitrage” • A view of how scholars define their relationships with their communities – Other academics, practitioners, students • A relationship involving negotiation, mutual respect, and collaboration to produce a learning community. • Studying complex problems with (not for) practitioners and other stakeholders

Major Activities in an Engaged Scholarship Study Context: Complexity, research team organization, stakeholders’ perspectives Research Design Develop variance or process model to study theory Theory Building Create, elaborate & justify a theory by abduction, deduction & induction Model n D ry es eo ig Involve knowledge experts in relevant disciplines & functions Th Involve methods experts & people providing access & information R g es ea in ild rc h Bu g in A. Van de Ven lv Involve intended audience to interpret meanings & uses So Problem Solving Communicate, interpret & negotiate findings with intended audience. rm ul Fo ob le m m le ob Pr Iterate & Fit at io n Theory Pr Solution Reality Problem Formulation Situate, ground, diagnose & infer the problem up close and from afar Involve those who experience & know the problem

Potential Benefits of Participatory Research • Results are relevant to interests, circumstances, and needs of those who would apply them • Results are more immediately actionable in local situations for people and/or practitioners • Generalizable findings more credible to people, practitioners and policy makers elsewhere because they were generated in partnership with people like themselves • Helps to reframe issues from health behavior of individuals to encompass system and structural issues. Green LW, Mercer SL. Am J Public Health Dec.

Differences Between Inside & Outside Inquiry-A philosophical digression Roger Evered & Meryl Reis Louis, Alternative perspectives in the organizational sciences: “Inquiry from the inside” and “inquiry from the outside, ” Academy of Management Review, 6, 3 (1981), p. 389.

Is Action Research Scientific? • No, when judged by positivist science • It doesn’t meet the “covering law” criterion – Actions derive meaning from the end pursued, not prior associations – Planned interventions and social systems cause variables to be dependent on context – Actions are seldom discrete events Sussman

Is Action Research Scientific? • Yes, given different philosophical viewpoints – Praxis: the art of acting upon the conditions one faces in order to change them – Hermeneutics: the interpretation of languages, culture, and history (no knowledge is possible without presupposition) – Existentialism: asserts the importance of human choice and values, with respect to action, avoids causal explanations – Pragmatism: shifted the criterion of truth to the practical consequences for adopting a particular stmt. – Process philosophies: you cannot step into the same social system twice, organizations constantly change, they are different – Phenomenology: insists on the primacy of immediate subjective experience is the basis for knowledge • It generates knowledge which is contingent on a particular situation and which develops the capacity of members of an organization to solve their own problems Sussman

Engaged Scholarship is based on a Critical Realist Philosophy of Science • There is a real world out there, but our understanding of it is limited • All facts, observations & data are theory laden • Social science has no absolute, universal, error-free truths or laws • No form of inquiry can be value free & impartial; each is value full • Knowing a complex reality demands use of multiple perspectives • Robust knowledge is invariant (in common) across multiple models • Models that better fit the problems they are intended to solve are selected, producing an evolutionary growth of knowledge. Van de Ven

Minimizing Potential Pitfalls Avoiding false negatives and false positives: – Systematic search for disconfirming evidence – Distinguish between theory failure and implementation failure – Understanding context: implementation environment, participant characteristics and other interventions Some approaches: – Addressing through design – eg experimental and quasi-experimental designs – Addressing through data collection – eg Beneficiary Assessment – Addressing through iterative analysis and collection – eg Mixed methods, Contribution Analysis, Multiple Levels and Lines of Evidence (MLLE), List of Possible Causes (LOPC) and General Elimination Methodology (GEM) P. Rogers, Royal Melbourne Institute of Technology. www. prgaprogram. org/riw/files/papers/Rogers%20 material%20 for%20 workshop. ppt.

Two key checks • Use of a devil’s advocate or critical friend. Remember the problem of confirmation bias – you are likely to be more enthusiastic about evidence that confirms your pet ideas than about evidence that undermines it! Get someone to try and be critical and find difficulties with your research – then fix or (if unfixable) discuss the problems. • Triangulation – compare results from different sources. Applies to data, methods, observers, theories (Robson, 2002: 174). Michael Wood http: //userweb. port. ac. uk/~woodm/rm/rm. ppt

Bridging Research and Practice Improved Understanding Improved Impact Strategic EDUCATIONAL Supports User-Inspired Research-Inspired Reflection Strategic POLICY Supports Existing Understanding Current Practices Dale A. Blyth, Ph. D. , U. Minn.

The Bridge (not the Pipeline) from Research to Practice and Back • If we want more evidence-based practice, we need more practice-based evidence. • The importance of practitioners and policymakers in shaping the research questions. • Practitioners and their organizations represent the structural links (and barriers) to addressing the important determinants of health behavior at each level. Engage them, not at passive recipients, but as partners… *Green, L. W. From research to “best practices” in other settings and pop J Health Behavior 25: 165 -178, April-May 2001. Full text: www. ajhb. org/25

Strategies for building relationships between the three cultures (researchers, practitioners, and policy makers): • Ongoing, regular communication between researchers and users: (eg, conduct spontaneous and planned exchanges and face to face meetings, partnerships, best practice demonstrations, workshops and seminars) • Involve practitioners and decision makers early and throughout all stages of the research process • Generate knowledge in a shared manner • Allow sufficient time for people to understand fully and connect their needs to what is being proposed/discussed • Utilise opportunities for researchers to ‘shadow’ policymakers and vice versa • Include decision makers in the governing or consulting bodies of research centres • Sources: Black & Donald (2001); Davies et al. (2000 b); Fixsen et al. (2005); Innvaer et al. (2002); Lomas (2000); Nutbeam (2003); Percy. Smith et al. (2002); Pyra (2003). KERRY LEWIG, FIONA ARNEY AND DOROTHY SCOTT

Summary • The cases illustrate that there are opportunities for alignment and linkage between research and operations (practitioners and managers) despite their distinct cultures, differing world views, and unique terminology. – Each mode of knowledge generation has its own problems. (the rigor vs relevance issue) • There are various points along the research and operation paths of knowledge generation and translation where the goals and objectives of the researcher and manager can be aligned. – It is at these points of alignment where each can optimally inform the other.

Reference List 1. Greiner AC, Knebel E. Health professions education: A bridge to quality - Institute of Medicine. Washington: National Academies Press; 2003. 2. Lewig K, Arney F, Scott D. Closing the research-policy and research-practice gaps: Ideas for child and family services. Family Matters 2006; 74: 12 -19. 3. Chaffin M, Friedrich B. Evidence-based treatments in child abuse and neglect. Children and Youth Services Review 2004; 26: 1097 -1113. 4. Benbasat I, Zmud RW. Empirical research in information systems: the practice of relevance. MIS Quarterly 1999; 23(1). 5. Mason RO. MIS Experiments: a pragmatic perspective. In: Benbasat I, editor. The Information Systems Research Challenge: Experimental Research Methods. Cambridge, MA: Harvard Buiness School; 1989. 6. Davenport TH, Markus ML. Rigor vs. relevance revisited: response to Benbasat and Zmud. MIS Quarterly 1999; 23(1). 7. Robey D, Markus ML. Beyond rigor and relevance: producing consumable research about information systems. Information Resources Management Journal 1998; 11. 8. Keen PGW. Relevance and rigor in information systems research: improving quality, confidence, cohesion and impact. In: Nissen HE, Heinz KL, Hirschleim R, editors. Information Systems Research: Contemporary Approaches and Emergent Traditions. North Holland; 1991. 9. Gibbons M, Limoges C, Nowotny H, Schwartzman S, Scott P, Trow M. The New Production of Knowledge: The Dynamics of Science and Research in Contemporary Societies. London. : Sage; 1994. 10. Reason P, Bradbury H. Handbook of Action Research. London: Sage; 2006. 11. Van de Ven A. Engaged Scholarship: A Guide for Organizational and Social Rsearch. Oxford University Press, 2007 12. Jaber R, Braksmajer A, Trilling J. Group Visits: A Qualitative Review of Current Research. Journal of American Board of Family Medicine 2006; 19: 276 -290. 13. Kirsh SR, Watts S, Pascuzzi K et al. Shared medical appointments based on the chronic care model: a quality improvement project to address the challenges of patients with diabetes with high cardiovascular risk. Quality Safety in Health Care 2007; 16: 349 -353. 14. Kirsh S, Lawrence R, Aron D. Tailoring an Intervention to the Context and System Redesign Related to the Intervention: Case Study of Implementing Shared Medical Appointments for Diabetes. Implementation Science 2008; 3: 34.