Скачать презентацию Qualitative Methods For Research Dr Susan Gasson College
124f54ed0956194e8b5a18db54a6b12e.ppt
- Количество слайдов: 47
Qualitative Methods For Research Dr Susan Gasson College of Information Science & Technology Drexel University Email: sgasson@cis. drexel. edu
Agenda What is qualitative research? p Issues of rigor and differences from quantitative research p Methods for qualitative analysis p n n Data collection methods Analysis methods A Study of Knowledge Management in a Boundary-Spanning, Global IS Devt. Group p Rigor and validity issues p Exercise: coding qualitative data p Useful resources and references p
What is qualitative analysis? p p p Non-quantifiable (or non-quantified) data are analyzed using a variety of methods, to understand patterns in the data. Whereas quantitative data are analyzed statistically, qualitative data are organized, categorized (coded) and then analyzed through inferential reasoning processes. Organization of qualitative data involves identification of relevant data samples, e. g. n n n sections from tape-recorded interviews time-stamped episodes from a video-recorded activity field notes from observed behavior in the situation being studied).
Example: Coding Observations Categorize a description of the voting process in a specific country. p Focus is on p (i) how the vote-counting process works, (ii) the reliability of the process (iii) the role of technology. p Code each new idea in the printout (may be a sentence or may be a paragraph) with n n Category code (may have >1) Attribute(s) of the category
Examples: Coding Voting Description Observation Category Code Attribute Code The officials check that no one person's ? ? ? vote is used more than once, and tally up the total number of ballot papers issued in order to help verify that all the ballots make it safely to the count ? ? ? Note that the count can be observed in ? ? ? the count room by the candidates and their agents; no press or news organization is allowed access, though they can typically watch from a balcony ? ? ? p Focus on: (i) How the vote-counting process works, (ii) The reliability of the process (iii) The role of technology (can you make any observations from this data? ).
Example: Coding Voting Description Observation Category Code Attribute Code The officials check that no one person's Vote-counting vote is used more than once, and tally process up the total number of ballot papers issued in order to help verify that all Reliability the ballots make it safely to the count Manual Note that the count can be observed in Vote-counting the count room by the candidates and process their agents; no press or news organization is allowed access, though Reliability they can typically watch from a balcony Visible Secure Trustworthy
Coding Scheme Process (of vote-counting) n n n Manual vs. electronic Hidden vs. visible Auditable vs. no-paper-trail Reliability (of the process) n n n Secure vs. insecure Trustworthy vs. untrustworthy Objective vs. partisan Technology (role of) n n n Registering vote Counting votes Tallying totals
Philosophical Questions 1. What are you measuring, in a scientific experiment? n Does it exist independently of your perception? n Is it universal? n Is it true? 2. What are you measuring, in an interview or observation study of people performing daily work? n Does it exist independently of your perception? n Is it universal? n Is it true? 3. If you have 5 different researchers performing the same study, will they reach the same conclusions?
Research Paradigms in IS & Info. Science 1. Positivist Research n n Positivists generally assume that reality is objectively given and can be described by measurable properties which are independent of the observer (researcher) and his or her instruments. Positivist studies generally attempt to test theory, to increase the predictive understanding of phenomena (hypothesis testing). 2. Interpretive/Constructivist Research n n Interpretive researchers start out with the assumption that “reality” is socially constructed. Phenomena can be understood only through the meanings that people assign to them, accessed via social constructions such as language, consciousness, & shared meanings. Interpretive research does not predefine dependent and independent variables, but focuses on the full complexity of human sense making in context as the situation emerges. 3. Critical Research n n Critical researchers assume that social reality is historically constituted and that people’s ability to change their social and economic circumstances is constrained by various forms of social, cultural and political domination. Critical research focuses on the oppositions, conflicts and contradictions in organizations and society. It is emancipatory in intent: it seeks to eliminate causes of alienation and domination.
The Research Life-Cycle In Theory Generation Tests/extends theory Generates/explores theory
Positivist vs. Interpretivist Beliefs Positivist / Functionalist Real-world phenomena & Ontological (beliefs about the relationships exist independently of the individual’s perceptions nature of reality) Interpretive / Constructivist Phenomena & relationships are viewed as social constructs by which an individual makes sense of the external world/reality Epistemological Natural laws govern all aspects Rules governing behavior in various of existence. These laws may be situations are dependent on context. (beliefs about observed from outside the Inferred relationships between knowledge & how situation and abstracted to contextual factors and observed we know reality) provide generally-applicable behaviors may be transferred to models and theories. similar situations. The behavior of individuals en Human beings have complete Human Nature (how we account masse (with exceptions that can autonomy: their actions are be explained by a lack of dictated by free will (which may be for human rationality or variance from the constrained by external forces). So behavior) mean) can be viewed as they do not according to any determined by the situation. laws of rational behavior. Methodological Researchers derive generalizable Researchers infer transferable, indepth subjective accounts of (beliefs about how models or theories of behavior through the analysis of smallsituations, that analyze we apply inquiry scope findings from large observations from small samples in methods) samples and systematic methods great detail. The presence of the to construct scientific theories observer is accounted for. regarding the “real world”.
Constructivism: The Hermeneutic Circle Hermeneutics is (literally) the interpretation of a text: n its intent n its content, and n its context. The whole (the big picture) The parts (analysis of minutiae or components) Methodologically, the assemblage of an understanding of the “whole” through an analysis of its parts, e. g. WHOLE PART General/typical case Instance of complicated case Learning process Instances of learning Decision process Instances of decision making Gadamer, H-G (1989), "Text and Interpretation, " in Dialogue and Deconstruction: The Gadamer-Derrida Encounter, edited/translated by D. P. Michelfelder and R. E. Palmer, SUNY Press, Albany, NY, pp 21 -51.
Use Of Multiple Methods p p p Most often (but not always), the term “qualitative research” refers to qualitative content analysis, performed interpretively. Tenet of interpretivism is that researcher “interprets” data. So can use multiple qualitative methods for both data collection and data analysis, e. g. n n Data collection: observation, formal interviews, interactive (facilitated analysis) interviews and workshops, document analysis, investigative surveys, etc. Data analysis: qualitative coding (using different sets of constructs, to examine different aspects of the data), inferential analysis (usually simple frequency coconcurrence), statistical analysis, discourse analysis, etc.
Use Of Mixed Methods p p The use of mixed methods indicates the comparison of findings across multiple data collection techniques and analysis methods. This approach n n p Provides multiple perspectives of the research problem Guards against limiting the scope of the inquiry Yields a stronger substantiation of the derived constructs (Cavaye, 1995; Eisenhardt, 1989; Orlikowski, 1994; Wolfe, 1994). Mixed methods may (but does not have to) combine qualitative and quantitative analysis.
Qualitative Data Collection Vs. Qualitative Analysis DATA ANALYSIS Qualitative Quantitative Interpretive content analysis Search for and presentation of studies. meaning in quantitative results. Hermeneutics, Explanations of findings n Interpretation of statistical results Phenomenology, n Graphical displays of data Grounded Theory. n Naming factors/clusters in factor analysis & cluster analysis Quantitative Post-positivist Content Analysis Turning words into numbers: n Word Counts, Free Lists, Pile Sorts, etc. n Statistical analysis of text frequencies; code cooccurrence Positivist Research: Statistical & mathematical analysis of numeric data (e. g. regression). Multivariate analysis. Source: Bernard, H. R. (1996) ‘Qualitative Data, Quantitative Analysis’, CAM, The Cultural Anthropology Methods Journal, Vol. 8 no. 1, available at http: //www. analytictech. com/borgatti/qualqua. htm
Contributions of Qualitative Research The contribution of qualitative research studies in IS can be: p The development of concepts n p The generation of theory n p e. g. “automate vs. informate" (Zuboff, 1988) e. g. Orlikowski & Robey (1991): organizational consequences of IT. The drawing of specific implications n e. g. Walsham & Waema (1994): the relationship between design and development and business strategy. p The contribution of rich insight n e. g. Suchman (1987): contrast of situated action with planned activity and its consequences for the design of organizational IT. Walsham, G. (1995) ‘Interpretive Case Studies In IS Research: Nature and Method’, European Journal of Information Systems, No. 4, pp 74 -81
Distributed Knowledge Coordination Across Virtual Organization Boundaries Dr Susan Gasson Edwin M. Elrod Drexel University
Knowledge Management For Virtual Collaboration Organizational KM view Knowledge-as-process p p Knowledge processes p are embedded within n Best practices (tacit knowledge), n Contexts (localized knowledge) and n Genres of communication (legitimate knowledge). Effective knowledge management depends on p sharing understanding that is only meaningful in the context and community of practice within which it is applied. KM Systems View Knowledge-as-thing Knowledge can be defined independently of human action. n Knowledge can be divorced from practice n Knowledge can be abstracted into rules or algorithms, independent of context n Knowledge can be defined objectively. Effective KM depends on knowledge capture, codification & transfer across many different places and many different Co. Ps. How do we resolve this tension?
Research Question How are different forms of knowledge managed and coordinated across the boundaries of a virtual, global organization?
e. Commerce Group Functional Boundaries Executive Management Vendor Projects Europe Technical Operations Client Facing Applications Backend Applications Financial & Client Performance Evaluation
Corporate and Geographic Boundaries e. Serv. Corp e. Commerce e. Serv. Corp EU Operations Vendor. Corp e. Serv. Corp EU Customer Service e. Serv. Corp Asia Pacific e. Serv. Corp N. American Operations e. Serv. Corporate Parent. Corp
Field Observations p p p Researchers observe & transcribe telephone conferences and other (face-to-face) meetings; Supplemented with monthly ad hoc interviews with management team. Sample statistics through June 2006 n 338 conference calls/group meetings; p p p n n p Average length: 0 : 30 Shortest: 0: 04 Longest: 1: 35 8 group interviews. Over 1000 pages of transcription Longitudinal, ethnographic, exploratory
Thematic Analysis Of Meetings (Initial) p p p p Thematic analysis: What are the most common themes? n Categories of behavior or phenomena, meaningful in context of the study. Are there notable exceptions? n E. g. individuals who do not discuss specific themes or who say very different things about particular topics? What concept-categories or event-categories can be identified ? n What is the range of views expressed with regard to a topic? Can you identify any sub-categories? n Variations on your themes, further distinctions/qualifications? What language is used? n Are there common synonyms or metaphors that indicate a specific meaning or category of behavior? What respondent characteristics are associated with particular views? n Do people with different expertise express different views? What patterns emerge, across various samples, or over time?
Knowledge Sharing (Johnson, et al, 2002) (Polanyi, 1958) (Zack, 2001) Knowledge Sharing Form Boundary Object Mechanism Repositories Standardized Forms, Methods, Procedures Models Know. What Know. Why Know. How Whoknowswhat Observed knowledge translation and transformational activities. Maps (Star, 1989) (Carlile, 2002)
Standardized Procedures Know-How Make work practices explicit through discussion and debate. Knowledge Sharing Form Boundary Object Mech. Know. What Know. Why Know. How Whoknowswhat Repositories Std Forms, . . . Models Maps Ms Corp. Sys: Some system reports have problems. Mr Vendor. Tech: This was fixed in acceptance, but it didn't move with the release. Mr EVP: How many times does this happen? About 50%. Why are we paying
Maps Know-Why Establish boundaries of e. Commerce group. Knowledge Sharing Form Boundary Object Mech. Know. What Know. Why Know. How Whoknowswhat Repositories Std Forms, . . . Models Maps Mr Client. Sys: It turns out that a vendor that the EU office has – is one that everyone else uses. Mr EVP: Yes and develops stuff for everyone else and shares the information. It depends whether we consider that a system for … constitutes a competitive advantage, Ms Europe: I think that outcome analysis and project sourcing has to become a strategic area. ●●●
Maps Who-Knows-What Identify relevant stakeholders in other groups. Knowledge Sharing Form Boundary Object Mech. Know. What Know. Why Know. How Repositories Std Forms, . . . Models Maps Ms Europe: Mr Support and June visited the French vendor, so I have asked them to do a write-up for us, so that we understand Formal to take some what the issues are etc. and if there is an opportunity knowledge sharing of the stuff like the product site, like the project bank for Europe, since it’s already built. But we need to look at the how we host it, where we do it – so I have asked them to write it up for us. Informal, distributed, social context Mr EVP: OK, let them write it up. Then let’s talk about it – you, me and Mr Client. Sys. …The reason I want to discuss this other stuff you, me and Mr Client. Sys - is that I want to make sure that whatever they put together, you have vetted. With a broader understanding of the global perspective than they might have. . Whoknowswhat
Concept Map Early Themes From Analysis of Meetings Project Collaboration & Knowledge Distribution Problem Too complex for one person to understand Problem emerges thro’ negotiation Informal, distributed social context of project Organization Diverse set of global groups collaborate according to focus Formal knowledge often local and undocumented Project definition is ad hoc (memorydependent) Project goals are subjective: various groups & individuals define project in different ways Project Knowledge Who-knows-what more important than who-can-do-what Project roles & responsibilities change frequently Knowledge located in people’s heads Group memory of project changes Definition of project changes frequently – little coordination or persistence of knowledge (group memory)
Analytical Framework: Categorize Collaborations By Modes of Organizational Problem-Solving Well-Structured Problems p p p Clear problem-structure defines change requirements Unambiguous goals for change Knowledge accessed via pattern recognition (problem-solvers in similar domains develop repertoire of solutions). Ill-Structured Problems p p p Uncertain problem-structure indicates multiple alternative solutions Need to bound and structure problem to analyze requirements (complexity reduction) Explore unfamiliar knowledge-domains through consultation with experts to resolve ambiguity re change-goals and scope. Wicked Problems p p Problem emerges: has no objective definition, boundary, or structure Stakeholders see partial subsets multiple goals for change Problem, solutions, scope of inquiry, and relevant expertise are negotiated (equivocality reduction). Explore emergent knowledge-domains thro’ iterative cycles of inquiry.
Three Spans of Collaboration (i) Local coordination of projects p Core e-Commerce group manage project: define goals, scope, timescales, deliverables, and rationale p Boundaries: functional, role, geographic. (ii) Conjoint agency p Core e-Commerce group control project: act as hub, incorporating knowledge/expertise from external groups p e-Commerce define goals, scope, and responsibilities p Collaboration with hardware or software vendors, other e. Serv. Corp business units, client project groups (iii) Distributed Collaboration p e-Commerce group part of a web of collaborating groups p Goals, scope, system definitions, business-process changes negotiated, implemented, and evaluated jointly p e-Commerce group subject to joint or external projectleadership by groups from e. Serv. Corp, Parent. Co. , associated companies, or vendors.
Knowledge coordination strategy depends on problem coordinationdistance Problem-Solving Mode Well-structured problems Ill-structured problems Wicked problems Problem-Coordination Distance Collaboration Span Local Coordination Conjoint Agency Distributed Collaboration
Relative Incidence of Problems
Modes of Organizational Problem-Solving Well-Structured Problems Local Situation interpretation: Coordination stories & analogies create shared resource to identify similar problems Conjoint Agency Scope interpretation: stories & analogies communicate rules, evaluation-criteria, responsibilities at boundary Coordinating division of Distributed Coordination labor: functional domainexpert roles and social network leveraged for knowledge exchange Ill-Structured Problems Wicked Problems Group identity construction: plans, processes & checklists formalize procedural memory Framing collective strategy: group agrees evolving goals of change, to clarify approach to problem Delegated knowledgeleadership: domain expert roles assumed. Rules & procedures at coordinate knowledge transfer at boundary Defining a collective response: delegated boundary-spanner locates knowledge & controls evolving boundary procedures Managing external networks of influence: group domain-experts jointly formulate problem, negotiate group responsibilities Collective knowledge networking: leader negotiates group role; group members become expert in evolving set of knowledge-domains
Conclusions and Contributions p Knowledge is coordinated by means of a web of: n n n Functional and domain-expert roles Distributed knowledge resources Imposed or negotiated procedures. p Knowledge coordination strategy depends on problem coordination-distance. This concept combines organizational span of coordination with problem-type. p Central role of a cohesive group identity: n n n Informs semi-autonomous decision making by group members Provides conceptual patterns for action at group boundaries Adapted collaboratively through distributed, improvisational sense making to deal with novel situations.
Two Dimensions of KM Coordination
KMS Implications p Knowledge Management Systems must expand beyond communicating management decisions to embrace distributed, emergent, collaborative decision formation: n n n p p Well-structured problems require rule-based KMS. Ill-structured problems require adaptive KMS. Wicked problems require evolutionary & dynamic KMS, supplemented by human contact. KMS must be supplemented with face-to-face mechanisms that permit social networks to be formed and maintained. KMS must be supplemented with face-to-face mechanisms that permit domain expertise to be acquired and translated across domains.
Analyzing Qualitative Data Principles and Practice(!)
Qualitative data coding p p Data are be transcribed into a textual form (recommended) and/or analyzed in its raw form (e. g. video/audio, with items of interest identified by time-stamp). Data analysis (coding) can take two forms: n Data are classified according to a conceptual schema or a theoretical model, which leads to explanations dependent upon, or the further development of the conceptual model n Data are classified according to patterns that emerge from interpretation of the data. As themes and patterns emerge from the data, these are tested against further data samples to derive a substantive (grounded) theory.
A Question Q: If two researchers are presented with the same data, will they derive the same results if they use the same methods, applied rigorously? Let’s find out! p Organize in groups of three(-ish) people. p Discuss themes arising from coded data (10 minutes) p Present findings: 5 minutes per group
How to “Code” Data p RQ: What are differences in the ways that various types of IS professional or manager define the core problems & skills of IS design & development? p Read the transcript or data record through. n n n p Categorize (“code”) your observations n n n p Ask yourself “what is it that is going on here? ” Make notes about “themes” that you see in the data; Don’t attempt to be systematic/comprehensive at this point Relate category-codes to research question Define attributes of categories (attribute codes) Define categories and sub-categories (coding “families”) Ask “so what? ” n n Relate categories and their attributes to contextual factors and/or type of subject Draw conclusions about what the data tells you, in answer to the research question.
Issues With Qualitative Research How much data is enough? p How do you know that what you found is not what you were looking for? p Is it difficult to publish qualitative research studies? p Is qualitative research considered less acceptable than quantitative research? p Is this something that a Ph. D student should consider? p
Intercoder Reliability/Agreement p p p Intercoder reliability is a measure of agreement among coders in their coding of data High reliability scores indicate that n Categories are well-defined (agreed) and can be replicated by others applying the same schema, OR n Multiple coders are applying a pre-defined set of categories consistently, when coding data samples. Assess by comparing (co-coding) several data samples (e. g. 10) n Or analyze data from a pilot study to see what codes emerge across researchers before main study starts Measures of intercoder agreement): n Coefficient of reliability (Holsti, 1969, p. 140) n Scott’s pi (Holsti, 1969, p. 140) n Cohen’s kappa (Krippendorff, 1980, p. 138) n Agreement coefficient (Krippendorff, 1980, p. 138) n Composite reliability (Holsti, 1969, p. 137) Good website: http: //astro. temple. edu/~lombard/reliability/
Summary: Issues in Qualitative Research p p Qualitative research methods are used differently by researchers working within various philosophical approaches and various qualitative traditions. Data collection methods include action research, case studies, ethnography. Data analysis methods include statistical sampling of coded data and the inductive generation of relationships between variables. In the interpretive approach: n n n Rigor is achieved through comparison of findings across data samples and reflexivity. Validity is communicated through trustworthiness and subject validation of interpretations, rather than statistical significance. Can protect yourself against allegations of subjective interpretation (lack of rigor), by testing for co-coder reliability.
The “Qualitative – Quantitative Debate” Qualitative p p p p Constructivist/Interpretivist p Find answers to questions p Social science view p Explanatory p Goal: understand the p subject’s perspective, in context p Investigation oriented p Emergent themes and p issues Researcher is part of situation being studied BUT Quantitative Realist/Positivist Test hypotheses Natural science view Confirmatory Goal: find probabilities and correlations Verification oriented Controlled variables Researcher distanced from situation being studied § Differences are not as simple as this – it is possible to perform qualitative research in a positivist way, or quantitative analysis of interpreted findings. § Positivist research is also subjective – but the subjectivity occurs earlier in the research “life-cycle”, in selection of theory to be tested and research instrument(s).
References (Books and Articles on How-To “Do” Qualitative Research) Denzin, N. K. , and Lincoln, Y. S. [Eds. ] (2000) The Handbook of Qualitative Research. Sage Books. Eisenhardt, K. M. (1989) "Building Theories From Case Study Research, " Academy of Management Review (14: 4), pp 532 -550. Gasson, S (2003) ‘Rigor in Grounded Theory Research’, in M. Whitman and A. Woszczynski (Eds. ) Handbook for Info. Sys. Research, Idea Group, Hershey PA Gasson, S. (2009) ‘ Employing A Grounded Theory Approach For MIS Research’, in Dwivedi et al. (Eds. ), Handbook of Research on Contemporary Theoretical Models in Information Systems, Idea Group, Hershey PA. Glaser, B. G. & Strauss, A. L. (1967) The Discovery of Grounded Theory, Aldine Publishing, New York Guest, G. , Bunce, A. , & Johnson, L. (2006). How Many Interviews Are Enough? An Experiment With Data Saturation And Variability. Field Methods, 18(1), 59 -82. Lincoln, Y. S. and Guba, E. G. (1985), Naturalistic inquiry, Sage Publications CA Miles, M. B. and Huberman, A. M. (1994) Qualitative Data Analysis: An Expanded Sourcebook, (2 nd. Edition) Sage Publications, Thousand Oaks, CA Patton, M. Q. (2002). Qualitative research and evaluation methods (3 rd ed. ). Thousand Oaks, CA: Sage. . Strauss, A. L. , and Corbin, J. (1998) Basics of Qualitative Research: Grounded Theory Procedures And Techniques. 2 nd. edition, Sage Publications, Newbury Park, CA Yin, R. K. Case Study Research, Design and Methods, 3 rd ed. Newbury Park, Sage Publications, 2002.
More references (recommended examples) – References used in slides are given in notes to slides Barley, S. (1990) ‘Images Of Imaging: Notes on Doing Longitudinal Field Work’, Organization Science, Vol. 1, No. 3, pp 220 -247 Cavaye, A. L. M. "User Participation In System Development Revisited, " Information & Management (28: 5) 1995, pp 311 -323. Checkland, P. (1981) Systems Thinking, Systems Practice, John Wiley & Sons, Chichester. Newman, M. , and Robey, D. (1992) "A Social Process Model of User-Analyst Relationships, " MIS Quarterly (16: 2) 1992, pp 249 -266. Orlikowski, W. J. & Robey, D. (1991) ‘Information Technology and the Structuring of Organizations', Information Systems Research, Vol. 2, No. 2, pp 143 -169 Schutz, A. (1962) Collected papers Vol. I. The problem of social reality. Martinus Nijhoff, The Hague. Suchman, L. (1987) Plans And Situated Action, Cambridge University Press, MA, USA Tannen, D. "What's In A Frame? " in: Framing in Discourse, D. Tannen (ed. ), Oxford University Press, Oxford, UK, 1993. Van Maanen, J. (1988) Tales of the Field, University of Chicago Press, Chicago, IL Walsham, G. (1995) ‘Interpretive Case Studies In IS Research: Nature and Method’, European Journal of Information Systems, No. 4, pp 74 -81 Wolfe, R. A. "Organizational Innovation: Review Critique and Suggested Research Direction, " Journal of Management Studies (31: 3) 1994, pp 405 -431. Yin, R. K. Case Study Research, Design and Methods, 2 nd ed. Newbury Park, Sage Publications, 1994.
Resources ISWORLD Qualitative Research website: http: //www. qual. auckland. ac. nz/ CAQDAS Qualitative Research resources – lots of software! http: //caqdas. soc. surrey. ac. uk/resources. htm University of Georgia – Qualitative Research Site: http: //www. qualitativeresearch. uga. edu/Qual. Page/ Ethnographic & Qualitative Methods Course Resources Discourse Analysis (Deborah Tannen, 2004): http: //www. lsadc. org/fields/index. php? aaa=discourse. htm Good discussion of inter-coder reliability in content analysis http: //www. temple. edu/sct/mmc/reliability/ Some freeware for qualitative data analysis - p Audacity is an audio editor which will record sounds, play sounds, import, edit and export WAV, AIFF, Ogg Vorbis, and MP 3 files p Express Scribe provides professional audio playback control software p Atlas/ti -- cut-down but usable demo of qualitative analysis software My web-page – interesting readings for Ph. D students: http: //www. ischool. drexel. edu/faculty/sgasson/IS-readings. html