Planning for Research Success William R Kinney Jr

Planning for Research Success William R. Kinney, Jr. Ph. D Project – ADSA Making a Difference Denver, Colorado August 6, 2011

Outline • Planning process overview • X, Y, V, and Z - a framework for planning • The scholarly researcher’s problem • Threats to research validity (Runkle and Mc. Grath meet Cook and Campbell) • Hints on how to get your paper published in a top-tier scholarly journal and other suggestions

1. Planning process overview Nothing you don’t know or couldn’t figure out with slight effort. • “Facts” are real world observables. • “Problems” are facts or relationships between facts that you don’t like or understand. • “Theories” are ideas about causal relationships between facts (or what causes the problem “facts”) • “Hypotheses” are predictions of real world observables that should occur if your theory is descriptive of the real world.

Getting started Suppose that you have an idea (a problem): you observe undesirable “facts” or peculiar “facts” or claims • What research barriers must be overcome: – Availability of: Causal theories? Data? Estimation methods? – Research design (how to combine the above)? – Exposition (if you can’t explain it, you fail)? • Who will want to read your paper (and why)? • What is your comparative advantage? Hint: “One gets the biggest potatoes on the first pass through the field” (Irish agricultural economics principle per Frank O’Connor, University of Iowa)

Auditing research domain Accounting Auditing KPS KS Professional structure Laws, regs, governance Contracts/ incentives Professional standards Firm organization, mores External enforcement Culture, markets, traditions

An idea source: the Sarbanes. Oxley Act of 2002 • Result of 25 year economic, technological, social, and professional/regulatory changes • Challenges – Relevance of GAAP – Reliability of auditing standards – Trustworthiness of auditors, independent directors, standards setters, financial intermediaries and employment contracts that bind them • Mandates a new (3 rd) role for accounting. . . this is a big new potato -- new states, new actions need new theories to explain/understand

2. A Framework Y = f ( X, Vs, Zs ) Y = phenomenon to be explained X = your (new) theory about a cause of Y Vs = prior causes of Y Zs = contemporaneous causes of Y

How does X (treatment) get there? Experiments vs. Archival studies ? X 0 {V-3, V -2, V -1 } Y 1 Z 0 Random assignment or independent of V vs. Self selection or V(s) determine X

3. Scholarly Researcher’s Problem: = risk that data incorrectly “accepts” new theory = risk that data incorrectly “rejects” new theory = true size of X effect on Y = residual variation given research design (i. e. , after effects of Vs and Zs) n = available sample size All five are related through a single, simple formula

The Sample Size formula: n = ( (Z + Z ) . ) 2

Researcher’s Problem (continued) is fixed at. 05 or. 10 by journal editors * * is the researcher’s risk of failing (you want to minimize ) = f ( X : Y relation) = f ( Vs, Zs) n is semi-fixed by data availability or cost = f ( n ) - - + -

Graphically. . . (small , small okay) _ Y| H 0, n, _ Y| HA, n, 0 Accept H 0 Reject H 0 Y

Graphically. . . (large , large okay) _ Y| HA, n, _ Y| H 0, n, 0 Accept H 0 Reject H 0 Y

Graphically . . . (small , large yikes!) _ Y| HA, n, _ Y| H 0, n, 0 Y Accept H 0 Reject H 0

4. Analyze Threats to Validity using Predictive Validity (Libby) Boxes Independent Conceptual Theoretical Dependent 1 Theoretical Y (Y) X (X ) 2 Operational X 3 5 Operational Y 4 Control Other potentially influential variables Vs and Zs

It is believable that X causes Y if: • X and Y are correlated • Vs and Zs ruled out by design, including – Y causes X – X and Y caused by an omitted V or Z • Reason to believe that operational X and Y measure X and Y • Reason to believe that X : Y relation generalizes to other persons, times, and settings.

Validity threats linked to Libby boxes (and Vs and Zs) • Statistical Conclusion Validity (5) • Internal Validity (4) • Construct Validity (2 and 3) • External Validity ( 1 generalizes to ? ? )

Auditor independence and non-audit services: Was the U. S. government right? (Kinney, Palmrose, Scholz) Does an audit firm’s dependence on fees for FISDI, internal audit, and certain other services to an audit client reduce financial reporting quality? The answer is important because a) the Sarbanes-Oxley Act presumes so, banning such services to audit clients, and b) some registrants now voluntarily restrict tax and other legally permitted services. Using fee data from 1995 -2000 for restating and similar nonrestating registrants, we find no consistent association between fees for FISDI or internal audit services with restatements, but find significant positive association between unspecified services fees and restatements and significant negative association between tax services and restatements.

Libby boxes for KPS example. . . Independent Conceptual Auditor dependence on client Dependent 1 2 Operational Lower quality financial reporting 3 Non-audit fees Restatement 5 4 Control Industry, size, audit policies, acquisitions, etc.

Causal theory vs. Policy using Libby Boxes (causal theory testing) Independent Conceptual Theoretical Dependent 1 Theoretical + X (X ) Y (Y) 2 Operational Control Operational X 3 5 + Operational Y 4 Other potentially influential variables Vs and Zs

Causal Theory vs. Policy using Libby Boxes (policy testing) Independent Conceptual identical Operational New policy Dependent 1 + Desired behaviors 2 Operational X 3 5 0, < 0 Operational Y 4 Control Other potentially influential variables Vs and Zs Need creativity

5. Hint one: Your main contribution is: 1. New data 2. New estimation 3. New theory (or new problem) Whatever it is, Exploit it!

Hint Two: Broaden your contribution (and reader interest) by: 1. Making your theory elaborate 2. Using multi-methods and multi-measures 3. Generalizing your approach across contexts, disciplines, cultures, and time

Hint Three: In introducing your paper, tell the reader: 1. What (specific) problem will be addressed 2. Why the (specific) problem is important 3. How you will address the (specific) problem (and what you found) (One page, double-spaced, 12 pt. font, normal margins, and include an informative title!)

Order of importance of clear and compelling exposition Title Abstract Introduction Conclusions. . . rest of text

Referees are not (entirely) stoopid – • Sometimes: – They are right – They are misguided, but you misguided them – They are wrong, but remind you that you were making a different (and important) point – They are simply wrong and nothing need be done (extremely rare, in my experience) • Always “attend to” their comments – you will benefit.

Remember. . . • Plan for research success – write three paragraphs before you do the work – minimize b ex ante – maximize validity ex ante • Make your unique contribution apparent to all • Your present model of the world is simplified – be alert to revision via new problems, theories, data, and methods (read, take courses, attend seminars in potentially related areas for new theories, monitor data sources, scan for useful research tools)

References • Cook, and D. Campbell, Quasi-experimentation design & analysis issues for field settings. Houghton Mifflin Co. (New York) 1971 (Validity types). • Kinney, W. , "Empirical Accounting Research Design for Ph. D. Students, " The Accounting Review, April 1986 (3 paragraphs and integration). • Libby, R. , Accounting and human information processing: Theory and applications. Prentice-Hall, Inc. , (Englewood Cliffs) 1981 (Boxes). • Runkel, P. , and J. Mc. Grath, Research on human behavior – A systematic guide to method. Holt, Rinehart, and Winston, Inc. , (New York) 1972 (Boxes). • Simon, J. , and P. Burstein, Basic Research Methods in Social Science (3 rd ed. ). Random House, (New York) 1985 (Chapter 3 – X, Y, Vs, Zs).