The purpose of this course is to develop well-rounded skills in the art of theory building, research design, communicating research, and evaluating research. Each student will critically analyze articles in his/her chosen research area, and present these analyses in class. In addition, a term paper is required, which summarizes and synthesizes the chosen research area, by presenting a model of important variables and relationships, a critique of past research, and resulting guidance for future research. In the past, these term papers have served as foundations for dissertation literature review sections as well as "stand alone" published articles.

Edited Volumes

E. Mumford, R. Hirschheim, G. Fitzgerald, and T. Wood-Harper (editors), Research Methods in Information Systems, North-Holland, Amsterdam, September 1985, 320pp.

H-E. Nissen, R. Hirschheim, and H. Klein (editors), Information Systems Research: Contemporary Approaches and Emergent Traditions, North-Holland, Amsterdam, July 1991, 754pp.

Hirschheim’s Area Review Papers

R. Hirschheim, "The Effect of A Priori Views of the Social Implications of Computing: The Case of Office Automation", ACM Computing Surveys, Vol.18, No.2, June 1986, pp.165-195.

K. Lyytinen and R. Hirschheim, "Information Systems Failures: A Survey and Classification of the Empirical Literature", Oxford Surveys in Information Technology, Vol.4, 1987, pp.257-309.

R. Hirschheim and M. Newman, "Symbolism and Information Systems Development: Myth, Metaphor and Magic", Information Systems Research, Vol.2, No.1, March 1991, pp.1-34.

G. Zinkhan and R. Hirschheim, "Truth in Marketing Theory and Research: An Alternative Perspective", Journal of Marketing, Vol.56, No.2, April 1992, pp.80-88.

R. Hirschheim and H. Klein, "Realizing Emancipatory Principles in Information Systems Development: The Case for ETHICS", MIS Quarterly, Vol.18, No.1, March 1994, pp.83-109.

R. Hirschheim, H. Klein, and K. Lyytinen, "Exploring the Intellectual Structures of Information Systems Development: A Social Action Theoretic Analysis", Accounting, Management and Information Technologies, Vol.6, No.1/2, 1996, pp.1-64.

S. Smithson and R. Hirschheim, "Analyzing Information Systems Evaluation: Another Look at an Old Problem", European Journal of Information Systems, Vol. 7, No. 3, Sept. 1998, pp.158-174.

J. Iivari, R. Hirschheim and H. Klein, “A Paradigmatic Analysis Contrasting Information Systems Development Approaches and Methodologies”, Information Systems Research, Vol.9, No.2, June 1998, pp.164-193.

R. Hirschheim and K. Lyytinen, "Banning Organizational Secrecy Can Threaten Research Too", European Journal of Information Systems, Vol.3, No.2, April 1994, pp.83-84.

M. Lacity and R. Hirschheim, "Four Stories of Information Systems Insourcing", University of Houston, Working paper, 1998.

H. Klein and R. Hirschheim, "The Rationality of Value Choices in Information Systems Development", Foundations of Information Systems, 1999.

R. Sabherwal, R. Hirschheim, and T. Goles, "The Dynamics of Alignment: A Punctuated Equilibrium Model", Organization Science, Vol.12, No.2, 2001.

Review Article Exemplars

Dickson, G.W., Senn, J. and Chervany, N., Research in management information systems: The Minnesota experiments. Management Science. 23(9) 1977, pp.913-923.

Ives, Blake and Margrethe H. Olson. "User Involvement and MIS Success: A Review of Research", Management Science, Volume 30, Number 5, May 1984, pp.586-603.

Kling, R.. Social Analyses of Computing: Theoretical Perspectives in Recent Empirical Research. ACM Computing Surveys. 12(1), 1980, pp.61-110.

Markus, M.L. and Robey, D., "Information technology and organizational change: causal structure in theory and research," Management Science. 34(5) May 1988, pp.583-598.

Mason, R. and Mitroff, I.,"A Program for Research on Management Information Systems," Management Science, 19(5), 1973, pp.475-487.

Rao, V. S. and Jarvenpaa, S., "Computer Support of Groups: Theory-Based Models for GDSS Research.", Management Science, Volume 37, Number 10, October 1991, pp.1347-1362.

Ramanujan, S. and Cooper R.B. "A Human Information Processing Perspective on Software Maintenance", OMEGA, Vol. 22 No. 2 (1994).

Zmud, R., "Individual Differences and MIS Success: A Review of the Empirical Literature", Management Science, Volume 25, Number 10, October 1979, pp.966-979.

Theory

Bacharach, Samuel B. "Organizational Theories: Some Criteria for Evaluation", The Academy of Management Review, Volume 14, Number 4 (October 1989), pp.496-515

Berger, P. and Luckmann, T., The Social Construction of Reality, New York: Basic Books, 1967.

Keen, P., "Relevance and Rigor in Information Systems Research", In Nissen, H-E., Klein, H. and Hirschheim, R. (eds.) Information Systems Research: Contemporary Approaches and Emergent Traditions, North-Holland, Amsterdam, 1991, pp.27-49.

Klein, H.K. and Lyytinen, K.. The Poverty of Scientism. In Mumford, E., Hirschheim, R., Fitzgerald, G. and Wood-Harper, T. (Eds.), Research Methods in Information Systems. Amsterdam: North-Holland. 1985, pp.131-162.

Poole, Marshall Scott and Van de Ven, Andrew H. "Using Paradox to Build Management and Organization Theories", Academy of Management Review, Vol.14, No.4 (1989), pp.562-578.

Steinfield, Charles W. and Fulk, Janet "The Theory Imperative", in Janet Fulk and Charles Steinfield (editors) Organizations and Communications Technology, Sage Publications, Newbury Park, California, 1990, pp.13-25.

Whetten, David A. "What Constitutes a Theoretical Contribution?", The Academy of Management Review, Volume 14, Number 4 (October 1989), pp.490-495

Method

Andrews et al. A Guide for Selecting Statistical Techniques for Analyzing Social Science Data, Institute for Social Research, University of Michigan, 1981.

Baroudi, J. and Orlikowski, W., "The Problem of Statistical Power in MIS Research", MIS Quarterly, March 1989, pp.87-106.

Benbasat, I., Goldstein, D. and Mead, M., "The case research strategy in studies of information systems", MIS Quarterly, Vol.11, No.3, September 1987, pp.369-386.

Cooke, T. D. and Campbell, D. T. Quasi-experimentation: Design and Analysis Issues for Field Settings, Houghton Mifflin Company, 1979

Galliers, R. and Land, F., Choosing appropriate information systems research methodologies, Communications of the ACM, Vol.30, No.11, November 1987, pp. 900-902.

Galliers, R.D. "Choosing Appropriate Information Systems Research Approaches: A Revised Taxonomy", H.E. Nissen, H.K. Klein, and R. Hirschheim (editors), Information Systems Research: Contemporary Approaches and Emergent Traditions, Elsevier Science Publishers (North Holland), 1991, pp.327-345.

T. Goles and R. Hirschheim, "The Paradigm is Dead, the Paradigm is Dead…. Long Live the Paradigm: The Legacy of Burrell and Morgan", OMEGA, Vol. 28, May 2000, pp.249-268

Jenkins, A. M., "Research Methodologies and MIS Research", in E. Mumford et al. (editors), Research Methods in Information Systems, North Holland, 1985, pp.103-117

Kaplan, B. and Duchon, D., Combining qualitative and quantitative approaches in information systems research: a case study, MIS Quarterly, Vol.12, No.4, December 1988, pp. 571-586.

Lee, A., A scientific methodology for MIS case studies, MIS Quarterly, Vol.13, No.1, March 1989, pp. 33-50.

Lee, A.S., "Integrating positivist and interpretive approaches to organizational research", Organization Science, Vol. 2, No. 4, 1991, pp. 342-365

Mason, R., McKenney, J. and Copeland, D., "Developing an historical tradition in MIS research", MIS Quarterly, Vol.21, No.3, September 1997, pp.257-278.

Mason, R., McKenney, J. and Copeland, D., "An historical method for MIS research: Steps and assumptions", MIS Quarterly, Vol.21, No.3, September 1997, pp.307-320

Reviewing/Publishing

Lee, Allen S. "Reviewing a Manuscript for Publication", May 1995.

Zimmerman, Jerold "Publishing Journal Articles", working paper.

IS General: Exemplary

Adams, D., Nelson, R. and Todd, P., "Perceived usefulness, ease of use, and usage of information technology: a replication", MIS Quarterly, Vol.16, No.2, June 1992, pp.227-247

Barley, S., "Technology as an occasion for structuring: evidence from observations of CT scanners and the social order of radiology departments". Administrative Science Quarterly. Vol. 31, No. 1, March, 1986, pp.78-108.

Beath, C. and Orlikowski, W., "The Contradictory Structure of Systems Development Methodologies: Deconstructing the IS-User Relationship in Information Engineering", Information Systems Research, Vol.5, No.4, 1994, pp.350-377.

Belcher, L. and Watson, H., "Assessing the value of Conoco’s EIS", MIS Quarterly, Vol.17, No.3, September 1993, pp.255-270.

Boland, R. J., "The Process and Product of System Design", Management Science, Vol 28, No 9, 1978, pp.887-898.

Boland, R. J. and Day, W., The Process of System Design: A Phenomenological Approach, in Proceedings of the Third International Conference on Information Systems, (eds.) M. Ginzberg and C. Ross, (Ann Arbor, Michigan, 1982) pp.31-45.

Cats-Baril, W. and Jelassi, T., "The French videotex system Minitel: A successful implementation of a national information technology infrastructure", MIS Quarterly, Vol.18, No.1, March 1994, pp.1-20.

Cooper, R. (1988): "Review of Management Information Systems Research: A Management Support Perspective," Information Processing & Management. Vol. 24, No. 1, 1988, pp.73-102.

Chin, W. and Todd, P., "On the use, usefulness, and ease of use of structural equation modeling in MIS research: A note of caution", MIS Quarterly, Vol.19, No.2, June 1995, pp.237-246.

Culnan, M., The Intellectual Development of Management Information Systems, 1972-1982: A Co-Citation Analysis. Management Science. 32(2), February 1986, pp.156-172.

Culnan, M., Mapping the Intellectual Structure of MIS, 1980-1985: A Co-Citation Analysis. MIS Quarterly. 11(3), September 198, pp.341-353.

Culnan, M. and Swanson, E.B., Research in Management Information Systems, 1980-1984: Points of Work and Relevance. MIS Quarterly, 10(3) September 1986, pp.286-301.

Davis, F. "Perceived usefulness, perceived ease of use, and user acceptance of information technology", MIS Quarterly, Vol.13, No.3, Sept 1989, pp.319-340.

Feldman, M. and March, J., "Information in Organizations as Signal and Symbol," Administrative Science Quarterly, he reader cannot follow what is being said.

Hitt, L. and Brynjolfsson, E., "Productivity, business profitability, and consumer surplus: Three different measures of information technology value", MIS Quarterly, Vol.20, No.2, June 1996, pp.121-142.

Ives, B., Hamilton, S. and Davis, G., A framework for research in computer-based management information systems. Management Science. 26(9) 1980, pp. 910-934.

Jarvenpaa, S., The importance of laboratory experimentation in IS research, Communications of the ACM, Vol.31, No.12, December 1988, pp.1502-1504.

Lee, A., "Electronic Mail as a Medium for Rich Communication: An Empirical Investigation Using Hermeneutic Interpretation", In Proceedings of the Fourteenth International Conference on Information Systems, J. DeGross, R. Bostrom, and D. Robey (eds.), Orlando, Florida, 1993, pp.13-22. (also in MIS Quarterly, Vol.18, No.2, June 1994, pp.143-158)

Markus, M. L., "Power, Politics and MIS Implementation", Communications of the ACM, Vol. 26, No. 6, June 1983, pp. 430-444.

McKenney, J., Mason, R., and Copeland, D., "Bank of America: The crest and trough of technological leadership", MIS Quarterly, Vol.21, No.3, September 1997, pp.321-353.

Moris, A., Kasper, G. and Adams, D., "The effects and limitations of automated text condensing on reading comprehension performance", Information Systems Research, Vol.3, No.1, March 1992, pp.17-35.

Newman, M. and Sabherwal, R., "Determinants of commitment to information systems development: A longitudinal investigation", MIS Quarterly, Vol.20, No.1, March 1996, pp.23-54.

Ngwenyama, O. and Lee, A., "Communication richness in electronic mail: Critical social theory and the contextuality of meaning", MIS Quarterly, Vol.21, No.2, June 1997, pp.145-168.

Orlikowski, W., "Integrated Information Environment or Matrix of Control? The Contradictory Implications of Information Technology, Accounting, Management, and Information Technologies, Vol 1, No 1, 1991, pp. 9-42.

Orlikowski, W., The Duality of Technology: Rethinking the Concept of Technology in Organizations", Organizational Science, Vol. 3, No. 3, August 1992, pp.398-427.

Orlikowski, W. and Baroudi, J., "Studying Information Technology in Organizations: Research Approaches and Assumptions," Information Systems Research. 2(1) March 1991, pp.1-28.

Orlikowski, W. and Robey, D., "Information Technology and the Structuring of Organizations", Information Systems Research, 2(2), June 1991, pp.143-169.

Pinsonneault, A. and Kraemer, K., "The impact of information technology on middle managers", MIS Quarterly, Vol.17, No.3, September 1993, pp.271-292.

Porra, J., Colonial Systems, Information Colonies and Punctuated Prototyping, Ph. D. thesis, Department of Computer Science, University of Jyvaskyla, 1996.

Todd, P. and Benbasat, I., "An experimental investigation of the impact of computer based decision aids on decision making strategies", Information Systems Research, Vol.2, No.2, June 1991, pp.87-115.

Truman, G. and Baroudi, J., "Gender differences in the information systems managerial ranks: An assessment of potential discriminatory practices", MIS Quarterly, Vol.18, No.2, June 1994, pp.129-142.

Umanath, N. and Scamell, R., "An experimental evaluation of the impact of data display format on recall performance", Communications of the ACM, Vol.31, No.5, May 1988, pp.562-571.

Walsham, G., "Organizational metaphors and information systems research", European Journal of Information Systems, Vol.1, No.2, 1991, pp. 83-94.

Weill, P., "The relationship between investment in information technology and firm performance: a study of the valve manufacturing sector", Information Systems Research, Vol.3, No.4, December 1992, pp.307-333.

Grade

Article Reviews 30%

Class Participation 30%

Area Review 40%

Article Reviews

Each student will review 3 published articles in his or her chosen research area. These reviews will be written and take the form of that suggested by Allen Lee (see Readings). Each review will also be presented to the class. One week prior to class presentation, copies of the article will be duplicated and distributed by the presenter to the rest of the class. Written reviews will be distributed to the class at the beginning of the presentation. I will comment on your written reviews.

Class Participation

All students are responsible for reading, critically analyzing, and participating in discussions on every paper presented in class.

Area Review

Each student will write a paper that offers a critical analysis of his/her chosen research area. These analyses will summarize and synthesize the articles he/she presented in addition to at least five other important papers in that research area. The result will be a model or survey of important variables and relationships, a critique of past research, and resulting guidance for future research. In the past, this work has served as a foundation for dissertation literature review sections and published papers.

Course Schedule
 

Week

Topic

Feb 18 Course Overview; Writing Review Articles

Feb 25 Theory

March 4 Method

March 11 Paper Presentations - 1

March 25 Paper Presentations - 2

April 1 Paper Presentations - 3

April 8 Paper Presentations - 4

April 15 Paper Presentations - 5

April 22 Paper Presentations - 6

April 29 Area Review Presentations - 1

May 6 Area Review Presentations - 2

May 13 Area Review Papers Due

Guidelines for Critical Paper Reviews

The purpose of the following guideline is to summarize some key ideas on how to approach undertaking a critical paper review. The guideline is not a cookbook, but rather a very brief introduction to the art of critical reviewing. It will hopefully stimulate questions on your part and force you to think about how to review and evaluate any paper you read. On the whole, I am not sure that there can be a definitive introduction to reviewing, because writing a good review is a creative process, just like writing a good paper. However, the following key points may help as a rough checklist to get started.

Keep in mind, when I speak of ‘critical’ I don’t mean ‘negative’. Many reviewers feel to be ‘critical’ they have to tear a paper apart. When reviewing a paper, you should not only be searching for flaws in the paper, but also how such flaws could/should be overcome. In other words, try to be constructive as well. How could it be improved? What would its value be if it was improved? Identify both strengths and weaknesses. The ultimate aim is to contribute to quality control: safeguarding the audience interest, but also to help the author to make a contribution. Nobody can write a good paper in the first round. Papers mature into worthwhile products only through critical review. Don’t let your ego conceal the weaknesses in your own writing.

A.) Introductory issues

1. What is the paper’s purpose? Is it clearly stated early in the paper? If not, can you intelligently guess at the purpose? If it’s not clear, interpret the purpose in your own words and make the point that the author needs to be clearer about this. The burden to be clear about the purpose is on the author! If the purpose is ambitious, are the limitations made clear? Sometime it is appropriate to talk about the problem rather than the purpose of a paper.

2. Often, even if the problem or purpose is stated clearly, the author needs to convince the audience that it is an important one; that it is worth the readers’ time to reading the paper. Hence ask, how legitimate is it to take up this problem and how well is the problem motivated? Has the author made the problem seem important and in need of research? Have other people addressed the same problem and missed important angles or have circumstances changed so that prior approaches are no longer valid or appropriate?

3. How well does the structure fit with the purpose? For a longer paper, the author should preview its structure near the end of the introduction.

4. Who is the audience? Is the style and language appropriate for the audience or has the author not thought through who his/her audience is and what might be of interest and understandable to the selected audience? (Is the audience an academic one, a practitioner one, or both?)

B.) Content

1. What is or are the (potential) contribution(s) to the problem? Or, how much of the purpose is achieved?

2. Did the author relate the paper’s line of reasoning and contributions or results with pertinent prior contributions? Is the literature review reasonably adequate (complete and clear)? Does the paper fit within a particular research stream?

3. Which claims or solutions are advanced? What types of evidence are used to support the claims? Is counter-evidence considered? How penetrating (thorough and sharp) is the reasoning to weigh conflicting evidence? Are crisp and enlightening examples or analogies suggested to help grasp difficult points?

Basically two types of evidence exist: empirical (i.e. cases, multiple observations as in surveys, statistical samples; anything coming from the 5 senses) and non-empirical (personal beliefs, contemplation, concepts; anything not coming from the senses). Another important aspect to evidence is that of coherence; more specifically, coherence with an established body of knowledge. For example, the claim of the Virgin Mary’s conception is coherent with Catholic theology, but not with biology. A biological discussion of this claim will employ different definitions and different types of arguments than a theological one.

4. Are some meaningful conclusions or recommendations advanced? Meaningful in whose eyes? In other words: If the paper’s main line of argument is accepted, what difference does it make? Again, what are significant differences will vary with different disciplines or cultures.

5. Is the basic argument used throughout the paper consistent and believable?

C.) Presentation and Structure

1. Is there a clear structure, one that has a place for everything (leaves nothing out that is of importance) and also puts everything in one place (avoids redundancies)?

This idea is related to the principles of unity and cohesion. Unity requires that everything said in the paper relates to the overall topic (no orphaned ideas) and that all ideas or claims are supported (no widows). Cohesion means everything hangs together in a logical flow of ideas or claims, supporting arguments and examples. For a paper to be coherent it should use important terms with one consistent meaning and not use different words for the same concept. For example, if the word IS is introduced as referring to both organizational and technical elements, then a different word is needed for the HW and SW supporting an IS (it could be "computer system" or "technical subsystem"). If the author uses words inconsistently, coherence suffers and the reader cannot follow what is being said.

2. Is the language clear and intelligible for the audience? (The style of writing is different for experts, the educated public, or the general public). Generally the papers you will read and write are for scholars with an interest and prior background in IS. But you may be writing for a practitioner audience that will require a different language. Correct grammar and spelling, and good style need to be observed.

3. Does the writing flow easily or is it something to "fight through"? Does the writing style keep the readers’ interest or put him/her to sleep? Keep in mind certain subjects may be inherently more difficult to discuss and hence the notion of "fighting through" might be different for different audiences and different reference disciplines.

4. Is there an appropriate use of figures and tables? Could the paper’s presentation be enhanced by using additional rhetorical vehicles, such a metaphors?

D.) Theoretical Foundation

1. What is the reference discipline? Different reference disciplines will expect (require) different literature bases; different ways of arguing (i.e. rhetorical styles and examples); different values and beliefs; and so on.

2. Is there a good fit between the problem and the reference discipline? (Would another discipline provide more insight for the chosen audience?)

3. Is the paper faithful to the chosen referent discipline? (Is the underlying model, structure or framework employed by the paper an appropriate application, subset, or extension of a theory in the reference discipline?)

E.) Research Design

1. Is the paper’s research design appropriate for the problem; and for the reference discipline?

2. Would alternative research designs provide greater insight?

3. Has the paper exhibited good execution of the research design (internal validity, appropriate use of research methods and techniques, etc.)?

F.) Results

1. How are the paper’s results presented? Do the results support the conclusions (e.g., external validity)?

2. Does the paper’s conclusions successfully address the so-called "so what" problem? In other words, can the author defend his/her results from the counter-argument that the results prove nothing? That the results are trivial?

3. Are the results presented consistent and believable? Are the arguments used intelligible and believable?

Intelligibility and believability are inextricably linked, and ultimately relate to the ‘validity’ of the research. Although all research projects must produce results which are intelligible and believable for the community of scholars to bestow the label "contribution to the state of knowledge" upon them, the criteria are perhaps more subjective in interpretive (i.e. qualitative) projects. Nevertheless, we have attempted to address these issues as best we could. How does the paper address the issue of ‘believability’? In the Lacity/Hirschheim paper on "Insourcing", believability is addressed in two ways: (i) by providing direct quotations from the data the authors hope the reader can agree, or at least understand, their stated interpretation; (ii) they argue that the ultimate picture "makes sense" and is a reasonable basis by which to think about insourcing. Intelligibility, on the other hand, is addressed by considering two needs: (i) does the picture add insight and/or provide a new way of thinking about insourcing; and (ii) is the overall picture coherent, making sure there are no major holes in the picture that is painted about insourcing. All papers must address the issues of intelligibility and believability in a satisfactory fashion.