EMPIRICAL RESEARCH METHODS

Research Methods Class, March 8, 15 & 22, 2002 Daniel Moody, Visiting Associate Professor, NTNU & Senior Research Fellow, Monash University (Melbourne, Australia) What is Empirical Research?
Empirical research methods are a class of research methods in which empirical observations or data are collected in order to answer particular research questions. While primarily used in academic research, they can also be useful in answering practical questions.

Research Questions, Theoretical Models and Hypotheses

Empirical research normally starts with some a priori theory, which the researcher develops to try to explain and/or predict what happens in the real world. The purpose of the research is to test the theory and possibly refine it. In some cases, research is conducted to develop theory (the grounded theory approach), but this is much harder (and more risky) to do.

Research Questions
Initially a research question is formulated. For example: Is programming language A more effective than programming language B? What are the critical success factors in implementing an ERP system?

Theoretical Models
To be empirically tested, the research question will need to be transformed into a theoretical model, consisting of theoretical constructs (latent variables), causal relationships and measures (observed variables). The theoretical model is generally developed based on analysis of the literature. The theoretical model forms the basis both for collecting and analysing data, and may be modified as a result of the research.

Hypotheses
A hypothesis defines an expected relationship between variables (based on causal relationships in the theoretical model), which can be empirically tested.

Qualitative vs Quantitative Methods

Empirical research methods can be divided into two categories: Quantitative research methods: such methods collect numerical data (data in the form of numbers) and analyse it using statistical methods. Qualitative research methods: such methods collect qualitative data (data in the form of text, images, sounds) drawn from observations, interviews and documentary evidence, and analyse it using qualitative data analysis methods.

Page 1

Qualitative methods tend to be more appropriate in the early stages of research (exploratory research) and for theory building. Quantitative methods tend to be more appropriate when theory is well developed, and for purposes of theory testing and refinement. In practice, no research method is entirely qualitative or quantitative (Yin, 1994). For example, a survey may collect qualitative data using open ended questions as well as quantitative data using closed questions; an experiment may include observations of participant behaviour as well as measures of response time and accuracy; a case study may incorporate quantitative data (e.g. system usage statistics) as well as qualitative data (e.g. interviews with users). The selection of an appropriate research method is critical to the success of any research project, and must be driven by the research question and the state of knowledge in the area being studied. In general, a combination of research methods may be most effective in achieving a particular research objective. For example, when a subject area is not well understood, qualitative methods may be used to build theory and testable hypotheses. This theory may then be tested using quantitative methods such as surveys and experiments. Mixing qualitative and quantitative research methods is called triangulation of method. While most researchers develop expertise in one style, the two types of methods have different, complementary strengths and when used together can lead to a more comprehensive understanding of a phenomenon.

Quantitative Methods
The most common quantitative methods are: Experiment: apply a treatment, measure results (before and/or after): this is the only method that can demonstrate causal relationships between variables. Experimental research is associated with the traditional scientific method (the Newtonian model of science). Survey: ask questions (face to face interview, telephone, mail, internet) Historical data: look for patterns in historical data (e.g. IT investment patterns)

While quantitative methods tend to result in more convincing scientific evidence, they are generally more difficult to apply in a real world context (field setting).

Qualitative Methods
The most common qualitative methods are: Case study: observations carried out in a real world setting (e.g. a software development project, an operating theatre in a hospital). The objective is to immerse yourself in the situation and gain a holistic understanding of the phenomena in its natural setting. Action Research: apply a research idea in practice, evaluate results, modify idea (cross between an experiment and case study)

Qualitative methods tend to be applied more easily in real world settings, but lack internal validity (alternative explanations of results; lack of control) and external validity (usually concerned with a single case: limits generalisability to other settings). Also, interpretation of data is by nature much more subjective than quantitative methodsit is easy to read what you want into the data.

Page 2

Data Analysis
Used to determine whether your theory is supported or not supported. Quantitative data analysis: use of statistical methods to identify patterns and relationships in the data. Qualitative data analysis: analysis is more subjective, and relies heavily on the researchers knowledge and experience to identify patterns, extract themes and make generalisations.

Topic Outline
Class 1 (March 8): Empirical Research Methods
Topics covered: Empirical research methods Quantitative vs qualitative methods Theory building Internal and external validity

Class 2 (March 15): Quantitative Research Methods

Topics covered: Experiments Surveys Historical analysis

Class 3 (March 22): Qualitative Research Methods

Topics covered: Case study Action research

Additional References
These represent definitive references on particular research topics. While not necessary in this subject, they are essential reading if you intend to use any of these techniques in your future research.

Theory Building
DUBIN, R. (1978): Theory Building (Revised Edition), The Free Press, New York.

Experimental Research
CAMPBELL, D.T. and STANLEY, J.C. (1966): Experimental and Quasi-Experimental Designs for Research, Houghton Mifflin College.

Survey Research
DILLMAN, D.A. (2000): Mail and Internet Surveys: The Tailored Design Method (2nd ed.), John Wiley & Sons, New York.

Page 3

Case Study Research

YIN, R. K. (1994) Case Study Research: Design Methods. Second Edition, Sage Publications, San Francisco

Action Research
Action Research Resources: a web site with a wide collection of resources on action research (including How to Do an Action Research Thesis) http://www.scu.edu.au/schools/gcm/ar/arhome.html

Statistical Analysis
PEDHAZUR, E.J. and SCHMELKIN, L.P. (1991): Measurement, Design and Analysis: An Integrated Approach, Lawrence Erlbaum Associates Inc, Hillsdale, NJ.

Qualitative Data Analysis

MILES, M.B. and A.M. HUBERMAN (1994): Qualitative data analysis: An expanded source book, 2nd ed. Thousand Oaks: Sage Publications

Page 4