Elements of Research Design


Outline

  1. Kinds of Objectives
  2. Causality
  3. Experiments and Field Studies
  4. Longitudinal vs Cross-Sectional

Kind of Objectives

While real studies are usually combinations of all three types, in principle, the objectives of studies can be classified into three basic types: exploratory, descriptive and deductive (theory-testing).

Theory-Testing

Also known as "deductive", "hypothesis-testing", and "predictive". The idea here is to test a theory by working out some of the specific implications of the theory (call these hypotheses), and then collecting data to see if the hypotheses are supported or not.

Normally, we engage in theory testing studies only after exploratory work has already been done, or we are borrowing general theories developed in other areas and applying them in a new setting. In either case, we obviously need to have a pretty well worked out theory before we begin the study.

Exploratory Study

Also known as "inductive" or "theory-building". In this kind of study, we don't begin with a theory. Instead, we collect data that, after analysis, we will use to develop a theory. After we develop the theory, we might then design a study to test the theory.

Of course, even in exploratory studies we have to have some idea about how things work, otherwise we wouldn't know what data to collect. For example, we might be interested in understanding why certain members of an organization have higher morale than others. So among other things we administer some questionnaires asking people about their employment history, the nature of their jobs, characteristics of their personality, their relationships with others, and so on. Then, in the analysis stage, we see if any of these things is related to morale. Once we find out what is related and what is not, we can start thinking about a theory that explains these statistical relationships.

Descriptive Study

A descriptive study is similar to an exploratory study in that we do not attempt to test hypotheses. Often, they are used in settings where a theory of how variables are related is already in place, but specific values for each of the variables are needed for specific cases in order to take some action. For example, a company wants to re-organize for maximum efficiency. One thing it needs to do is allocate people to jobs to match the kinds of customers it has. So if most of the customers are government agencies, this calls for a different internal sales, distribution and manufacturing structure than if most of the customers are individual consumers. So a survey of customers provides basic data on which to make organizational decisions. Similarly, if an organization is considering adopting a new benefits package that costs more but has new features that might be attractive, the organization needs to know what the needs of the employees are to determine whether the package makes sense for them. For example, if the main feature of the more expensive package is a domestic partner program (that's where homosexual partners of employees are entitled to health insurance just like heterosexual spouses), it makes sense to find out how many gay & lesbian employees the firm has. If it doesn't have any, it might not make sense to get the package. (However, new employees are hired everyday, and there are ethical issues here that transcend cost/benefit analyses -- it's just an example!)

Relation to How the Data are Used

Applied/Consulting Basic/Academic
Theory-Testing Infrequent Common
Exploratory Common Infrequent
Descriptive Common Rare

 

Causality

In contrast to your textbook, I would say that all studies (except descriptive) are interested in causality. What varies is how hard they try. All theories are causal. So theory-testing studies are basically about testing whether it's true that X causes Z and why. Similarly, the purpose of most exploratory studies is to develop a causal theory, which can then be tested at a later time.

However, (this a big however) there is no statistical technique that can determine causality. We can observe correlations (associations, relationships among variables) but we must infer causality. On the other hand, some kinds of studies make this inference a lot more sure than others. In particular, longitudinal studies and better for establishing causality than cross-sectional studies, and lab experiments are better than field studies.

Longitudinal vs Cross-Sectional Studies

A longitudinal study is where we follow the units of analysis (say, employees) over time, and measure key variables at different points in time. For example, we might measure morale before and after a promotion. A cross-sectional study is where we collect data only once from each unit of analysis.

Because longitudinal studies are much more difficult and expensive to perform, we often try to use cross-sectional studies to answer questions which we would really rather ask in a longitudinal setting. For example, instead of waiting for people to age so we can see the effect of age on some variable, we can do a cross-sectional study in which people of all different ages are measured on the same variable so we can see if older people have different values on the variable than do younger people.

Field Studies, Field Experiments and Lab Experiments

A field study is a study in which the researcher goes to a research site and observes and asks questions, but does not change anything. It's like a naturalist observing wildlife without doing anything like setting out food to attract animals, or making obstacles to see how the animals react.

A field experiment is a study in which you make changes in the independent variable to see how it affects the dependent variable, but otherwise you leave everything in its natural state. For example, an ethologist (someone who studies animal behavior), might put sugar water at different distances from a bee nest in order to observe how the difference in distance affects the dance that the bees do on returning home to communicate where food is.

A lab experiment is a study in which you make changes in the independent variable, and then control all the other variables so that only the variable of interest could possibly affect the outcome. For example, if you are interested in the effects of seeing an inspirational film on taking a math test, you can recruit some experimental subjects to come to your theatre, then randomly assign half to see the film, and the other half to see some other film, then give them the test right after the films.

Experiments are much better for inferring causality than observational studies because they control for more confounding variables. For example, if you were using an observation study to look at the effects of the inspirational film, you could just give the test to thousands of people, then ask them if they happened to have seen that particular film. Then you could compare the average score for people who had seen it with the score of people who had not seen it. The trouble with this is that we don't know why people chose to see or not see the film. That reason could be related to math or general study skills. For example, the inspirational film might have been more likely to draw devout, middle-class Christians than drugged-out, school-hating teenagers. The people going to such a film on their own might be the same people who would do better on math tests anyway, so if they did do better on the test, it might not be because the movie itself inspired them.


Copyright 1996 Stephen P. Borgatti Revised: June 24, 1997 Home Page