You are on page 1of 38

LECTURE SIX

ELEMENTS OF RESEARCH DESIGN
COMPILES AND PREPARED BY SIR IMRAN ZAIDI 1

1 OBSRVATION Board area of research interest identified

THE RESEARCH PROCESS
4 3 5 THEORETICAL FRAMEWORK GENERATION OF HYPOTHESES 6 SCIENTIFIC RESEARCH DESIGN 7 DATA COLLECTION ANALYSIS AND INTERPRETATION

PROBLEM DEFINITION Research problem delineated

Variables clearly identified and labeled
8 DEDUCTION Hypotheses substantiated? Research question answered?

2 PRELIMINARY DATA GATHERING Interviewing Literature survey

No

Yes

9 Report Writing

10 Report Presentation

11 Managerial Decision Making

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

2

THE RESEARCH DESIGN
A research design is a plan, structure and strategy of investigation so conceived as to obtain answers to research questions or problems. The plan is the complete scheme or program of the research. It includes an outline of what the investigator will do from writing the hypothesis and their operational implication to the final analysis of data.
COMPILES AND PREPARED BY SIR IMRAN ZAIDI 3

Feel for data Exploration Description Hypothesis testing PROBLEM STATEMENT Contrived Noncontrived 2. Extent of researcher Interference Minimum: Studying events as they normally occur Moderate: Minimum amount of interference Maximum: High degree of control and artificial settings Study setting MEASURMENT Measurement and measures Operational definition items (measure) Scaling Categorizing Coding DATA ANALYSIS 1. Sampling design Time horizon One-Shot (cross-sectional) Multishot (longitudinal) Data-Collection method Probability/ nonprobability Sample Size (n) Observation Interview Questionnaire Physical measurement Unobtrusive 3. Goodness or data Unit of analysis (Population to be studied) Individuals Dyads Groups Organizations Machines etc. Hypotheses testing COMPILES AND PREPARED BY SIR IMRAN ZAIDI 4 .THE RESEARCH DESIGN DETAILS OF STUDY Purpose of the study Types of Investigation Establishing: -Casual relationships -Correlations -Group differences.

PURPOSE OF THE STUDY EXPLORATORY STUDY An exploratory study is undertaken when not much is known about the situation in hand. but more information is needed for developing a viable theoretical framework. Exploratory studies are also necessary when some facts are known. COMPILES AND PREPARED BY SIR IMRAN ZAIDI 5 . or no information is available on how similar problems or research issues have been solved in the past.

This might help him to elicit further information or decide right away on the types of individuals who should be made ineligible for loans in the future. For instance. and number of business courses taken. It would include details of their average age.DESCRIPTIVE STUDY A descriptive study is undertaken in order to ascertain and be able to describe the characteristics of the variables of interests in a solution. age groupings. Example A bank manager wants to have a profile of the individuals who have loan payments outstanding for 6 months and more. fulltime/part-time employment status. nature of occupation. sex composition. earnings. and the like. COMPILES AND PREPARED BY SIR IMRAN ZAIDI 6 . a study of a class in terms of the percentage of members who are in their senior and junior years. number of semesters left until graduation. can be considered as descriptive in nature.

HYPOTHESES STUDY Studies that engage in hypotheses testing usually explain the nature of certain relationships. then sales will also go up. the manager would like to know the nature of the relationship that can be established between advertising and sales by testing the hypothesis: If advertising is increased. the sales of the company will increase. if he doubles the advertising dollars. or establish the differences among groups or the independence of two or more factors in a solution. Here. Example A marketing manager wants to know. COMPILES AND PREPARED BY SIR IMRAN ZAIDI 7 .

Correlational study: When the researcher is interested in delineating the important variables associated with the problem.TYPES OF INVESTIGATION CAUSAL VERSUS CORRELATIONAL Causal study: The study in which the researcher wants to delineate the cause of one or more problems is called a causal study. and chewing tobacco associated with cancer? If so. which of these contributes most to the variance in the dependent variable? COMPILES AND PREPARED BY SIR IMRAN ZAIDI 8 . drinking. the study is called a correlational study. Example A causal study question: Does smoking cause cancer? A correlational study question: Are smoking and cancer related? OR Are smoking.

A correlational study is conducted in the natural environment of the organization with minimum interference by the researcher with the normal flow of work. COMPILES AND PREPARED BY SIR IMRAN ZAIDI 9 .EXTENT OF RESEARCHER INTERFERENCE The extent of interference by the researcher with the normal flow of work at the workplace has a direct bearing on whether the study undertaken is causal or correlational.

COMPILES AND PREPARED BY SIR IMRAN ZAIDI 10 . whereas most rigorous causal are done in contrived lab settings.STUDY SETTING: CONTRIVED AND NONCONTRIVED Noncontrived settings: If organizational research be done in the natural environment where work proceeds normally. Correlational studies are invariably conducted in noncontrived settings. the research is in noncontrived settings. contrived settings: If organizational research be done in artificial environment the research is in contrived settings.

FIELD EXPERIMENT: If cause and effect relationships are studied with some amount of researcher interference. LAB EXPERIMENT: If the researcher explores cause and effect relationship not only exercising a high degree of control but in an artificial and deliberately created settings (contrived). the study is field experiment (contrived). but still in the natural settings where work continues in the normal environment.CONTRIVED AND NONCONTRIVED SETTINGS 1. COMPILES AND PREPARED BY SIR IMRAN ZAIDI 11 2. FIELD STUDY: If various factors are examined in the natural settings in which daily activities going on as normal with minimal researcher interference. . the study is field study (noncontrived). 3.

certificates of deposit. Research here is done in a noncontrived setting with no interference with the normal work routine. COMPILES AND PREPARED BY SIR IMRAN ZAIDI 12 .EXAMPLE OF FIELD STUDY A bank manager wants to analyze the relationship between interest rates and bank deposit patterns of clients. This is a field study where the bank manager has merely taken the balances in various types of accounts and correlated them to the changes in interest rates. and interest-bearing checking accounts) as interest rates changed. She tries to correlate the two by looking at deposits into different kinds of accounts (such as savings.

She select branches within a 60-mile radius for the experiment. and 10% in the third. But it is possible that some other factors might affect the findings. The above would be a field experiment since nothing but the interest rate in manipulated. so that the interest savings relationships are not influenced by some third factors. COMPILES AND PREPARED BY SIR IMRAN ZAIDI 13 . In the fourth branch. Within the week. she would be able to determine the effects. The banker may not have been aware of this fact while setting up the experiment. all four branches chosen would be more or less compatible in size. For 1 week only. For example. deposit patterns. if any. with all activities occurring in the normal and natural work environment.EXAMPLE OF FIELD EXPERIMENT The bank manager now wants to determine the cause-and-effect relationship between interest rate and the inducements it offers to clients to save and deposit money in the bank. and the like. Hopefully. despite the attraction of a good interest rate. the interest rate remains unchanged at 5%. of interest rates on deposit mobilization. she advertise the annual rate for new certificates of deposit received during that week in the following manner: the interest rate would be 9% in one branch. number of depositors. one of the areas may have more retirees who many not have additional disposable income that they could deposit. 8% in another.

and keeps the interest at the lowest rate of 1% for group 4. In this lab experiment with the contrived settings. COMPILES AND PREPARED BY SIR IMRAN ZAIDI 14 . which they are told they might utilize to buy their needs or save for the future. beyond a doubt. inasmuch as the setting is difficult. Because of this she wants to create an artificial environment and trace the true cause and effect relationship. She offers them an incentive. the researcher interference has been maximal. keeping in step with the increasing rates of interest. She splits them into four groups and gives each one of them amount of $1. the independent variable has been manipulated. 9% for group 3. 8% for group 2.EXAMPLE OF LAB EXPERIMENT The bank manager now wants to establish the causal connection between interest rates and saving. she would be able to established a cause and effect relationship between interest and the disposition to save. If the banker finds that the savings by the four groups increase progressively. She has also chosen subjects with similar backgrounds and exposure to financial matters (business students). Here the manager has created an artificial laboratory environment and has manipulated the interest rates for savings. or both.000. and most external contaminating factors such as age and experience have been controlled. She recruit 40 students who are all business majors in their final year of study and are more or less of the same age. interest on what they save but manipulates the interest rates by offering a 6% interest rate on savings for group 1.

15 COMPILES AND PREPARED BY SIR IMRAN ZAIDI . Engage in a field experiment. then. Are there cost constraints? Both internal validity and external validity are important.Decision points for embarking on an experimental design Is tracing causal effects necessary? Yes and if No Do not undertake an experimental design study Internal validity is more important than external validity Engage in a lab experiment. Yes No Engage in a simpler experimental design. Generalizability is more important than internal validity. First do a Lab experiment. Engage in a more sophisticated design. a FIELD experiment.

COMPILES AND PREPARED BY SIR IMRAN ZAIDI 16 . then several two-person groups. then the data analysis will be done at the departmental level. If we compare the different cities of any country. then the unit of analysis would be at the group level. If we compare the different countries. If we compare different departments in the organization. etc.etc. If the problem statement is related to group effectiveness. then we are interested in individuals employees in the organization and would have to find out what we can do to raise their motivation. If the researcher is interested in studying two-person interactions. If the problem statement focuses on how to rates levels of employees in general. Here the unit of analysis is the individual. If we compare different organizations. then the data analysis will be at the country level. then the data analysis will be at the city level.UNITS OF ANALYSIS The unit of analysis refers to the level of aggregation (bunch) of the data collected during the subsequent data analysis stage. also known as dyads. then the data analysis will be done at the organizational level.

EXAMPLES 1. Data were collected from stock brokers between April and June of last year to study their concerns in a turbulent (beyond control) stock market. perhaps over a period of days or weeks or months. A drug company desirous of investing in research for a new obesity (reduction) pill conducted a survey among obese people to see how many of them would be interested in trying the new pill. are called one-shot or crosssectional studies. Data with respect to this particular research had not been collected before. This is a one-shot or crosssectional study to assess the likely demand for the new product.TIME HORIZON One Shot or Cross-Sectional Studies If data are gathered just once. nor will they be collected again from them for this research. 2. in order to answer a research question. COMPILES AND PREPARED BY SIR IMRAN ZAIDI 17 .

one could conclude that either the advertisement is ineffective or it will take a longer time to take effect. EXAMPLE One could study the sales volume of a product before and after an advertisement. the researcher might want to study employees’ behavior before and after a change in the top management. to the advertisement. If there is no increase in sales. but is carried longitudinally across a period of time.Multishot or Longitudinal Studies If the researcher might want to study people or phenomena at more than one point in time in order to answer the research question or when data on the dependent variable are gathered at two or more points in time to answer the research question. the study is not cross-sectional or of the one-shot kind. and provided other environmental changes have not impacted on the results. one could attribute the increase in the sales volume. Here. For instance. the studies are called longitudinal studies. COMPILES AND PREPARED BY SIR IMRAN ZAIDI 18 . if any. so as to know what effects the change accomplished. because data are gathered at two different points in time.

The type of investigation. Joyce Lynn. sales volume. 3. 2. and staff training. 4. • The purpose of study. Scenario A Ms. 6. profit margin. The extent of researcher interference. that is. the owner of small business (a women’s dress boutique). COMPILES AND PREPARED BY SIR IMRAN ZAIDI 19 . 5. give reason also: 1. The time horizon for the study.EXERCISE In the following scenarios indicate how the researcher should proceed in each case. The study settings. has invited a consultant to tell her how business is different from similar small businesses within a 60-mile radius with respect to use of the most modern computer technology. The unit of analysis. determine the following.

he would like to be assured that the idea would work. He asks a researcher to help him on this issue. COMPILES AND PREPARED BY SIR IMRAN ZAIDI 20 . pall Hodge.EXERCISE • Scenario B Mr. But before he actually does this. is concerned about the wide differences in their profit margins. He would like to try some incentive plans for increasing the efficiency levels of those restaurants that lag behind. the owner of several restaurants on the East Coast.

EXPERIMENTAL DESIGN COMPILES AND PREPARED BY SIR IMRAN ZAIDI 21 .

since we do not know how much of the total variation of Y was caused by the presence of the other factor A. it is possible that some other factor. it will not be possible to determine the extent to which Y occurred only because of X. COMPILES AND PREPARED BY SIR IMRAN ZAIDI 22 .CONTROLLING OF CONTAMINATING FACTORS When we postulate cause-and-effect relationships between two variables X and Y. might also influence the dependent variable Y. In such a case. says A.

the manager cannot prove that the special training alone caused greater effectiveness. some of the new secretaries might function more effectively than others. since the previous intermittent experience of some secretaries with the web is a contaminating factor. In this case. This might be done by not including in the experiment those who already have had some experience with the web. mainly or partly because they have had previous intermittent experience with the web. a Human Resource Development manager might arrange for special training to a set of newly recruited secretaries in creating web pages. then the learners’ previous experience has to be controlled. If the true effect of the training on learning is to be assessed.EXAMPLE OF CONTROL For instance. However. This is what we mean when we say we have to control the contaminating factors. COMPILES AND PREPARED BY SIR IMRAN ZAIDI 23 .

.e. the process by which individuals are drawn (i. Randomization In randomization. COMPILES AND PREPARED BY SIR IMRAN ZAIDI 24 . everybody has a known and equal chance of being drawn) and their assignment to any particular group (each individual could be assigned to any one of the groups set up ) are both random.CONTROLLING THE CONTAMINATING EXOGENOUS OR “NUISANCE” VARIABLES Matching Groups One way of controlling the contaminating or “nuisance” variables is to match the various groups by picking the confounding characteristics and deliberately spreading them across groups.

EXTERNAL VALIDITY To what extent would the result found in the lab setting be transferable or generalizable to the actual organizational or field settings? In other words. if we do find a cause-and-effect relationship after conducting a lab experiment. can we then confidently say that the same cause-and-effect relationship will also hold true in the organizational setting? COMPILES AND PREPARED BY SIR IMRAN ZAIDI 25 .INTERNAL VALIDITY Internal validity refers to the confidence we place in the causeand-effect relationship with in the lab settings.

thus affecting the internal validity. Time: t1 t2 t3 Independent variable Sales promotion Dependent variable Sales Dairy farmers’ advertisement Uncontrolled variable COMPILES AND PREPARED BY SIR IMRAN ZAIDI 26 . and this history of events would confound the cause-and-effect relationship between the two variables.FACTORS AFFECTING INTERNAL VALIDITY History Effects Certain events or factors that would have an impact on the independent variabledependent variable relationship might unexpectedly occur while the experiment is in progress.

Maturation Effects Cause-and-effect inferences can also be contaminated by the effects of the passage of time—another uncontrollable variable. Time: t1 t2 t3 Dependent variable Independent variable Enhanced technology Efficiency Increase Gaining experience and doing the job faster Uncontrolled variable COMPILES AND PREPARED BY SIR IMRAN ZAIDI 27 . Such contamination is called Maturation effects.

The difference between the posttest and the pretest scores is then attributed to the treatment. However. That is. and posttest. and not because of the treatment’s differential impact at the end. the very fact that respondents were exposed to the pretest might influence their responses on the posttest. then the treatment given. to test the effect of a treatment. administered. first a measure of the dependent variable is taken (the pretest). COMPILES AND PREPARED BY SIR IMRAN ZAIDI 28 .Testing Effects Frequently. subjects are given what is called a pretest (say. These might arise because of a change in the measuring instrument between pretest. and after that a second test. Instrumentations Effects Instrumentation effects are yet another source of threat to internal validity. a short questionnaire eliciting their feelings and attitudes). which would adversely impact on internal validity. called the posttest.

Mortality Another confounding factor on the causeand-effect relationship is the mortality or attrition of the members in the experimental or control group or both. COMPILES AND PREPARED BY SIR IMRAN ZAIDI 29 . as the experiment progresses.Selection Bias Effects The threat to internal validity could also come from improper or unmatched selection of subjects for the experimental and control groups.

This phenomenon of low scores tending to closer to the mean is known as “regression towards the mean” (statistical regression). COMPILES AND PREPARED BY SIR IMRAN ZAIDI 30 . We know from the law of probability that those with very low scores on a variable have a greater probability of showing improvement and scoring closer to the mean on the posttest after being exposed to the treatment.Statistical Regression The effect of statistical regression are brought about when the members chosen for the experimental group have extreme scores on the dependent variable to begin with. those with very high abilities would also have a greater tendency to regress towards the mean-they will score lower on the posttest than on the pretest. Likewise.

however. the effects of the treatment can be obtained by measuring the difference between the posttest and the pretest (O2-O1). Where O refers to some process of observation or measurement. Group Experimental group Pretest score Treatment X Posttest Score O1 O2 Treatment effect = (O2-O1) COMPILES AND PREPARED BY SIR IMRAN ZAIDI 31 .TYPES OF EXPERIMENTAL DESIGNS Pretest and Posttest Experimental Group Design An experimental group (without a control group) may be given a pretest exposed to a treatment. that testing and instrumentation effects might contaminate the internal validity. Here. history and maturation effects may also confound the results. If the experiment is extended over a period of time. and the X and Os in the row are applied to the same specific group. Note. X represents the exposure of a group to an experimental treatment. and then given a posttest to measure the effects of the treatment.

selection biases could contaminate the results. If the two groups are not matched or randomly assigned. There are at least two possible threats to validity in this design. The effects of the treatment are studied by assessing the difference in the outcomesthat is. Group Experimental group Control group Treatment X Outcome O1 O2 32 Treatment effect = (O2-O1) COMPILES AND PREPARED BY SIR IMRAN ZAIDI . Otherwise. Here is a case where the testing effects have been avoided because there is no pretest. Randomization would take care of this problem. the former alone being exposed to a treatment and not the latter. only a posttest.Posttests Only with Experimental and Control Groups Some experimental designs are set up with an experimental and a control group. the posttest scores of the experimental and control groups. to make sure that the two groups are matched for all the possible contaminating “nuisance” (unwanted) variables. Mortality (the drop out individuals from groups) can also confound the results. however. the true effects of the treatment cannot be determined by merely looking at the difference in the posttest scores of the two groups.

COMPILES AND PREPARED BY SIR IMRAN ZAIDI 33 . Measuring the difference between the differences in the post-and pretest scores of the two groups would give the net effects of the treatment. It is possible that those who drop out are in some way different from those who stay on until the end and take the posttest. mortality could offer a plausible (apparently valid) rival explanation for the difference between O2 and O1. The only difference between the two groups is that the former is exposed to a treatment whereas the latter is not. testing. Both groups have been exposed to both the pre-and posttests.g. some of the subjects in the experimental group may drop out before the end of the experiment. maturation. In experiments that take several weeks. or measuring the impact of technology advancement on effectiveness.. we have also controlled the effects of selection biases and statistical regression. If so. however. Mortality could. and both groups have been randomized. This is so due to the fact that whatever happened with the experimental group (e.Pretest and Posttest Experimental and Control Group Designs Two groups-one experimental and the other control-are both exposed to the pretest and the posttest. and instrumentation) also happened with the control group. Through the process of randomization. testing. as in the case of assessing the impact of training on skills development. history. pose a problem in this design. and instrumentation effects have been controlled. thus we could expect that the history maturation. and in measuring the net effects (the difference in the differences between the pre-and posttest scores) we have controlled these contaminating factors.

Pretest and posttest experimental and control group Group Experimental group Control group Pretest Treatment X Posttest O1 O3 O2 O4 Treatment effect = [(O2-O1) .(O4-O3)] COMPILES AND PREPARED BY SIR IMRAN ZAIDI 34 .

COMPILES AND PREPARED BY SIR IMRAN ZAIDI 35 . Here the effects of the treatment can be calculated in several different ways. This design. The other two groups will be given only the posttest. it is advisable to set up two experimental groups and two control groups for the experiment. This increases the internal validity of the results of the experimental design. is perhaps the most comprehensive and the one with the least number of problems with internal validity. To the extent that we come up with almost the same results in each of the different calculations.SOLOMON FOUR GROUP DESIGN To gain more confidence in internal validity in experimental design. One experimental group and one control group can be given both the pretest and the posttest. known as the Solomon four-group design. we can attribute the effects to the treatment.

3. COMPILES AND PREPARED BY SIR IMRAN ZAIDI 36 . 4.(O4-O3)] If all Es are similar. 2. Experimental Control Experimental Control O1 O3 X X O2 O4 O5 O6 Treatment effect (E) could be judged by: E= (O2-O1) E= (O2-O4) E= (O5-O6) E= (O5-O3) E= [(O2-O1) . SOLOMON FOUR GROUP DESIGN MODEL Group Pretest Treatment Posttest 1.. the cause-and-effect relationship is highly valid.

Group 2. these have had no impact at all. instrumentation. maturation. nor maturation. In other words. nor testing. nor mortality has had an impact. helps us to see whether or not history. the control group that was exposed to both the pre-and posttest. then it is established that neither history. or mortality threaten internal validity. This removes the statistical regression and selection biases. COMPILES AND PREPARED BY SIR IMRAN ZAIDI 37 . testing. It is important to note that subjects have been randomly selected and randomly assigned to groups. If scores O3 and O4 (pre-and posttest scores of group 2) remain the same. nor instrumentation. nor statistical regression.Solomon Four-Group Design and Threats to Internal Validity Let us examine how the threats to internal validity are taken care of in the Solomon four-group design. regression.

THANK YOU FOR YOUR CONCENTRATION COMPILES AND PREPARED BY SIR IMRAN ZAIDI 38 .