Research Methodology

PART 4 Experimental Design

M S Sridhar Head, Library & Documentation ISRO Satellite Centre Bangalore 560017
E-mail: sridhar@isac.gov.in & sridharmirle@yahoomail.com

Experimental Design
Synopsis 1. Introduction to Research &

Research methodology 2. Selection and formulation of research problem 3. Research design and plan 4. Experimental designs 5. Sampling and sampling strategy or plan 6. Measurement and scaling techniques 7. Data collection methods and techniques 8. Testing of hypotheses 9. Statistical techniques for processing & analysis of data 10. Analysis, interpretation and drawing inferences 11. Report writing

A.

Basic concepts & principles Validity of experiments Experimental design: types & steps Single case research: ABAB design Some experimental designs Classical Design (J.S.MILL) 1. Method of agreement 2. Negative canon of agreement 3. Method of difference Limitations of classical design B. Informal Experimental Designs 1. Before-and-after without control 2. After only with control 3. Before-and-after with control C. Formal Experimental Designs 1. Completely randomised design (CR) 2. Randomised block design (RB) 3. Latin square design (LS) 4. Factorial design: Simple & Complex
Research Methodology 4 2

M S Sridhar

Introduction
1. Basic design of logical proof 2. Helps testing hypotheses 3. Leads to inference on causality (finding causal relationship between variables) 4. Employs a set of control & experimental groups of units 5. Administers treatments / stimuli 6. Controlled observation of change & development in variables 7. Adequate control is the essence 8. Reduces bias & increases reliability
M S Sridhar

Experimental studies or hypothesis testing involve: - Causal relationship between variables - reduce bias & increase reliability - drawing inferences about causality THREE BASIC PRINCIPLES OF EXPERIMENTAL DESIGN ARE: REPLICATION - Experiment repeated several times for better results/ increasing statistical accuracy RANDOMISATION - Protects experiment against extraneous factors of chance LOCAL CONTROL - Known source of variability is made to widely vary deliberately so that its effects can be measured and eliminated from experimental error
Research Methodology 4 3

Basic Concepts
VARIABLE: A concept which can take on different quantitative values (absence of non-quantifiable attributes) DEPENDENT VARIABLE: Depends upon or a consequence of other variable INDEPENDENT VARIABLE: Antecedent to dependent variable EXTRANEOUS VARIABLE: Independent variable not related to the purpose but may affect the dependent variable & its effect on dependent variable is Experimental error CONTROL: To minimise the effects of extraneous independent variable CONFOUNDED RELATIONSHIP: is relationship between dependent & independent variable is not free from the influence of extraneous variable Contd..
M S Sridhar Research Methodology 4 4

Basic Concepts

contd.

RESEARCH HYPOTHESIS: a predictive statement relating independent variable to dependent variable (at least one dependent & one independent variable) EXPERIMENTAL HYPOTHESIS TESTING: independent variable is manipulated in the experiment NON-EXPERIMENTAL HYPOTHESIS TEST: independent variable not manipulated in the experiment EXPERIMENTAL GROUP: Group exposed to some novel or special condition / stimuli / treatments CONTROL GROUP: Group exposed to usual conditions TREATMENTS (STIMULI): Different conditions under which experimental & control groups are put / subjected to EXPERIMENT: Process of examining the truth of a statement, hypothesis (absolute or comparative experiment) EXPERIMENTAL UNITS: Pre-determined plots or blocks, where different treatments are used
M S Sridhar Research Methodology 4 5

The Validity of Experiments

INTERNAL: Do the treatment make a difference in the specific experiments THREATS: 1. History (Events occurring during the time) 2. Maturation (Subjects change) 3. Statistical regression effects (increase unreliability of measuring instruments & extraneous factors) 4. Testing sensitized subject 5. Instrumentation (human observers change in the skills & levels of concentration) 6. Selection (of subjects) Bias, cloudy effect interaction with other factors 7. Experimental mentality - Loss of subjects thro dropouts
M S Sridhar

EXTERNAL: To what populations they can be generalized THREATS 1. Failure to describe and verify independent variable explicitly 2. Lack of representativeness of available & target population 3. Psychological / Hawthorne effects of mere participation 4. Inadequate operationalising of dependent variables (No validity in non-experimental setting) 5. Sensitization of experimental conditions 6. Interaction effects of extraneous factors and experimental treatments Note: Without internal validity an experiment cannot possibly be externally valid. But the converse does not necessarily follow
Research Methodology 4 6

Experimental Design : Types & Steps

Experiments, Quasi-Experiments & Single-Case Research 1. Recall from Part 3 that Ex-post facto research is experimentation in reverse (i.e. start with groups that are already different) 2. Experimental research deliberately controls and manipulates the conditions which determine the events (i.e., making a change in independent variable and observing the effect of that change on dependent variable) 3. Most empirical studies in social and behavioral studies are quasi-experimental (i.e., groups have been constituted by means other than random selection)
M S Sridhar

Steps in Experimental Research

Identify and precisely define problem amenable to experimental methods 2. Formulate hypotheses by predicting relationships and deciding other variables to be excluded. Variables should be measurable, operationally defined and proxy variable must be a valid indicator of the hypothetical variable in which one is interested 3. Select appropriate levels at which to test the independent variable 4. Take account of the population to which results are to be generalized (sample size and sample methods) 5. Overcome validity problems ( appropriate instruments, tests and methods of analysis) 6. Pilot test the experimental procedures 7. During experiment follow tested and agreed-on procedure to the letter (exact timing, meticulous reading and checking of observations) Research Methodology 4
1.

A quasi experimental study of book reservation behaviour

It is quite natural to speculate that the attitude of the users about a book and their reservation behaviour are influenced by the background of those who suggested/ indented the book to the library and by those (in the organisation) interested in it. A quasi experimental study was carried out during October 1985 to see how far the decision of a user to reserve and use a book is influenced by the interest of other colleagues in it. A carefully selected 30 new books were added to the new arrivals display of the library as usual but with a tag on 16 of them with the names (and their sections/ divisions / projects) of the persons who originally indented/ suggested them. The same information about indenters of these books was also provided in the weekly list of additions, copies of which were distributed to all sections, divisions and projects just before the display started. The remaining 14 books fairly matching in subject and nature with the 16 books in the experimental group did not carry any such information about indenters either in the weekly list or on the display. After a week's display, there were 25 reservations for 16 books in the experimental group and 14 reservations for 14 books in control group. The average number of reservations per book during the week for books in the experimental group (where interest of other users/ colleagues are publicised) was 1.56 as against 1.00 for books in the controlled group. Thus a 56% extra reservations are attracted by making users aware of the fact that other users/colleagues are interested in them. However, there is not much difference in the attitude of the users about books suggested/indented by communication stars, supervisors and section/division heads compared to books suggested/indented by others. (Sridhar, M S.
Information behaviour of scientists and engineers. New Delhi: Concept Publishing Company, 1995, p 273-274).
M S Sridhar Research Methodology 4 8

Single-Case Research : ABAB Design

Involves continuous assessment of some aspect of human behaviour over a period of time (i.e., administration of measures on multiple occasions) Involves Intervention effects which are replicated in the same subject (s) over time Changes are made in the experimental conditions to which the client is exposed Examines the effects of an intervention by alternating the base line condition (i.e., phase A and phase B) Provides experimental technique for evaluating the interventions for the individual subject Intervention can be directed towards a particular person or group & replicated over time or across behaviour, situation or persons. Serves as an alternative to usual between-group design

Merits:
1. 2. 3.

Limitations:
1. Ambiguities introduced by trends & variations in baseline phase data 2. Generalizability of results from single case research Example: The effect that a teacher could have upon the disruptive behaviour of a mentally retarded adolescent boy in a special education class whose persistent talking disturbed his fellow classmates ( see diagrams on p 205M S Sridhar 6, Cohen and Manion, 1980) Research Methodology 4

Some Experimental Designs

A. CLASSICAL DESIGN (J.S.MILL) - BASIC DESIGN OF LOGICAL PROOF
1. METHOD OF AGREEMENT

ELEMENTS OF SITUATION X

A B C PRODUCE
ELEMENTS OF SITUATION Y

C D E

PRODUCE

Therefore
M S Sridhar

C Produces
Research Methodology 4

Z
10

Some Experimental Designs - Classical

2. NEGATIVE CANON OF AGREEMENT
The condition of non C is found to be associated with the observation of non Z

Contd...

3. METHOD OF DIFFERENCE (Classical design combined

both 1 & 2) ELEMENTS OF SITUATION X

AB C

PRODUCE

ELEMENTS OF SITUATION Y

AB

NON C

NON

PRODUCE

THEREFORE
M S Sridhar

PRODUCES

Z
11

Research Methodology 4

Some Experimental Designs - Informal

LIMITATIONS OF CLASSICAL DESIGN
1. Difficulty in recognising & controlling variables

Contd.

2. Causal relationship may not be clear C & Z are caused by unknown variables; A & B may also cause Z but absurd by unknown variables; C cause Z only in the presence of unknown variables;does not cause Z simply as an accident 3. Element of time may confuse experimental result in social sciences 4. The quantitative effect is not clear Proportion / magnitude & direction of cause & effect are missing

B. INFORMAL EXPERIMENTAL DESIGNS

Less sophisticated methods based on difference in magnitude 1. Before-and-after without control design 2. After only with control design 3. Before-and-after with control design
contd......
M S Sridhar Research Methodology 4 12

Some Experimental Designs - Formal

Contd.

(1) The one group pretest-post test (A pre-experimental design) Problems of extraneous variables (i.e., internal validity) (2) The pretest-post-test control group design (A true experimental design) Uses two groups with randomisation Problem of interaction effects & clouding effects (conditions incidentally having greater consequences) possibility of pretest sensitization (3) The non-equivalent control group design (a quasi experimental design) matching of elements in the group

C. FORMAL EXPERIMENTAL DESIGNS

Offers relatively more control & use precise statistical procedures for analysis 4. Completely randomised (CR) design 5. Randomised Block (RB) design 6. Latin square (LS) design 7. Factorial design
M S Sridhar Research Methodology 4 13

Some Experimental Designs - Formal Contd...

1. BEFORE-AND-AFTER WITHOUT CONTROL DESIGN
Single Test Group
Dependent variable is measured before & after treatment The effect of treatment is equal to difference in level of phenomenon (y-x) TREATMENT Therefore Treatment Effect = Y-X

Ignores extraneous factors due to passage of time

2.

AFTER-ONLY WITH CONTROL DESIGN

Uses two groups
Treatment introduced to experimental group Dependent variable is then measured in both the groups Treatment effect is equal to difference in measurement (Y-Z) EXPERIENTAL GROUP

TREATMENT

CONTROL GROUP NO TREATMENT Z Therefore Treatment effect = Y-Z (Assumes that two groups are identical)
M S Sridhar Research Methodology 4 14

Some Experimental Designs - Formal

3. BEFORE-AND-AFTER WITH CONTROL DESIGN

Contd.

Uses two groups Dependent variable is measured in both groups for an identical period Treatment introduced in experimental / test group (area) Dependent variable is again measured in both for an identical period TIME PERIOD I TIME PERIOD II

EXPTAL

GROUP

X A

TREATMENT NO TREATMENT

Y Z

CONTROL GROUP

Therefore Treatment Effect = (Y-X) - (Z-A)

- Lack of historical data, time or comparable control group / area are the limitations NOTE: Using more than two groups you can vary treatment intensities on experimental groups
M S Sridhar Research Methodology 4 15

Some Experimental Designs - Formal

Contd.

4. COMPLETELY RANDOMISED DESIGN Simplest possible design Involves principle of replication & principle of randomisation Subjects are randomly assigned to treatments Used when subjects are homogenous (experimental group) Groups could be unequal (replications) Uses one-way ANOVA A. TWO-GROUPS SIMPLE RANDOMISATION DESIGN Sample selected randomly from (one) population and assigned to experimental & control groups Two groups are given different treatments (A & B) on the independent variable EXPTAL
GROUP TREAT AON IND.VAR.

POP

RANDOM SAMPLE SELECTION

RANDOM ASSIGNMENT
CONTROL GROUP TREATBON IND.VAR.

LIMITATION: Extraneous variable is not controlled, e.g.: Treatment differences

M S Sridhar Research Methodology 4 16

Some Experimental Designs - Formal

4.
1.

Contd

2. 3. 4.

COMPLETELY RANDOMISED DESIGN contd B. RANDOM REPLICATION DESIGN Improved by incorporating Replication (i.e., no. of repetitions for each treatment are done) and (i) Provides for controlling differential effects of the extraneous independent variable & (ii) Randomises any individual differences among those conducting treatment Generally equal no. of items are put in each group Variables relating to both population characteristics are assumed to be randomly distributed among two groups An extension of two-group single randamised design
GROUPS 1E IND./ CAUSAL VAR. TREATMENT

2E 3E POP SAMPLE 4E (TREAT) (TREAT) 5C 6C 7C 8C NOTE: More than two treatments are also possible M S Sridhar
Research Methodology 4

RANDOM SELECTION RANDOM POP SAMPLE ASSIGNMENT (STUDY) (STUDY)

17

Some Experimental Designs - Formal

Contd

5. RANDOMISED BLOCK DESIGN i. Improvement over completely randomised design ii. Involves principle of local control iii. Number of subjects in a block = no. of treatments; each treatment appears same no. of times in each block iv. Each treatment appears same number of times in each block v. Subjects divided into groups & subjects are assumed to be homogeneous with respect to selected variable vi. One subject in each block is randomly assigned to each treatment vii. Extraneous factor is held at block level viii. uses two way ANOVA & F test ix. E.g. Three coupons & four blocks/ libraries by size BLOCK TREATMENT IND. LAN 1 IND. LAN 2 IND. LAN 3 Yj
M S Sridhar

Yi
LIB 4 11 8 5 8 MEAN 16 13 9 Yij=12.7
18

LIB 1 20 17 14 17

LIB 2 18 14 10 14

LIB 3 15 13 7 11.7

Research Methodology 4

Some Experimental Designs - Formal

6. LATIN SUQARE DESIGN
i. ii.

Contd

iii.

Widely used in agriculture research Field is divided into as many blocks as there are varieties of fertilisers and then each block is again divided into as many parts as there are varieties of fertilizers in such a way that each variety of fertiliser is used once only in each block. Four observations or measurements made for each fertiliser level & seed leading to 4X4 Matrix (16 measurements or observations) FERTILISER (BLOCKING FACTOR) BLOCKS 1 2 3 4 PARTS SEEDS (Treatment)

i ii iii

A B C

B C D

C D A

D A B

iv

iv. Two extraneous factors like fertility level and seed difference are (used) eliminated M S Sridhar
Research Methodology 4

19

Some Experimental Designs -Formal (LSD)

Contd

Uses two-way ANOVA technique. (As there are no df & mean, square error, cannot be used for 2X2) Assumes no interaction between blocking factor and treatments Requires no. of rows, columns & treatments to be equal For larger size (>10 treatments), rows & columns may not be homogenous & principle of local control is ineffective 1 2 3 4 Yi I II III IV Yj
M S Sridhar

A 8.7 B 7.5 C 14.0 D 11.3 10.375

B 9.2 C 12.7 D 9.2 A 8.7 9.950

C 11.6 D 4.6 A 5.1 B 4.0 6.325

D 9.1 A 7.3 B 6.7 C 12.9 9.000 9.225 Yij 8.9125 8.750 8.025 9.650

Research Methodology 4

20

Some Experimental Designs Formal

7. FACTORIAL DESIGN

contd

To Determine Effects of More Than One Varying Factors (I) Simple - Effect of 2 factors on dependent variable (ii) Complex - Effect of more than 2 factors on dependent variable (multifactor-factorial design) In a 2X2 design two treatments of experimental variable and two levels of control variable lead to four cells. Interaction between treatments & levels can be examined by drawing graphs of study 1 and study 2
STUDY 1 CONTROL (INTELLIGE NCE) LEVEL 1 (LOW) LEVEL 2 (HIGH) COLUMN MEAN Yj TRAING. TREAT A 15.5 35.8 25.6 TRAING. TREAT B 23.3 30.2 26.7 STUDY 2 ROW TRAING. MEAN Yi TREAT A 19.4 33.0 10.4 30.6 20.5
Research Methodology 4

TRAING. TREAT B 20.6 40.4 30.5

ROW MEAN Yi 15.5 35.5

M S Sridhar

21

Some Experimental Designs Formal

60 MEAN SCORE OF DEP.VAR. (ABILITY) 50 40 30 20 10 0 1 (LOW) B A A B STUDY 1 60 50 40 30 B 20 10 0 A

contd

STUDY 2

B A

2 1 2 (HIGH) (LOW) (HIGH) CONTROL LEVEL (INTELLIGENCE) NOTE: There is interaction between treatment and level in STUDY 1, but they are relatively independent in STUDY 2.
M S Sridhar Research Methodology 4 22

References
Anderson, Jonathan, et. al. Thesis and assignment writing. New Delhi: Wiley, 1970. 2. Best, Joel. Damned lies and statistics. California: University of California Press, 2001. 3. Best, Joel. More damned lies and statistics; how numbers confuse public issues. Berkeley: University of California Press, 2004 4. Body, Harper W Jr. et.al. Marketing research: text and cases. Delhi: All India Traveler Bookseller, 1985. 5. Booth, Wayne C, et. al. The craft of research. 2 ed. Chicago: The University of Chicago Press, 2003. 6. Chicago guide to preparing electronic manuscripts: For authors and publishers. Chicago: The University of Chicago Press, 1987. 7. Cohen, Louis and Manion, Lawrence. Research methods in education. London: Routledge, 1980. 8. Goode, William J and Hatt, Paul K. Methods on social research. London; Mc Graw Hill, 1981. 9. Gopal, M.H. An introduction to research procedures in social sciences. Bombay: Asia Publishing House, 1970. 10. Koosis, Donald J. Business statistics. New York: John Wiley,1972. 11. Kothari, C.R. Research methodology: methods and techniques. 2 ed., New Delhi: Vishwaprakashan, 1990.
M S Sridhar Research Methodology 4 23

1.

References
12. 13. 14. 15. 16. 17. 18. 19. 20. 21.
M S Sridhar

contd.

Miller, Jane E. The Chicago guide to writing about numbers. Chicago: the University of Chicago Press, 2004. Rodger, Leslie W. Statistics for marketing. London: Mc-Graw Hill, 1984. Salvatoe, Dominick. Theory and problems of statistics and econometrics (Schaums outline series). New York: McGraw-Hill, 1982. Spiegel, Murray R. Schauims outline of theory and problems of statistics in SI units. Singapore: Mc Graw Hill , 1981. Simpson, I. S. How to interpret statistical data: a guide for librarians and information scientists. London: Library Association, 1990. Slater, Margaret ed. Research method in library and information studies. London: Library Association, 1990. Turabian, Kate L. A manual for writers of term papers, theses, and dissertations. 6 ed. Chicago: The University of Chicago, 1996. Young, Pauline V. Scientific social surveys and research. New Delhi: Prentice-Hall of India Ltd., 1984. Walizer, Michael H and Wienir, Paul L. Research methods and analysis: searching for relationships. New York: Harper & Row, 1978. Williams, Joseph M. Style: towards clarity and grace. Chicago: The University of Chicago Press, 1995.
Research Methodology 4 24

About the Author

Dr. M. S. Sridhar is a post graduate in Mathematics and Business Management and a Doctorate in Library and Information Science. He is in the profession for last 36 years. Since 1978, he is heading the Library and Documentation Division of ISRO Satellite Centre, Bangalore. Earlier he has worked in the libraries of National Aeronautical Laboratory (Bangalore), Indian Institute of Management (Bangalore) and University of Mysore. Dr. Sridhar has published 4 books, 81 research articles, 22 conferences papers, written 19 course materials for BLIS and MLIS, made over 25 seminar presentations and contributed 5 chapters to books. E-mail: sridharmirle@yahoo.com, mirlesridhar@gmail.com, sridhar@isac.gov.in ; Phone: 91-80-25084451; Fax: 91-80-25084476.

M S Sridhar

Research Methodology 4

25