The Scientific Method

• 4 Steps
• Observe a phenomenon

Sources of Knowledge

• Formulate hypotheses – tentative statements of cause & effect • Further observation / experimentation to rule out alternative explanations • Refining & retesting of explanations

• Provides a check on the validity of its conclusions

Hypothesis Formation Library Research
Learning “what is known” about the target behavior Based on Lib. Rsh., propose some “new knowledge”

Research Design
Determine how to obtain the data to test the RH:

The “Research Loop”
• Novel RH:
Draw Conclusions
Decide how your “new knowledge” changes “what is known” about the target behavior

Data Collection
Choose sample, measures, method

• Replication • Convergence

Data Analysis Hypothesis Testing
Based on design properties and statistical results Appropriate statistical analysis

Scientific Explanations
• Accepted because they have the following properties:
• Empirical: based on objective & systematic observation with the senses • Rational: follows rules of logic, consistent with known facts • Testable: able to be verified, disproved • Parsimonious: uses fewest assumptions • General: apply to broad circumstances • Tentative: admittedly imperfect • Rigorously evaluated

Scientific Explanations
• Mechanistic
• Describes the physical components and the chain of cause and effect through which the behavior is produced • Describes how it works (but not why)

• Functional

• Describes something in terms of what is it does, but not how it happens

• Need both for a full understanding of the concept

Research Hypotheses
• The whole research process revolves around hypotheses -- literature reviews to form them, designs to generate data to be analyzed to test them, replication and convergence of them, etc. • A research hypothesis is a tentative explanation or “guess” about the target behavior
• What you will find when you complete your research and data analysis

Basic vs. Applied Research
• Basic research
• Conducted in order to test theories or empirical positions – basic goal is to obtain general information about a phenomenon – not really interested in applying finding to real-world situations

• Applied research
• Purpose is to investigate a problem that occurs in the real

Sources of Research Ideas • Research starts with an interesting
idea • Experience
• Unsystematic Observation
• You read/see/hear/experience something, and ask WHY?

• Systematic Observation
• Planned observation of real world behavior • Previous observations of others (published research) • Own research • Internet research

Scientific Theories
• Theory - a partially verified statement of a scientific relationship that cannot be directly observed (Martin, 1985)

• Theory

Sources of Research Ideas
• A developed set of assumptions and rules about the causes of behavior
• Research can involve applying a theory to a novel situation • Testing competing theories about behavior

Classifying Theories
• Can be classified along 3 dimensions
• Quantitative vs. qualitative • Level of description • Domain

• Quantitative Theory
• Theory expressed in mathematical terms • Formulas used to explain behavior

• Qualitative Theory
• States relationship between variables in verbal rather than mathematical terms

Classifying Theories
Level of description • Descriptive Theory
• Merely describes the relationships among variables – it does not explain the relationship

• Analogical Theory
• Explains the relationships among variables through analogies to something that is well-understood
• E.g., Memory as a computer

Classifying Theories
• Fundamental Theory
• Proposes a new structure to explain the relationships among variables • Not only describes behavior, it explains behavior – not through analogies but through a new structure • Highest level of theory
• E.g., Cognitive dissonance theory

Classifying Theories
The theory’s domain (scope) • The range of situations to which the theory may be applied
• E.g., Cognitive dissonance – applicable to many situations

• Ability to Account for Data • Explanatory Relevance

A Good Theory

• Account for existing data and wellestablished facts within its domain • Offer good grounds for believing that the phenomenon would occur under specified conditions • Theory must define logical links between variables • It must be capable of failing some empirical test

• Testability

A Good Theory
• Prediction of Novel Events
• Should predict phenomena that the theory was not specifically designed to account for but that are within its domain

• Parsimony
• Should explain phenomena within its domain with the fewest possible assumptions and the simplest terms possible

Testing Theories
• Confirmational Strategy
• Look for evidence to confirm predictions from a theory • Important part of theory testing, but has limits • Confirmation does not prove a theory is correct • Confirmation may occur when predictions are too loosely defined

Testing Theories
• Disconfirmational Strategy
• Using a positive research result to disconfirm a theory’s predictions

* Should use both strategies
• Begin with confirmational
• Can it explain the phenomena?

• Move to disconfirmational
• Do unexpected outcomes happen?

• Theory is tested and modified based on outcome of research and then tested again • Cycle of testing and modification continues until theory adequately accounts for behavior • Several alternative explanations can be tested with an experiment
• Some ruled out • New experiment tests remaining alternatives • Continue until only one alternative remains

Strong Inference

Good Questions
• Not too broad
• Operational definitions – describe how a variable will be measured
− Limits generalizability

• Ask IMPORTANT questions
• Clarify theoretical or empirical issues
• Support one hypothesis/theory over another

Good Questions

• Address important practical issues • Probably unimportant if:
• Answer already firmly established • Small effects of no theoretical interest • No reason to believe that 2 variables are related

Purposes of Literature Review
• To fully describe the results from prior research
• What is the “state of the knowledge”?

• To clearly state the purposes of the study • To clearly state the hypotheses, which should follow logically from the literature review
• Purpose is to address some need

• Primary vs. Secondary Sources
• A

Sources of Research Info

primary source includes a full report of a research study, including methodological details
* Primary sources are preferred because secondary sources may be biased or author may not have discussed something you may think is important.

• A secondary source summarizes information from a primary source (e.g. review article)
• Exception – meta-analysis

• Books

• General textbooks or specialized professional publications • Anthologies - assemble papers that an editor feels are important in a field
• Possibility for editor’s bias

• Most useful in early stages of literature search • Should be used with caution - may not undergo rigorous review; info

Sources Cont.
• Scholarly Journals
• Current research and theoretical thinking • Refereed vs. nonrefereed journal * Prefer refereed sources • You can evaluate the quality of a journal by • Consulting Journals in Psychology • Consulting the Social Science Citations Index
• Using the method of authority

Sources Cont.
• Conventions and Professional Meetings
• Gatherings of researchers to present findings • Provide the most up-to-date info • Advantages of attending convention
• Information is at the frontiers of science • Meet others in your field and exchange ideas

Evaluating a Research Article: The Introduction
• Has relevant research been adequately reviewed? • Are assertions supported with the appropriate citations? • Are the purposes of the study clearly stated? • Are the hypotheses clearly stated, and do they flow logically from the info in the introduction?

Evaluating a Research Article: The Method Section
• Was the nature of the subject sample specified? • Does the design of the study allow an adequate test of the hypotheses? • Are there any methodological flaws that might affect the validity of the results? • Is sufficient detail presented to allow one to replicate the study?

Evaluating a Research Article: The Results Section
• Did the statistically significant effects support or refute the hypotheses? • Are the differences reported large or small? • Were the appropriate statistics used? • Do the tables, figures, and text match?

Evaluating a Research Article: The Discussion Section
• Do the conclusions presented match the results reported? • If the author speculates about implications of results, does he or she stray too far from the results reported? • How well do the results mesh with existing theory and empirical data? • Does the author point the way to directions for future research?

Factors Affecting the Quality of Research Information
• Statistical Significance
• Journals typically do not publish findings that do not meet the minimum .05 level of statistical significance • File drawe r p henomenon: Findings that don’t reach significance at .05 end up in the file drawer, perhaps masking true effects

• Causal relationships

• One variable influences another • Unidirectional vs. bidirectional

• Correlational relationships
• Changes in one variable associated with changes in another variable (variables covary) • Can’t say one change causes the other change * Our ability to distinguish between the 2 types of relationships depends on the level of control in the study

Correlational Research
• Form of nonexperimental research • Goals
• Determine whether two variables covary • Establish direction, magnitude, forms of the relationship

• No variables are manipulated
• Instead just observed

Correlational Research
Descriptive Purposes • Determine whether a relationship exists between 2 variables Predictive Purposes • Knowledge of value of one variable can help us predict value of related variable
• Predic to r v aria ble used to predict the value of a Criterion variable

Correlation & Causality
• Correlational research ca nno t be used to establish caus al re lat ion ship s among variables – correlation does not equal causation • Causality: 3 conditions to say that A causes B

• A precedes B • A is related to B • All alternative explanations have been ruled out

• Third variable problem

Obstacles to Causal Explanations
• An unmeasured variable may account for changes in both variables • Examples
• Both observed variables may vary together although they are not directly related • Aggressive video games and aggression – aggressive personality • Ice cream sales and crime – hot weather

• There are ways to statistically account for the third variable problem

• The directionality problem

Obstacles to Causal Explanations
• When a causal relationship does exist, it is often hard to know the direction of the effect • Does A cause B, B cause A, or both? • Examples
• Playing aggressive video games and aggression • Weight and frequency of exercise

• Gathering data in the early stages of research
• Identify possible causal relationships which can then be tested experimentally

When to Use Correlational Research

• Inability to manipulate variables
• Manipulating independent variable may be impossible or unethical

• Relating naturally occurring variables

Experimental Research
• Involves high degree of control over variables
• Allows us to establish causal relationships

• An Independent Variable is manipulated
• A variable whose values are chosen and set by the experimenter • Participants must be exposed to at least two levels of this variable

• A Dependent Variable is measured

Experimental Research
• The variable whose value you observe (the outcome) • The value of the dependent variable depends on the participants’ behavior

• The most basic experiment consists of an experimental and a control group

• The exper im ental gr oup receives the treatment • The cont rol group does not receive the treatment • Assignment to these groups/conditions must be completely random

Experimental Research
• Control over extraneous variables
• Extraneous variables = other variables besides the IV that may affect the DV • May mask any influence of IV on DV or may produce chance differences in DV that are unrelated to IV • To control these possible effects:
• Hold extraneous variables constant • Random assignment of participants to conditions

Research Settings
• Depends on: costs, convenience, ethical considerations, research question • The laboratory setting
• Anywhere other than where the behavior actually occurs – a formal lab, a classroom, etc. • Affords greatest control over extraneous variables

• Simulations

Research Settings
• Attempt to recreate the real world in a lab setting • When attempting to control extraneous variables and interested in generalizing results to real-world settings

• Realism is an issue

• Mundane real ism : How well does a simulation mimic the real world event being simulated • Experi mental real ism : How engaging (psychologically) is the simulation for participants * Experimental realism is more important than mundane realism

Research Settings
• The field setting
• Study conducted in a real world environment – where the behavior actually occurs • Field experiment: Manipulate variables in the field
• Could be nonexperimental, or contain all the characteristics of a true experiment • Easily generalizable to the real world • Little control over confounding

Internal Validity
• Internal validity
• The degree to which your design tests what it was intended to test • The degree to which inferences about whether variations in the independent variable cause variations in the dependent variable are warranted • The extent to which the research design adequately tests your hypothesis

• History

Threats to Internal Validity
• Events may occur between multiple observations • E.g., High profile news story

• Maturation
• Participants may become older or fatigued

• Repeated Testing
• Taking a pretest can affect results of a later test

• Instrumentation

Threats to Internal Validity
• Changes in instrument calibration or observers may change results

• Statistical Regression
• Subjects may be selected based on extreme scores

• Biased Subject Selection
• Subjects may be chosen in a biased fashion

Threats to Internal Validity
• Experimental Mortality
• Differential loss of subjects from groups in a study may occur

• These threats make it impossible to tell if results (changes in DV) are due to changes in the IV

• External validity - the degree to which results can be accurately generalized beyond your sample and research setting
• Other participants, tasks, stimuli, settings & times

External Validity

• Population: Will the results generalize to other persons or animals ?

• Will a study of college students generalize to your target population of “consumers”? • Will a study of chronically depressed patients transfer to those who are acutely depressed?

Components of External Validity • Setting: Will the findings apply to
other settings ?
• Will a laboratory study generalize to what happens in the classroom? • Will a study in a psychiatric hospital generalize to a out-patient clinic? • Will a study conducted in 1965 generalize to today? • Will a study conducted today still be useful 10 years from now? … 5 years from now?

• Societal/Temporal: Will the findings continue to apply?

Components of External Validity
• Task/Stimuli: Will the results generalize to other tasks or stimuli ?
• Usually the participant is “doing something” that directly or indirectly generates the behavior that is being measured • What do I learn about “consumer decision making” from a study that asks participants to select the best “widgit”? • Will research using visual illusions inform us about the perception of everyday objects ?

Threats to External Validity
• Degree of control
• Data obtained in highly controlled lab settings may not generalize to real-world situations

• Sample used
• Results may apply only to subjects representing a unique group (i.e. college students)

Threats to External Validity
• Reactive effects of experimental arrangements
• Participants’ knowledge that they are research subjects may affect results (demand characteristics)

• Multiple treatment interference
• Exposure to early treatments may affect responses to later treatments

• Trade-off characterization

Internal vs. External Validity
• Impossible to promote both internal and external validity within a single study • Steps taken to increase internal validity may decrease external validity and vice versa • Must choose which will be emphasized • Internal validity (control) • External validity

• Internal validity may be more important in basic research, external validity  in applied research
• Which is more important depends on the reasons why the research is being conducted

Internal vs. External Validity

• Internal validity an important precursor
• Without causal interpretability (from

Sign up to vote on this title
UsefulNot useful

Master Your Semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master Your Semester with a Special Offer from Scribd & The New York Times

Cancel anytime.