Research Methods and Program Evaluations Study Guide

Research Methods/Program Evaluation: Study Guide CHD
STUDY GUIDE: RESEARCH METHODS & PROGRAM

EVALUATIONS
Chapter 1
M/C Question 1
High quality research has which of the following characteristics?
 based on the work of others

M/C Question 2
How are research questions most often described?
 may arise from our everyday life experiences

M/C Question 3
Which of the following best describes a hypothesis?
 posits a clear relationship between different factors

M/C Question 4
What can always be a result of a well-run study?
 providing valuable information

M/C Question 5
Experimental methods can typically be described by which of the following?
 cause-and-effect

M/C Question 6
Michael hands out a survey to find out the average age and schooling level of his class. What type of
research is this?
 descriptive
M/C Question 7
Correlational research provides information about which of the following?
 the relationship between two or more things
M/C Question 8
What type of research do you use to determine the causal relationship between variables?
 experimental research

M/C Question 9
Identify and select the correct order of steps in scientific inquiry.
4
 formulating a hypothesis, collecting relevant information, testing the hypothesis, working

with the hypothesis

M/C Question 10
Qualitative research can be defined by which of the following?
 examines human behavior in the contexts in which they occur
M/C Question 11
Which of the following is true of the scientific method of inquiry?
 systematic process that is used to answer questions

M/C Question 12
Which of the following is a characteristic of experimental research?
 high level of control over variables

M/C Question 13
Nonexperimental research methods consist of which of the following?
 only describe characteristics of existing phenomenon

M/C Question 14
What is another name for research that occurs “post hoc”?
 quasi-experimental
M/C Question 15
Jacob is interested to see if there is a relationship between student absentee rate and national test scores.
Which would be the appropriate research method to use?
 correlational

M/C Question 16
Emily’s school has implemented an early intervention program for at-risk students. In order to monitor
its effectiveness on student performance, she would use which research method?
 quasi-experimental
M/C Question 17
Hannah assigns children to different teaching method groups and tests their math performance after six
weeks. This is an example of what type of research?
 experimental

5
M/C Question 18
In true experimental designs, what does research attempt to isolate and control?
 factors not tested that could be responsible for any effects except the ones being tested
M/C Question 19
What is the major difference between applied and basic research?
 basic research has no immediate application
M/C Question 20
What is a characteristic of quasi-experimental research?
 assignment to groups based on some pre-determined characteristic

M/C Question 21
Which of the following methods provides information about past trends?
 historical
M/C Question 22
What is another term for basic research?
 pure research

M/C Question 23
What is one characteristic of poor research?
 a political activity
M/C Question 24
How many steps are included in the research process?
 8

M/C Question 25
What is the most important step in the research process?
 asking the question

M/C Question 26
Madison is interested in how many of the children in her school come from single-parent, intact, and
blended families. What method of research would she use?
 descriptive

M/C Question 27
Matthew thinks that there is a relationship between parenting style and self-esteem in children. Which
research method should he use?
6
 correlational
M/C Question 28
Samantha is interested in studying the relationship between gender differences and verbal ability. This is
an example of what type of research?
 correlational

M/C Question 29
Ann divides her classes into two groups—a treatment group and a non-treatment group—to test whether
her new method of teaching subtraction is better than her old method. This is an example of what type of
research?
 experimental research

M/C Question 30
The scientific method is described in the textbook as ________.
 a standard sequence of steps in formulating and answering a question

CHAPTER 2
M/C Question 1
When variables compete to explain the same effects, what are they sometimes called?
 confounding
M/C Question 2
In a study of the effect of the amount TV viewing on children’s aggressiveness, amount of TV viewing
would be what type of variable?
 independent variable

M/C Question 3
In a study of the effect of pleasure reading on vocabulary level, the vocabulary level is considered what
type of variable?
 dependent variable

M/C Question 4
Factorial designs are experiments that can best be defined by which of these statements?
 have more than one independent variable

M/C Question 5
Which type of designs includes more than one independent variable?
7
 factorial designs

M/C Question 6
What is another term for the independent variable?
 treatment variable

M/C Question 7
How is the independent variable different from the dependent variable?
 the independent variable is manipulated during the experiment to understand the
effects of this manipulation on the dependent variable

M/C Question 8
A control variable can best be defined by which of the following statements?
 influences the dependent variable

M/C Question 9
What type of variable has an unpredictable impact on the dependent variable?
 extraneous variable

M/C Question 10
What type of variable masks the true relationship between the independent and dependent variables?
 moderator variable

M/C Question 11
The null hypothesis represents which of the following statements?
 no relationship between the variables under study

M/C Question 12
Which type of study may NOT have an implied null hypothesis?
 descriptive

M/C Question 13
Why is a null hypothesis said to be implied?
 It’s never directly tested.

M/C Question 14
Which is the key criterion for selecting a dependent variable?
 sensitivity of the variable to changes in the independent variable

8
M/C Question 15
Which of the following best describes independent variables?
 manipulated to assess the effect of the treatment

M/C Question 16
The interpretation of “differences are significant” means that the differences found
are_________________.
 probably not due to chance
M/C Question 17
If you read that a study’s finding was significant at the p < .05 level, you could conclude that there is a
less than 5% probability that _____________________________.
 the results were due to chance
M/C Question 18
Why is it important to choose a representative sample of the population?
 maximize generalizability of results

M/C Question 19
There will be no relationship between children's time in day care and later academic achievement. This
is an example of which of the following?
 a null hypothesis
M/C Question 20
The best dependent variable is defined by which of the following characteristics?
 sensitive to changes in the treatment

M/C Question 21
A research hypothesis ________________.
 is represented by Roman symbols

M/C Question 22
The significance level reported in a research study can be explained by which of the following?
 risk associated with not being 100% confident the difference is due to the treatment
M/C Question 23
Which of the following is another term for dependent variable?
 criterion variable

9
M/C Question 24
Dr. Flemming found a statistically significant relationship between gender and aggressive playground
activity. Dr. Flemming can conclude which of the following?
 The effect of gender on aggressive playground activity is likely not due to chance.
M/C Question 25
What is the term often associated with the random variability introduced into every study as a function
of the group of subjects participating, as well as many other unforeseen factors?
 chance
M/C Question 26
A nondirectional research hypothesis is similar to a directional hypothesis in what way?
 both reflect differences between groups

M/C Question 27
The null hypothesis always refers to the __________, whereas the research hypothesis always refers to
the _______________.
 population, sample
M/C Question 28
Which of the following is a characteristic of a well-written research hypothesis?
 is testable

M/C Question 29
In testing whether riding a bicycle at least 30 minutes each day reduces weight, what would be the
independent variable?
 amount of time bicycling each day

M/C Question 30
The researcher notices that the vocabulary level is also associated with intelligence level, so he
determines that the research design must hold the level of intelligence constant to get a good idea of the
relationship between pleasure reading and vocabulary level. Intelligence here is an example of what type
of variable?
 control variable
CHAPTER 3A
M/C Question 1
Which of the following is a fatal error in research problem selection?
 sticking with the first idea that comes to mind
10

M/C Question 2
Which of the following is a good way to find a research topic?
a) personal experience
b) getting an idea from your advisor
c) looking for the next step in the research process
d) all of the above

M/C Question 3
Why is a clear research question useful?
 it makes more efficient use of your time when searching for references

M/C Question 4
Which type of source is the “meat and potatoes” of the literature review?
 primary sources
M/C Question 5
Which type of source is NOT typically included in a literature review?
 general sources
M/C Question 6
What use do secondary sources provide?
a) finding a scholarly summary of research
b) finding further sources of research
c) sometimes cited in your literature review
d) all of the above

M/C Question 7
What are the most important primary sources of information about a topic?
 journals

M/C Question 8
What is the average rejection rate for the top journals?
 80%
M/C Question 9
A research abstract contains which of the following?a one-paragraph summary of a journal article
 a one-paragraph summary of a journal article
11
M/C Question 10
Provide an example of a source published annually that provides information about the United States.
 The Statistical Abstract of the United States

M/C Question 11
What is one of the advantages of using online search services?
 you are not likely to miss very much

M/C Question 12
Which question is NOT used to judge a research study?
 Outline: Did the researcher use an outline for the literature review?
M/C Question 13
How would you use the Google phonebook to conduct research?
 by using the name and university of a researcher to find contact information

M/C Question 14
How would you define the Internet?
 network of networks
M/C Question 15
What is the most common use of the Internet?
 e-mail

M/C Question 16
What is a place where information can be posted and shared among Internet users called?
 electronic newsgroup

M/C Question 17
A well-written hypothesis:
 reflects a theory or body of literature upon which it is based

M/C Question 18
What is the first step for preparing a successful research proposal?
 reviewing the literature
M/C Question 19
What is the most popular search engine?
 Google

12
M/C Question 20
Which is the most comprehensive guide to general literature?
 The Reader's Guide, Full Text Mega Edition
M/C Question 21
Which of the following is a general source?
 Lexis/Nexis Academic

M/C Question 22
Which source should NOT be cited in a formal literature review?
 Newsweek

M/C Question 23
Which source allows the researcher to locate articles which cite a specific article?
 Social Sciences Citation Index

M/C Question 24
Peer review means that of a journal submission usually requires a minimum of how many reviewers?
 3
M/C Question 25
What might a bibliographic database program help you to do?
 organize your references and generate a final list for use in your manuscript

M/C Question 26
Which service allows you to secure a copy of a journal that your library does not carry, but another
library does carry?
 Interlibrary Loan System

M/C Question 27
What is the best way to find a particular home page?
 use a search engine

M/C Question 28
Once information is collected, what is the first step in writing a literature review?
 read other literature reviews

M/C Question 29
What statement best defines the term meta-search engine?
 a search engine that provides results of exploring many search engines all at once
13
M/C Question 30
Which is NOT a hint to help you write your literature review?
 make sure it is testable
CHAPTER 3B
M/C Question 1
Forcing someone to participate in a study is an example of what ethical issue/concern?
 coercion

M/C Question 2
The top of the to-do list when it comes to participating in the research process is the requirement to:
 Have the human subjects committee review and approve the procedures.

M/C Question 3
What is the primary role of the institutional review board?
 Evaluate the risk to participants.

M/C Question 4
What is another name for participants?
 subjects

M/C Question 5
What should be the focus of research studies?
 risks and benefits

M/C Question 6
Who are the most commonly used population in research studies?
 college students

M/C Question 7
What is the most important requirement in a research study?
 The signed informed consent form.

M/C Question 8
What is not contained in the letter sent to potential participants?
 Once involved, that participants have to remain in the study.

14
M/C Question 9
What is the best way to maintain confidentiality in a research study?
 Minimize the number of people who see or handle the data.

M/C Question 10
What is not true in the debriefing process of a research study?
 Provide participants with detailed information on the study’s intent.

M/C Question 11
What is the last ethical principle in conducting research?
 sharing benefits
M/C Question 12
Confidentiality refers to:
 Anything learned is not shared.

M/C Question 13
What is the least observed ethical principle in conducting research?
 sharing benefits

M/C Question 14
What is the best definition of an institutional review board?
 a group of people found in most public institutions and private agencies who render a
judgment as to whether experiments carried out in their facilities are free from risk
M/C Question 15
What does the ethical principle of “sharing benefits” mean?
 all groups involved in a study should eventually be given an opportunity to receive any
treatments that were found to be effective in the study

True or False

T/F Question 16
Because the usefulness of the information is so great, it is considered ethical to secretly record
conversations between therapists and their clients.
 False

M/C Question 17
Which is NOT a component of informed consent?
 a computer simulation of the data
15

M/C Question 18
Which of the following information should be provided in an informed consent form for any study
involving human participants?
 the procedures involved in the study
M/C Question 19
The Tuskegee Syphilis Study is often used as an example of:
 a study that did not consider ethical considerations
T/F Question 20
You can quote and analyze online information without asking for permission as long as the information
is officially and publicly archived, not password protected and nothing on the site prohibits the use of
this information.
 True
M/C Question 21
Which of the following is NOT considered a basic principle of ethical behavior in research?
 a well-stated hypothesis

M/C Question 22
When must a researcher be sure to debrief participants in an experiment?
 debriefing must be done in experiments where one group of participants is asked to do
something for a reason other than that which they are told

M/C Question 23
What should be done if a treatment is suspected to have adverse effects?
 Use a small well-informed sample.

M/C Question 24
What is the first step a researcher takes to ensure that ethical principles are maintained?
 A computer simulation in which data are constructed and subjected to the effects of
various treatments.

M/C Question 25
What is the third step a researcher takes to ensure that ethical principles are maintained?
 If the treatment involves risk, be absolutely sure that the risks are clear to the
participants and other parties.

16
M/C Question 26
What is the second step a researcher takes to ensure that ethical principles are maintained?
 When the treatment is deemed harmful, try to locate a population that has already been
exposed to the treatment.

M/C Question 27
What is the last step a researcher takes to ensure that ethical principles are maintained?
 Have an institutional review board assess the information regarding the research study
and approve the study.
M/C Question 28
Who passed the National Research Act?
 U.S. Congress
M/C Question 29
Which agency requires that all people conducting research on human subjects have training in research
ethics?
 National Institute of Health

M/C Question 30
Who released the Belmont Report?
 National Commission
CHAPTER 4
M/C Question 1
In which sampling technique does each member of the population have an equal and independent chance
of being selected to be part of the sample?
 simple random sampling

M/C Question 2
In which sampling technique does every kth name on the list get selected?
 systematic sampling
M/C Question 3
Which sampling technique can assure that the profile of the sample matches the profile of the
population?
 stratified sampling

M/C Question 4
In which sampling technique are units of individuals selected rather than the individuals themselves?
17
 cluster sampling
M/C Question 5
What type of sampling is usually the easiest to do?
 convenience sampling

M/C Question 6
What type of sampling selects people with the desired characteristics until the desired number is
reached?
 quota sampling
M/C Question 7
Which of the following statements is a definition of sampling error?
 the difference in characteristics between the sample and the population sampled from
M/C Question 8
In a study focused on group difference, what is the general rule of thumb regarding the number of
individuals per group?
 30

M/C Question 9
When might you need a larger sample to represent the population?
 the amount of variability within groups is high
 the difference between the two groups is great
 both a and c

M/C Question 10
If you are mailing out surveys for your research, you should increase your sample size by what percent?
 40% - 50%

M/C Question 11
What is the purpose of a table of random numbers?
 randomly assign people to groups

M/C Question 12
What does generalizability refers to?
 results that may be applied to different populations in different settings
M/C Question 13
A table of random numbers _________________________________.
18
 can be used to create a stratified sample

M/C Question 14
What is the most unbiased tool that can be used in random selection?
 table of random numbers

M/C Question 15
When is a stratified sample an appropriate sampling technique?
 individuals in a population are not equal

M/C Question 16
Which of the following is NOT a probability sampling method?
 quota sampling

M/C Question 17
Which of the following statements is true of systematic sampling?
 not all members of the population have an equal chance to be selected

M/C Question 18
Which one of the following is a characteristic of probability sampling?
 Subjects are determined by chance factors.

M/C Question 19
Which sampling technique is the most time-consuming?
 simple random sampling

M/C Question 20
Which sampling technique should be used when the population's members differ from one another on
several important characteristics?
 stratified random sampling
M/C Question 21
Which of the following explains the use of probability sampling?
 the likelihood of any one member of a population being selected is known

M/C Question 22
Estimating sample size requires that you know which of the following?
 overall differences estimated between groups

19
M/C Question 23
What is a disadvantage of quota sampling?
 The degree of generalizability is questionable.

M/C Question 24
How might a researcher minimize sampling error?
 randomly assigning individuals to groups
M/C Question 25
Which of the following selection processes is the most random?
 using a table of random numbers to select from the list of possible participants

M/C Question 26
Sally selects classrooms of first graders to study the effects of teaching styles on math achievement. This
is an example of ________________________.
 cluster sampling

M/C Question 27
Susan calls every 10th name in the student phone book to ask them to participate in her study. This is an
example of ________________________.
 stratified sampling

M/C Question 28
Sampling error can be defined by which of the following statements?
 can be minimized by using simple random sampling

M/C Question 29
Which of the following statements is true with respect to sample size?
 the greater the number of subgroups of interest to the study, the larger the sample

M/C Question 30
Which of these statements is true regarding cluster sampling?
 a great time-saver compared to simple random sampling
CHAPTER 5
M/C Question 1
Which is considered the least precise level of measurement?
 nominal

20
M/C Question 2
Which is considered the most precise level of measurement?
 ratio

M/C Question 3
Which of the following variables is an example of the nominal level of measurement?
 gender

M/C Question 4
Which variable would be considered to be measured at the ordinal level?
 racing results
M/C Question 5
The interval level of measurement would include which of the following?
 IQ score

M/C Question 6
The ratio level of measurement would include which of the following?
 amount of books owned

M/C Question 7
Which level of measurement conveys the most information?
 ratio

M/C Question 8
Which of the following is NOT an example of reliability?
 the test doing what it is supposed to do

M/C Question 9
A score that you would actually record is a(n) _______________.
 observed score

M/C Question 10
What are the two components of error scores?
 trait error and method error

M/C Question 11
A measure of how stable a test is over time is an example of which of the following?
 test-retest reliability
21
M/C Question 12
_________________ describes a measure used to compare two different tests with the same group of
participants to see how closely correlated the two sets of scores are with each other.
 Parallel-forms reliability

M/C Question 13
Which coefficient reflects the occurrence of perfect reliability?
 + 1.00

M/C Question 14
Two trained professionals observe the behavior of children in a classroom. They each rate observed
behaviors using the same form and the number of items that were rated the same is calculated. This is an
example of which type of reliability?
 inter-rater reliability
M/C Question 15
If the measures associated with a test are said to be consistent, you might conclude the measure is
______________________.
 reliable

M/C Question 16
When we calculate reliability, we know the observed score. What are the two unknown components of
the reliability equation?
 true and error scores

M/C Question 17
Inter-rater reliability measures consistency _________________.
 from rater to rater

M/C Question 18
What would the inter-rater reliability be for a 50-item measure in which the number of agreements
between Rater 1 and Rater 2 was 45?
 0.90

M/C Question 19
Which of the following would be considered a continuous variable?
 length
22
M/C Question 20
If the reason for the difference between the true and observed score is a characteristic of the person
taking the test, the resulting decrease in reliability will be attributed to ____________.
 trait error
M/C Question 21
Which of the following is concerned with monitoring estimates of present performance and predictions
of future performance?
 criterion validity

M/C Question 22
A measure of how well a test estimates a criterion is a concern of _________________.
 concurrent validity

M/C Question 23
Dr. Baylor is concerned that the items on his test really do adequately represent the universe of items
from which he chose. Dr. Baylor's concern is one of ________________________.
 content validity

M/C Question 24
Which of the following statements is false?
a) Validity refers to the results of a test not the test itself.
b) Validity progresses from low to high degrees rather than all or none.
c) A test can lack validity and still be a reliable measure.
d) A test can lack reliability and still be a valid measure.

M/C Question 25
What is the name of the individual credited with devising levels of measurement in which different
measurement outcomes can be classified?
 Stevens

M/C Question 26
What is one way to obtain a criterion validity estimate?
 select a criterion and correlate scores on the test with scores on the criterion
M/C Question 27
What levels of measurement define most variables in behavioral and social science research?
 nominal and ordinal

M/C Question 28
Which one of the following is NOT an important criterion for increasing reliability?
23
 use subjective scoring procedures

M/C Question 29
Which of the following is an example of method error?
 bias in grading

M/C Question 30
Which is the least frequently established form of reliability?
 internal consistency
CHAPTER 7

M/C Question 1
Which of the following is NOT a step in data collection?
 the selection of a sample

M/C Question 2
Which is a pitfall of using an optical scanner?
 It is expensive.

M/C Question 3
What is the best code to use when coding gender?
 1, 2

M/C Question 4
Which is NOT one of the ten commandments of data collection?
 destroy original data when your study is finished

M/C Question 5
Which of the following measures is NOT a measure of central tendency?
 mass
 mean
 median
 mode
M/C Question 6
What is the mean of 5, 10, 15, and 20?
 12.5

24
M/C Question 7
What is the median of 2, 2, 2, 4, 6, 6, 6, and 6?
 5

M/C Question 8
What is the mode of 1, 1, 3, 4, and 6?
 1

M/C Question 9
What is the range of 5, 8, 10, 14, and 11?
 9
M/C Question 10
What is the average amount that each score varies from the mean of the set of scores?
 the standard deviation
M/C Question 11
What is another way the mode can be described?
 occurs most frequently

M/C Question 12
What do you call research data that has not been analyzed and is still in an unorganized format?
 raw data

M/C Question 13
When a normal distribution is represented by a histogram, where would most scores fall?
 in the middle of the distribution

M/C Question 14
Which measure of central tendency provides the most information?
 mean
M/C Question 15
Which measure of central tendency is most often used when the data has extreme scores?
 median
M/C Question 16
How can descriptive statistics be defined?
 describing elements of a distribution

25
M/C Question 17
Which measure of variability is the most direct way to measure how dispersed a set of scores is?
 range
M/C Question 18
When computing the standard deviation, what does it mean when the sum of the deviation about the
mean is NOT 0?
 The mean of the data set was computed incorrectly.

M/C Question 19
Which measure of variability describes the average deviation of each score from the mean?
 mean

M/C Question 20
What measure would be used to compute the average gender of subjects?
 mode

M/C Question 21
Which measure would be used to determine the average age of college freshmen?
a) mean
M/C Question 22
Which measure would be used to describe the average class ranking of algebra students?
 median
M/C Question 23
Which measure would be used to describe how far a test score is from the average test score?
 standard deviation
M/C Question 24
In a set of scores with a mean of 50 and a standard deviation of 5, what raw score is represented by a z
score of 1.00?
 55
M/C Question 25
In a set of scores with a mean of 100 and a standard deviation of 15, what raw score is represented by a z
score of 2.00?
 130

M/C Question 26
In a normal distribution, what percentage of scores fall between the mean and a z score of +1.00?
26
 34%

M/C Question 27
What is another phrase used to describe a bell-shaped curve?
 normal curve

M/C Question 28
You are interviewing classmates to find out their political party affiliation. When you chose the measure
of central tendency to use in presenting your results, which should you choose?
 median
M/C Question 29
When you have ordinal data (such as class rank), which is the best measure of central tendency to use?
 median

M/C Question 30
When you have interval or ratio data (such as age in years) what is typically the best measure of central
tendency to use?
 mean
CHAPTER 8
M/C Question 1
What does a level of significance of .05 mean?
 there is a 5% chance of Type I error

M/C Question 2
What is the correct order of steps in testing a hypothesis?
 statement of null hypothesis, setting level of risk, selection of test statistic
M/C Question 3
What is the statistical test for examining the significant difference between two means from two
unrelated groups?
 t-test for independent means

M/C Question 4
If the two groups you are comparing are related to each other, what test should you use?
 t-test for dependent means
27
M/C Question 5
When examining whether group differences occur on more than one dependent variable, what should be
used?
 MANOVA

M/C Question 6
When examining the significance of the relationship between two variables, what test should you use?
 t-test for the correlation coefficient

M/C Question 7
To compare findings from many different studies that use the same dependent variable, what technique
is used?
 meta-analysis

M/C Question 8
What technique reduces the number of variables in a construct?
 factor analysis

M/C Question 9
Which of the following statements is true?
 Meaningfulness is better than significance.

M/C Question 10
When investigating a relationship between two variables, what is the most acceptable initial explanation
of the relationship?
 chance

M/C Question 11
When a researcher draws conclusions about a population based on the results of a test on a sample, he or
she is most likely using which of the following?
 inferential statistics
M/C Question 12
What is the crucial element in making an accurate inference?
 obtaining a representative sample of the population

M/C Question 13
Which of the following states that regardless of the shape of the population, the means of all the samples
selected will be normally distributed?
28
 central limit theorem

M/C Question 14
One of the keys to successful operation of the central limit theorem is that the sample size must be
___________________.
 greater than 30

M/C Question 15
If Helen mistakenly accepts her null hypothesis when it should be rejected, what has occurred?
 Type II error
M/C Question 16
How can Type II errors be reduced?
 increasing sample size
M/C Question 17
The level of significance is similar to which of the following?
 a Type I error

M/C Question 18
What does the central limit theorem enable researchers to do?
 generalize the results from a sample to a population without knowing the exact nature of
the population’s distribution

M/C Question 19
Which of the following has the largest potential to decrease the accuracy of an inference made to a
population based on a sample?
 sampling error
M/C Question 20
The critical value is__________________________________.
 the value needed to reject the null hypothesis
M/C Question 21
The critical value decreases when_______________________________.
 the sample size increases
M/C Question 22
What assumption forms the basis for inference?
 A researcher can select a sample that is similar to the population from which it came.
29
M/C Question 23
If Dr. Robinson rejects the null hypothesis after observing a test statistic which exceeds the critical value
at the .05 level, there is_____________________________________________.
 a 5% chance that the null hypothesis is actually true

M/C Question 24
The minimum or maximum value that one would expect the test statistic to yield if the null hypothesis is
true is an example of which of the following?
 critical value

M/C Question 25
What can you conclude if the obtained value of a test statistic exceeds the critical value?
 The null hypothesis can be rejected.

M/C Question 26
In order to determine the critical value, which two pieces of information are needed?
 degrees of freedom and level of significance

M/C Question 27
If Professor Columbo is interested in the general trend of research findings regarding infant visual
attention and childhood intelligence, what technique would be most amenable to his inquiry?
 Meta-analysis

M/C Question 28
What is the standard metric used in most meta-analytic studies called?
 effect size

M/C Question 29
Which of the following statements regarding statistical significance is FALSE?
 When your results are found to be statistically significant, you can conclude that they will
have a meaningful effect in the clinic.
M/C Question 30
The first step for conducting a meta-analysis is ____________.
 collect as many studies as possible (or at least a representative group)
30
CHAPTER 9
M/C Question 1
If you are trying to describe current events, what type of research are you conducting?
 descriptive research

M/C Question 2
What are unstructured questions also known as?
 open-ended questions

M/C Question 3
What is the second step in developing an interview?
 select an appropriate sample

M/C Question 5
When conducting an interview, what should you be careful NOT to do?
 begin the interview cold

M/C Question 5
Which is the weakest correlation?
a) + 0.76
b) + 0.21
c) - 0.01
d) - 0.88

M/C Question 6
What would a correlation of .35 be called?
a) weak
M/C Question 7
What type of research tool can be either informal or highly structured?
 interview

M/C Question 8
What is the basic tool of survey research?
 interview

31
M/C Question 9
Which technique establishes validity in survey research?
 interviewing respondents twice

M/C Question 10
Which of the following is the potential weakness of survey interviewing?
 interviewer bias

M/C Question 11
Which of the following is an example of a closed-ended interview question?
 Did you vote for Gerald Ford?

M/C Question 12
Samantha wanted to investigate the relationship between the amount of time her students engaged in
outside leisure reading and their participation in class discussions. Her study would study be classified
as _________________.
 correlational

M/C Question 13
Which correlation coefficient indicates the strongest relationship?
a) 0.00
b) +0.50
c) -0.70
d) +0.60

M/C Question 14
What is the best way to begin an interview?
 Warm-up with some general conversation.

M/C Question 15
If your research does NOT include a treatment or control group, what type of research are you
conducting?
 descriptive research

M/C Question 16
Which interview question is open-ended?
 How do you feel about tests?

M/C Question 17
What is the third step of developing an interview?
32
 developing the questions
M/C Question 18
Which of the following is NOT an advantage of interviews?
 They are relatively inexpensive as they do not require much time.

M/C Question 19
Which of the following is NOT a type of survey research?
 observation

M/C Question 20
Which of the following statements regarding correlational research is false?
 Correlational research is a very effective means of establishing casual relationships.
M/C Question 21
Which of the following shows an example of how a correlation coefficient may be written?
 rxy = 0.53

M/C Question 22
What is the most frequently used measure of relationships?
 Pearson product moment
M/C Question 23
Of the following pieces of information, which is NOT needed to compute a correlation coefficient?
 the level of significance

M/C Question 24
What would a correlation of 0.70 be considered?
 strong

M/C Question 25
In a study, the correlation coefficient for W and X is -0.25. The correlation coefficient for X and Y is
0.20. The coefficient of determination for X and Z is 0.16. The coefficient of alienation for Y and Z is
0.90. Which pair of variables has the highest amount of variance accounted for?
 X and Z
M/C Question 26
Which of the following statements is false?
a) A correlation of .8 may be interpreted as very strong.
b) The higher the correlation is, the larger the “jump” in explained variance.
33
c) A scattergram is a visual representation of a correlation between two variables.

d) The absolute value of the correlation coefficient is more important than the + or – sign.
M/C Question 27
Survey research would be most appropriate for which of the following research questions?
a) parents’ attitudes towards sex education in schools

M/C Question 28
If you square a correlation coefficient and subtract this number from 1.00, what is the result?
 the coefficient of alienation
 the amount of variance NOT explained by the variance in each variable
 both a and c

M/C Question 29
Which of the following could be a potential problem in conducting survey research?
 response rate

M/C Question 30
Which of the following examples shows a positive or direct correlation?
 A study showing the correlation between ice cream sales and heat is +.78.
CHAPTER 10

M/C Question 1
What type of research is sometimes called “research without the numbers”?
 qualitative

M/C Question 2
Which would NOT be considered a source of information for qualitative research?
 tests

M/C Question 3
What “objects” are open to interpretation?
 physical artifacts

M/C Question 4
Which is a method used to study an individual as intensely as possible?
 case study
34

M/C Question 5
How would you describe an ethnography?
 explores a culture
M/C Question 6
What are original artifacts considered?
 primary sources

M/C Question 7
Which of the following is better to use?
 primary sources

M/C Question 8
What is another term for “authenticity”?
 validity
M/C Question 9
What is another term for historical research?
 historiography
M/C Question 10
What is the first step in conducting historical research?
 defining a topic or problem
M/C Question 11
What is the second step in conducting historical research?
 formulating a hypothesis

M/C Question 12
The fourth step in conducting historical research is ___________________________.
 evaluating authenticity and accuracy of evidence

M/C Question 13
The fifth step in conducting historical research is ______________________________.
 integrating data

M/C Question 14
What is data that is derived from sources at least once removed from the original event?
 secondary sources
35

M/C Question 15
What is the criteria of authenticity also known as?
 external criticism

M/C Question 16
Which of the following statements about qualitative research would be considered false?
 It is much less complex than quantitative research because numbers are not extensively
involved.

M/C Question 17
Which of the following questions would NOT be considered to be an external criticism?
a) “Was this document written when claimed?”
b) “Was this document found where one might expect it?”
 “Do the numbers listed in this document seem plausible based on other sources?”
c) “Is this document truly written by the person it appears to be written by?”
M/C Question 18
What is another term for accuracy?
 internal criticism

M/C Question 19
Which of the following is NOT a characteristic of participant observation?
a) It is a primary method used in historical research.
M/C Question 20
Which of the following is NOT a qualitative computer research tool?
 SPSS

M/C Question 21
You have a diary thought to be written by a confederate soldier during the Civil War. A handwriting
expert believes the writing matches other samples that have already been traced to this man. A carbon-
dating expert can verify the age of the document. The diary gives an account of a battle, but you have
other documents that contradict some of the events as written. Based on this information, what should
you question about the diary?
 accuracy

M/C Question 22
Which type of research has only one subject?
 case study
36

M/C Question 23
Which of the following is NOT characteristic of a case study?
 It is a quick method of obtaining information.

M/C Question 24
How might case studies be superior to other methods of research?
 They allow for very detailed examinations of data.
M/C Question 25
What is one of the limitations of the case-study method?
 cause-and-effect links cannot be established

M/C Question 26
What is the third step in conducting historical research?
 gathering data

M/C Question 27
The last step in conducting historical research is __________________.
 interpreting the results

M/C Question 28
In what situation would the researcher be adjacent to the environment, but NOT actively involved in the
process being studied?
 direct observation

M/C Question 29
Which is NOT a main function of focus groups?
 in-depth information gathering on one participant

M/C Question 30
Which of the following is NOT characteristic of ethnographic research?
 Ethnographers develop a research question prior to initiating their research methods just as
empirical researchers do.
CHAPTER 11
M/C Question 1
Which of the following is NOT one of the general categories of experimental design?
 pseudo-experimental
37
M/C Question 2
In which of the following is randomization NOT possible?
 post-testing of participants
M/C Question 3
Which design uses no control group and no random selection?
 pre-experimental

M/C Question 4
What is an example of a true experimental design?
 pretest posttest control group design

M/C Question 5
What term is synonymous with internal validity?
 control

M/C Question 6
Which is a threat to internal validity?
 maturation

M/C Question 7
Which is a threat to external validity?
 pretest sensitization

M/C Question 8
Dr. Evans was surprised that many of his participants scored much lower than expected on his pretest.
When he observed a significant increase in scores on the posttest, he was cautious in attributing the
effect to his treatment alone. Which of the following threats to internal validity was Dr. Evans concerned
about?
 regression

M/C Question 9
Which will NOT help control for extraneous variables?
 using a control group
M/C Question 10
What does the term ANCOVA mean?
 analysis of covariance
M/C Question 11
Experimental research methods are used _______________________.
38
 to describe some aspect of a certain variable

M/C Question 12
In an experimental research study, which group does NOT receive treatment?
 control group

M/C Question 13
What is the proper order of the following designs if they are to be listed from the one with the least
amount of control over variables to the most?
 one-group pretest posttest, quasi-experimental, pretest posttest control group, Solomon
four-group

M/C Question 14
True experimental designs are NOT characterized by ______________________________.
 descriptive statistics

M/C Question 15
Most threats to ____________________ are taken care of by the experiment’s design, while most
threats to _____________________ need to be taken care of by the designer of the experiment.
 internal validity, external validity
M/C Question 16
Jim was coming down with a cold and decided to take his friend's advice and drink hot lemon juice with
a dash of salt to help his throat. Jim's experimentation can be categorized as what type of research?
 one-shot case study

M/C Question 17
The experimental design that uses one group, a treatment, and a posttest is called ________________.
 one-shot case study design
M/C Question 18
The experimental design that uses one group and a pretest and posttest is called
____________________.
 one-group pretest posttest design

M/C Question 19
To what does the term internal validity refer?
the degree to which the results of a study can be attributed to the manipulation of the
independent variable

39
M/C Question 20
Susan has twelve subjects drop out of her study. Which threat to internal validity must she consider?
 mortality

M/C Question 21
Many researchers devise “sham” treatments for their control groups to undergo, meaning that the control
group interacts with researchers in some way that is not thought to impact the dependent variable, or the
outcome. This is done to weaken the argument that the treatment group improvement may have occurred
in part simply due to the attention this group received from the researchers. These researchers are
attempting to control for which of the following?
 the Hawthorne effect

M/C Question 22
Which of the following threats to external validity happens when subjects receive unintended
treatments?
 multiple treatment interference

M/C Question 23
Which of the following threats to external validity happens when subjects discover the experiment's
purpose?
 reactive arrangements
M/C Question 24
Which of the following threats to external validity happens when the subjects learn information based on
the pretest?
 pretest sensitization

M/C Question 25
Which of the following threats to external validity occur if the researcher interacts differently with
different subjects?
 experimenter effects
M/C Question 26
Besides randomization, which of the following is a technique used to equalize initial group differences?
 using analysis of covariance
M/C Question 27
The Persian Gulf War might influence internal validity in what way?
 history
40
M/C Question 28
What type of threat to internal validity is associated with practice effects?
 testing

M/C Question 29
Which experimental design involves one experimental group and three control groups?
 Solomon four-group design

M/C Question 30
What is one disadvantage of the Solomon four-group design?
 It is time-consuming.
CHAPTER 12

M/C Question 1
When you cannot use a pretest, what would be the appropriate design to use?
 static group comparison
M/C Question 2
What is the most serious threat to internal validity in nonequivalent control group design?
 selection

M/C Question 3
If nonequivalent control group design had random selection and assignment, what type of design would
it be?
 pretest posttest control group design

M/C Question 4
Which allows for in-depth examination of specific behaviors?
 single-subject design

M/C Question 5
Which of the following notes that the treatment is reintroduced?
 ABAB

M/C Question 6
What is the primary advantage of ABAB design?
 addressing ethical concerns

41
M/C Question 7
Why use nonequivalent control group design?
 because you cannot assign subjects to groups

M/C Question 8
In what design is the cause-and-effect argument strongest?
 experimental design

M/C Question 9
In quasi-experimental research, when will the cause of differences between groups occur?
 it has already occurred

M/C Question 10
Which of the following statements is false regarding quasi-experimental methodology?
 It is more powerful than experimental research.
M/C Question 11
What is the major limitation of the quasi-experimental design?
 the preassignment to groups

M/C Question 12
What measure must be established prior to the treatment variable to measure its effectiveness?
 baseline

M/C Question 13
The term nonequivalent in “nonequivalent control group design” refers to the nonequivalence of
__________________________.
 groups before the experiment begins

M/C Question 14
If, while using the nonequivalent control group design, the researcher discovers that groups differ on the
pretest measure, one technique which can be used to statistically equalize the groups on the pretest is
____________________.
 Analysis of Covariance (ANCOVA)

M/C Question 15
If it is not possible to randomize or administer a pretest, what is the best choice of design?
 static group design

42
M/C Question 16
What is one suggested limitation of the single subject design?
 external validity

M/C Question 17
Single-subject designs have roots in which of the following views of development?
 behavioral

M/C Question 18
In the above example, if Dr. Moore intervenes with reinforcement for one month, withholds treatment
for one month but then re-intervenes, Dr. Moore's design is best represented as __________________.
 ABAB
M/C Question 19
What is one criticism of the AB single-subject design?
 may be unethical to not apply a facilitative treatment

M/C Question 20
Which design is represented below?
Group 1 Treatment Posttest

Group 2 No Treatment Posttest
 static group comparison

M/C Question 21
Professor Russel wants to examine the effectiveness of his new reading instruction method for third
graders. He locates two third grade classes in the same school. He assesses all students’ current reading
ability at the start of the school year. He then has one classroom utilize his new instruction method while
the other classroom uses the traditional method. At the end of the school year, Professor Russell assesses
reading ability once again and compares group differences. The design of this study exemplifies which
of the following?
 nonequivalent control group design

M/C Question 22
Which design is presented below?
Group 1 Pretest Treatment Posttest
Group 2 Pretest No Treatment Posttest
 nonequivalent control group design
43
M/C Question 23
Which of the following sequences exemplifies the method of a single-subject design?
 measure behavior prior to treatment, apply treatment, withdraw treatment

M/C Question 24
What type of research method assesses changes in behavior in one group of subjects at more
than one point in time?
 longitudinal

M/C Question 25
What is one drawback of longitudinal studies?
 They can have a high mortality rate.

M/C Question 26
Dr. Goldstein studied the sleep patterns of five groups of subjects aged 10, 15, 20, 25, and 30 to
determine if there were any differences in pattern following adolescence. How would this type of study
be classified?
 cross-sectional

M/C Question 27
Which study design uses more than one test period prior to initiating any treatment and uses a
comparison between one subject who receives a treatment and one who does not?
 multiple baseline design
M/C Question 28
Longitudinal studies examine changes that occur with age; what do cross-sectional studies
examine?
 age differences

M/C Question 29
Compared to longitudinal studies, cross-sectional studies ______________________.
 are much less expensive
M/C Question 30
If you want to answer a research question regarding whether a particular group of subjects experiences
any significant changes in behavior as they age, but you do not have the time or money for a traditional
longitudinal study, which method would be acceptable?
 follow-up
44
BASIC TERMINOLOGY CHEAT SHEET GRID

A set of related ideas that explain events that have occurred and predict events that
Theory – may happen.
Research – A process through which new knowledge is discovered.
A shared general philosophy regarding how questions about human behavior should
be answered, including a standard sequence of steps in formulating, and answering a
Scientific Method - question.
Hypothesis – An objective extension of the question that was originally posed.
Chance – What you can't control in an experiment, luck, fate.
Nonexperimental Rsrch – Describes relationships between variables, but cannot test causal relationships
Descriptive Research – Describes the characteristics of an existing phenomenon.
Categorized as nonexperimental research; examines human behavior in the social,

Qualitative Research – cultural, and political contexts in which they occur.
Provides some indication as to how two or more things are related to one another or
what they have in common, or how well a specific outcome might be predicted by one
Correlational Research – or more pieces of information.
Participants are assigned to groups by the researcher based on some criterion; can
Experimental Research – examine cause and effect relationships.
Participants in the study have been indirectly assigned to groups based on some
characteristic beyond the researcher's control; not able to determine cause and effect
Quasi-experimental Rsrch– relationships.
Sometimes called pure research, this research has no immediate application at the
Basic Research – time the research is conducted.
Applied Research – Research that has an immediate application at the time it is completed.

45

Research Methods and Program Evaluations Study Guide

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Research Methods and Program Evaluations Study Guide

Uploaded by

Copyright:

Available Formats

Research Methods/Program Evaluation: Study Guide CHD

STUDY GUIDE: RESEARCH METHODS & PROGRAM

 formulating a hypothesis, collecting relevant information, testing the hypothesis, working

 can be used to create a stratified sample

 use subjective scoring procedures

 central limit theorem

 developing the questions

c) A scattergram is a visual representation of a correlation between two variables.

 to describe some aspect of a certain variable

Group 1 Treatment Posttest

BASIC TERMINOLOGY CHEAT SHEET GRID

Research – A process through which new knowledge is discovered.

Hypothesis – An objective extension of the question that was originally posed.

Chance – What you can't control in an experiment, luck, fate.

Descriptive Research – Describes the characteristics of an existing phenomenon.

Categorized as nonexperimental research; examines human behavior in the social,

You might also like