1.

RELIABILITY AND VALIDITY

• CONCEPTS AND CONSTRUCTS IN
RESEARCH

2. RESEARCH QUESTIONS AND HYPOTHESIS

SUBMITTED TO : MS. MEHREEN KAMRAN

SUBMITTED BY : MARIYA YOUSAF 003

ILMA NADEEM 006

ZUBIA SYED 023

LAIBA ISRAR 033

DATE : 17/05/20
 ASSIGNMENT 1

VALIDITY

The term validity refers to whether or not a test measures what it intends to measure. On a
test with high validity the items will be closely linked to the test’s intended focus. For many
certification and licensure tests this means that the items will be highly related to a specific
job or occupation. If a test has poor validity, then it does not measure the job-related content
and competencies it ought to. There are several ways to estimate the validity of a test,
including content validity, construct validity, criterion-related validity (concurrent &
predictive ) and face validity.

TYPES OF VALIDITY

CONTENT VALIDITY
Content validity refers to the connections between the test items and the subject-related tasks.
The test should evaluate only the content related to the field of study in a manner sufficiently
representative, relevant, and comprehensible.
CONSTRUCT VALIDITY:
It implies using the construct correctly (concepts, ideas, notions). Construct validity seeks
agreement between a theoretical concept and a specific measuring device or procedure.
 For example, a test of intelligence nowadays must include measures of multiple
intelligences, rather than just logical-mathematical and linguistic ability measures.
CRITERION-RELATED VALIDITY
Also referred to as instrumental validity, it states that the criteria should be clearly defined by
the teacher in advance. It has to take into account other teachers´ criteria to be standardized
and it also needs to demonstrate the accuracy of a measure or procedure compared to another
measure or procedure which has already been demonstrated to be valid.
PREDICTIVE VALIDITY:
This is another statistical approach to validity that estimates the relationship of test scores to
an examinee's future performance as a master or non-master. Predictive validity considers the
question, "How well does the test predict examinees' future status as masters or non-
masters?" For this type of validity, the correlation that is computed is based on the test results
and the examinee’s later performance. This type of validity is especially useful for test
purposes such as selection or admissions.
CONCURRENT VALIDITY
Concurrent validity is a statistical method using correlation, rather than a logical method.
Examinees who are known to be either masters or non-masters on the content measured by
the test are identified before the test is administered. Once the tests have been scored, the
relationship between the examinees’ status as either masters or non-masters and their
performance (i.e., pass or fail) is estimated based on the test. This type of validity provides
evidence that the test is classifying examinees correctly. The stronger the correlation is, the
greater the concurrent validity of the test is.
FACE VALIDITY
Like content validity, face validity is determined by a review of the items and not through the
use of statistical analyses. Unlike content validity, face validity is not investigated through
formal procedures. Instead, anyone who looks over the test, including examinees, may
develop an informal opinion as to whether or not the test is measuring what it is supposed to
measure. While it is clearly of some value to have the test appear to be valid, face validity
alone is insufficient for establishing that the test is measuring what it claims to measure.
 RELIABILITY
Reliability is the extent to which an experiment, test, or any measuring procedure shows the
same result on repeated trials. Without the agreement of independent observers able to
replicate research procedures, or the ability to use research tools and procedures that produce
consistent measurements, researchers would be unable to satisfactorily draw conclusions,
formulate theories, or make claims about the generalizability of their research.

EQUIVALENCY RELIABILITY
Equivalency reliability is the extent to which two items measure identical concepts at an
identical level of difficulty. Equivalency reliability is determined by relating two sets of test
scores to one another to highlight the degree of relationship or association. For example, a
researcher studying university English students happened to notice that when some students
were studying for finals, they got sick. Intrigued by this, the researcher attempted to observe
how often, or to what degree, these two behaviors co-occurred throughout the academic year.
The researcher used the results of the observations to assess the correlation between
“studying throughout the academic year” and “getting sick”. The researcher concluded there
was poor equivalency reliability between the two actions. In other words, studying was not a
reliable predictor of getting sick.
STABILITY RELIABILITY
Stability reliability (sometimes called test, re-test reliability ) is the agreement of measuring
instruments over time. To determine stability, a measure or test is repeated on the same
subjects at a future date. Results are compared and correlated with the initial test to give a
measure of stability. This method of evaluating reliability is appropriate only if the
phenomenon that the test measures is known to be stable over the interval between
assessments. The possibility of practice effects should also be taken into account.
INTERNAL CONSISTENCY
Internal consistency is the extent to which tests or procedures assess the same characteristic,
skill or quality. It is a measure of the precision between the measuring instruments used in a
study. This type of reliability often helps researchers interpret data and predict the value of
scores and the limits of the relationship among variables. For example, analyzing the internal
reliability of the items on a vocabulary quiz will reveal the extent to which the quiz focuses
on the examinee’s knowledge of words.
INTER-RATER RELIABILITY
Inter-rater reliability is the extent to which two or more individuals (coders or raters) agree.
Inter-rater reliability assesses the consistency of how a measuring system is implemented. For
example, when two or more teachers use a rating scale with which they are rating the
students’ oral responses in an interview (1 being most negative, 5 being most positive). If one
researcher gives a "1" to a student response, while another researcher gives a '5'
obviously the inter-rater reliability would be inconsistent. Inter-rater reliability is dependent
upon the ability of two or more individuals to be consistent. Training, education and
monitoring skills can enhance inter-rater reliability.
Intra-rater reliability is a type of reliability assessment in which the same assessment is
completed by the same rater on two or more occasions. These different ratings are then
compared, generally by means of correlation. Since the same individual is completing both
assessments, the rater's subsequent ratings are contaminated by knowledge of earlier ratings.

CONCEPTS

To understand and communicate information about objects and events, there must be a
common ground on which to do it. Concepts serve this purpose. A concept is a generally
accepted collection of meanings or characteristics associated with certain events, objects,
conditions, situations, and behaviors. Classifying and categorizing objects or events that have
common characteristics beyond any single observation creates concepts. We abstract such
meanings from our experiences and use words as labels to designate them. For example, we
see a man passing and identify that he is running, walking, skipping, crawling, or hopping.
These movements all represent concepts. We also have abstracted certain visual elements by
which we identify that the moving object is an adult male, rather than an adult female or a
truck or a horse.
We design hypotheses using concepts. We devise measurement concepts by which to test
these hypothetical statements. We gather data using these measurement concepts. The
success of research hinges on
(1) how clearly we conceptualize and
(2) how well others understand the concepts we use. For example, when we survey people on
the question of customer loyalty, the questions we use need to tap faithfully the attitudes of
the participants
The challenge is to develop concepts that others will clearly understand. We might, for
example, ask participants for an estimate of their family’s total income. This may seem to be
a simple, unambiguous concept, but we will receive varying and confusing answers unless we
restrict or narrow the concept by specifying:
• Time period, such as weekly, monthly, or annually.
• Before or after income taxes.
• For head of family only or for all family members.
• For salary and wages only or also for dividends, interest, and capital gains.
• Income in kind, such as free rent, employee discounts, or food stamps.

CONSTRUCTS
Concepts have progressive levels of abstraction that is, the degree to which the concept does
or does not have something objective to refer to. Table is an objective concept. We can point
to a table, and we have images of the characteristics of all tables in our mind. An abstraction
like personality is much more difficult to visualize. Such abstract concepts are often called
constructs. A construct is an image or abstract idea specifically invented for a given research
and/or theory-building purpose.

The Role of Constructs

A construct is an abstract idea inferred from specific instances that are thought to be related.
Typical marketing constructs are brand loyalty, satisfaction, preference, awareness,
knowledge. Research objectives typically call for the measurement of constructs. There are
customary methods for defining and measuring constructs.
EXAMPLES
• Aggression
• Love
• Intelligence
 ASSIGNMENT 2

FORMULATING RESEARCH QUESTIONS AND HYPOTHESIS

✓ Introduction
✓ Processes involved before formulating the hypotheses.
✓ Definition
✓ Nature of Hypothesis
✓ Types
✓ How to formulate a Hypotheses in
• Quantitative Research
• Qualitative Research
✓ Testing and Errors in Hypotheses
✓ Summary

o INTRODUCTION

The research structure helps us create research that is :

QUANTIFIABLE VERIFIABLE REPLICABLE DEFENSIBLE

Corollaries among the model, common sense & paper format

Model Common Sense Paper Format

Research Question Why Intro

Develop a Theory Your Answer Intro

Identify Variables (if applicable) How Method

Identify hypotheses Expectations Method

Test the hypotheses Collect/Analyze data Results

Evaluate the Results What it Means Conclusion

Critical Review What it doesn’t Mean Conclusion

Most research projects share the same general structure, which could be represented in the
shape of an hourglass.

The “Hourglass” notion of research

BEGIN WITH BROAD QUESTIONS

NARROW DOWN, FOCUS IN

OPERATIONALIZE

OBSERVE

ANALYZE DATA

REACH CONCLUSIONS

GENERALIZE BACK TO QUESTIONS

o PROCESSES INVOLVED BEFORE FORMULATING THE HYPOTHESES.

Some of the methods that are included for research formulation are

✓ Where does the problem origination or discovery begin?

• Previous Experience
• Triggered Interest
• Potential problem fields
✓ Criteria of problems and problem statement
✓ Goals & Planning
✓ Search, Explore & Gather the Evidence
✓ Generate creative and logical alternative solutions

Making the educated guess- the hypothesis!

o DEFINITION OF HYPOTHESIS

• “Hypotheses are single tentative guesses, good hunches – assumed for use in devising
theory or planning experiments intended to be given a direct experimental test when
possible”. (Eric Rogers, 1966)

• “A hypothesis is a conjectural statement of the relation between two or more variables”.

(Kerlinger, 1956)

• “Hypothesis is a formal statement that presents the expected relationship between an

independent and dependent variable.”(Creswell, 1994)

• “A research question is essentially a hypothesis asked in the form of a question.”

• “It is a tentative prediction about the nature of the relationship between two or more
variables.”

• “A hypothesis can be defined as a tentative explanation of the research problem, a

possible outcome of the research, or an educated guess about the research outcome.”
(Sarantakos, 1993: 1991)

• “Hypotheses are always in declarative sentence form, an they relate, either generally
or specifically , variables to variables.”

• “An hypothesis is a statement or explanation that is suggested by knowledge or

observation but has not, yet, been proved or disproved.” (Macleod Clark J and Hockey
L 1981)

o NATURE OF HYPOTHESIS
• It can be tested – verifiable or falsifiable
• Hypotheses are not moral or ethical questions
• It is neither too specific nor to general
• It is a prediction of consequences
• It is considered valuable even if proven false

An Example...

Imagine the following situation:

You are a nutritionist working in a zoo, and one of your responsibilities is to develop a
menu plan for the group of monkeys. In order to get all the vitamins they need, the
monkeys have to be given fresh leaves as part of their diet. Choices you consider include
leaves of the following species: (a) A (b) B (c) C (d) D and (e) E. You know that in the wild
the monkeys eat mainly B leaves, but you suspect that this could be because they are safe
whilst feeding in B trees, whereas eating any of the other species would make them
vulnerable to predation. You design an experiment to find out which type of leaf the
monkeys actually like best: You offer the monkeys all five types of leaves in equal
quantities, and observe what they eat.

There are many different experimental hypotheses you could formulate for the monkey
study. For example:
When offered all five types of leaves, the monkeys will preferentially feed on B leaves.
This statement satisfies both criteria for experimental hypotheses. It is a

•Prediction: It predicts the anticipated outcome of the experiment

•Testable: Once you have collected and evaluated your data (i.e. observations of what the
monkeys eat when all five types of leaves are offered), you know whether or not they ate
more B leaves than the other types.

Incorrect hypotheses would include:

When offered all five types of leaves, the monkeys will preferentially eat the type they like
best.
This statement certainly sounds predictive, but it does not satisfy the second criterion: there is
no way you can test whether it is true once you have the results of your study. Your data will
show you whether the monkeys preferred one type of leaf, but not why they preferred it (i.e.,
they like it best). I would, in fact, regard the above statement as an assumption that is inherent
in the design of this experiment, rather than as a hypothesis.
When offered all five types of leaves, the monkeys will preferentially eat B leaves because
they can eat these safely in their natural habitat.
This statement is problematic because its second part ('because they can eat these safely in their
natural habitat') also fails to satisfy the criterion of testability. You can tell whether the
monkeys preferentially eat baobab leaves, but the results of this experiment cannot tell you
why.
In their natural habitat, howler monkeys that feed in B trees are less vulnerable to
predation than monkeys that feed on A, C, D, or E.

This is a perfectly good experimental hypothesis, but not for the experiment described in
the question. You could use this hypothesis if you did a study in the wild looking at how
many monkeys get killed by predators whilst feeding on the leaves of A, B etc. However,
for the experimental feeding study in the zoo it is neither a prediction nor testable.

When offered all five types of leaves, which type will the monkeys eat preferentially?

This is a question, and questions fail to satisfy criterion #1: They are not predictive
statements. Hence, a question is not a hypothesis.
o TYPES OF HYPOTHESIS

✓ NULL HYPOTHESES

Designated by: H0 or HN

Pronounced as “H oh” or “H-null”

✓ ALTERNATIVE HYPOTHESES (DIRECTIONAL HYPOTHESIS)

Designated by: H1 or HA

• NON DIRECTIONAL HYPOTHESIS

The Null Hypothesis represents a theory that has been put forward, either because it is believed
to be true or because it is to be used as a basis for argument, but has not been proved. A null
hypothesis represents the traditional approach: it makes a prediction that in the general
population, no relationship or no significant difference exists between groups on a variable.
The wording is, “There is no difference (or relationship)” between the groups.
• Has serious outcome if incorrect decision is made!

The Alternative Hypothesis is a statement of what a hypothesis test is set up to establish. The
investigator makes a prediction about the expected outcome, basing this prediction on prior
literature and studies on the topic that suggest a potential outcome.
• Opposite of Null Hypothesis.
• Only reached if H0 is rejected.
• Frequently “alternative” is actual desired conclusion of the researcher!

EXAMPLE ….
In a clinical trial of a new drug, the null hypothesis might be that the new drug is no better, on
average, than the current drug.
We would write H0: there is no difference between the two drugs on average.
The alternative hypothesis might be that:
the new drug has a different effect, on average, compared to that of the current drug.
We would write H1: the two drugs have different effects, on average.
the new drug is better, on average, than the current drug.
We would write H1: the new drug is better than the current drug, on average.
We Give Special Consideration To The Null Hypothesis...
• This is due to the fact that the null hypothesis relates to the statement being tested,
whereas the alternative hypothesis relates to the statement to be accepted if /
when the null is rejected.

• The final conclusion, once the test has been carried out, is always given in terms of
the null hypothesis. We either 'reject H0 in favor of H1' or 'do not reject H0'; we never
conclude 'reject H1', or even 'accept H1'.

• If we conclude 'do not reject H0', this does not necessarily mean that the null
hypothesis is true, it only suggests that there is not sufficient evidence against H0 in
favor of H1; rejecting the null hypothesis then, suggests that the alternative hypothesis
may be true.

Another type of alternative hypothesis is Nondirectional—a prediction is made, but the exact
form of differences (e.g., higher, lower, more, less) is not specified because the researcher does
not know what can be predicted from past literature. Thus, the investigator might write, “There
is a difference” between the two groups. An example follows which incorporates both types of
hypotheses

Example….

Nondirectional and Directional Hypotheses

Sometimes directional hypotheses are created to examine the relationship among variables
rather than to compare groups. For example, Moore (2000) studied the meaning of gender
identity for religious and secular Jewish and Arab women in Israeli society. In a national
probability sample of Jewish and Arab women, the author identified three hypotheses for study.
The first is nondirectional and the last two are directional.

H1: Gender identity of religious and secular Arab and Jewish women are related to different
sociopolitical social orders that reflect the different value systems they embrace.

H2: Religious women with salient gender identity are less socio-politically active than secular
women with salient gender identities.

H3: The relationships among gender identity, religiosity, and social actions are weaker among
Arab women than among Jewish women.
o FORMULATING A HYPOTHESIS

The formulation of the hypothesis basically varies with the kind of research project

conducted:

• QUALITATIVE
• QUANTITATIVE

Can also be devided into,

✓ QUALITATIVE APPROACH

The use of Research Questions as opposed to objectives or hypothesis, is more frequent.

Characteristics Of Qualitative Research Questions;

• Use of words- what or how.

Specify whether the study: discovers, seeks to understand, explores or describes the
experiences.

• Use of non-directional wording in the question.

These questions describe, rather than relate variables or compare groups.

• The questions are under continual review and reformulation-will evolve and change during
study.
• The questions are usually open-ended, without reference to the literature or theory.

Use of a single focus.

THE RULES OF QUALITATIVE RESEARCH

Kleining offers four rules for a scientific and qualitative process of approaching understanding
to reality.

Rule 1 (refers to subject / researcher)

"Prior understandings of the phenomenon to be researched should be seen as provisional and

should be transcended with [the discovery of] new information with which they are not
consistent." (1982: 231)

Rule 2 (refers to the object of study)

"The object is provisional; it is only fully known after the successful completion of the process
of discovery." (1982: 233)

Rule 3 (refers to action in relation to the subject of research, hence to data collection)

"The object should be approached from "all" sides; rule of the maximum variation of
perspectives." (1982: 234)

Rule 4 (refers to the evaluation of information gathered, hence to data analysis)

"Analysis of the data for common elements." (1982: 237)

Example….

Qualitative Central Questions From a Case Study

Padula and Miller (1999) conducted a multiple case study that described the experiences of
women who went back to school, after a time away, in a psychology doctoral program at a
major Midwestern research university. The intent was to document the women’s experiences,
providing a gendered and feminist perspective for women in the literature. The authors asked
three central questions that guided the inquiry:

(a) How do women in a psychology doctoral program describe their decision to return to
school?

(b) How do women in a psychology doctoral program describe their reentry experiences? And

(c) How does returning to graduate school change these women’s lives?

(Padula & Miller, 1999, p. 328)

✓ QUANTITATIVE APPROACH

In survey projects the use of research questions and objectives is more frequent

In experiments the use of hypotheses are more frequent

Represent comparison between variables

relationship between variables

Characteristics Of Quantitative Research Questions And Hypotheses

• The testable proposition to be deduced from theory.

• Independent and dependent variables to be separated and measured separately.
• To be either writing-questions, or objectives or hypotheses, but not a combination.
• Consider the alternative forms for writing and make a choice based on the audience for the
research

Here is an Example of a script for a quantitative research question:

Does _________ (name the theory) explain the relationship between_________ (independent
variable) and _________ (dependent variable), controlling for the effects of _________
(control variable)?

Alternatively, a script for a quantitative null hypothesis might be as

follows:

There is no significant difference between _________ (the control and experimental groups on
the independent variable) on _________(dependent variable).

✓ GENERATION OF
HYPOTHESIS

Problem statements
become research
hypotheses when
constructs are
operationalized.
Example:

Consider the example of a simple association between two variables, Y and X.

1. Y and X are associated (or, there is an association between Y and X).

2. Y is related to X (or, Y is dependent on X).

3. As X increases, Y decreases (or, increases in values of X appear to effect reduction in values

of Y).

The first hypothesis provides a simple statement of association between Y and X. Nothing is
indicated about the association that would allow the researcher to determine which variable, Y
or X, would tend to cause the other variable to change in value.

The second hypothesis is also a simple statement of association between Y and X, but this
time it may be inferred that values of Y are in some way contingent upon the condition of the
X variable.

The third hypothesis is the most specific of the three. Not only does it say that Y and X are
related and that Y is dependent on X for its value, but it also reveals something more about the
nature of the association between the two variables.

o TESTING & CHALLENGING

The degree of challenge to the hypothesis will depend on the type of problem and its
importance. It can range from just seeking “a good enough” solution to a much more rigorous
challenge.

The term “challenging” may include

• Verification
• Justification
• Refutability
• Validity
• Rectification
• Repeatability
• Falsification
There are two possibilities

1. Nothing Happened the Null Hypothesis - Ho

2. Something Happened the Alternative Hypothesis - H1

Hypothesis testing is a four-step procedure:

1. Stating the hypothesis (Null or Alternative)

2. Setting the criteria for a decision

3. Collecting data

4. Evaluate the Null hypothesis

✓ ERRORS IN HYPOTHESES

Two types of mistakes are possible while testing the hypotheses.

Type I

Type II

• Type I Error:

A type I error occurs when the null hypothesis (H0) is wrongly rejected.

For example, A type I error would occur if we concluded that the two drugs produced different
effects when in fact there was no difference between them.

• Type II Error:

A type II error occurs when the null hypothesis H0, is not rejected when it is in fact false.

For example: A type II error would occur if it were concluded that the two drugs produced the
same effect, that is, there is no difference between the two drugs on average, when in fact they
produced different ones.
To generalize:

DECISION

T REJECT H0 DON’T REJECT H0

U H0 TYPE I ERROR RIGHT DECISION

H1 RIGHT DECISION TYPE II ERROR

• A type I error is often considered to be more serious, and therefore more important to avoid,
than a type II error.

✓ SUMMARY

“Research questions and hypotheses become “signposts” for explaining the purpose of the
study & guiding the research...”, Creswell

A hypothesis is an explanation, tentative and unsure of itself, for specific phenomena about
which you have questions.

A well-crafted hypothesis very often suggests the best way to perform the research and gives
you clues as to your research design.

There are different types of hypotheses.

• Deductive
• Inductive

Research Hypothesis can either be non-directional or directional. There exists a hypothesis

that is opposite of the positively stated one, i.e. the null hypothesis
Thus to conclude it would be fitting to say “hypothesis is perhaps the most powerful tool,
man has invented to achieve dependable knowledge” – Fred Kerlinger...