RESEARCH METHODOLOGY

CHAPTER 4: ANALYSIS

Assoc. Prof Sallehuddin Muhamad

UTM Razak School
Research Design
 Overview of Research Methodologies

• Qualitative Research
– Ethnography (human culture), Case Study, Grounded
Theory, Autobiography, Participatory Action Research,
Phenomenology (each grounded in a specific discipline
and philosophical assumptions)
• Quantitative Research
– Survey methods, Experiments
• Mixed Methods
– Draw from qualitative and quantitative methods
 TESTING METHOD

• Before data can be collected and analysed, researcher must

prepare testing method.

• Typically, the test method can be categorized as laboratory

experiment or non-laboratory experiment (survey).

• Conceptual Research and Empirical Research usually use

laboratory experiment to gather data. Descriptive Research,
Correlative Research and Analytical Research do not require
laboratory experiment.
 Laboratory Experiment

• Many researches in engineering and science require

laboratory experiment.
• Experiments are conducted:
– To find empirical relationship between variables
– To prove theory
– To understand phenomena.
– To find root cause of a problem
– To predict the effect of a cause.
• Experiment is expensive and time consuming to
conduct.
 Laboratory Experiment - continue

• Taguchi Method offers an effective way to conduct

laboratory experiment.
• It investigate how different parameters affect the
mean and variance of a process performance
characteristic.
– Based on a Design of Experiments (DOE) methodology for
determining parameter levels
 Laboratory Experiment - continue

• DOE is an important tool for designing processes and

products
– A method for quantitatively identifying the right inputs and
parameter levels for making a high quality product or
service
• The Taguchi method is best used when there are an
intermediate number of variables (3 to 50), few
interactions between variables, and when only a few
variables contribute significantly.
 Laboratory Experiment - continue

• As the number of factors in a two level factorial

design increases, the number of runs for even a
single replicate of the 2k design becomes very large.
– For example, a single replicate of an 8 factor two level
experiment would require 256 runs!
• Taguchi Method will be able to reduce the number
experiment without jeopardizing the quality of the
results.
• Refer to texts on Taguchi Method for further
information.
 Taguchi Method Sample

• The strength of a welding depends on welding current,

welding speed and distance from work piece. Table 1 shows 9
variables. To conduct the research 29 experiments are
required.
• However by using L9 Level Taguchi Orthogonal Array, only 9
experiments are needed as shown in Table 2.
 Non - Laboratory Experiment

• Research which do require laboratory experiment,

data can be collected through:
– Interviews
– Focus groups
– Participant observation (field notes)
– Video
– Text and Image analysis (documents, media data)
– Historical data
– Published reports
 Quantitative Research
Generate Hypothesis

Construct Survey Questions Hypothesis Check

Initial Survey and Data Collection

Conclusion

Check Question Validity

Conduct Full Scale Survey

Data Analysis
 Qualitative Research
Identify Parameters to Measure

Construct Survey Questions

Survey and Data Collection

Converting Data into Information

Recheck to Information

Deduction

Validation by Subject Matter Expert

 Dependent, Non-dependent and
Extraneous Variable
• Dependent or Endogenous variables:
– When the change in one variable depends on the change
in other variable
Eg. Demand ----- Price (independent)

• Independent or Exogenous variable

– The variable that causes the change in the dependent
variable is known as independent or exogenous variable.
Eg. Demand (Dependent) ------- Price ,Income
Here demand is a dependent variable while price / income is an
independent variable.
 Dependent, Non-dependent and
Extraneous Variable
• Extraneous variable:
– The independent variable which is not directly related to
the purpose of the study but affects the dependent
variable.
– The influence caused by the extraneous variable on the
dependent value is technically known as “Experimental
Error”
– Good Research design should minimize the effect for
Extraneous variables.
Data Collection
 Introduction

• Data Collection is the systematic gathering of

information (data) for a particular purpose from
various sources. (Various sources can be
questionnaires, interviews, observations existing
records and electronic devices).
• Two Important sources of Information are,
– Primary Data
– Secondary Data
 Introduction - continue

• Primary Data:
– Refers to the data collected for the first time (Original data)
– Example: Proceedings from conferences meetings,
interviews, survey.
• Secondary Data:
– Refers to the data that have already been collected and
used earlier by somebody or some agency.
– Example: Online database, Historical books etc.
 Method of Collecting Primary Data

1. Direct Personal Interviews:

– A face to face contact where interviewer asks set of
questions regarding the study and tries to get the desired
information.
– Guided topic with open questions to gather more
information.
– Suitable to gather information from experts or people with
authority in the area of interest.
– Reconfirm the information gathered to ensure accuracy.
– Usually use in Qualitative Method.
 Method of Collecting Primary Data -
continue
1. Direct Personal Interviews:
– Advantage
• Information are more accurate (interviewee can be reexamined or
Cross-examined to obtain the information
– Disadvantage
• Expensive and time consuming when the number of informants is
large.
• Greater chance of personnel bias and prejudice. (coaching)
 Method of Collecting Primary Data
- continue
2. Information From Correspondents:
– Correspondents collect information and transfer or
transmit the information to the central office where the
data are processed. Eg. News paper Agencies.
– Advantages
• Cheap
• Can be used to get regular information at regular intervals (daily,
weekly or monthly).
– Disadvantages:
• Lacks Accuracy.
 Method of Collecting Primary Data
- continue
3. Mailed or On-line Questionnaire Method:
– A list of questions pertaining to the survey is known as
“Questionnaire”. It is prepared and sent to various
informants by post, email or on-line.
– Advantage
• Easily adopted in large populations and when the informants are
wide spread over a large geographical area.
• Relatively cheap & timely.
– Disadvantage
• Lacks Accuracy because the information may not be correct.
• Required computer knowledge to setup on-line survey.
• Difficulty getting cooperation from informants
 Method of Collecting Primary Data
- continue
4. Observation Method:
– Human behaviors are observed and measured using video
or instruments without observers interference.
– Observation time varies.
• eg. Time Study on worker performance
– Advantage
• True results obtained
– Disadvantage
• Bias, not natural, not consistent.
 Method of Collecting Secondary Data

• Selecting a particular source of Secondary Data

depends on:
– Purpose & Scope of the study.
– Availability of time.
– Availability of Resources.
– The degree of Accuracy desired.
– Statistical tools to be used.
– Sources of Information (data).
Data Sampling
 Introduction

• In many cases it is impractical to study all population

in the area of interest.
• It is not only expensive but require a long time to
complete.
• However, the sampling process allows us to make
inferences or comprehensiveness of the findings
based on careful observation of a small part of the
subject (sample) of the large population.
 Introduction - continue

• Advantages of sampling
– Saves time because fewer items are collected and
processed.
– There is reduction in cost and in man power since only few
items are studied.
– More reliable results can be obtained
– More detailed information can be obtained by sample
survey.
– Results can be tested since more scientific
– The organization and administration is easy in sample.
survey
 Introduction - continue

• Disadvantages of sampling
– If sampling is not carefully planned & executed, the
conclusions may be false.
– If the sample taken from the population is not the right
representative, the result may be false or misleading.
– If there is a lack of experts to plan, execute and analyze
the samples, the result would be unsatisfactory.
– There may be personal bias & prejudice in choosing the
sampling techniques.
– If the size of the sample is not appropriate, it leads to
untrue conclusions.
 Sampling Terminology

• POPULATION: is a large group of people, objects or events

studied.
• SAMPLE: is the small group that obtained from the
population.
• SUBJECT: is the individuals in the sample.
• TARGET POPULATION: is the large group of people, objects or
events targeted from research findings.
• PARAMETER: is the measured characteristics of the
population.
 Sampling Process

DEFINITION OF POPULATION

SAMPLE SELECTION

ESTIMATING PARAMETERS OF POPULATION BASED

STATISTICAL MEASURES OBTAINED FROM SAMPLE
 Scientific Sampling

• There are four scientific and systematic sampling

methods for getting good samples that can be
representative of the population, namely:
1. Random Sampling Method
a. Simple Random
b. Systematic Random
2. Stratified Random Sampling Method
3. Cluster Sampling
4. Purposive Sampling
 Simple Random Sampling Method

• In this method, the subject is selected from the

selected population using random selection.
• Each subject in the population has an equal chance
to be selected as a sample.
• If the population is small and the required sample
also small, the easiest way is to put all the names
(identities) subjects in a container and draw done.
• However, if population is large, the Random Number
Table is used.
– Method: Give the number (identity) to the subject and
subject selection is done by the numbers on the table of
random numbers.
Simple Random Sampling Method -
continue
 Simple Random Sampling Method -
continue
How to use a random number table
– Let's assume that we have a population of 185 students
and each student has been assigned a number from 1 to
185. Suppose we wish to sample 5 students
– Since 185 is a three digit number, we need to use the first
three digits of the numbers listed on the chart.
– We randomly point to a spot on the chart selected 20631
in the first column. The first three digit 206 is larger than
our population with that number, we go to the next
number 89990. Again that number is larger than our
population, so we continue at the top of the next column.
 Simple Random Sampling Method -
continue
How to use a random number table.
– As we work down the column, we find that the first
number to match our population 10005 on the chart.
Student number 100 would be in our sample. Continuing
down the chart, we see that the other four subjects in our
sample would be students 049, 082, 153, and 164.
 Systematic Random Sampling
Method
• This sampling method is similar with the simple random
method except that each subject did not has an equal chance
to be selected as a sample.
• Procedures to obtain systematic random sample according to
the method is as follows:
– Get a list of all members of the population randomly selected and listed
alphabetically.
– If there are patterns on the subject and it is an important items in the
study, using Simple Random Sampling Method.
– If there is no pattern, dividing the number of subjects in the population
with the required number of subjects in the sample yield a number “n”.
– Starting with this number as the subject first and then pick the subjects
for each double the number "n" so that you choose the number of
subjects required.
 Systematic Random Sampling
Method - continue
• Example:
Say there are 600 subjects in the population while you only
need 30 subjects in the sample. The procedure may be made
are as follows:
• Divide 600 by 30 equals 20
• Therefore, the smallest number to choose subjects are
subjects numbered 20, followed by the subjects numbered
40, 60, until the 30th subject numbered 600.
 Stratified Random Sampling Method

• Stratified sampling technique is generally used when the

population is heterogeneous.
• The entire population is divided into sub population (sub
groups) (i.e. the sub population being homogenous).
• This method is more reliable & accurate.
• Example:
– 50 students of a school having 1000 students on a total
were selected and interviewed on the interest in music.
– The students were grouped based on their age, 7 years, 8
years, 9 years, 10 years and 11 years.
– From each age group and sex, 5 students were chosen and
totally 50 students were interviewed.
 Cluster Sampling

• The entire population is sub divided into mutually exclusive

groups or clusters
• Simple Random sampling is applied and the need clusters are
selected for the study.
• If all the elements found in the selected cluster is taken for
the study, then it is one stage cluster sampling.
• If random sampling is applied in selecting elements found
within the clusters, it is two stage cluster sampling.
 Purposive Sampling

• This sampling is usually used in the study the type of action.

• In this method the researchers determined that the
respondent in accordance with the objectives of the study;
thus the sampling is biased.
• Although the findings of actions using purposive sampling
was not representative of the population but it can provide
an initial overview of the field of study.
 Sample Size

• What is the sample size required for a study? This

question is difficult to answer.
• Selection of sample size usually depends on:
a. Type of population
b. Research design
 Sample Size - continue

a. Type of Population
• If there are subjects in this population is homogeneous, the
study does not require a large sample size.
• However, if the population studied was heterogeneous,
then the sample size should be large so that researchers
acquire characteristics that represent the population.
 Sample Size - continue

b. Research Design
• Typically, quantitative research requires a relatively large
sample size compared with qualitative research.
• For example, research using quantitative statistical factor
analysis requires a minimum of 300 cases.
• While the case study of a qualitative research requires a
much smaller sample.
• However, as a guide, a research with larger sample size is
better and more like a statistical sample with population
parameter.
Sample Size Table for Quantitative
Research
 Sample Limitation

• Sampling is not appropriate when we need

information from each subject contained in the
population.

• For instance, if we want to make a census of the

population that requires information for each
individual in the population.
 Sample Error

• Sampling error may occur during sample selecting and efforts

must be taken to reduce the impact of this error.
• Sampling error means the difference between the sample
information (statistical) and population (parameter).
• Some of the factors that caused the error is:
a. Choosing subjects as research samples because the
subjects are readily available.
b. Choosing subjects from the experimental group and the
control group of different populations.
c. Choosing too small sample size too while the statistical
analysis used require large sample size.
 Data Analysis
(Quantitative and Qualitative)
 How to Analyze Data?

• Need to consider the type of data collected

– categorical
– Continuous
– mixed

Dr. Deryck D. Pattron, Ph.D. 47

 Tests Used on Categorical Data

According to Dr. Deryck D. Pattron, Ph.D, these are some of the

tests:
• Chi-squared test - to determine the relationship between
variables
• Fisher’s test - to compare two unpaired groups
• McNemar’s test - to analyze a matched case-control study
• Kappa - to quantify interafter agreement
• Wilcoxon test - to compare one group to a hypothetical value
• Kruskal-Wallis test - to compare three or more groups
• Spearman test - to compare association between variables

48
 Tests Used on Continuous Data

• T test - to compare one or two groups

• ANOVA (Analysis of Variance) - to compare three or
more groups
• Pearson test - to compare the association between
variables
• Refer to statistical book for further explanation.

49
 Data Analysis

• The purpose of data analysis is to bring order to the

data
• Characteristics of data:
– Thick, rich descriptions
– Voluminous
– Unorganized
• Perspectives on analysis and interpretation
– No single way to gain understanding of phenomena
– Numerous ways to report data
 Data Analysis During Data Collection

• Data analysis is an ongoing process throughout the

entire research project
– Analysis begins with the very first interaction between the
researcher and the participants
– This is a very important perspective given the interpretive
nature of the analysis and the emergent nature of
qualitative research designs
• Informal steps involve gathering data, examining
data, comparing prior data to newer data, and
developing new data to gain perspective
 Data Analysis After Data Collection

• Data management
– Creating and organizing data collected during the study
– Purposes
• Organize and check data for completeness
• Start the analytical and interpretive process
– No meaningful analysis can be done without effective data
management
 Data Analysis After Data Collection

• Data management (continued)

– Suggestions
• Write dates on all notes
• Sequence all notes with labels
• Label notes according to type
• Make photocopies of all notes
• Organize computer files into folders according to data
types and stages of analysis
• Make backup copies of files
• Read through data to make sure it is legible and
complete
• Begin to note potential themes and patterns that
emerge
 Data Analysis After Data Collection

• Three formal steps to analyze data

– Reading and taking note
– Describing the context and participants
– Classifying and interpreting
• Issues
– Scoring procedures
– Tabulation and coding
– Use of computers
 Scoring Procedures

• Instructions
– Standardized tests detail scoring instructions
– Teacher-made tests require the delineation of scoring
criteria and specific procedures
• Types of items
– Selected response items - easily and objectively scored
– Open-ended items - difficult to score objectively with a
single number as the result
 Tabulation and Coding

• Tabulation is organizing data

– Identifying all information relevant to the analysis
– Separating groups and individuals within groups
– Listing data in columns
• Coding
– Assigning names to variables
• EX1 for pre-test scores
• SEX for gender
• EX2 for post-test scores
 Tabulation and Coding

• Reliability
– Concerns with scoring by hand and entering data
– Machine scoring
• Advantages
– Reliable scoring, tabulation, and analysis
• Disadvantages
– Use of selected response items
 Computerized Analysis

• Need to learn how to calculate descriptive statistics

by hand
– Creates a conceptual base for understanding the nature of
each statistic
– Exemplifies the relationships among statistical elements of
various procedures
• Use of computerized software
– SPSS-Windows
– Other software packages
 Descriptive Statistics

• Purpose – to describe or summarize data in a

parsimonious manner
• Four types
– Central tendency
– Variability
– Relative position
– Relationships
 Descriptive Statistics

SCORE
• Graphing data – a 5

frequency polygon 4

– Vertical axis represents

3
the frequency with
which a score occurs 2

– Horizontal axis
Frequency 1
represents the scores Std. Dev = 1.63
Mean = 6.0

themselves 0
3.0 4.0 5.0 6.0 7.0 8.0 9.0
N = 16.00

SCORE
 Central Tendency

• Purpose – to represent the typical score attained by

subjects
• Three common measures
– Mode
– Median
– Mean
 Central Tendency

• Mode
– The most frequently occurring score
– Appropriate for nominal data
• Median
– The score above and below which 50% of all scores lie
(i.e., the mid-point)
– Characteristics
• Appropriate for ordinal scales
• Doesn’t take into account the value of each and every score in the
data
 Central Tendency

• Mean
– The arithmetic average of all scores
– Characteristics
• Advantageous statistical properties
• Affected by outlying scores
• Most frequently used measure of central tendency
• Formula for
– Mode
– Median
– Mean

Objectives 4.2, 4.3, & 4.4

 Variability

• Purpose – to measure the extent to which scores are

spread apart
• Four measurements:
– Range
– Quartile deviation
– Variance
– Standard deviation
 Type of Data Distribution

• A bell shaped curve reflecting the normal distribution

of many variables of interest.
• Skewed distribution:
– Positive – many low scores and few high scores
– Negative – few low scores and many high scores
– Relationships between the mean, median, and mode
• Positively skewed – mode is lowest, median is in the
middle, and mean is highest
• Negatively skewed – mean is lowest, median is in the
middle, and mode is highest
 Measures of Relative Position

• Purpose
– indicates where a score is in relation to all other scores in
the distribution
• Characteristics
– Clear estimates of relative positions
– Possible to compare students’ performances across two or
more different tests provided the scores are based on the
same group
 Measures of Relative Position

• Types
– Percentile ranks – the percentage of scores that fall at or
above a given score
– Standard scores – a derived score based on how far a raw
score is from a reference point in terms of standard
deviation units
a. z score
b. T score
c. Stanine (STAndard NINE)
 Measures of Relative Position

A. z score
– The deviation of a score from the population mean in
standard deviation units
– The basic standard score from which all other standard
scores are calculated.
– In order to use z score, the four things must be available:
• The population mean.
• The population standard deviation.
• The sample mean.
• The sample size.
 Measures of Relative Position

z score (continued)
– Characteristics:
• Mean = 0
• Standard deviation = 1
• Positive if the score is above the mean and negative if it is below
the mean
• Relationship with the area under the normal curve
– Possible to calculate relative standings like the percent
better than a score, the percent falling between two scores,
the percent falling between the mean and a score, etc.
 Measures of Relative Position

B. T score
• Definition:
– T-Scores are a transformation of raw scores into a standard form, where
the transformation is made when there is no knowledge of the
population's mean and standard deviation.
– The scores are computed by using the sample's mean and standard
deviation, which is our best estimate of the population's mean and
standard deviation.
– It is used to determine:
a. whether there are differences between two groups on the same
variable;
b. whether a group's mean (average) value is greater or less than
some standard;
c. whether the same group has different mean (average) scores on
different variables.
 Measures of Relative Position

C. Stanine (STAndard NINE)

• Definition:
– Stanine is a method of scaling test scores on a nine-point standard scale
with a mean of five (5) and a standard deviation of two (2).
– The underlying basis for obtaining Stanine is that a normal distribution
is divided into nine intervals, each of which has a width of 0.5 standard
deviations excluding the first and last. The mean lies at the centre of the
fifth interval.
 Measures of Relationship

Correlation
• Purpose – to provide an indication of the relationship between
two variables
• Characteristics of correlation coefficients
– Strength or magnitude – 0 to 1
– Direction – positive (+) or negative (-)
• Types of correlation coefficients – dependent on the scales of
measurement of the variables
– Spearman rho – ranked data
– Pearson r – interval or ratio data
 Calculating Descriptive Statistics

• Using SPSS Windows

– Means, standard deviations, and standard scores
• The DESCRIPTIVE procedures
• Interpreting output
– Correlations
• The CORRELATION procedure
• Interpreting output