This action might not be possible to undo. Are you sure you want to continue?
https://www.scribd.com/doc/88989530/060TechniquesofDataAnalysis
04/27/2014
text
original
Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman
Director
Centre for Real Estate Studies
Faculty of Engineering and Geoinformation Science
Universiti Tekbnologi Malaysia
Skudai, Johor
Objectives
Overall: Reinforce your understanding from the main
lecture
Specific:
* Concepts of data analysis
* Some data analysis techniques
* Some tips for data analysis
What I will not do:
* To teach every bit and pieces of statistical analysis
techniques
Data analysis – “The Concept”
Approach to desynthesizing data, informational,
and/or factual elements to answer research
questions
Method of putting together facts and figures
to solve research problem
Systematic process of utilizing data to address
research questions
Breaking down research issues through utilizing
controlled data and factual information
Categories of data analysis
Narrative (e.g. laws, arts)
Descriptive (e.g. social sciences)
Statistical/mathematical (pure/applied sciences)
AudioOptical (e.g. telecommunication)
Others
Most research analyses, arguably, adopt the first
three.
The second and third are, arguably, most popular
in pure, applied, and social sciences
Statistical Methods
Something to do with “statistics”
Statistics: “meaningful” quantities about a sample of
objects, things, persons, events, phenomena, etc.
Widely used in social sciences.
Simple to complex issues. E.g.
* correlation
* anova
* manova
* regression
* econometric modelling
Two main categories:
* Descriptive statistics
* Inferential statistics
Descriptive statistics
Use sample information to explain/make
abstraction of population “phenomena”.
Common “phenomena”:
* Association (e.g. σ
1,2.3
= 0.75)
* Tendency (leftskew, rightskew)
* Causal relationship (e.g. if X, then, Y)
* Trend, pattern, dispersion, range
Used in nonparametric analysis (e.g. chi
square, ttest, 2way anova)
Examples of “abstraction” of phenomena
Trends in property loan, shop house demand & supply
0
50000
100000
150000
200000
Year (1990  1997)
Loan t o propert y sect or (RM
million)
32635.8 38100.6 42468.1 47684.7 48408.2 61433.6 77255.7 97810.1
Demand f or shop shouses(unit s) 71719 73892 85843 95916 101107 117857 134864 86323
Supply of shop houses(unit s) 85534 85821 90366 101508 111952 125334 143530 154179
1 2 3 4 5 6 7 8
0
50,000
100,000
150,000
200,000
250,000
300,000
350,000
B
a
t
u
P
a
h
a
t
J
o
h
o
r
B
a
h
r
u
K
l
u
a
n
g
K
o
t
a
T
i
n
g
g
i
M
e
r
s
i
n
g
M
u
a
r
P
o
n
t
i
a
n
S
e
g
a
m
a
t
District
N
o
.
o
f
h
o
u
s
e
s
1991
2000
0
2
4
6
8
10
12
14
0

4
1
0

1
4
2
0

2
4
3
0

3
4
4
0

4
4
5
0

5
4
6
0

6
4
7
0

7
4
Age Category (Years Old)
P
r
o
p
o
r
t
i
o
n
(
%
)
Demand (% sales success)
120 100 80 60 40 20 0
P
r
i
c
e
(
R
M
/
s
q
.
f
t
o
f
b
u
i
l
t
a
r
e
a
)
200
180
160
140
120
100
80
Examples of “abstraction” of phenomena
Demand (% sales success)
120 100 80 60 40 20
P
r
i
c
e
(
R
M
/
s
q
.
f
t
.
b
u
i
l
t
a
r
e
a
)
200
180
160
140
120
100
80
10.00 20.00 30.00 40.00 50.00 60.00
10.00
20.00
30.00
40.00
50.00
100.00
80.00
60.00
40.00
20.00
0.00
20.00
40.00
60.00
80.00
100.00
D
i
s
t
a
n
c
e
f
r
o
m
R
a
k
a
i
a
(
k
m
)
Distance from Ashurton (km)
%
prediction
error
Inferential statistics
Using sample statistics to infer some
“phenomena” of population parameters
Common “phenomena”: causeandeffect
* Oneway r/ship
* Multidirectional r/ship
* Recursive
Use parametric analysis
Y1 = f(Y2, X, e1)
Y2 = f(Y1, Z, e2)
Y1 = f(X, e1)
Y2 = f(Y1, Z, e2)
Y = f(X)
Examples of relationship
Coefficients
a
1993.108 239.632 8.317 .000
4.472 1.199 .190 3.728 .000
6.938 .619 .705 11.209 .000
4.393 1.807 .139 2.431 .017
27.893 6.108 .241 4.567 .000
34.895 89.440 .020 .390 .697
(Constant)
Tanah
Bangunan
Ansi lari
Umur
Fl o_go
Model
1
B Std. Error
Unstandardi zed
Coeffi ci ents
Beta
Standardi zed
Coeffi ci ents
t Si g.
Dependent Vari abl e: Ni l aism
a.
Dep=9t – 215.8
Dep=7t – 192.6
Which one to use?
Nature of research
* Descriptive in nature?
* Attempts to “infer”, “predict”, find “causeandeffect”,
“influence”, “relationship”?
* Is it both?
Research design (incl. variables involved). E.g.
Outputs/results expected
* research issue
* research questions
* research hypotheses
At postgraduate level research, failure to choose the correct data
analysis technique is an almost sure ingredient for thesis failure.
Common mistakes in data analysis
Wrong techniques. E.g.
Infeasible techniques. E.g.
How to design exante effects of KLIA? Development
occurs “before” and “after”! What is the control treatment?
Further explanation!
Abuse of statistics. E.g.
Simply exclude a technique
Note: No way can Likert scaling show “causeandeffect” phenomena!
Issue Data analysis techniques
Wrong technique Correct technique
To study factors that “influence” visitors to
come to a recreation site
“Effects” of KLIA on the development of
Sepang
Likert scaling based on
interviews
Likert scaling based on
interviews
Data tabulation based on
openended questionnaire
survey
Descriptive analysis based
on exante postante
experimental investigation
Common mistakes (contd.) – “Abuse of statistics”
Issue Data analysis techniques
Example of abuse Correct technique
Measure the “influence” of a variable
on another
Using partial correlation
(e.g. Spearman coeff.)
Using a regression
parameter
Finding the “relationship” between one
variable with another
Multidimensional
scaling, Likert scaling
Simple regression
coefficient
To evaluate whether a model fits data
better than the other
Using R
2
Many – a.o.t. BoxCox
_
2
test for model
equivalence
To evaluate accuracy of “prediction” Using R
2
and/or Fvalue
of a model
Holdout sample‟s
MAPE
“Compare” whether a group is
different from another
Multidimensional
scaling, Likert scaling
Many – a.o.t. twoway
anova, _
2
, Z test
To determine whether a group of
factors “significantly influence” the
observed phenomenon
Multidimensional
scaling, Likert scaling
Many – a.o.t. manova,
regression
How to avoid mistakes  Useful tips
Crystalize the research problem → operability of
it!
Read literature on data analysis techniques.
Evaluate various techniques that can do similar
things w.r.t. to research problem
Know what a technique does and what it doesn‟t
Consult people, esp. supervisor
Pilotrun the data and evaluate results
Don‟t do research??
Principles of analysis
Goal of an analysis:
* To explain causeandeffect phenomena
* To relate research with realworld event
* To predict/forecast the realworld
phenomena based on research
* Finding answers to a particular problem
* Making conclusions about realworld event
based on the problem
* Learning a lesson from the problem
Data can‟t “talk”
An analysis contains some aspects of scientific
reasoning/argument:
* Define
* Interpret
* Evaluate
* Illustrate
* Discuss
* Explain
* Clarify
* Compare
* Contrast
Principles of analysis (contd.)
Principles of analysis (contd.)
An analysis must have four elements:
* Data/information (what)
* Scientific reasoning/argument (what?
who? where? how? what happens?)
* Finding (what results?)
* Lesson/conclusion (so what? so how?
therefore,…)
Example
Principles of data analysis
Basic guide to data analysis:
* “Analyse” NOT “narrate”
* Go back to research flowchart
* Break down into research objectives and
research questions
* Identify phenomena to be investigated
* Visualise the “expected” answers
* Validate the answers with data
* Don‟t tell something not supported by
data
Principles of data analysis (contd.)
Shoppers Number
Male
Old
Young
6
4
Female
Old
Young
10
15
More female shoppers than male shoppers
More young female shoppers than young male shoppers
Young male shoppers are not interested to shop at the shopping complex
Data analysis (contd.)
When analysing:
* Be objective
* Accurate
* True
Separate facts and opinion
Avoid “wrong” reasoning/argument. E.g.
mistakes in interpretation.
Introductory Statistics for Social Sciences
Basic concepts
Central tendency
Variability
Probability
Statistical Modelling
Basic Concepts
Population: the whole set of a “universe”
Sample: a subset of a population
Parameter: an unknown “fixed” value of population characteristic
Statistic: a known/calculable value of sample characteristic
representing that of the population. E.g.
μ = mean of population, = mean of sample
Q: What is the mean price of houses in J.B.?
A: RM 210,000
J.B. houses
μ = ?
SST
DST
SD
1
= 300,000
= 120,000
2
= 210,000
3
Basic Concepts (contd.)
Randomness: Many things occur by pure
chances…rainfall, disease, birth, death,..
Variability: Stochastic processes bring in
them various different dimensions,
characteristics, properties, features, etc.,
in the population
Statistical analysis methods have been
developed to deal with these very nature
of real world.
“Central Tendency”
Measure Advantages Disadvantages
Mean
(Sum of
all values
†
no. of
values)
 Best known average
 Exactly calculable
 Make use of all data
 Useful for statistical analysis
 Affected by extreme values
 Can be absurd for discrete data
(e.g. Family size = 4.5 person)
 Cannot be obtained graphically
Median
(middle
value)
 Not influenced by extreme
values
 Obtainable even if data
distribution unknown (e.g.
group/aggregate data)
 Unaffected by irregular class
width
 Unaffected by openended class
 Needs interpolation for group/
aggregate data (cumulative
frequency curve)
 May not be characteristic of group
when: (1) items are only few; (2)
distribution irregular
 Very limited statistical use
Mode
(most
frequent
value)
 Unaffected by extreme values
 Easy to obtain from histogram
 Determinable from only values
near the modal class
 Cannot be determined exactly in
group data
 Very limited statistical use
Central Tendency – “Mean”,
For individual observations, . E.g.
X = {3,5,7,7,8,8,8,9,9,10,10,12}
= 96 ; n = 12
Thus, = 96/12 = 8
The above observations can be organised into a frequency
table and mean calculated on the basis of frequencies
= 96; = 12
Thus, = 96/12 = 8
x 3 5 7 8 9 1 0 1 2
f 1 1 2 3 2 2 1
Ef 3 5 1 4 2 4 1 8 2 0 1 2
Central Tendency–“Mean of Grouped Data”
House rental or prices in the PMR are frequently
tabulated as a range of values. E.g.
What is the mean rental across the areas?
= 23; = 3317.5
Thus, = 3317.5/23 = 144.24
Rental (RM/month) 135140 140145 145150 150155 155160
Midpoint value (x) 137.5 142.5 147.5 152.5 157.5
Number of Taman (f) 5 9 6 2 1
fx 687.5 1282.5 885.0 305.0 157.5
Central Tendency – “Median”
Let say house rentals in a particular town are tabulated as
follows:
Calculation of “median” rental needs a graphical aids→
Rental (RM/month) 130135 135140 140145 15550 150155
Number of Taman (f) 3 5 9 6 2
Rental (RM/month) >135 > 140 > 145 > 150 > 155
Cumulative frequency 3 8 17 23 25
1. Median = (n+1)/2 = (25+1)/2 =13
th
.
Taman
2. (i.e. between 10 – 15 points on the
vertical axis of ogive).
3. Corresponds to RM 140
145/month on the horizontal axis
4. There are (178) = 9 Taman in the
range of RM 140145/month
5. Taman 13
th
. is 5
th
. out of the 9
Taman
6. The interval width is 5
7. Therefore, the median rental can
be calculated as:
140 + (5/9 x 5) = RM 142.8
Central Tendency – “Median” (contd.)
Central Tendency – “Quartiles” (contd.)
Upper quartile = ¾(n+1) = 19.5
th
.
Taman
UQ = 145 + (3/7 x 5) = RM
147.1/month
Lower quartile = (n+1)/4 = 26/4 =
6.5 th. Taman
LQ = 135 + (3.5/5 x 5) =
RM138.5/month
Interquartile = UQ – LQ = 147.1
– 138.5 = 8.6
th
. Taman
IQ = 138.5 + (4/5 x 5) = RM
142.5/month
“Variability”
Indicates dispersion, spread, variation, deviation
For single population or sample data:
where σ
2
and s
2
= population and sample variance respectively, x
i
=
individual observations, μ = population mean, = sample mean, and n
= total number of individual observations.
The square roots are:
standard deviation standard deviation
“Variability” (contd.)
Why “measure of dispersion” important?
Consider returns from two categories of shares:
* Shares A (%) = {1.8, 1.9, 2.0, 2.1, 3.6}
* Shares B (%) = {1.0, 1.5, 2.0, 3.0, 3.9}
Mean A = mean B = 2.28%
But, different variability!
Var(A) = 0.557, Var(B) = 1.367
* Would you invest in category A shares or
category B shares?
“Variability” (contd.)
Coefficient of variation – COV – std. deviation as
% of the mean:
Could be a better measure compared to std. dev.
COV(A) = 32.73%, COV(B) = 51.28%
“Variability” (contd.)
Std. dev. of a frequency distribution
The following table shows the age distribution of secondtime home buyers:
x^
“Probability Distribution”
Defined as of probability density function (pdf).
Many types: Z, t, F, gamma, etc.
“Godgiven” nature of the real world event.
General form:
E.g.
(continuous)
(discrete)
“Probability Distribution” (contd.)
Dice1
Dice2
1 2 3 4 5 6
1 2 3 4 5 6 7
2 3 4 5 6 7 8
3 4 5 6 7 8 9
4 5 6 7 8 9 10
5 6 7 8 9 10 11
6 7 8 9 10 11 12
“Probability Distribution” (contd.)
Values of x are discrete (discontinuous)
Sum of lengths of vertical bars Ep(X=x) = 1
all x
Discrete values
Discrete values
“Probability Distribution” (contd.)
2.00 3.00 4.00 5.00 6.00 7.00
Rental (RM/sq.ft.)
0
2
4
6
8
F
r
e
q
u
e
n
c
y
Mean = 4.0628
Std. Dev. = 1.70319
N = 32
▪ Many real world phenomena
take a form of continuous
random variable
▪ Can take any values between
two limits (e.g. income, age,
weight, price, rental, etc.)
“Probability Distribution” (contd.)
P(Rental = RM 8) = 0 P(Rental < RM 3.00) = 0.206
P(Rental < RM7) = 0.972 P(Rental > RM 4.00) = 0.544
P(Rental > 7) = 0.028 P(Rental < RM 2.00) = 0.053
“Probability Distribution” (contd.)
Ideal distribution of such phenomena:
* Bellshaped, symmetrical
* Has a function of
μ = mean of variable x
σ = std. dev. Of x
π = ratio of circumference of a
circle to its diameter = 3.14
e = base of natural log = 2.71828
“Probability distribution”
μ ± 1σ = ? = ____% from total observation
μ ± 2σ = ? = ____% from total observation
μ ± 3σ = ? = ____% from total observation
“Probability distribution”
* Has the following distribution of observation
“Probability distribution”
There are various other types and/or shapes of
distribution. E.g.
Not “ideally” shaped like the previous one
Note: Ep(AGE=age) ≠ 1
How to turn this graph into
a probability distribution
function (p.d.f.)?
“ZDistribution”
(X=x) is given by area under curve
Has no standard algebraic method of integration → Z ~ N(0,1)
It is called “normal distribution” (ND)
Standard reference/approximation of other distributions. Since there
are various f(x) forming NDs, SND is needed
To transform f(x) into f(z):
x  µ
Z =  ~ N(0, 1)
σ
160 –155
E.g. Z =  = 0.926
5.4
Probability is such a way that:
* Approx. 68% 1< z <1
* Approx. 95% 1.96 < z < 1.96
* Approx. 99% 2.58 < z < 2.58
“Zdistribution” (contd.)
When X= μ, Z = 0, i.e.
When X = μ + σ, Z = 1
When X = μ + 2σ, Z = 2
When X = μ + 3σ, Z = 3 and so on.
It can be proven that P(X
1
<X< X
k
) = P(Z
1
<Z< Z
k
)
SND shows the probability to the right of any
particular value of Z.
Example
Normal distribution…Questions
Your sample found that the mean price of “affordable” homes in Johor
Bahru, Y, is RM 155,000 with a variance of RM 3.8x10
7
. On the basis of a
normality assumption, how sure are you that:
(a) The mean price is really ≤ RM 160,000
(b) The mean price is between RM 145,000 and 160,000
Answer (a):
P(Y ≤ 160,000) = P(Z ≤ )
= P(Z ≤ 0.811)
= 0.1867
Using , the required probability is:
10.1867 = 0.8133
Always remember: to convert to SND, subtract the mean and divide by the std. dev.
160,000 155,000
\3.8x10
7
Ztable
Normal distribution…Questions
Answer (b):
Z
1
=  =  = 1.622
Z
2
=  =  = 0.811
P(Z
1
<1.622)=0.0455; P(Z
2
>0.811)=0.1867
P(145,000<Z<160,000)
= P(1(0.0455+0.1867)
= 0.7678
X
1
 μ
σ
145,000 – 155,000
\3.8x10
7
X
2
 μ
σ
160,000 – 155,000
\3.8x10
7
Normal distribution…Questions
You are told by a property consultant that the
average rental for a shop house in Johor Bahru is
RM 3.20 per sq. After searching, you discovered
the following rental data:
2.20, 3.00, 2.00, 2.50, 3.50,3.20, 2.60, 2.00,
3.10, 2.70
What is the probability that the rental is greater
than RM 3.00?
“Student‟s tDistribution”
Similar to Zdistribution:
* t(0,σ) but σ
n→∞
→1
* ∞ < t < +∞
* Flatter with thicker tails
* As
n→∞
t(0,σ) → N(0,1)
* Has a function of
where I=gamma distribution; v=n1=d.o.f; t=3.147
* Probability calculation requires information on
d.o.f.
“Student‟s tDistribution”
Given n independent measurements, x
i
, let
where μ is the population mean, is the sample
mean, and s is the estimator for population
standard deviation.
Distribution of the random variable t which is
(very loosely) the "best" that we can do not
knowing σ.
“Student‟s tDistribution”
Student's tdistribution can be derived by:
* transforming Student's zdistribution using
* defining
The resulting probability and cumulative
distribution functions are:
“Student‟s tDistribution”
where r ≡ n1 is the number of degrees of freedom, ∞<t<∞,I(t) is the gamma function,
B(a,b) is the beta function, and I(z;a,b) is the regularized beta function defined by
f
r
(t) =
=
F
r
(t) =
=
=
Forms of “statistical” relationship
Correlation
Contingency
Causeandeffect
* Causal
* Feedback
* Multidirectional
* Recursive
The last two categories are normally dealt with
through regression
Correlation
“Coexist”.E.g.
* left shoe & right shoe, sleep & lying down, food & drink
Indicate “some” coexistence relationship. E.g.
* Linearly associated (ve or +ve)
* Codependent, independent
But, nothing to do with CAE r/ship!
Example: After a field survey, you have the following
data on the distance to work and distance to the city
of residents in J.B. area. Interpret the results?
Formula:
Contingency
A form of “conditional” coexistence:
* If X, then, NOT Y; if Y, then, NOT X
* If X, then, ALSO Y
* E.g.
+ if they choose to live close to workplace,
then, they will stay away from city
+ if they choose to live close to city, then, they
will stay away from workplace
+ they will stay close to both workplace and city
Correlation and regression – matrix approach
Correlation and regression – matrix approach
Correlation and regression – matrix approach
Correlation and regression – matrix approach
Correlation and regression – matrix approach
Test yourselves!
Q1: Calculate the min and std. variance of the following data:
Q2: Calculate the mean price of the following lowcost houses, in various
localities across the country:
PRICE  RM „000 130 137 128 390 140 241 342 143
SQ. M OF FLOOR 135 140 100 360 175 270 200 170
PRICE  RM „000 (x) 36 37 38 39 40 41 42 43
NO. OF LOCALITIES (f) 3 14 10 36 73 27 20 17
Test yourselves!
Q3: From a sample information, a population of housing
estate is believed have a “normal” distribution of X ~ (155,
45). What is the general adjustment to obtain a Standard
Normal Distribution of this population?
Q4: Consider the following ROI for two types of investment:
A: 3.6, 4.6, 4.6, 5.2, 4.2, 6.5
B: 3.3, 3.4, 4.2, 5.5, 5.8, 6.8
Decide which investment you would choose.
Test yourselves!
Q5: Find:
(AGE > “3034”)
(AGE ≤ 2024)
( “3539”≤ AGE < “5054”)
Test yourselves!
Q6: You are asked by a property marketing manager to ascertain whether
or not distance to work and distance to the city are “equally” important
factors influencing people‟s choice of house location.
You are given the following data for the purpose of testing:
Explore the data as follows:
• Create histograms for both distances. Comment on the shape of the
histograms. What is you conclusion?
• Construct scatter diagram of both distances. Comment on the output.
• Explore the data and give some analysis.
• Set a hypothesis that means of both distances are the same. Make
your conclusion.
Test yourselves! (contd.)
Q7: From your initial investigation, you belief that tenants of
“lowquality” housing choose to rent particular flat units just
to find shelters. In this context ,these groups of people do
not pay much attention to pertinent aspects of “quality
life” such as accessibility, good surrounding, security, and
physical facilities in the living areas.
(a) Set your research design and data analysis procedure to address
the research issue
(b) Test your hypothesis that lowincome tenants do not perceive
“quality life” to be important in paying their house rentals.
Thank you
Objectives
Overall: Reinforce your understanding from the main lecture Specific: * Concepts of data analysis * Some data analysis techniques * Some tips for data analysis
What I will not do: * To teach every bit and pieces of statistical analysis techniques
Data analysis – “The Concept”
Approach to desynthesizing data, informational, and/or factual elements to answer research questions Method of putting together facts and figures to solve research problem Systematic process of utilizing data to address research questions Breaking down research issues through utilizing controlled data and factual information
social sciences) Statistical/mathematical (pure/applied sciences) AudioOptical (e.Categories of data analysis Narrative (e.g. adopt the first three. arguably. arguably. laws. telecommunication) Others Most research analyses. and social sciences .g.g. The second and third are. applied. arts) Descriptive (e. most popular in pure.
etc. * correlation * anova * manova * regression * econometric modelling Two main categories: * Descriptive statistics * Inferential statistics . events. Simple to complex issues. phenomena.Statistical Methods Something to do with “statistics” Statistics: “meaningful” quantities about a sample of objects. persons. E. Widely used in social sciences.g. things.
2.g.3 = 0.Descriptive statistics Use sample information to explain/make abstraction of population “phenomena”.g. if X.75) * Tendency (leftskew. rightskew) * Causal relationship (e. σ1. Y) * Trend. ttest.g. chisquare. range Used in nonparametric analysis (e. dispersion. then. 2way anova) . pattern. Common “phenomena”: * Association (e.
of houses 200000 150000 100000 50000 0 Loan t o propert y sect or (RM million) Demand f or shop shouses (unit s) Supply of shop houses (unit s) 71719 85534 73892 85821 85843 90366 95916 101508 101107 111952 117857 125334 134864 143530 86323 154179 300.000 200.000 100.1 1991 2000 0 Year (1990 .1997) Trends in property loan.6 3 42468. shop house dem and & supply 200 14 12 Price (RM/sq.6 7 77255.8 2 38100.000 No.2 6 61433.1 4 47684.000 250.000 50.7 8 97810.000 150.000 1 32635.Examples of “abstraction” of phenomena 350. ft of built area) 180 Proportion (%) 10 8 6 4 2 0 160 140 120 100 04 10 1 4 20 2 4 30 3 4 40 4 4 50 5 4 60 6 4 70 7 4 80 0 20 40 60 80 100 120 Age Category (Years Old) Demand (% sales success) Ba tu J o Pa ho h a rB t ah r Kl u Ko ua ta ng Ti n M ggi er si ng M u Po ar n Se tian ga m at District .7 5 48408.
00 40.00 60.00 50.00 40.00 80 20 40 60 80 100 120 10.00 20.00 % prediction error 100.00 80.00 60.00 Distance from Rakaia (km) 160 40.00 0.00 100.Examples of “abstraction” of phenomena 200 180 50.00 140 30.00 60.00 100 10.00 20.00 80.00 30.00 120 20.00 40.00 20.00 Distance from Ashurton (km) Demand (% sales success) .
e2) * Recursive Y1 = f(X. e1) Y2 = f(Y1. Z. Z. e2) Use parametric analysis . e1) Y2 = f(Y1.Inferential statistics Using sample statistics to infer some “phenomena” of population parameters Common “phenomena”: causeandeffect * Oneway r/ship Y = f(X) * Multidirectional r/ship Y1 = f(Y2. X.
317 3.241 .472 1.893 6.567 .6 Coefficientsa Unstandardized Coefficients B Std.728 11. .017 .020 Model 1 (Constant) Tanah Bangunan Ansilari Umur Flo_go t 8.393 1.8 Dep=7t – 192.390 Sig.632 4.199 6. Dependent Variable: Nilaism .440 Standardized Coefficients Beta .139 .000 .108 239.190 .895 89.807 27.Examples of relationship Dep=9t – 215.705 . Error 1993.938 .108 34.000 .697 a.619 4.000 .000 .209 2.431 4.
variables involved). “predict”. failure to choose the correct data analysis technique is an almost sure ingredient for thesis failure. Outputs/results expected * research issue * research questions * research hypotheses At postgraduate level research.Which one to use? Nature of research * Descriptive in nature? * Attempts to “infer”. find “causeandeffect”. “relationship”? * Is it both? Research design (incl.g. . E. “influence”.
g. E. How to design exante effects of KLIA? Development occurs “before” and “after”! What is the control treatment? Further explanation! Abuse of statistics.Common mistakes in data analysis Wrong techniques. E. E. Issue Data analysis techniques Wrong technique To study factors that “influence” visitors to come to a recreation site “Effects” of KLIA on the development of Sepang Likert scaling based on interviews Correct technique Data tabulation based on openended questionnaire survey Descriptive analysis based on exante postante experimental investigation Likert scaling based on interviews Note: No way can Likert scaling show “causeandeffect” phenomena! Infeasible techniques.g.g. Simply exclude a technique .
twoway anova.Common mistakes (contd.) Multidimensional scaling.) – “Abuse of statistics” Issue Data analysis techniques Example of abuse Measure the “influence” of a variable on another Finding the “relationship” between one variable with another To evaluate whether a model fits data better than the other To evaluate accuracy of “prediction” “Compare” whether a group is different from another To determine whether a group of factors “significantly influence” the observed phenomenon Using partial correlation (e.t. Likert scaling Multidimensional scaling.o. manova.o. Likert scaling . BoxCox 2 test for model equivalence Holdout sample‟s MAPE Many – a. Spearman coeff. 2. regression Using R2 and/or Fvalue of a model Multidimensional scaling.o.t. Likert scaling Using R2 Correct technique Using a regression parameter Simple regression coefficient Many – a. Z test Many – a.t.g.
Evaluate various techniques that can do similar things w.Useful tips Crystalize the research problem → operability of it! Read literature on data analysis techniques. to research problem Know what a technique does and what it doesn‟t Consult people. esp. supervisor Pilotrun the data and evaluate results Don‟t do research?? .How to avoid mistakes .t.r.
Principles of analysis Goal of an analysis: * To explain causeandeffect phenomena * To relate research with realworld event * To predict/forecast the realworld phenomena based on research * Finding answers to a particular problem * Making conclusions about realworld event based on the problem * Learning a lesson from the problem .
) Data can‟t “talk” An analysis contains some aspects of scientific reasoning/argument: * Define * Interpret * Evaluate * Illustrate * Discuss * Explain * Clarify * Compare * Contrast .Principles of analysis (contd.
…) Example .Principles of analysis (contd.) An analysis must have four elements: * Data/information (what) * Scientific reasoning/argument (what? who? where? how? what happens?) * Finding (what results?) * Lesson/conclusion (so what? so how? therefore.
Principles of data analysis Basic guide to data analysis: * “Analyse” NOT “narrate” * Go back to research flowchart * Break down into research objectives and research questions * Identify phenomena to be investigated * Visualise the “expected” answers * Validate the answers with data * Don‟t tell something not supported by data .
) Shoppers Male Old Young Female Old Young More female shoppers than male shoppers More young female shoppers than young male shoppers Young male shoppers are not interested to shop at the shopping complex Number 6 4 10 15 .Principles of data analysis (contd.
g.) When analysing: * Be objective * Accurate * True Separate facts and opinion Avoid “wrong” reasoning/argument. E. mistakes in interpretation.Data analysis (contd. .
Introductory Statistics for Social Sciences Basic concepts Central tendency Variability Probability Statistical Modelling .
? A: RM 210.B. E.000 DST . houses μ=? = 210.000 2 = 120.Basic Concepts Population: the whole set of a “universe” Sample: a subset of a population Parameter: an unknown “fixed” value of population characteristic Statistic: a known/calculable value of sample characteristic representing that of the population.g. μ = mean of population.000 SD 3 SST J.B.000 1 = 300. = mean of sample Q: What is the mean price of houses in J.
features. birth. . disease.. in the population Statistical analysis methods have been developed to deal with these very nature of real world.Basic Concepts (contd... characteristics. Variability: Stochastic processes bring in them various different dimensions. death. etc. properties.) Randomness: Many things occur by pure chances…rainfall.
g. group/aggregate data) Unaffected by irregular class width Unaffected by openended class Unaffected by extreme values Easy to obtain from histogram Determinable from only values near the modal class Disadvantages Affected by extreme values Can be absurd for discrete data (e.g.“Central Tendency” Measure Mean (Sum of all values † no. Family size = 4.5 person) Cannot be obtained graphically Needs interpolation for group/ aggregate data (cumulative frequency curve) May not be characteristic of group when: (1) items are only few. (2) distribution irregular Very limited statistical use Median (middle value) Mode (most frequent value) Cannot be determined exactly in group data Very limited statistical use . of values) Advantages Best known average Exactly calculable Make use of all data Useful for statistical analysis Not influenced by extreme values Obtainable even if data distribution unknown (e.
= 12 Thus.9.8.g.7. = 96/12 = 8 .8.5.10. X = {3. = 96/12 = 8 The above observations can be organised into a frequency table and mean calculated on the basis of frequencies x f f 3 1 3 5 1 5 7 2 8 3 9 2 10 12 2 1 14 24 18 20 12 = 96. E.7.Central Tendency – “Mean”. For individual observations. n = 12 Thus.10.12} = 96 . .9.8.
5 150155 152.5 155160 157.5/23 = 144.Central Tendency–“Mean of Grouped Data” House rental or prices in the PMR are frequently tabulated as a range of values.5 6 885.5 Number of Taman (f) 5 9 1282.24 .5 What is the mean rental across the areas? = 23.5 145150 147.g.5 Thus. = 3317. Rental (RM/month) Midpoint value (x) 135140 137.0 1 157. = 3317. E.5 fx 687.0 2 305.5 140145 142.
the median rental can be calculated as: 140 + (5/9 x 5) = RM 142.8 . Median = (n+1)/2 = (25+1)/2 =13th. is 5th. Taman 2. The interval width is 5 7. There are (178) = 9 Taman in the range of RM 140145/month 5. Taman 13th. between 10 – 15 points on the vertical axis of ogive).Central Tendency – “Median” Let say house rentals in a particular town are tabulated as follows: Rental (RM/month) 130135 135140 140145 15550 150155 Number of Taman (f) Rental (RM/month) Cumulative frequency 3 >135 3 5 > 140 8 9 > 145 17 6 > 150 23 2 > 155 25 Calculation of “median” rental needs a graphical aids→ 1. out of the 9 Taman 6. (i. Corresponds to RM 140145/month on the horizontal axis 4.e. Therefore. 3.
) .Central Tendency – “Median” (contd.
5th. Taman UQ = 145 + (3/7 x 5) = RM 147.5 + (4/5 x 5) = RM 142.5 th.1 – 138. Taman IQ = 138.6th.5/5 x 5) = RM138.5 = 8.Central Tendency – “Quartiles” (contd. Taman LQ = 135 + (3.5/month .5/month Interquartile = UQ – LQ = 147.) Upper quartile = ¾(n+1) = 19.1/month Lower quartile = (n+1)/4 = 26/4 = 6.
“Variability” Indicates dispersion. = sample mean. spread. variation. The square roots are: standard deviation standard deviation . xi = individual observations. μ = population mean. and n = total number of individual observations. deviation For single population or sample data: where σ2 and s2 = population and sample variance respectively.
3. 2.0. different variability! Var(A) = 0. 3.28% But.557.6} * Shares B (%) = {1. 1. 1. 3.0.0.8. Var(B) = 1.9.1. 2.5.“Variability” (contd.9} Mean A = mean B = 2.) Why “measure of dispersion” important? Consider returns from two categories of shares: * Shares A (%) = {1.0.367 * Would you invest in category A shares or category B shares? . 2.
deviation as % of the mean: Could be a better measure compared to std.) Coefficient of variation – COV – std. dev. COV(B) = 51.28% .“Variability” (contd.73%. COV(A) = 32.
of a frequency distribution The following table shows the age distribution of secondtime home buyers: x^ .“Variability” (contd. dev.) Std.
General form: (continuous) (discrete) E. gamma. .“Probability Distribution” Defined as of probability density function (pdf). “Godgiven” nature of the real world event.g. F. t. Many types: Z. etc.
) Dice1 Dice2 1 2 3 4 5 6 2 3 4 5 6 7 3 4 5 6 7 8 4 5 6 7 8 9 5 6 7 8 9 10 6 7 8 9 10 11 1 2 3 4 5 6 7 8 9 10 11 12 .“Probability Distribution” (contd.
) Discrete values Discrete values Values of x are discrete (discontinuous) Sum of lengths of vertical bars p(X=x) = 1 all x .“Probability Distribution” (contd.
price.) 6 Frequency 4 2 Mean = 4.00 3.) .00 7.0628 Std.00 5. = 1. etc. weight. Dev. income.00 6.00 4.“Probability Distribution” (contd. rental.g. age.) 8 ▪ Many real world phenomena take a form of continuous random variable ▪ Can take any values between two limits (e.ft.70319 N = 32 0 2.00 Rental (RM/sq.
972 P(Rental 7) = 0.) P(Rental = RM 8) = 0 P(Rental < RM7) = 0.“Probability Distribution” (contd.544 P(Rental < RM 2.053 .00) = 0.00) = 0.206 P(Rental RM 4.00) = 0.028 P(Rental < RM 3.
“Probability Distribution” (contd. symmetrical μ = mean of variable x * Has a function of σ = std. Of x π = ratio of circumference of a circle to its diameter = 3. dev.) Ideal distribution of such phenomena: * Bellshaped.14 e = base of natural log = 2.71828 .
“Probability distribution” μ ± 1σ = ? μ ± 2σ = ? μ ± 3σ = ? = ____% from total observation = ____% from total observation = ____% from total observation .
“Probability distribution” * Has the following distribution of observation .
E.f. Note: p(AGE=age) ≠ 1 How to turn this graph into a probability distribution function (p.g.d.“Probability distribution” There are various other types and/or shapes of distribution.)? Not “ideally” shaped like the previous one .
SND is needed To transform f(x) into f(z): xµ Z = .g.= 0.96 < z < 1.“ZDistribution” (X=x) is given by area under curve Has no standard algebraic method of integration → Z ~ N(0.4 Probability is such a way that: * Approx.926 5. 95% 1. Since there are various f(x) forming NDs. 1) σ 160 –155 E. 68% 1< z <1 * Approx. 99% 2.~ N(0.1) It is called “normal distribution” (ND) Standard reference/approximation of other distributions.58 < z < 2.96 * Approx.58 . Z = .
Z = 3 and so on.e. When X = μ + σ.“Zdistribution” (contd. Z = 2 When X = μ + 3σ.) When X= μ. Z = 1 When X = μ + 2σ. Z = 0. i. It can be proven that P(X1 <X< Xk) = P(Z1 <Z< Zk) SND shows the probability to the right of any particular value of Z. Example .
is RM 155.1867 Using Ztable .8133 160.1867 = 0.000 Answer (a): P(Y ≤ 160.000) = P(Z ≤ ) = P(Z ≤ 0.811) 3.8x107. subtract the mean and divide by the std. dev.8x107 = 0.000 Always remember: to convert to SND. how sure are you that: (a) The mean price is really ≤ RM 160. On the basis of a normality assumption.000 with a variance of RM 3. the required probability is: 10.Normal distribution…Questions Your sample found that the mean price of “affordable” homes in Johor Bahru.000 155. Y.000 (b) The mean price is between RM 145.000 and 160. .
0455. P(Z2>0.000) = P(1(0.000<Z<160.7678 X2 .811 σ 3.622 σ 3.= 0.000 – 155.8x107 P(Z1<1.1867 P(145.1867) = 0.000 .μ 160.000 – 155.Normal distribution…Questions Answer (b): Z1 = .= .000 Z2 = .622)=0.8x107 X1 .= 1.0455+0.= .811)=0.μ 145.
70 What is the probability that the rental is greater than RM 3.00. 3.00. 2.00.60. 3.20.50. 2. 2.10.3. 3. 2. 2. you discovered the following rental data: 2.20.00? .Normal distribution…Questions You are told by a property consultant that the average rental for a shop house in Johor Bahru is RM 3.20 per sq.50. After searching.
v=n1=d. =3.o.1) * Has a function of where =gamma distribution.σ) → N(0.f.σ) but σn→∞→1 * ∞ < t < +∞ * Flatter with thicker tails * As n→∞ t(0. .o.“Student‟s tDistribution” Similar to Zdistribution: * t(0.f.147 * Probability calculation requires information on d.
. and s is the estimator for population standard deviation. xi. Distribution of the random variable t which is (very loosely) the "best" that we can do not knowing σ. let where μ is the population mean. is the sample mean.“Student‟s tDistribution” Given n independent measurements.
“Student‟s tDistribution”
Student's tdistribution can be derived by: * transforming Student's zdistribution using
* defining The resulting probability and cumulative distribution functions are:
“Student‟s tDistribution”
fr(t) =
=
Fr(t) = = = where r ≡ n1 is the number of degrees of freedom, ∞<t<∞,(t) is the gamma function, B(a,b) is the beta function, and I(z;a,b) is the regularized beta function defined by
Forms of “statistical” relationship
Correlation Contingency Causeandeffect * Causal * Feedback * Multidirectional * Recursive The last two categories are normally dealt with through regression
E. Interpret the results? . food & drink Indicate “some” coexistence relationship. * left shoe & right shoe. you have the following data on the distance to work and distance to the city of residents in J.g. nothing to do with CAE r/ship! Example: After a field survey.Correlation “Coexist”. sleep & lying down. E.g. area.B. independent But. * Linearly associated (ve or +ve) Formula: * Codependent.
if Y. + if they choose to live close to workplace. they will stay away from city + if they choose to live close to city. NOT X * If X. then. then. then. ALSO Y * E. then. NOT Y.Contingency A form of “conditional” coexistence: * If X. then.g. they will stay away from workplace + they will stay close to both workplace and city .
Correlation and regression – matrix approach .
Correlation and regression – matrix approach .
Correlation and regression – matrix approach .
Correlation and regression – matrix approach .
Correlation and regression – matrix approach .
variance of the following data: PRICE .RM „000 SQ.RM „000 (x) NO. in various localities across the country: PRICE . M OF FLOOR 130 137 128 390 140 241 342 143 135 140 100 360 175 270 200 170 Q2: Calculate the mean price of the following lowcost houses. OF LOCALITIES (f) 36 3 37 14 38 10 39 36 40 73 41 27 42 20 43 17 .Test yourselves! Q1: Calculate the min and std.
4. 45). What is the general adjustment to obtain a Standard Normal Distribution of this population? Q4: Consider the following ROI for two types of investment: A: 3. 5. 3.8.Test yourselves! Q3: From a sample information. 5.6. 4. . 6.6.8 Decide which investment you would choose.3. 5.6.2.5. 4. a population of housing estate is believed have a “normal” distribution of X ~ (155.2.5 B: 3. 4. 4. 6.2.
Test yourselves! Q5: Find: (AGE > “3034”) (AGE ≤ 2024) ( “3539”≤ AGE < “5054”) .
. You are given the following data for the purpose of testing: Explore the data as follows: • Create histograms for both distances. Comment on the shape of the histograms. What is you conclusion? • Construct scatter diagram of both distances. Make your conclusion. • Explore the data and give some analysis. Comment on the output. • Set a hypothesis that means of both distances are the same.Test yourselves! Q6: You are asked by a property marketing manager to ascertain whether or not distance to work and distance to the city are “equally” important factors influencing people‟s choice of house location.
these groups of people do not pay much attention to pertinent aspects of “quality life” such as accessibility.Test yourselves! (contd. . In this context . you belief that tenants of “lowquality” housing choose to rent particular flat units just to find shelters.) Q7: From your initial investigation. security. and physical facilities in the living areas. good surrounding. (a) Set your research design and data analysis procedure to address the research issue (b) Test your hypothesis that lowincome tenants do not perceive “quality life” to be important in paying their house rentals.
Thank you .
This action might not be possible to undo. Are you sure you want to continue?