Data Collection and Analysis:

Interpretation and providing solution

1
 Have you ever collected data so far?

What is your experience and how you did it?

2
Data Collection – Quantitative and
Qualitative
 After
 developing the instruments
 Pilot and Main (actual)
 determine the data sources
 In case of experimental , analytical or predictive

Setting the objectives of the design artifacts and later
after the construction of the artifact.
 Requires sufficient preparation - has to be reported

3
Developing Data Collection Instruments
 Two general approaches (in case of people respondent)
 Adopting from similar previous researches
 Mention how and what part is adopted
 Crafting yourself based / inline with specific objectives
 No orphan question or objectives should be observed
 And pilot study to validate

 Describe procedures of data sources accessing and

acquisition (in case of non-human data sources)

4
 Actual Data Collection
• One should carefully plan the data collection as this
is the departure for execution of the research
• Pre-data collection
– Training of Data Collectors might be crucial
– Supporting letters might be necessary
• Post-data collection
– Editing of returned questionnaires
• The data you have collected may be presented using
–5 Tabular methods
Cont…
 Followed by analysis
 In strict sense in case of quantitative but not in qualitative
 In qualitative you collect- analyze- collect…
 Until “saturation “

 Analysis could start

 by arranging; presentation
 by description (like in NLP and DM)
 By structuring requirements

6
Exploring and Organizing a Data Set
 Before employing any statistical procedure, develop
habit of looking closely at your data and exploring
various ways of organising them – detect patterns
 Example : reading test scores of 11 children.
Ruth, 96; Robert, 60; Chuck, 68; Margaret, 88; Tom,
56;
Mary, 92; Ralph, 64; Bill, 72; Alice, 80; Adam, 76;
Kathy, 84
 What do you see? Arrange and look for patterns
 Careful researchers discover everything possible
about their data, whether the information is
immediately useful or not
Arranging The Data
 Alphabetical arrangement – look for meaning or
pattern, no matter how trivial
 Observable fact – highest score was earned by a girl
and that the lowest score was earned by a boy –
although silly and meaningless, it’s an observable
fact, and it may come in handy at a future time
 Symmetrical pattern – same sex arrangement -
 Separate girls from boys
 The graph shows dramatic trends – the girls’ scores increase
as we proceed through the alphabet, and the boys’ scores
decrease
Dramatic Trends
 Not only is there a divergence of trends, the scores are
equidistant from one another. Each score is 4 points either
above or below the preceding one
 May have no relevance to research but it represents dynamics
within the data
 The researcher should be aware of the dynamics, the
phenomena, that are active within the data, whether those
phenomena are important to the purpose of the research or not
 The astute (clever at seeing quickly how to gain an advantage)
researcher overlooks nothing
 Another arrangement of the data would have probably made a
different meaning more apparent.
Fundamental Guideline for Looking
at the Data
 Whatever the researcher does with the data to prepare it for
inspection or interpretation will affect the meaning that those
data reveal
 Therefore, every researcher should be able to provide a clear,
logical rationale for the procedure used to arrange and
organise the data
 There is no rationale whatever for arranging the test scores
according to the children’s first names
 Had their last names being used, which would have been
equally illogical, we would still have seen that the girls had
higher scores than the boys, but we would not necessarily have
seen the diverging trends.
Drawing Conclusions from the Data
 Questions from the example
 Why were all the scores of the girls higher than those
of the boys?
 Why is this algorithm working better?
 Why were the intervals between each of the scores
equidistant for both boys and girls?
 Knowledge springs from questions like these
 But must be careful not to make snap judgments
Drawing Conclusions from the Data
 Even the most thorough research can go
astray(wrong) at the point of drawing conclusions
 The example – might conclude that girls read better
than boys – not thinking carefully
 Reading is a complex and multifaceted skill
Organising Data to Make Them
Easier to Think About and Interpret
 Another example: Joe’s daily grade at high school in the order
they were earned – a simple linear sequence
92, 69, 91, 70, 90, 89, 72, 87, 73, 86, 85, 75, 84, 76, 83, 83, 77,
81, 78, 79
 Arrange the grades in a two-dimensional table
 Read vertically the grades on Monday, Wednesday, and Friday
– they are considerably higher than those on Tuesday and
Thursday
 Look at successive scores in each column – they get
progressively worse on Mondays, Wednesdays, and Fridays,
but progressively better on Tuesdays and Thursdays
 The researcher should explore the data further – it may have a
cause for behaving that way
 Tabular methods of data presentation
• Tabulated data can be more easily understood than
facts
• They help facilitate statistical treatment of data
• When data are tabulated, all unnecessary details and
repetitions are avoided.
• Type of tables
– Simple (one way) table: shows one characteristic
– Two-way table: shows two characteristics
– Higher order table: shows three or more
characteristics

14
 Tabular
Tabularmethods
methodsofofdata
datapresentation
presentation
(Frequency
(Frequencydistributions)
distributions)
•• Steps
Steps
–– Begin
Beginby byarranging
arrangingthe thedata
datafrom
fromsmallest
smallestto tolargest
largest
–– Count
Countvalues
valuesthat
thatrepeat
repeatbybymaking
makingtallies
tallies
–– Group
Groupobservations
observationswith withcomparable
comparablemagnitude
magnitude
–– Stop
Stopthe theclassification
classificationwhen
whenyou
youare
aresure
surethat
thatthe
thefirst
firstand
and
the
thelast
lastclasses
classesrespectively
respectivelyconsist
consistthe
thesmallest
smallestand
andlarges
larges
values
values
–– Indicate how many many values
values and
are included in a class
Note:
Note:
IfIfthe
thenumber
numberof ofclasses
classeskkhas
hasbeen
beenfixed,
fixed,then
thenclass
classwidth
width
may
maybe befixed
fixedasasww==range/k
range/k

15
 Graphical methods of data presentation
• Data in a frequency distribution can be presented
graphically or diagrammatically
• Graphs are the natural choice to represent
continuous data
• For discrete or qualitative data, we have
– Pie chart (multiply relative frequency by 3600
– Pictogram (use of pictures)
– Bar graph (class limit and Abs. frequency)

16
 Graphical methods … cont’d
• For continuous data, we have
– Histogram (class boundary and abs. freq.)
– Frequency polygon (Class mark and abs freq.)
– Cumulative frequency graph (class mark and
cumulative frequency)

17
Analysis
 Statistical descriptive and inferential or experiments for
quantitative
 Understanding numbers is very much important
 Parameter /Variable settings (assumptions..)
 Understanding meanings through pattern matching,
content analysis, time series analysis … using coding
 Providing “Thick” description is important including rival
explanations.
 Design and Construction of an artifact
 Following standards and principles is important
 May need input from qualitative type of analysis

18
Interpretation and Discussion
 Explain the results in light of previous literatures and
theories.
 No clear distinction in case of qualitative
 Involves demonstration and evaluation in case of design
research
 May be required to collect data and analysis with the same
procedures as we have seen before

19
Example (In design science )

22
24
25
26
THE TOOLS OF RESEARCH- with
respect to data collection and analysis

 Tools are chosen to facilitate research tasks

 Some researchers need special tools indigenous to
particular discipline
 The concern here is with the general tools of research that
the majority of researchers, regardless of discipline and
situation, typically need to derive meaningful and insightful
conclusions from the data they collect
 Be careful not to equate tools of research with the
methodology of research
THE TOOLS OF RESEARCH
 Research tool = specific mechanism or strategy the
researcher uses to collect, manipulate, or interpret data.

 Research methodology = general approach the researcher

takes in carrying out the research project; to some extent,
this approach dictates the particular tools the researcher
selects.
THE TOOLS OF RESEARCH
 Statisticsis always ancillary to basic research
 However to insist the use of statistics will deny
valid research in non-quantitative investigation
THE TOOLS OF RESEARCH
There are six general tools of research:
1. The library and its resources
2. The computer and its software
3. Techniques of measurement
4. Statistics
5. The human mind
6. Language
1. The Library and Its Resources
 Scholar should know its principal resources and
understand its classification system, and find the shortest
route to the information it contains.
 You learn the library by using the library
 Feel the mood of research in a library
 Libraries have manual
 Learn where the various holdings are located
 Catalogue is the heart of the library – books , films,
filmstrip, tapes, phonograph records, microfilm, maps,
pictures, slides, CDs, …
Gateway to UPM Library
UPM Library Catalogue
The Treasury of Knowledge
 After selecting research problem, the library is the FIRST
place to clarify the dimension of the problem
 Learn what others have done in the area or in corollary
investigation
 To receive ideas that help to sharpen the focus of research
UPM Online Database
ACM Digital Library
2. The Computer and its Software – Tool of
Research
a. Taking advantage of the Internet
The World Wide Web (WWW) is the world of knowledge.
Web browsers: Mozilla Firefox, Netscape, IE, etc.
Web site: journals, publishers, organisations, individuals, etc.
Search engine: google, yahoo, Alta Vista, etc.
b. E-Mail
Faster and to individual or a group of people.
Asking questions to authors, experts, etc.
Facilitate collaboration among people.
Attached file (reports, etc.)
c. News
List servers: E-discussion group.
Many groups with particular interests.
3. Measurement – Tool of Research
 Researchers strive for objectivity: not influenced by own
perceptions, impressions, and biases.
 Therefore, must identify systematic way of measuring a
phenomenon
 Old proverb – if it exists, then it can be measured
 If it is researchable, then data must be measurable
 Many ways we can measure data
 Measuring data has nothing to do with physical measurement
Quantifying
 Measurement is quantifying of any phenomenon,
substantial or insubstantial, concrete or abstract, and
involves the comparison of the data being measured to a
pre-established standard.

 Quantifying mean “how much”, how many”, “to what

degree” you think of the world and its manifestations
through the data observed in terms of magnitude and
significance.
Scales of Measurement
 In 1946, Stevens suggested a hierarchy of levels of
measurement, widely adopted by statisticians and
researchers as means of classifying data
 Four types of measurements, nominal, ordinal, interval,
and ratio

 Nominal means “name” – can measure data by assigning

name to data
 Can measure a group of children by dividing into two
groups – boys and girls
Ordinal
 Think of the quantity being measured in terms of the
symbol < and >, higher or lower, greater or lesser,
younger or older
 Always an asymmetrical relationship
 Level of education grossly on ordinal, unschooled,
primary, secondary, college, graduate
 Work force, unskilled, semi-skilled, skilled
Interval
 Two features : 1) equal units of measurement
2) zero point established arbitrarily
 Most familiar of interval scale is the thermometer
 What is the freezing point and the boiling point for
Celsius and Fahrenheit?? 0 (100) and 32 (212)
 Common use is the rating scale - 5 point measurement of
academic teaching effectiveness
 Can determine mode, mean, std deviation, t-test, F-test,
product moment correlation
Ratio
 A scale that measures in terms of equal intervals and
absolute zero point of origin
 Difference between interval and ratio: thermometer, we
cannot say 80oF is twice as warm as 40oF, because it does
not originate from point of absolute zero. If it is, then we
cannot measure temperature below zero
Ratio
 True zero as “the total absence of the quantity being
measured”, cannot measure minus distance
 Can be used for determining the geometric mean, the
harmonic mean, the percent variation and all other
inferential statistical analysis
Measurement Summary
 If you can say that:
 one object is different from another, you have a nominal
scale
 One object is bigger or better or more of anything than
another, you have an ordinal scale
 One object is so many units (degrees, inches) more than
another, you have an interval scale
 One object is so many times as big or bright or tall or
heavy as another, you have a ratio scale
Validity and Reliability
 Two very important considerations
 Validity is concerned with the soundness, the effectiveness of
the measurement instrument
 Example, a standardize test, what does the test measure? Does
it, in fact, measure what it is supposed to measure? How well,
how comprehensively, how accurately does it measure?
 Scale that measure professor’s availability, “always available”
– what does “always” mean?
Validity and Reliability
 Validity looks to the end results – are we really measuring
what we think we are measuring?
 Reliability – what accuracy does the measure (test,
instrument, inventory, questionnaire) what it is intended to
measure?
 4. Statistics as a Tool of Research
 All tools are MORE suitable for some purposes than for others.
 Example : screw driver was designed to INSERT and remove
screws – BUT people often used it for punching holes, scratch
away paints, etc…..misuse……
 So, too, with statistics
 Statistics can be a powerful tool when used correctly (for specific
kind of data & research questions)
 BUT can be misleading when applied in other contexts.
 4. Statistics as a Tool of Research
 More useful in some academic disciplines than in others.
 REMEMBER, the statistical values obtained are never the
end of research nor the final answer to research problem.
 The final question is “What do the data indicate” not what is
their numerical configuration.
 Statistics give information about the data BUT a
conscientious researcher is not satisfied until the MEANING
of this information is revealed.
 Primary Function of Statistics
1) Describe the data
2) Draw inference from the data

 Descriptive Statistics summarize the general nature of the

data, average, variability, closeness of two or more
characteristics, etc.
Primary Functions of Statistics
 Inferential Statistics help in making decision about the data:
decide whether the differences observed between two groups in
an experiment are large enough to be attributed to the
experimental intervention rather than to a once-in-a-blue-moon
fluke.

 Both involve summarizing data in some ways and create

entities that have no counterpart in reality.

 Example : students work 24, 22, 12, and 16 hours per week.
The average is 18.5 but NO student work exactly 18.5
hours/week.
5. The Human Mind – Tool of Research

 Statistics can tell us the centre, the spread, relationship of

data BUT cannot interpret and arrive at a logical
conclusion or meaning.
 Only mind can do.
 Mind is the most important tool.
 Nothing equals its powers of comprehension, integrative
reasoning and insight.
The Human Mind
 Strategies to make use of the human mind to better
understand include :

1. Deductive logic
2. Inductive reasoning
3. Scientific method
4. Critical thinking
5. Collaboration with others
Critical Thinking
 During LR don’t just accept research findings and
theories at face value.
 Scrutinize for faulty assumptions, questionable
logic, weaknesses in methodology, inappropriate
statistical analyses, and unwarranted conclusions.
 Good researchers engage in critical thinking.
 Involves evaluating information or arguments in
terms of their accuracy and worth.
 Critical Thinking
 Take a variety of forms, depending on the context.
1. Verbal reasoning – Understanding and evaluating the
persuasive techniques found in oral and written language.
2. Argument analysis – Discriminating between reasons that
do and do not support a particular conclusion.
3. Decision making – Identifying and judging several
alternatives and selecting the best alternative.
4. Critical analysis of prior research.
Critical Analysis of Prior Research
 Evaluating the value of data and research results in terms of the
methods used to obtain them and their potential relevance to
particular conclusion.
 Consider these questions
1. Was an appropriate method used to measure a particular
outcome?
2. Are the data and results derived from a relatively large number
of people, objects, or events?
3. Have other possible explanations or conclusions been
eliminated?
4. Can the results obtained in one situation be reasonably
generalized to other situations?
Collaboration with Others
 More heads are better than one.
 A researcher has certain perspectives, assumptions, and
theoretical biases – not to mention holes in knowledge
about subject matter – that limit research approaches of a
project.
 Need to bring colleagues who have perspectives,
backgrounds, and areas of expertise somewhat different –
more cognitive resources to tackle research problem and
how to find meaning.
Collaboration with Others
 Can be equal partners or
 Simply offer suggestions and advice.
 Graduate students themselves are the key players.
 Typically they are assigned an advisor or advisory
committee.
 Prudent (careful) student selects committee that will
make genuine contribution.
Make sure that your data collection
and analysis is logical!!!

62