You are on page 1of 131

Research Methods in Science and Engineering

Nezih Pala
Electrical & Computer Engineering
Florida International University

Research Methods in Science and Engineering


2 Purpose

• To make attendees familiar with the process of rigorous research in an


academic environment.
• To encourage attendees to critically evaluate research papers they
read.
• To outline the processes required to undertake a research project.

Research Methods in Science and Engineering


3 Topics

1. Overview of the Research Process


2. Literature Search
3. Report Writing, Data Collection & Presentation
4. Statistical Analysis of Data and Sampling
5. Making a Presentation
6. Survey Research Methods
7. Review

Research Methods in Science and Engineering


4 Topic 1

Overview of the Research Process

Research Methods in Science and Engineering


5 What is Research?
• Creative work undertaken on a systematic basis in order to increase the
stock of knowledge and the use of this stock of knowledge to devise new
applications
• Discovery of new things that can been independently verified by other
professionals.
• Something new to humanity (not just new to you or your group).

Research Methods in Science and Engineering


6 Good & Bad Research Examples
• Case 1 A high school research paper
• Case 2 A good idea
• Case 3 Tested outcomes for a new idea

Research Methods in Science and Engineering


7 The Scientific Method

Prior
knowledge An idea
The Outcome is Recognised
as a Major contributionReport,
Submit

to the field Thesis,


Journal
Discovery or
Conference
Paper

Independent verification:
literature, experiment,
Independent verification:
numerical model,
literature,
analytical Assessors
numericalmodel,
model,etc
analytical model, etc

Research Methods in Science and Engineering


8 The Research Community

• All use the same scientific method.


• All follow the same ethical principles.
• All use the same language and terms.
• All provide information to the world-wide community reported in a full
and open manner.
• All acknowledge the previous work of others.

Research Methods in Science and Engineering


9 Publications and Referencing

• The archival literature (must be printed somewhere and unalterable).


• Must be reviewed by independent professionals before publication.
• Must be complete so others can reproduce the results.

• These three form the basic validity test!

Research Methods in Science and Engineering


10 Types of Publication

• Scientific papers (refereed journal and conference papers)


• Trade articles
• Newspaper articles
• Infomercials
• Advertisements

You must only rely on refereed papers in accredited journals and conferences.

Research Methods in Science and Engineering


11 How can you tell?

• Length of title
• References (and their quality)
• Author’s name and affiliation
• Evidence that the paper has been reviewed and revised.
• Date of submission & date of publication.
• The paper includes a review of previously published work.
• Conclusion contains a critical reflection on the contents of the article.

Research Methods in Science and Engineering


12 Topic 2

Literature Search

Research Methods in Science and Engineering


13 Literature Review

• Who has done what and how?


• What is their plan for “further work”?
• Have they reported more recent work in a conference?
• What opportunities are available for confirming the results of others
and expanding their results and conclusions?

Research Methods in Science and Engineering


14 What is Literature?
Literature is a collection of all the scholarly writings on a topic.

These writings can be in the form of scholarly


• peer reviewed articles,
• conference proceedings,
• dissertations
• books,
• and other sources alike.

Research Methods in Science and Engineering


15 Why Literature Review?

A
C

Research Methods in Science and Engineering


16 Why Literature Review?
Research Process
• Understand topic
• reduces time spent looking for information
• helps locate a ‘niche’ in the literature
• helps clarify the scope of your research topic
• helps define and “refine” the research question(s)
• helps find data and research methods
• maximizes quality and appropriateness of results
• helps identify experts/ important works in the field
• Develop your own ideas

Literature Review
Research Methods in Science and Engineering
17 Types of Literature Review
Part of a larger work

e.g. Introduction to a e.g. Thesis / Dissertation


journal article

Selective Comprehensive

e.g. Course assignment e.g. Review article

Stand alone work

Research Methods in Science and Engineering


18 Key Words

• Essential for searching the literature.


• Must be both general and specific.

• Prominent researchers’ names.

• Other useful keywords

Research Methods in Science and Engineering


19 Searching the Web
• Google scholar http://scholar.google.com/

Research Methods in Science and Engineering


20 Searching the Web
• Google scholar http://scholar.google.com/

Research Methods in Science and Engineering


21 Searching the Web
• Journals and publisher’s indexes
– IEEE Xplore digital library http://ieeexplore.ieee.org/Xplore/home.jsp

Research Methods in Science and Engineering


22 Searching the Web
• Journals and publisher’s indexes
– Thomson Reuters Web of Science : https://apps.webofknowledge.com/
– Elsevier http://www.elsevier.com/wps/find/journal_browse.cws_home
– and many more.

Research Methods in Science and Engineering


23 Dissertation Search

Research Methods in Science and Engineering


24 IP Searching
• Patents
– US: http://www.uspto.gov/
– https://www.google.com/?tbm=pts&gws_rd=ssl
– World: http://www.wipo.int/pctdb/en/
– Country Based Searching http://www.wipo.int/ipdl/en/resources/links.jsp

Research Methods in Science and Engineering


25 Citation Managers
Using proper citation style allows us to give credit to the creators of the material we are using. It is
how we use information responsibly and respectfully.

By using citations, our claims and theories become more authorized and credible because we are
providing supporting evidence from other sources. Citations also allow us to be honest about our
contributions and avoid plagiarism.

Citation managers allow researchers to save and organize their citations as well as generate
bibliographies and works cited lists. Citations from numerous databases can be imported into
Citation managers.

Research Methods in Science and Engineering


26 Citation Managers

Research Methods in Science and Engineering


27 Social Networks for Researchers

Research Methods in Science and Engineering


28 Social Networks for Researchers

Research Methods in Science and Engineering


29 The process

Topic: Research: Work with Brain:


Choose, explore Collect Read articles, take
focus. information notes, shape ideas

Keep track of Write:


citations Revise, proof-read,
(RefWorks etc.) include biography

Research Methods in Science and Engineering


30 How to get most out of it?

• Collect & read relevant literature


• Provide an overview of significant literature
• Highlight key concepts & papers
• Describe
• Summarize
• Compare & contrast
• Critically evaluate
• Analyze
• Organize

Research Methods in Science and Engineering


31 Quality of a paper
• Incremental advance compared to paradigm shift.
• Lateral translation research.
• Number of citations says it all:

Research Methods in Science and Engineering


32 How to get most out of it?

Refutes
Supports

Research Methods in Science and Engineering


33 How to get most out of it?

• Discover relationships between sources


• Identify major themes and concepts
• Identify critical gaps and disagreements

Your goal should be further and integrate and synthesize what you find in the
literature into something new. Ideally, you will create your own conceptual map
or outline of the literature on your topic.

Research Methods in Science and Engineering


34 How to get most out of it?

Research Methods in Science and Engineering


35 How to get most out of it?

Research Methods in Science and Engineering


36 Activity

• Use http://scholar.google.co.id/ and enter the key words from the


paper you have.

• Did you find it?


• What else did you find?

Research Methods in Science and Engineering


37 Activity

• Find some scientific terms in your paper, and check the definition.
(Why not wikipedia?)
• Key word searches, key word selection.
• Definition of terms.

Research Methods in Science and Engineering


38 Topic 3

Report Writing

Research Methods in Science and Engineering


39 Publication delays

• Check your paper and see the submission date and the publication
date.
• This delay may mean that this team has moved forward with their
research. Following their suggestions for further work might have you
gazumped.
• Conferences often have a 6 month delay between abstract submission
and the conference presentation.

Research Methods in Science and Engineering


40 Planning for an outcome

• What is convincing “proof”?


• What is the evidence you will provide?
– Data
– Sampling techniques
– Accuracy.
• Who is interested in this research?
• Where will you release (publish/present) your research results?

Research Methods in Science and Engineering


41 Anticipating problems

• Team planning meetings


– Circulate outcomes immediately following the meeting
– Action items
• Equipment calibration
• Reliable power
• Preventing Data loss

Research Methods in Science and Engineering


42 Publication of Data

• Internal report?
• Choosing a conference
• Choosing a journal

Research Methods in Science and Engineering


43 Journal rankings

• Impact factor
• Half life
• Citations (Google, ISI Thomson Web of Knowledge, Scopus, etc)
http://scholar.google.co.id/
• Weaknesses of the ranking systems
• H index – The number of papers that have more than that number of
citations for person.

Research Methods in Science and Engineering


44 Research Planning

• Concurrent Engineering
– Assembling the equipment
– Arranging access to the site
– Writing the paper draft
– Choosing the journal
• Concurrent Research

Research Methods in Science and Engineering


45 Using the right language

• Definition of terms (standards, standard usage, standard methods of analysis).


• Standard Measurement Procedures
• Standard values (eg copper conductivity)
• Spelling (US English or UK English?), Lexicon and naming conventions.
• Key words in publications

• This is vital for accurate electronic searching of indexes.

Research Methods in Science and Engineering


46 The title

• 10-15 words is most common.


• Must be sufficiently specific.

Research Methods in Science and Engineering


47 The Abstract – an example

• High speed electronic beam switching is a desirable feature of


smart antennas.

Research Methods in Science and Engineering


48 The Abstract – an example

• High speed electronic beam switching is a desirable feature of


smart antennas. Most smart antennas are too large for most
applications and require significant power during normal
operations.

Research Methods in Science and Engineering


49 The Abstract – an example

• High speed electronic beam switching is a desirable feature of


smart antennas. Most smart antennas are too large for most
applications and require significant power during normal
operations. A thirteen element switched parasitic antenna was
optimised for gain, speed and beam coverage.

Research Methods in Science and Engineering


50 The Abstract – an example

• High speed electronic beam switching is a desirable feature of


smart antennas. Most smart antennas are too large for most
applications and require significant power during normal
operations. A thirteen element switched parasitic antenna was
optimised for gain, speed and beam coverage. Antenna
characteristics were determined at 1.8 GHz by finite element
modelling and measurements on a prototype.

Research Methods in Science and Engineering


51 The Abstract – an example

• High speed electronic beam switching is a desirable feature of smart


antennas. Most smart antennas are too large for most applications and
require significant power during normal operations. A thirteen element
switched parasitic antenna was optimised for gain, speed and beam
coverage. Antenna characteristics were determined at 1.8 GHz by finite
element modelling and measurements on a prototype. The antenna had a
gain of +9.8 dBi, a footprint of less than one half wavelength squared and
was switched ion less than 100 ms.

Research Methods in Science and Engineering


52 The Abstract – an example

• High speed electronic beam switching is a desirable feature of smart


antennas. Most smart antennas are too large for most applications and
require significant power during normal operations. A thirteen element
switched parasitic antenna was optimised for gain, speed and beam
coverage. Antenna characteristics were determined at 1.8 GHz by finite
element modelling and measurements on a prototype. The antenna had a
gain of +9.8 dBi, a footprint of less than one half wavelength squared and
was switched ion less than 100 ms. This is a better performance
compared to previous antennas.

Research Methods in Science and Engineering


53 The Abstract – a general guide

• 2 sentences on the wider field – context and significance.


• 2 sentences on the research method
• 2 sentences on the results and conclusions.

Research Methods in Science and Engineering


54 Scientific writing style - Do’s and Don’ts

• Past tense
• Third person
• Usually timing of events is not included unless it is essential to data
collection.
• Sections and subsections (one level? two level? three level?).
• Quotes from other authors – not common!

Research Methods in Science and Engineering


55 Creating equations

• There are standard symbols for quantities (eg V=IR).


• There are standard forms for scalar symbols (often lower case, italics, not-bold)
and vector symbols (upper-case, bold).
• The symbols must be the same font on every occasion used in the equations
and in the main text.
• All symbols must be defined.
• MS Equation editor allows for equation creation.
• There are standard upper-case and lower-case type settings.

Research Methods in Science and Engineering


56 Data Collection & Presentation

Research Methods in Science and Engineering


57 Types of Data

• Quantitative data (numerical)


– Integers (eg animal counts, packets received, bit error rate)
– Non-integers (eg analog sensor output)
• Qualitative data (descriptive words)
• Binary data (yes/no, success/failure, present/absent etc)
• Scalar information (1D, 2D, 3D, nD)
• Vector information (1D, 2D, 3D, nD)

Research Methods in Science and Engineering


58 Quantitative Data

• Kelvin’s First Law of Measurement: “The measurement must not alter


the event being measured”.
– Microwave current measurements?
– The impedance of an antenna?

Research Methods in Science and Engineering


59 Data Presentation

• Plots (2D and 3D), histograms, pie charts, tables of numbers.


• Printed papers usually black and white (lines distinguished by dots, dashes, ellipse,
legend etc)
• Colour in power point slides and web publishing.
• For comparison plot more than one data set on the same graph using the same
scale.
• Images and flow charts.
• Interpolation and extrapolation.
• Curve fitting (covered in later lectures)
• Contour plots.

Research Methods in Science and Engineering


60 Plotting and analysis tools

• MS EXCEL (Chart Wizard - 4 steps) - demonstration


• Matlab (plot, subplot, contour, quiver, etc)

Research Methods in Science and Engineering


61 Graphing Guidelines

• Always plot discrete points clearly.


• Do not join points unless you have a continuous mathematical
function.
• To compare data plot several lines on the same axes.
• Consider including error bars on all points

Research Methods in Science and Engineering


62
3.5
3
2.5

Voltage (mV)
2

1.5
1
0.5
0
0 2 4 6 8 10 12
Time (secs)

3.5 3.5

X
3

X
3
2.5 2.5
Voltage (mV)

Voltage (mV)
2 2
1.5 1.5
1 1
0.5 0.5
0 0
0 2 4 6 8 10 12 0 2 4 6 8 10 12
Time (secs) Time (secs)

Research Methods in Science and Engineering


63 Matlab scalar 2D plots
8
contourf image
50 7

40 6

30
5

20
4

10
8
3
6 8
4 6
4
2 2
2
0 0

1
1 2 3 4 5 6 7

surf
50 mesh 1

2
40

3
30

4
20
5
10
8 6
6 8
4 6
7
4
2 2
0 0 8

1 2 3 4 5 6 7

Research Methods in Science and Engineering


64 Matlab vector 2D plots
quiver
8

North-south 5

(meters) 4

1
1 2 3 4 5 6 7 8 9 10 11

East-west (meters)

Research Methods in Science and Engineering


65 Qualitative Data

• This can be a challenge as everyone will use a different description.


• One approach is to convert qualitative data to quantitative data (eg
rate from very bad to very good on a score of 1 to 10).

Research Methods in Science and Engineering


66 Decision Matrix

Vehicle Cost Size Warranty Delivery time Comfort Total Score

Mazda 3 6 8 7 8 8 37
Mazda 2 8 6 7 7 6 34

Ford Focus 6 7 7 8 7 35
Honda 6 6 5 6 5 28
Toyota
Camry 4 8 6 7 8 33
VW 2 6 5 3 7 23

Research Methods in Science and Engineering


67 Decision Matrix - Histogram
40
35 Mazda 3
30
Mazda 2
25
Ford Focus
Score 20
Honda
15
Toyota Camry
10
5 VW
0

rt
y
st

ze

re
nt

tim

fo
Co

co
Si

ra

lS
ry
ar

Co
ive
W

ta
To
l
De

Survey Questions

Research Methods in Science and Engineering


68 Data Collection

• Asking the right questions without leading the person (survey


instruments - questionaires).
• Use redundant questions that always need a positive response
(discussed in a later lecture).
• Survey results (Is 35% return good enough?).

Research Methods in Science and Engineering


69 Flow Charts (MS Word)

Initiate equipment

Yes/No?

Stop process

Research Methods in Science and Engineering


70 Activity

• Plotting analysis using MS eXcel.


• Flow chart using MS word.

Research Methods in Science and Engineering


71 Topic 5

Statistical Analysis and Sampling

90
80
70
60
50 East
40 West
30 North
20
10
0
1st Qtr 2nd Qtr 3rd Qtr 4th Qtr

Research Methods in Science and Engineering


72 Normal Distribution

From: http://mathbits.com/MathBits/TISection/Statistics2/normaldistribution.htm

Research Methods in Science and Engineering


73 Experimental error?

• How does this compare with your results?


• Is your result significant statistically?

Research Methods in Science and Engineering


74 Linear correlation

• Need to fit a line to your data? Quote the linear correlation coefficient
(linear regression)

2.5

2
Sample

1.5

0.5

0
0 5 10 15 20
y = 0.1199x + 0.2876
Voltage
R2 = 0.9498

Research Methods in Science and Engineering


75 Sampling

• Population – every possible candidate.


• Sample population – a small number of candidates selected from the
population.
• It is impossible to know from an examination of your sample alone, if
your sample is representative of the whole population.

Research Methods in Science and Engineering


76 Examples:

• In Australia the total population over 18 years votes in an election.


• Before the election, the press like to take a small sample the population to estimate the
likely outcome of an election. This is called “polling”.
• They hope that the sample is representative of the entire population.
• How do they select a representative sample for a telephone poll?
– Post code?
– Telephone book?
– In the street or shopping centre?
– etc

Research Methods in Science and Engineering


77 All samples may be biased

• Why?
– Age?
– Shyness/openness?
– Work times (shift workers)?
– etc

Research Methods in Science and Engineering


78 Example

• 6 people live in a single house


• We want to randomly select 2 to get an idea of the use of mobile phones in the
house.
• To do this we could:
visit at 10am on a week day.
visit at 3pm on a week day.
telephone at 8pm on a week day.
visit on Saturday morning at 10am.
Visit on Sunday afternoon at 3pm.
– etc

Research Methods in Science and Engineering


79 We ask the question:

• How do you rate your use of a mobile phone on a scale of 1 to 10?


– 10 means very continuously (>20 hours per week)
– 1 means never (<30 minutes per week)

Research Methods in Science and Engineering


80 We have the following opinions

Mary stays at home, goes shopping and drives children to school at 8am and pick up at
4
3pm.
Fred drives to work for night shift. Leaves at 7pm and comes home at 6am. 2
Asif is a 9am – 5pm office worker who rides the train. 5
Sri is a part time sales person drives around the city from 10am to 2 pm. 8
Chen cycles to University 9am and back at 3pm. 7
Rocco is retired and stays in the house all day. 1

Average value is 4.5/10

Research Methods in Science and Engineering


81 How many possibilities?

• If we select 2 people from the total population of n people we


have P combinations where

2!(n  2)!
P
n!
• ! indicates factorial where 5! = 5x4x3x2x1.
• For a population of 6 we have 15 possibilities.

Research Methods in Science and Engineering


82 There are 15 different combinations

• Lowest result from a sample of two people would be Rocco and Fred
(2 and 1) – Mean is 1.5/10.
• Highest sample of two would be Sri and Chen (7 and 8) – Mean is
7.5/10.
• 5 combinations lie between 4 and 5
• 11 combinations lie between 3 and 6
• 13 combinations lie between 2 and 7
• 15 combinations lie between 1 and 8

Research Methods in Science and Engineering


83 Compromise required

• The greater the need for a very accurate result, the smaller the chance
of fulfilling this, even with the best method of approach.

Research Methods in Science and Engineering


84 Sampling Strategies

• Clustered Sampling: Select a sample from only those parts of the population which are
relevant; eg chose only those people who use the road at peak hour.
• Stratified Sampling: Select a sample proportionally to those who are likely to use the road at
peak hour and those that don’t. (4/6 use at peak hour and 2/6 don’t, so use a sample of 3, two
who travel at peak hour and one that does not)
• Destructive Sampling: If the sample is destroyed by sampling (i.e. their mind is changed), then
clearly you should not sample all people.

Research Methods in Science and Engineering


85 Chassis strength testing

• A production line of note book computers produces 2000 units per


day.
• The company is required to strength-test to failure15 samples every
day.
• How do we select those samples?

Research Methods in Science and Engineering


86 The Monte-Carlo Method

• A random sampling technique to define the effect of a large number of


parameters on an outcome. (Usually between 0.1% and 1% of total
population).
• Usually applied to complex systems described by mathematics.
• One randomly selects the parameters and calculates the outcome.
• Used in optimisation.

Research Methods in Science and Engineering


87 Random Sampling

• How can I choose a team of 6 people randomly from this class?


– Family name?
– Student number?
– Seating location in the class?
– Every third person?
• Every person must have an equal probability of being chosen.

Research Methods in Science and Engineering


88 Random Numbers

1 0.5175 0.2455 0.9670 0.7566 0.3222


6 0.3234 0.0239 0.0048 0.6207 0.3796
11 0.4670 0.0300 0.3014 0.6453 0.6414
16 0.3208 0.8862 0.4546 0.3273 0.6023
21 0.0936 0.8864 0.8905 0.1542 0.0377
26 0.8704 0.9132 0.8435 0.1844 0.3351 Excel function
31 0.4451 0.5474 0.2504 0.4552 0.0782
36 0.1478 0.1726 0.7339 0.5332 0.5440
41
46
0.6520
0.9420
0.4870
0.8144
0.8396
0.4230
0.1624
0.9258
0.4911
0.2879
=rand()
51 0.8824 0.9366 0.7085 0.4091 0.2527
56 0.6609 0.5831 0.4059 0.0312 0.4393
61 0.2039 0.5489 0.5263 0.1673 0.6586
66 0.1703 0.4718 0.5256 0.5651 0.3256
71 0.0161 0.7533 0.0915 0.9854 0.0017
76 0.1654 0.3323 0.4037 0.1403 0.9727
81 0.1091 0.1725 0.7821 0.3336 0.1009
86 0.3612 0.5130 0.2648 0.3091 0.3184
91 0.5611 0.3804 0.3079 0.3543 0.9555
96 0.9638 0.8282 0.1850 0.1629 0.3493

Research Methods in Science and Engineering


89 Sample Rate

• Number of samples per second.


• In a digital recording sensor system this might be obvious initially, but there may
be “overheads” when you need time to send and/or store data.
• In an analog system this is regulated by the filter response (eg mechanical
needle, DMM update speed, noise reduction filter).
• Over-sampling and under-sampling.
• Nyquist sampling (twice the maximum frequency of interest).

Research Methods in Science and Engineering


90 Topic 5

Making a Presentation

Research Methods in Science and Engineering


91 Preparing a Power Point Presentation

• Maximum number of slides – one per minute!


• Optimal number of slides – one per 2 minutes
• Use slides as a reminder of what you will say.
• During your presentation, do not read what is on the slides.
• 100 words maximum on each slide.

Research Methods in Science and Engineering


92 Preparing a Power Point Presentation

• Font size? (large!)


• Graphs? (large!)
• Colours? (clearly distinguishable, high contrast, minimal background colour – not
dark)
• Movies? (check on the presentation computer before your talk – usually they
don’t work!)
• Pictures? (not too dark)
• Lighting? (Keep the room lights up so you can see the audience)

Research Methods in Science and Engineering


93 Images

• You MUST acknowledge the source of image if it is not yours including


– MS word image library (in this presentation)
– Pictures taken from web sites
– Pictures taken from colleagues
– Graphs taken from papers etc

Research Methods in Science and Engineering


94 Organisation: 10 minute talk

• Title slide (Name and affiliation) 1


• Outline slide (Major sections) 1
• Introduction (Wider research context) 1
• Main text (method, apparatus, results) 4-6
• Conclusions 1
• References 1

Research Methods in Science and Engineering


95 Nervous?

• Hints for overcoming nervousness:


• Memorise the first 2-3 sentences (opening sentences).
• Make sure you have key words on your power point to trigger your
memory.
• Do not start speaking until the title slide is visible to the audience.

Research Methods in Science and Engineering


96 Being Polite! Before you speak

• Introduce yourself to the session chair before the session starts.


• Load your presentation before the session starts.
• Wait for the chair to introduce you before you speak.
• Switch off your mobile telephone.

Research Methods in Science and Engineering


97 Being Polite! During your talk

• Thank the chairperson for the introduction.


• Speak clearly
• Pretend you are talking to the back row of seats in the room (project your voice).
• Acknowledge your co-authors in Slide 1.
• Rigidly stick to the allocated presentation time.
• Clearly indicate the presentation is finished by a slide and say “thank you” to the
audience.
• Do not invite questions from the audience. (This is the role of the chair person)

Research Methods in Science and Engineering


98 Being Polite! After your talk

• Go quickly back to your seat.


• Do not discuss your paper with others during the next talk.
• If necessary, leave the room (politely – do not slam the door).
• Once the session is complete, thank the chair person.

Research Methods in Science and Engineering


99 Why References?

• For scientific rigour.


• In case someone in the audience has made a major contribution to the
field.
• So the audience can follow up on your previous publications.

Research Methods in Science and Engineering


100 Topic 6

Survey Research Methods

Research Methods in Science and Engineering


101

• This is about how to prepare and analyse a survey


(questionaire)

Research Methods in Science and Engineering


102 “Sick building” Survey

• The research question:


• Do you think that working in this building is making you feel sick?

Research Methods in Science and Engineering


103 Designing a Survey

• Role of the researcher


– Develop the research plan
– Design the survey instrument
– Select the sample population
– Issue/distribute the survey
– Prompt the sample population for responses
– Analyse the data
– Generate conclusions

Research Methods in Science and Engineering


104 Who are the stake-holders

• Selecting the sample population


– Who are the stake-holders?
– What’s in it for them? (No interest can mean no completion)
• Random selection from a large population
• Inclusion –
– Those that are keen to participate will respond
– Are they a biased sample?
• Exclusion
– Will people be offended if they are not asked to respond?

Research Methods in Science and Engineering


105 Who are the stake-holders

• You must be able to defend your sample population selection

Research Methods in Science and Engineering


106 Anonymous Responses

• Arguments for “yes” – Anonymous


– Sample population might be less influenced by who is asking the questions
– Respondents might be less concerned about others learning of their opinions
• Arguments for “no” – Non-anonymous
– Who will you send the results to?
– Who will you send the reward (chocolates) to?
– How do you know who to follow up about returning the survey?

Research Methods in Science and Engineering


107 Confidentiality

• You need to ensure that confidentiality is assured before the survey is


sent out.
• Consider using an independent third party to administer the survey.
• I have been asked to complete a survey which asked for sufficient
personal information to be identified uniquely.
• How will you report “free” comments?

Research Methods in Science and Engineering


108 Feedback

• It is assumed that your sample population (and the full population) will
want access to the results.
• You must explain how will this be done at the beginning of the survey.

Research Methods in Science and Engineering


109 Sample Time lines

• Week 1: Pre-survey letter of introduction (Wider research context and


brief research plan)
• Week 2: Survey send out
• Week 3: Mid-survey reminder letter
• Week 4: Last minute final reminder
• Week 6: Post-survey analysis report completed

Research Methods in Science and Engineering


110 Rating system – 5 point scale

• Strongly disagree 1
• Disagree 2
• Neutral 3
• Agree 4
• Strongly agree 5

• Neutral allows respondents to “sit on the fence”

Research Methods in Science and Engineering


111 Rating system – 4 point scale

• Strongly disagree 1
• Disagree 2
• Agree 3
• Strongly agree 4

• This forces respondents to show positive or negative attitudes.

Research Methods in Science and Engineering


112 Topics for “Sick building” survey

• General personal well being


• Lighting
• Ventilation
• Noise and vibration
• Odour
• Electromagnetic radiation
• Security
• Demographics of respondents

Research Methods in Science and Engineering


113 Hints for writing questions

• Keep is very simple – avoid jargon


• Use one concept per question – avoid multiple concepts
• Keep wording positive – avoid negative words and phrases, double negatives
• The first question should be the “over-all question” – Never place a controversial question at
the beginning.
• Place demographics questions at the end – Demographics at the beginning can raise
suspicions.
• Keep related questions together – Difficult for the respondent to remain coherent
• Use three questions per topic – Do not over question, don’t waste people’s time.

Research Methods in Science and Engineering


114 Statement wording
• I don’t feel well most of the time (negative wording).
• I enjoy good health.
• I am satisfied with the ventilation and the lighting environment (double-barrelled
question).
• I am satisfied with the ventilation.
• I am satisfied with the lighting.
• The University does not do a bad job of keeping us informed about work place health
and safety issues. (double negative)
• The University does a good job of keeping us informed about work place health and
safety issues.
• Many students feel ill as soon as they walk into the building. (projecting the feelings of
others).
• Students enjoy working in this building.

Research Methods in Science and Engineering


115 Judgemental versus Observational

• This work environment is just as good as other places where I have


worked.
• I am happy with this work environment.
• The University listens and acts on student and staff concerns about
the building environment.
• I am satisfied with the University’s response to student concerns about
the building environment.

Research Methods in Science and Engineering


116 Judgemental versus Observational

• This work environment is just as good as other places where I have


worked.
• I am happy with this work environment.

• What if you asked both statements to be rated?


• The conclusions would be different

Research Methods in Science and Engineering


117 Reverse scoring

• Q10: I am not happy with this work environment. (1 – 5)


• Q35: I am happy with this work environment. (1 – 5)

• You would need to reverse score Q10 for proper statistics.


• The dangers include:
– Donkey vote gives confusion (What do you do if you get 5 for both?)
– Was the question misread?
– Was the respondent annoyed?

Research Methods in Science and Engineering


118 Sample Open ended questions and comments

• Please identify at least three things that cause you concern in


this work environment.
• Please identify at least three things that you like about this work
environment.

Research Methods in Science and Engineering


119 Reporting

• Calculate averages and statistics for each theme.


• Construct a Histogram and report the mean value
• E.g. 80% rated the noise environment neutral or better.
• Or: 20% indicated that the noise environment was not good.
• Report selective quotes on open questions.

Research Methods in Science and Engineering


120 Missing Data

• Did the respondent simply forget one question?


• Maybe the question was not relevant to that person?
• Was the question too personal?
• Was the question confusing? Could it have been scored as a 1 for one
interpretation and a 5 using another interpretation.

Research Methods in Science and Engineering


121 Accuracy and Reliability

• On a 5 point scale there are 5 possible answers.


• Your mean value for the sample population can be expressed to
several decimal places.
• How many places are significant?
• Return to Normal Distribution statistics based on z score.

Research Methods in Science and Engineering


122 References

• Connolly, P.M. & Connolly, K.G., 2004, Employee opinion


questionaires, Pfeiffer.
• Rosenfeld, P., Edwards, J.E., & Thomas, M.D., (eds), 1993, Improving
organizational surveys, SAGE Pub.
• Images from MS Word Clip Art.

Research Methods in Science and Engineering


123 Review

Research Methods in Science and Engineering


124 1. The Research Process

• Independent verification of results.


• Designing the experiment for outcomes
• Journal rankings

Research Methods in Science and Engineering


125 2. Literature Search

• Using the web etc

Research Methods in Science and Engineering


126 3. Report writing, Data Collection & Presentation

• Abstract
• Referencing
• Equations
• Figures
• Conclusions and Further work
• Qualitative and quantitative data
• Plotting techniques for multi-dimensional data

Research Methods in Science and Engineering


127 4. Statistical Analysis and Sampling

• Regression analysis
• How to select a random sample.

Research Methods in Science and Engineering


128 5. Making a Presentation

Research Methods in Science and Engineering


129 6. Survey research methods

• How to create and analyse a survey.

Research Methods in Science and Engineering


130 Why this presentation?

• To develop an understanding of the scientific environment in which


research is conducted.

Research Methods in Science and Engineering


131 Student Evaluation of Course

• Survey!

Research Methods in Science and Engineering

You might also like