You are on page 1of 135

Master of System Engineering Faculty of Engineering Gadjah Mada University

Research Methodology and Project Proposal Preparation


Adhy Kurniawan Faculty of Engineering Gadjah Mada University

Adhy Kurniawan
1987-1990 SMA 3 Semarang 1990-1991 Fac. Of Economy, Diponegoro University, Semarang S1(1991-1996) Civil Engineering Dept. Gadjah Mada Univ. S3(1998-2003) Swiss Federal Institute of Technology Lausanne (EPFL), Swiss Post Doct (nov.2005-sept.2006) Kyoto University, Japan

My Goals for Course


That each of you develop an intuition for the fundamental principles of research methodology That we have an enjoyable semester learning together

Lecture and Homework


Lecture
Presentation and discussion

Homework
Your chance to practice using the concepts presented in class Teamwork vs. Individual work?

References
All of literature concerning: Res Met Marczyk, DeMatteo, Festinger. 2005, Essentials of Research Design and Methodology, John Wiley and Sons. Day and Gastel, 2006, How to write and Publish a Scientific Report, Greenwood Press Metodologi Riset, Etc.

List of students
Alif Ardy Saputra, Geodesi UGM Anik FR, TL, ITB Ashri Uswatun, TFisika,UGM Ayi Fajarwati, TL, ITB Corry Agustina, Perenc Wil, TA, UGM Dwi Astuti, TKimia, UGM Elva Nur , TF, UGM Erika Kezia, TL, ITB Fitri Wijayanti, Fisika, UNS I Nyoman Kusuma, TF, UGM Ihsan Hasan, T Industri, UII, Ihwan Ghazali, T Industri, UAD Iin Lestari, TL, ITB M Sony Abertiawan, TL, ITB Maria Auliana, T Sipil, UGM Norma Pradipta, TArsitektur, UGM Satrya Alrizki, TGeofisik, ITB Tatag Lindu Bhakti, TFisika, UGM

Contents
The aims of research, the research topic, title and research problem, literature review, research design: population and sampling types, types of quantitative research designs, validity of conclusions, data-collecting methods and measuring instruments in quantitative research, qualitative research designs, data analysis and interpretation of results, report writing and the research proposal, ethical consideration on research.

OVERVIEW OF SCIENCE AND THE SCIENTIFIC METHOD


science can be defined as a methodological and
systematic approach to the acquisition of new knowledge. This definition of science highlights some of the key differences between how scientists and nonscientists go about acquiring new knowledge. Specifically, rather than relying on mere casual observations and an informal approach to learn about the world, scientists attempt to gain new knowledge by making careful observations and using systematic, controlled, and methodical approaches (Shaughnessy & Zechmeister, 1997).
Shaughnessy, J. J., & Zechmeister, E. B. (1997). Research methods in psychology (4th ed.). Boston: McGraw Hill.

In addition, scientific knowledge is not based on the opinions, feelings, or intuition of the scientist. Instead, scientific knowledge is based on objective data that were reliably obtained in the context of a carefully designed research study. In short, scientific knowledge is based on the accumulation of empirical evidence (Kazdin, 2003a)
Kazdin, A. E. (2003a). Methodology: What it is and why it is so important. In A. E. Kazdin ( Ed.), Methodological issues and strategies in clinical research (3rd ed., pp. 522). Washington, DC: American Psychological Association.

The defining characteristic of scientific research is the scientific method . First described by the English philosopher and scientist Roger Bacon in the 13th century, it is still generally agreed that the scientific method is

the basis for all scientific investigation.


The scientific method is best thought of as an approach to the acquisition of new knowledge, and this approach effectively distinguishes science from nonscience.

The Scientific Method


The development of the scientific method is usually credited to Roger Bacon, a philosopher and scientist from 13th-century England, although some argue that the Italian scientist Galileo Galilei played an important role in formulating the scientific method. Later contributions to the scientific method were made by the philosophers Francis Bacon and Ren Descartes.

Although some disagreement exists regarding the exact characteristics of the scientific method, most agree that it is characterized by the following elements:
Empirical approach Observations Questions Hypotheses Experiments Analyses Conclusions Replication

Empirical Approach
The scientific method is firmly based on the empirical approach. The empirical approach is an evidence-based approach that relies on direct observation and experimentation in the acquisition of new knowledge (see Kazdin, 2003a). In the empirical approach, scientific decisions are made based on the data derived from direct observation and experimentation. Contrast this approach to decision making with the way that most nonscientific decisions are made in our daily lives. For example, we have all made decisions based on feelings, hunches, or gut instinct. Additionally, we may often reach conclusions or make decisions that are not necessarily based on data, but rather on opinions, speculation, and a hope for the best. The empirical approach, with its emphasis on direct, systematic, and careful observation, is best thought of as the guiding principle behind all research conducted in accordance with the scientific method.

Observations
An important component in any scientific investigation is observation. In this sense, observation refers to two distinct conceptsbeing aware of the world around us and making careful measurements. Observations of the world around us often give rise to the questions that are addressed through scientific research. For example, the Newtonian observation that apples fall from trees stimulated much research into the effects of gravity. Therefore, a keen eye to your

surroundings can often provide you with many ideas for research studies.

Questions
After getting a research idea, perhaps from making observations of the world around us, the next step in the research process involves translating that research idea into an answerable question. The term answerable is particularly important in this respect, and it should not be overlooked. It would obviously be a frustrating and ultimately unrewarding endeavor to attempt to answer an unanswerable research question through scientific investigation. It is therefore important to formulate a research question that can be answered through available scientific methods and procedures.

Hypotheses
The next step in the scientific method is coming up with a hypothesis, which is simply an educatedand testableguess about the answer to your research question. A hypothesis is often described as an attempt by the researcher to explain the phenomenon of interest. Hypotheses can take various forms, depending on the question being asked and the type of study being conducted. A key feature of all hypotheses is that each must make a prediction. Remember that hypotheses are the researchers attempt to explain the phenomenon being studied, and that explanation should involve a prediction about the variables being studied. These predictions are then tested by gathering and analyzing data, and the hypotheses can either be supported or refuted on the basis of the data.

Two types of hypotheses with which you should be familiar are the null hypothesis and the alternate (or experimental) hypothesis. The null hypothesis always predicts that there will be no differences between the groups being studied. By contrast, the alternate hypothesis predicts that there will be a difference between the groups. For example, the null hypothesis would predict that the exercise group and the no-exercise group will not differ significantly on levels of cholesterol. The alternate hypothesis would predict that the two groups will differ significantly on cholesterol levels. Homework: Individual Please try to find one example. About the null

Experiments
After articulating the hypothesis, the next step involves actually conducting the experiment (or research study). For example, if the study involves investigating the effects of exercise on levels of cholesterol, the researcher would design and conduct a study that would attempt to address that question. As previously mentioned, a key aspect of conducting a research study is measuring the phenomenon of interest in an accurate and reliable manner. In this example, the researcher would collect data on the cholesterol levels of the study participants by using an accurate and reliable measurement device. Then, the researcher would compare the cholesterol levels of the two groups to see if exercise had any effects.

Accuracy vs. Reliability


When talking about measurement in the context of research, there is an important distinction between being accurate and being reliable. Accuracy refers to whether the measurement is correct, whereas reliability refers to whether the measurement is consistent. An example may help to clarify the distinction. When throwing darts at a dart board, accuracy refers to whether the darts are hitting the bulls eye (an accurate dart thrower will throw darts that hit the bulls eye). Reliability, on the other hand, refers to whether the darts are hitting the same spot (a reliable dart thrower will throw darts that hit the same spot). Therefore, an accurate and reliable dart thrower will consistently throw the darts in the bulls eye. As may be evident, however, it is possible for the dart thrower to be reliable, but not accurate. For example, the dart thrower may throw all of the darts in the same spot (which demonstrates high reliability), but that spot may not be the bulls eye (which demonstrates low accuracy).

Analyses
After conducting the study and gathering the data, the next step involves analyzing the data, which generally calls for the use of statistical techniques. The type of statistical techniques used by a researcher depends on the design of the study, the type of data being gathered, and the questions being asked. It is important to be aware of the role of statistics in conducting a research study. In short, statistics help researchers minimize the likelihood of reaching an erroneous conclusion about the relationship between the variables being studied.

Conclusions
After analyzing the data and determining whether to reject the null hypothesis, the researcher is now in a position to draw some conclusions about the results of the study. For example, if the researcher rejected the null hypothesis, the researcher can conclude that the phenomenon being studied had an effect a statistically significant effect, to be more precise.

If the researcher rejects the null hypothesis in our exercise-cholesterol example, the researcher is concluding that exercise had an effect on levels of cholesterol.

It is important that researchers make only those conclusions that can be supported by the data analyses. Going beyond the data is a cardinal sin that researchers must be careful to avoid.

Replication
One of the most important elements of the scientific method is replication. Replication essentially means conducting the same research study a second time with another group of participants to see whether the same results are obtained. The same researcher may attempt to replicate previously obtained results, or perhaps other researchers may undertake that task.

Replication illustrates an important point about scientific researchnamely, that researchers should avoid drawing broad conclusions based on the results of a single research study because it is always possible that the results of that particular study were an aberration. In other words, it is possible that the results of the research study were obtained by chance or error and, therefore, that the results may not accurately represent the actual state of things. However, if the results of a research study are obtained a second time (i.e., replicated), the likelihood that the original studys findings were obtained by chance or error is greatly reduced.

What are the three general goals of scientific research?

Answer:
description, prediction, and understanding/explaining

What Exactly is Research?


we will focus on two of the most common types of research
correlational research and experimental research

Correlational research:
In correlational research, the goal is to determine whether two or more variables are related. (By the way, variables is a term with which you should be familiar. A variable is anything that can take on different values, such as weight, time, and height.) For example, a researcher may be interested in determining whether age is related to weight. In this example, a researcher may discover that age is indeed related to weight because as age increases, weight also increases. If a correlation between two variables is strong enough, knowing about one variable allows a researcher to make a prediction about the other variable.

It is important to point out, however, that a correlation or relationshipbetween two things does not necessarily mean that one thing caused the other.To draw a causeand-effect conclusion, researchers must use experimental research. .

Experimental research:
In its simplest form, experimental research involves comparing two groups on one outcome measure to test some hypothesis regarding causation. For example, if a researcher is interested in the effects of a new medication on headaches, the researcher would randomly divide a group of people with headaches into two groups. One of the groups, the experimental group, would receive the new medication being tested. The other group, the control group, would receive a placebo medication (i.e., a medication containing a harmless substance, such as sugar, that has no physiological effects).

Experimental research:
Besides receiving the different medications, the groups would be treated exactly the same so that the research could isolate the effects of the medications. After receiving the medications, both groups would be compared to see whether people in the experimental group had fewer headaches than people in the control group. Assuming this study was properly designed (and properly designed studies will be discussed in detail in later chapters), if people in the experimental group had fewer headaches than people in the control group, the researcher could conclude that the new medication reduces headaches.

Task
Compose your own brief research proposal. Try to determine your research topic for MST final project Format:
1. In MS Word 2. In Power point

Task/assignment next week


Review 1 International Publication (Journal, Conference paper,etc) related to Renewable energy Compose the summarize of your review Format:
1. In MS Word 2. In Power point

Purpose of the research proposal


1. To inform the reader of nature of your proposed research.
What is the problem? What is its extent?

2. To convince the reader, especially supervisors and reviewers, of the value of your proposed research.
Is this project worth the time and money? Will it make a difference to the world?

Purpose of the research proposal


3. To demonstrate your expertise and competency in a particular area of study.
Do you have the qualifications to conduct this research? Have you informed yourself of the existing theory and data relevant to your topic? Do you have the necessary skills to conduct the research?

Purpose of the research proposal


4. To plan the research project and provide a step-by-step guide to the tasks necessary for its completion.
What are the key stages of the work? What are the priorities? How do the various components fit together?

5. To request support from individuals and agencies who provide supervision, oversight or funding for the research project.
What kinds of support does the project need? Are all participants properly protected?

Purpose of the research proposal


6. To contract with the agencies and individuals involved, including supervisors, foundations and participants in the research team.
How will tasks be assigned and resources expended? What does each contribute to the collective endeavor?

First things first


1. 2. 3. 4. 5. Basics Topic ideas Typical methodologies Common pitfalls Getting started and putting it all together 6. Questions/discussion

Basic steps of a research project


Find a topicWhat, When Formulate questionsWhat, Why Define populationWho, When Select design & measurementHow Gather evidenceHow Interpret evidenceWhy Tell about what you did and found out

Selecting a Research Topic


What are some considerations when selecting a research topic?

Considerations in Selecting a Topic


Personal interest / Passion Importance / Contribution to the field Newness / Relevance Feasibility
Tradeoff between rigor and practicality Time constraints Ethical constraints Organizational support Economic factors Availability of Subjects

Sources of Research Topics


Peer-reviewed journals in your field Personal experiences Work setting experiences Existing literature
Recommendations for future research

Refining Your Topic


Refinement needed for effective and efficient research
Narrow your topic Identify a theoretical framework Specifically and unambiguously define terms State research questions and hypotheses

Refining Your Topic (contd)


A literature review will help you
See if your idea has been tried Include all relevant constructs Select instruments Anticipate common problems

Components of a Concept Paper


Title page Introduction Nature of the Problem Background and Significance of the Problem Preliminary Literature Review Initial Research Question or Questions

Components of a Concept Paper (contd)


Brief Description of Methodology and Research Design Anticipated Outcomes Timeline References

The Literature Review

What is a Literature Review?


According to Creswell (2005), a review of the literature is a written summary of journal articles, books and other documents that describes the past and current state of information, organizes the literature into topics and documents a need for a proposed study. (pp. 79)
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research

Focusing on Empirical Research


What does Empirical Mean? Primary Sources
Original Research Article

Secondary Sources
Newspapers Book chapters Television/Radio Magazines Wikepedia

Empirical Research
All empirical research is inherently flawed
Limitations

1. Sampling

Generalizability Representative Measurement Error Social Desirability Grasping the Whole Problem

2. Measurement

3. Problem Identification

Literature Reviews
Well-written analytical narrative that brings a reader up-to-date on what is known on a given topic, but also provide fresh insights that advance knowledge
Resolve conflicts between studies Identify new ways to interpret research results Creating a path for future research

Anecdotal Reports
A description of an event or experience that happened to be noticed
No control No comparison

Review of Key Elements of Previous Definition


The LR is a summary of research:
It is not a list of found research but a coherent and articulate account of past and current research findings Suggestion: read 2 or 3 LRs in order to become familiar with summary styles

Review of Key Elements of Previous Definition (contd)


The sources typically are journal articles, books and other documents that describe past and present status of research in a given field: The LR should be exhaustive and as

current as possible.

How many articles?


There is no set number. As long as the search is exhaustive and focused on the research topic, the review will be acceptable.

Review of Key Elements of Previous Definition (contd)


How far back should one search? A reasonable and widely accepted timeframe includes research conducted during the past 10 years. Important studies (i.e., studies that had a significant impact on the field of study) should also be mentioned even if these go beyond the mentioned timeframe.

Review of Key Elements of Previous Definition (contd)


The LR should be organized:
The review should not only be coherent, but should organize the studies reviewed under themes or topics. The reviewer is a guide and should be able to provide readers with an in-depth and current status of research in a given area. This aspect is essential for readers to understand what the reviewer found during the search.

Review of Key Elements of Previous Definition (contd)


The LR should document the need for a proposed study:

Studies should not duplicate research that has been already done. Even in cases when research is duplicated (replicated is the appropriate term), one is responsible for documenting the need for replication, e.g., need to explore the same methodology with a different group or population, or need to change methodology with the same group.

Creswells 5 steps to Conduct a Literature Review


Step 1: Identify Key Terms or Descriptors
Extract key words from your title (remember, you may decide to change the title later) Use some of the words other authors reported in the literature
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research

Step 1: Identify Key Terms or Descriptors (contd)


Use the Thesaurus of ERIC Descriptors to look for terms that match your topic: go to www.eric.ed.gov and in Search select Descriptors (from Thesaurus) Scan both electronic and library journals from the past 10 years and look for key terms in the articles
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research

Creswells 5 steps to Conduct a Literature Review (contd)


Step 2: Locate Literature
Use academic libraries, do not limit your search to an electronic search of articles Use primary and secondary sources. A primary source is research reported by the researcher that conducted the study. A secondary source is research that summarizes or reports findings that come from primary sources
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research

Step 2: Locate Literature (contd)


It is best to report mostly primary sources (p. 82) Search different types of literature: summaries, encyclopedias, dictionaries and glossaries of terms, handbooks, statistical indexes, reviews and syntheses, books, journals, indexed publications, electronic sources, abstract series, and databases
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research

Creswells 5 steps to Conduct a Literature Review (contd)


Step 3: Critically Evaluate and Select Literature

Rely on journal articles published in national journals Prioritize your search: first look for refereed journal articles, then, non-refereed articles, then books, then conference papers, dissertations and theses and then papers posted to websites

Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research

Step 3: Critically Evaluate and Select Literature (contd)


Look for research articles and avoid as much as possible opinion pieces Blend qualitative and quantitative research in your review

Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research

Creswells 5 steps to Conduct a Literature Review (contd)


Step 4: Organize the Literature
Create a file or abstract system to keep track of what you read. Each article you read should be summarized in one page containing
Title (use APA to type the title so that you can later copy-paste this into the References section of your paper) Source: journal article, book, glossary, etc.
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research

Step 4: Organize the Literature (contd)


Research problem: one or two lines will suffice Research Questions or Hypotheses Data collection procedure (a description of sample characteristics can be very handy as well) Results or findings of the study

Sort these abstracts into groups of related topics or areas which can then become the different sections of your review
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research

Creswells 5 steps to Conduct a Literature Review (contd)


Step 5: Write a Literature Review
Types of Reviews:
Thematic Review: a theme is identified and studies found under this theme are described. Major ideas and findings are reported rather than details.

Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research

Step 5: Write a Literature Review (contd)


Study-by-study Review: a detailed summary of each study under a broad theme is provided. Link summaries (or abstracts) using transitional sentences. Must be organized and flow coherently under various subheadings. Avoid string quotations (i.e., lengthy chunks of text directly quoted from a source)

Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research

Preliminary Literature Review


This succinct review of current literature should:
Provide further contextual background Reveal issues related to your study Describe similar problems in other organizations Provide significance to your approach to the study

Guidelines on Style, Mechanics, and Language Usage

Does your draft follow the logic or idea that is presented in your intro and title? Avoid overusing direct quotations, especially long ones Check style manual for correct use of citations
(Doe, 2005); Doe (2005); (Doe & Smith, 2005); Doe and Smith (2005); (Black, 2005; Brown, 2006; Yellow, 2007)

Guidelines on Style, Mechanics, and Language Usage


Avoid using synonyms for recurring words
This is not creative writing and stay consistent with terminology
Group I, Phoenix Cohort, Experimental Group

Spell out all acronyms when first using them

Traditional - American Psychological Association (APA) Non-traditional - Collective Efficacy (CE)

Yes - Do NOT use contractions; No Dont use contractions Coined terms should be set off by quotes

Guidelines on Style, Mechanics, and Language Usage Avoid the following:


Slang cool Colloquialisms thing >> item or feature Idioms rise to the pinnacle >> to become prominent

Use great care to avoid Plagiarism

What needs to be included in the Literature review.


Provides contextual background Reveals related issues Reviews similar problems elsewhere Provides significance to your approach to the study Includes major/seminar research articles pertaining to study Written in an integrated manner Uses peer-reviewed research Includes a Reference section

Writing Your Research Question(s)


Reflect the problem that the researcher wants to investigate Can be formulated based on theories, past research, previous experience, or the practical need to make data-driven decisions in a work environment

Writing Your Research Question(s) (contd)


Are vitally important because they, in large part, dictate what type of statistical analysis is needed, as well as what type of research design may be employed A research question should address only 1 concept Question must be measurable

Types of Questions Asked


Once you have identified the topic of study, you will need to consider the type of question you want answered and how it will be answered Two paradigms Quantitative Paradigm Generally attempt to quantify variables of interest. Questions frequently address how well or how much.

Types of Questions Asked


Qualitative Paradigm there are times when we wish to know not how many or how well, but simply how. (Shulman, 1988, pg. 7)

Class Exercise
Now youre ready to formulate your own research question(s) Sample questions: Is there a relationship between participation in an Elluminate chat session and course grade? How do 5th grade students experience the anticipation of standardized testing?

Research Questions
From Topic to Research Question A good research topic asks a clear, concise question. Asking a research question helps you keep a tight focus on your topic. Tweaking Your Research Question A good research topic is broad enough to allow you to find plenty of material, but narrow enough to fit within the size and time constraints of your paper.
If your topic is either too broad or too narrow, consider adding or eliminating the following elements:

Time Period, century, decade, future, Population Type, age, gender, nationality, species, Geographic Location country, state, region, Point of View economic, social, cultural, biological

Assignment 2 Components
(see syllabus for details)

Title Page Nature of the Problem Background and Significance of the Problem Literature Review Research Questions References

Topic ideas
Online chat reference
Types of questions
Subject? Type? # of turnaways*

Difference in discourse
In-person vs. chat

Partnership studies
Similar libraries with same software

Topic Ideas
E-book usage Usability studies of
Online tutorial(s) My Library portals

Analysis of library web sites or library instruction sites or pathfinders by best practices Student learning outcomes in LI programs

Types of methodologies
QuaLitative Measures
Descriptive Numbers not the primary focus Interpretive, ethnographic, naturalistic

QuaNtitative Measures
N for numbers Statistical Quantifiable

QuaLitative measures
Content Analysis
Analyzed course syllabi of library use through discipline and level (Rambler) Studied online tutorials, applying best practices recommendations (Tancheva)

QuaLitative Measures
Discourse Analysis
Analyzed student responses in writing and discussions to a short film & compared findings to parallel study with LIS grad Ss (Vandergrift) Discussed how participants experience & use the library (Von Seggern & Young) Studied why students use the Internet and how much time they use it (Wilson)

Focus Groups

QuaLitative Measures
Interviews
Studied 25 HS students web use for research assignments (Lorenzen) Looked at what type of information first year students need and how they go about acquiring it (Seamans) Observed students as they conducted online research & noted their activities (Dunn) Retrieval of discarded cheat sheets to analyze academic misconduct (Pullen et. al.)

Observation (obtrusive)

Observation (Unobtrusive)

QuaLitative Measures
Think Aloud Protocols
Studied how users navigate a library web site (Cockrell & Jayne)

Usability testing
Examined students mental models of online tutorials (Veldof & Beavers)

QuaNtitative measures
CompareThings Count Things Survey People About Things

QuaNtitative measures
Comparison studies
Experimental and control groups Instructional methodologies (Colaric; Cudiner & Harmon) Program assessment using before/after analysis of research papers(Emmons & Martin)

QuaNtitative measures
Pre & Post Tests (Van Scoyoc) Measures & Scales
Bosticks Library Anxiety Scale (Onwuegbuzie & Jiao; Van Scoyoc) Procrastination Assessment Scale (Onwuegbuzie & Jiao)

QuaNtitative measures
Numeric Studies
Citation AnalysisBibliometrics (Dellavalle) Webometrics (Bar-Ilian)

Ready Made Data Sets


National Survey of Student Engagement (Whitmire) College Student Experiences Questionnaire (Kuh and Gonyea) The Web

Internet Archive (Ryan, Field & Olfman) Electronic journals (Dellavalle)

Library server logs

Common Pitfalls
Problems with population
Sampling?
Representativeness? Self-selection?

Research Problem #1

A study assessing student learning outcomes in 2 broad categories (concepts, techniques) by examining student research journals in 1 section of an elective information literacy course in fall semester.

Research Problem #4

A 2004 article on a library use and services satisfaction study that used as its measurement tool a survey given to every nth person entering the library building on 40 randomly selected days throughout the school year.

Research Problem #5

An outcomes assessment research project of a 5 year old IL program in which all incoming freshmen must participate. Total student population on campus is divided between 32% freshmen to senior (or 4 year) and 68% transfer students.

Common Pitfalls
Problems with operationalization
Defining of what is measured

Research Problem #2
An experimental study that proposes a fund allocation formula for academic library collections based on the following:
average of overall book price + average of overall serial prices * degree level (10 for undergraduate to 30 for doctorate) / the number of students enrolled in degree program as majors + the total number of faculty in the department * three * total number of students in program. (OAB + OAS) * D/(Sn +(Fn*3))*Sn
N.B. Not a standard formula

Research Problem #3 A newspaper article you read just the other day stated that in a recently published study done at a major U.S. university, researchers found that domestic violence affects 1 in every 4 women.

Research Problem #4 A 2004 article on a library use and services satisfaction study that used as its measurement tool a survey given to every nth person entering the library building on 40 randomly selected days throughout the school year.

Research Problem #5
Over a one year period, researchers studied the occurrence of turn-aways in a virtual reference service and noted that the significantly high occurrence of turn-aways indicates increased need for virtual reference service.

Common Pitfalls
Problems with generalizability
False conclusions Transformations

Research Problem #1

A study assessing student learning outcomes in 2 broad categories (concepts, techniques) by examining student research journals in 1 section of an elective information literacy course in fall semester.

Research Problem #7 A survey of faculty found that the majority of those interviewed interacted most with librarians at the reference desk. The researchers concluded that most faculty view librarians in a servile role.

Keep In Mind That


No study is perfect All data is dirty is some way or another; research is what you do with that dirty data (Manuel) Measurement involves making choices

Be Critical About Numbers


(Best 2001)

Every statistic is a way of summarizing complex information into relatively simple numbers. (Best) How did the researchers arrive at these numbers? Who produced the numbers and what is their bias? How can key terms be defined & in how many different ways?

Be Critical About Numbers


How was the choice for the measurement made? What type of sample was gathered & how does that affect result? Is the statistical result interpreted correctly? If comparisons are made, are they appropriate? Are there competing statistics?

Getting Started
Read to learn; read to analyze
About research methodology Studies on similar topics Interesting studies Non-library studies

Getting Started
Finding a topic neednt be traumatic
Work projects Research studies
P&T overhaul Library GO Bond Proposal Project Library workshop trends User repair strategies

Getting Started
Data collection involves agreement & consent Forge partnerships At some point you will need to leave the comfort zone of reading and literature gathering and

Just get out and do it!

Questions?

Research methodology
Quantitative Methods Qualitative procedures

Quantitative Methods

A definition
A survey or experiment that provides as output a quantitative or numeric description of some fraction of the population, called the sample.

Components of a survey method


The survey design The population and sample The instrumentation Variables in the study Data analysis

The survey design


Purpose of the survey The research question Type of survey
Cross sectional Longitudinal

Form of data collection

The population and sample


Description of the population Sampling design
Single stage Multistage Stratified

Sample selection

The instrumentation
The instrument (tool)
Existing New

Rating scale
1. 2. 3. 4. 5.

Likert scale: Rating the Items. 1-to-5 rating scale where:


= strongly unfavorable to the concept = somewhat unfavorable to the concept = undecided = somewhat favorable to the concept = strongly favorable to the concept

Pilot Administration

Postal survey email

Variables and analysis


The research question Variable in the research

E.g. Number of years of academic study E.g. How many years of study in a University
As an undergraduate? As a postgraduate?

The questions in the instrument

Data analysis

Steps Bias in the data

Statistics, e.g. mean, standard deviation etc.

Non-response

Components of an experimental method


Subjects Instruments and materials The experimental design

Subjects
Selection
Conveniently Random (RCT)

Group assignment
Random Matched. E.g. Size
Ability, Age

Variables
Dependent Independent

Randomized Controlled Trial (RCT)


A true experiment, in which the researcher randomly assigns some patients to at least one maneuver (treatment) and other patients to a placebo, or usual treatment. Key features = the classic way to evaluate effectiveness of drugs (or exercise, diet, counseling). Patients are followed over time (Prospective). If properly done, an RCT can be used to determine cause and effect

Instrumentation and Materials


Description Validation
Pilot Content validity Prediction validity

Materials

The experimental design


Type
Pre-experimental
No control group

Quasi-experimental
Control group, but not randomly assigned

Single subject design (over time) Pure experiment Repeated measures


Change groups

Overview of Qualitative Research Design


Historical routes in anthropology
Generates new understanding by naming and framing concepts and themes Removes bias by questioning preconceived assumptions of the social group under study Promotes neutrality through adoption by the researcher of nave stance or critical discussion, challenges pre-conceived assumptions of both the researcher and the social group under study Produces new understanding about the world, changes the way power, culture and social interaction are understood

Data Collection in Qualitative Research


Observation (Videoed, non-participant, semiparticipant and participant observation, field notes) Interviews (individual and group - known as focus groups, tape recorded and transcribed, field notes) Secondary data analysis (using written material collected for purposes other than research) Questionnaires (unstructured, postal, interviews) A mixture of all four

Questions in Qualitative Research


In qualitative research questions are open-ended. Sometimes a check list or topic guide will be used by the researcher to ensure all the relevant areas are covered. This is known as semi-structured data collection. It is used in all four methods of data collection Sometimes the only guide is the topic itself and the researcher collects verbatim or naturally occurring data. This is known as unstructured data collection. It is used in all four methods of data collection

Sampling in Qualitative Research


The sampling method of choice is theoretical sampling (queuing behaviour) However, often this is not possible and people resort to convenience sampling (students) and snowball sampling (mental health in black and ethnic minority communities) Neither of the latter two methods are considered strong but maybe all that can be achieved. Research must be viable.

Data Analysis in Qualitative Research


Read and re-read data, become engrossed in it. Identify themes: common, conflicting, minority

Test themes across the data set, where are they common, under what circumstances are they found, not found. This sets the parameters on the interpretation and generalisation of data Get more than one person to analyse the data independently then together
Demonstrate trustworthiness in data analysis

Examples
Biographical continuity Nursing routines as a method of managing a transient workforce

Qualitative research
Interpretative research Process orientated Researcher(s) are the primary data collection instrument Descriptive research Outputs are an inductive process

References
MSc project web pages
http://www.comp.glam.ac.uk/gis/start.asp?whatfile=gis/gis rc/msc-proj.htm

Creswell, J. W. (1994) Research design : qualitative and quantitative approaches. - Thousand Oaks, Calif.; London : Sage Publications, ISBN 0803952546