DATA COLLECTION

Data collection is the process of gathering and measuring information on variables of interest, in an
established systematic fashion that enables one to answer stated research questions, test hypotheses,
and evaluate outcomes. The data collection component of research is common to all fields of study
including physical and social sciences, humanities, business, etc. While methods vary by discipline, the
emphasis on ensuring accurate and honest collection remains the same.

The importance of ensuring accurate and appropriate data collection

Regardless of the field of study or preference for defining data (quantitative, qualitative), accurate data
collection is essential to maintaining the integrity of research. Both the selection of appropriate data
collection instruments (existing, modified, or newly developed) and delineated instructions for their
correct use reduce the likelihood of errors occurring.

Consequences from improperly collected data include

 inability to answer research questions accurately

 inability to repeat and validate the study
 distorted findings resulting in wasted resources
 misleading other researchers to pursue fruitless avenues of investigation
 compromising decisions for public policy
 causing harm to human participants and animal subjects

While the degree of impact from faulty data collection may vary by discipline and the nature of the
investigation, there is the potential to cause disproportionate harm when these research results are used
to support public policy recommendations.

Issues related to maintaining the integrity of data collection:

The primary rationale for preserving data integrity is to support the detection of errors in the data
collection process, whether they are made intentionally (deliberate falsifications) or not (systematic or
random errors).

Most, Craddick, Crawford, Redican, Rhodes, Rukenbrod, and Laws (2003) describe ‘quality assurance’
and ‘quality control’ as two approaches that can preserve data integrity and ensure the
scientific validity of study results. Each approach is implemented at different points in the research
timeline (Whitney, Lind, Wahl, 1998):

1. Quality assurance - activities that take place before data collection begins

2. Quality control - activities that take place during and after data collection

Quality Assurance

Since quality assurance precedes data collection, its main focus is 'prevention' (i.e., forestalling problems
with data collection). Prevention is the most cost-effective activity to ensure the integrity of data collection.
This proactive measure is best demonstrated by the standardization of protocol developed in a
comprehensive and detailed procedure manual for data collection. Poorly written manuals increase the
risk of failing to identify problems and errors early in the research endeavor. These failures may be
demonstrated in some ways:

 Uncertainty about the timing, methods, and identity of the person(s) responsible for reviewing
data
 A partial listing of items to be collected
  Unclear description of data collection instruments to be used instead of rigorous step-by-step
instructions on administering tests
 Failure to identify specific content and strategies for training or retraining staff members
responsible for data collection
 Obscure instructions for using, making adjustments to, and calibrating data collection equipment
(if appropriate)
 No identified mechanism to document changes in procedures that may evolve throughout the
investigation.

An important component of quality assurance is developing a rigorous and detailed recruitment and
training plan. Implicit in training is the need to effectively communicate the value of accurate data
collection to trainees (Knatterud, Rockhold, George, Barton, Davis, Fairweather, Honohan, Mowery,
O'Neill, 1998). The training aspect is particularly important to address the potential problem of staff who
may unintentionally deviate from the original protocol. This phenomenon, known as ‘drift’, should be
corrected with additional training, a provision that should be specified in the procedures manual.

Given the range of qualitative research strategies (non-participant/ participant observation, interview,

archival, field study, ethnography, content analysis, oral history, biography, unobtrusive research) it is
difficult to make generalized statements about how one should establish a research protocol to facilitate
quality assurance. Certainly, researchers conducting non-participant/participant observation may have
only the broadest research questions to guide the initial research efforts. Since the researcher is the main
measurement device in a study, many times there are little or no other data collecting instruments.
Indeed, instruments may need to be developed on the spot to accommodate unanticipated findings.

Quality Control

While quality control activities (detection/monitoring and action) occur during and after data collection,
the details should be carefully documented in the procedures manual. A clearly defined communication
structure is a necessary pre-condition for establishing monitoring systems. There should not be any
uncertainty about the flow of information between principal investigators and staff members following the
detection of errors in data collection. A poorly-developed communication structure encourages lax
monitoring and limits opportunities for detecting errors.

Detection or monitoring can take the form of direct staff observation during site visits, conference calls, or
regular and frequent reviews of data reports to identify inconsistencies, extreme values, or invalid codes.
While site visits may not be appropriate for all disciplines, failure to regularly audit records, whether
quantitative or quantitative, will make it difficult for investigators to verify that data collection is
proceeding according to procedures established in the manual. In addition, if the structure of
communication is not delineated in the procedures manual, the transmission of any change in
procedures to staff members can be compromised

Quality control also identifies the required responses, or ‘actions’ necessary to correct faulty data
collection practices and also minimize future occurrences. These actions are less likely to occur if data
collection procedures are vaguely written and the necessary steps to minimize recurrence are not
implemented through feedback and education (Knatterud, et al, 1998)

Examples of data collection problems that require prompt action include:

 errors in individual data items

 systematic errors
 violation of protocol
 problems with individual staff or site performance
 fraud or scientific misconduct

In the social/behavioral sciences where primary data collection involves human subjects, researchers are
taught to incorporate one or more secondary measures that can be used to verify the quality of
information being collected from the human subject. For example, a researcher conducting a survey
might be interested in gaining a better insight into the occurrence of risky behaviors among young adults
as well as the social conditions that increase the likelihood and frequency of these risky behaviors.

To verify data quality, respondents might be queried about the same information but asked at different
points of the survey and in several different ways. Measures of ‘Social Desirability’ might also be used to
get a measure of the honesty of responses. Two points need to be raised here, 1) cross-checks within the
data collection process and 2) data quality being as much an observation-level issue as it is a complete
data set issue. Thus, data quality should be addressed for each measurement, for each observation, and
the entire data set.

Each field of study has its preferred set of data collection instruments. The hallmark of laboratory
sciences is the meticulous documentation of the lab notebook while social sciences such as sociology and
cultural anthropology may prefer the use of detailed field notes. Regardless of the discipline,
comprehensive documentation of the collection process before, during, and after the activity is essential to
preserving data integrity.

Sources:

Knatterud.,G.L., Rockhold, F.W., George, S.L., Barton, F.B., Davis, C.E., Fairweather, W.R., Honohan, T.,
Mowery, R, O’Neill, R. (1998). Guidelines for quality assurance in multicenter trials: a position
paper. Controlled Clinical Trials, 19:477-493.

Most, .M.M., Craddick, S., Crawford, S., Redican, S., Rhodes, D., Rukenbrod, F., Laws, R. (2003). Dietary
quality assurance processes of the DASH-Sodium controlled diet study. Journal of the American Dietetic
Association, 103(10): 1339-1346.

Whitney, C.W., Lind, B.K., Wahl, P.W. (1998). Quality assurance and quality control in longitudinal
studies. Epidemiologic Reviews, 20(1): 71-80.

DATA COLLECTION METHODS

Quantitative and Qualitative Data collection methods

The Quantitative data collection methods rely on random sampling and structured data collection
instruments that fit diverse experiences into predetermined response categories. They produce results
that are easy to summarize, compare, and generalize. 

Quantitative research is concerned with testing hypotheses derived from theory and/or being able to
estimate the size of a phenomenon of interest.  Depending on the research question, participants may be
randomly assigned to different treatments.  If this is not feasible, the researcher may collect data on
participant and situational characteristics to statistically control for their influence on the dependent, or
outcome, variable. If the intent is to generalize from the research participants to a larger population, the
researcher will employ probability sampling to select participants. 

Typical quantitative data gathering strategies include:

 Experiments/clinical trials.
 Observing and recording well-defined events (e.g., counting the number of patients waiting in an
emergency at specified times of the day).
 Obtaining relevant data from management information systems.
  Administering surveys with closed-ended questions (e.g., face-to-face and telephone interviews,
questionnaires, etc).

Interviews

In Quantitative research (survey research), interviews are more structured than in Qualitative research.

In a structured interview, the researcher asks a standard set of questions and nothing more. (Leedy and
Ormrod, 2001)

Face-to-face interviews have a distinct advantage of enabling the researcher to establish rapport with
potential participants and therefore gain their cooperation. These interviews yield the highest response
rates in survey research. They also allow the researcher to clarify ambiguous answers and when
appropriate, seek follow-up information. Disadvantages include impractical when large samples are
involved time consuming and expensive. (Leedy and Ormrod, 2001)

Telephone interviews are less time-consuming and less expensive and the researcher has ready access
to anyone on the planet who has a telephone. Disadvantages are that the response rate is not as high as
the face-to-face interview as but considerably higher than the mailed questionnaire. The sample may be
biased to the extent that people without phones are part of the population about whom the researcher
wants to draw inferences.

Computer-Assisted Personal Interviewing (CAPI): is a form of personal interviewing, but instead of

completing a questionnaire, the interviewer brings along a laptop or hand-held computer to enter the
information directly into the database. This method saves time involved in processing the data, as well as
saving the interviewer from carrying around hundreds of questionnaires. However, this type of data
collection method can be expensive to set up and requires that interviewers have computer and typing
skills.

Questionnaires

Paper-pencil-questionnaires can be sent to a large number of people and saves the researcher time and
money. People are more truthful while responding to the questionnaires regarding controversial issues in
particular because their responses are anonymous. But they also have drawbacks. The majority of the
people who receive questionnaires don't return them and those who do might not be representative of the
originally selected sample. (Leedy and Ormrod, 2001)

Web-based questionnaires: A new and inevitably growing methodology is the use of Internet-based
research. This would mean receiving an e-mail on which you would click on an address that would take
you to a secure website to fill in a questionnaire. This type of research is often quicker and less detailed.
Some disadvantages of this method include the exclusion of people who do not have a computer or are
unable to access a computer. Also, the validity of such surveys is in question as people might be in a
hurry to complete them and so might not give accurate responses.
(http://www.statcan.ca/english/edu/power/ch2/methods/methods.htm)

Questionnaires often make use of Checklist and rating scales. These devices help simplify and quantify
people's behaviors and attitudes. A checklist is a list of behaviors, characteristics, or other entities that
the researcher is looking for. Either the researcher or survey participant simply checks whether each item
on the list is observed, present or true, or vice versa. A rating scale is more useful when a behavior needs
to be evaluated on a continuum. They are also known as Likert scales. (Leedy and Ormrod, 2001)

Qualitative data collection methods play an important role in impact evaluation by providing

information useful to understand the processes behind observed results and assess changes in people’s
perceptions of their well-being. Furthermore, qualitative methods can be used to improve the quality of
survey-based quantitative evaluations by helping generate evaluation hypotheses; strengthening the
design of survey questionnaires, and expanding or clarifying quantitative evaluation findings. These
methods are characterized by the following attributes:

 they tend to be open-ended and have less structured protocols (i.e., researchers may change the
data collection strategy by adding, refining, or dropping techniques or informants)
 they rely more heavily on interactive interviews; respondents may be interviewed several times to
follow up on a particular issue, clarify concepts or check the reliability of data
 they use triangulation to increase the credibility of their findings (i.e., researchers rely on multiple
data collection methods to check the authenticity of their results)
 generally, their findings are not generalizable to any specific population, rather each case study
produces a single piece of evidence that can be used to seek general patterns among different
studies of the same issue

Regardless of the kinds of data involved, data collection in a qualitative study takes a great deal of time.
The researcher needs to record any potentially useful data thoroughly, accurately, and systematically,
using field notes, sketches, audiotapes, photographs, and other suitable means. The data collection
methods must observe the ethical principles of research.

The qualitative methods most commonly used in evaluation can be classified in three broad categories: 

 in-depth interview
 observation methods
 document review

Source: Leedy, P. D., & Ormrod, J. E. (2005). Practical research (Vol. 108). Saddle River, NJ: Pearson
Custom.

A Step-by-Step Guide to Data Collection

Step 1: Define the aim of your research

Before you start the process of data collection, you need to identify exactly what you want to achieve. You
can start by writing a problem statement: what is the practical or scientific issue that you want to
address and why does it matter?

Next, formulate one or more research questions that precisely define what you want to find out.
Depending on your research questions, you might need to collect quantitative or qualitative data:

 Quantitative data are expressed in numbers and graphs and are analyzed through statistical
methods.
 Qualitative data is expressed in words and analyzed through interpretations and categorizations.

If you aim to test a hypothesis, measure something precisely, or gain large-scale statistical insights,
collect quantitative data. If you aim to explore ideas, understand experiences, or gain detailed insights
into a specific context, collect qualitative data. If you have several aims, you can use a mixed-methods
approach that collects both types of data.

Examples of quantitative and qualitative research aims

You are researching employee perceptions of their direct managers in a large organization.

 Your first aim is to assess whether there are significant differences in perceptions of managers
across different departments and office locations.
  Your second aim is to gather meaningful feedback from employees to explore new ideas for how
managers can improve.

You decide to use a mixed-methods approach to collect both quantitative and qualitative data.

Step 2: Choose your data collection method

Based on the data you want to collect, decide which method is best suited for your research.

 Experimental research is primarily a quantitative method.

 Interviews/focus groups and ethnography are qualitative methods.
 Surveys, observations, archival research, and secondary data collection can be quantitative or
qualitative methods.

Carefully consider what method you will use to gather data that helps you directly answer your research
questions.

Data collection methods

Method When to use How to collect data

Experiment To test a causal relationship. Manipulate variables and measure their effects on
others.

Survey To understand the general Distribute a list of questions to a sample online, in

characteristics or opinions of a person, or over the phone.
group of people.

Interview/focus To gain an in-depth Verbally ask participants open-ended questions in

group understanding of perceptions or individual interviews or focus group discussions.
opinions on a topic.

Observation To understand something in its Measure or survey a sample without trying to affect
natural setting. them.

Ethnography To study the culture of a Join and participate in a community and record
community or organization first- your observations and reflections.
hand.

Archival research To understand current or Access manuscripts, documents, or records from

historical events, conditions, or libraries, depositories, or the internet.
practices.

Secondary data To analyze data from populations Find existing datasets that have already been
collection that you can’t access first-hand. collected, from sources such as government
agencies or research organizations.
Step 3: Plan your data collection procedures
When you know which method(s) you are using, you need to plan exactly how you will implement them.
What procedures will you follow to make accurate observations or measurements of the variables you are
interested in?

For instance, if you’re conducting surveys or interviews, decide what form the questions will take; if you’re
experimenting, make decisions about your experimental design.

Operationalization
Sometimes your variables can be measured directly: for example, you can collect data on the average age
of employees simply by asking for dates of birth. However, often you’ll be interested in collecting data on
more abstract concepts or variables that can’t be directly observed.

Operationalization means turning abstract conceptual ideas into measurable observations. When

planning how you will collect data, you need to translate the conceptual definition of what you want to
study into the operational definition of what you will measure.

For an example of operationalization, you have decided to use surveys to collect quantitative data. The
concept you want to measure is the leadership of managers. You operationalize this concept in two ways:

 You ask managers to rate their leadership skills on 5-point scales assessing the ability to
delegate, decisiveness and dependability.
 You ask their direct employees to provide anonymous feedback on the managers regarding the
same topics.

Using multiple ratings of a single concept can help you cross-check your data and assess the  test
validity of your measures.

Sampling
You may need to develop a sampling plan to obtain data systematically. This involves defining
a population, the group you want to conclude about, and a sample, the group you will collect data from.

Your sampling method will determine how you recruit participants or obtain measurements for your
study. To decide on a sampling method you will need to consider factors like the required sample size,
accessibility of the sample, and timeframe of the data collection.

Standardizing procedures
If multiple researchers are involved, write a detailed manual to standardize data collection procedures in
your study.

This means laying out specific step-by-step instructions so that everyone in your research team
consistently collects data – for example, by conducting experiments under the same conditions and using
objective criteria to record and categorize observations.

This helps ensure the reliability of your data, and you can also use it to replicate the study in the future.
Creating a data management plan
Before beginning data collection, you should also decide how you will organize and store your data.

 If you are collecting data from people, you will likely need to anonymize and safeguard the data to
prevent leaks of sensitive information (e.g. names or identity numbers).
 If you are collecting data via interviews or pencil-and-paper formats, you will need to
perform transcriptions or data entry in systematic ways to minimize distortion.
 You can prevent loss of data by having an organization system that is routinely backed up.

Step 4: Collect the data

Finally, you can implement your chosen methods to measure or observe the variables you are interested
in.

Examples of collecting qualitative and quantitative data to collect data about perceptions of managers,
you administer a survey with closed- and open-ended questions to a sample of 300 company employees
across different departments and locations.

The closed-ended questions ask participants to rate their manager’s leadership skills on scales from 1–5.
The data produced is numerical and can be statistically analyzed for averages and patterns.

The open-ended questions ask participants for examples of what the manager is doing well now and what
they can do better in the future. The data produced is qualitative and can be categorized through  content
analysis for further insights.
To ensure that high-quality data is recorded systematically, here are some best practices:

 Record all relevant information as and when you obtain data. For example, note down whether or
how lab equipment is recalibrated during an experimental study.
 Double-check manual data entry for errors.
 If you collect quantitative data, you can assess the reliability and validity to get an indication of
your data quality.

Source: Bhandari, P., (June 5, 2020). A step-by-step guide to data collection.

https://www.scribbr.com/methodology/data collection/#:~:text=1%20Define%20the%20aim%20of
%20your%20research.%20Before,or%20observe%20the%20variables%20you%20are%20interested%20in.