Original Article

Teaching Public Administration

1–22
Clarification of research ª The Author(s) 2018
Reprints and permissions:
design, research methods, sagepub.co.uk/journalsPermissions.nav
DOI: 10.1177/0144739418775787
and research methodology: journals.sagepub.com/home/tpa

A guide for public

administration researchers
and practitioners
Sawsan Abutabenjeh
Department of Political Science and Public Administration, Mississippi State University, USA

Raed Jaradat
Industrial and Systems Engineering Department, Mississippi State University, USA

Abstract
Research design is a critical topic that is central to research studies in science, social
science, and many other disciplines. After identifying the research topic and for-
mulating questions, selecting the appropriate design is perhaps the most important
decision a researcher makes. Currently, there is a plethora of literature presenting
multiple approaches to the formulation of research design. Although the existence of
multiple approaches is a powerful source in the development of a research design, new
public administration (PA) researchers and students may see it as a source of confusion
because there is a lack of clarity in the literature about the approaches to research
design, research methods, and research methodology in the social sciences. The
purpose of this research paper is to lessen the confusion over research design and offer
a better understanding of these approaches. The comparison analysis obtained in this
research can provide guidance for PA researchers, students and practitioners when
considering the research design most appropriate for their study. To achieve the
research purpose, a comparison analysis was conducted to study the differences in
research design perspectives and approaches. Three dominant works related to
research design, by Earl R. Babbie, Elizabethann O’Sullivan et al., and John W. Creswell,

Corresponding author:
Sawsan Abutabenjeh, Department of Political Science and Public Administration Office, Mississippi State
University, 105 Bowen Hall, MS 39762, USA.
Email: sawsan.abutabenjeh@msstate.edu; Telephone: 001-662-325-1469
2 Teaching Public Administration XX(X)

were selected as representative of the wide array of approaches in the PA literature. In

identifying similarities and differences across the scholars’ approaches, the analysis
includes: (a) definitions of and steps in research design, and (b) the perspectives on
research methods and research methodology. The analysis showed that research
design approaches are convergent and divergent and that it is necessary for PA
researchers and students to be knowledgeable about the various research design
approaches before selecting a specific design for their research. An example scenario
was provided to show the impact of different perspectives on research design in the PA
literature.

Keywords
Research design, research methods, comparative study, research guide, methodological
confusions, public procurement, value of certification

Introduction
Scholars emphasize the importance of conducting research in the field of public
administration (PA) (Denhardt, 2001; Brewer, Douglas, Facer et al., 1999; Mosher,
1956, 1975) in order to build on current knowledge and theory related to PA (Lieb-
man,1963), to help understand governmental institutions’ work (Wright, Manigault and
Black, 2004), to guide PA practitioners and professionals, to inform the decision-making
process (Streib, Slotkin and Rivera, 2001; Wright et al., 2004), and to help current
students and emerging scholars to acquire knowledge by filling the current research gaps
and develop new lines of research (Aguado, 2009; Anderson and Harsell, 2005; Engbers
2016; Fitzpatrick, 2000).
To prepare current students and emerging scholars to conduct research, schools that
offer PA and public policy degrees offer research methods as core courses (Engbers,
2016; Fenwick, 1992; Roeder and Whitaker, 1993; US News and World Report, 2016).
While taking these classes, students learn how to conduct research, learn about the
research process, and get trained to produce an empirical work in the form of a project,
paper, thesis or dissertation, or manuscript. One of the important steps in conducting
research is to design the research. A research design is a blueprint to guide the research
process by laying out how a study will move from the research purpose/questions to the
outcomes. It is a comprehensive planning process used to collect and analyze data in
order to increase the understanding of a given topic. At a general level, the research
process consists of three primary stages: posing a question for examination, collecting
data to answer the question, and presenting an answer to the question. Although the
general research process is broadly similar across disciplines, this research will focus on
the research design and methods in the PA field.
A discussion among a group of PA doctoral students regarding the concept of research
design provided the impetus for writing this paper. They raised several key questions
concerning how they might proceed with respect to research design in PA, including:
(a) which definition should they adopt with respect to research design, (b) which
Abutabenjeh and Jaradat 3

approach to research design is most appropriate for their research, and (c) what are the
common themes among the current PA approaches to research design. From the stu-
dents’ perspective, the sheer amount of information about research design found in
textbooks, journal articles, and assorted guides was overwhelming. The contribution of
this research is to condense and simplify that information so there is less ambiguity
surrounding the variety of approaches used to describe research design in PA. If new
researchers cannot understand the different approaches to research design, then the
probability of constructing an inappropriate design is high. Our exploration of research
design is based on our premise that along with formulating research questions, selecting
the research design is one of the most critical aspects of the research process.
In the review of PA literature, we found that scholars have extensively evaluated
many articles published in prominent journals in an effort to study how researchers used
research design and method in their work (Forrester and Watson 1994; Houston and
Delevan 1990, 1994; McCurdy and Cleary, 1984; Stallings and Ferris, 1988; Wright,
Manigault and Black 2004). Other scholars have studied the quality of dissertation
research and its effect on the development of PA (Adams and White 1994; Brewer et al.
1998; Cleary 1992, 2000; Stallings, 1986; White, 1986). Another trend found in the
literature is related to data collection processes. Several scholars have evaluated the
methods of study and corresponding data collection and measurement methods for
articles and dissertations published in PA (Engbers, 2016; Gunn, 2017; Wright et al.,
2004). For instance, Engbers (2016) discussed that, given the rising importance of “data
driven decision-making,” it is necessary to place a greater emphasis on the best practices
for teaching PA research methods. Additionally, Gunn emphasized that teaching
research methods is in a stage of regeneration, and a broad range of social science
disciplines can learn from each other by recognizing useful applications from a wide
range of studies (Gunn, 2017).
Although many scholars point out the importance of research design, one theme
emerging in the literature is focused on the importance of providing clarification
regarding the ambiguity of the concept itself. Forrest (2017) asserted that there is con-
fusion about what social science research methods are. Furthermore, Perry and Kraemer
(1986), who adopted Kaplan’s definition of research methodologies to review and
evaluate articles related to research methodologies in management from 1975 to 1984,
stated that “we need to clarify some definitional ambiguities. What, precisely, do we
mean by research methodology?” (Perry and Kraemer 1986: 215). There certainly
appears to be room to provide some clarity for understanding the different approaches
and their implications for research design.
Several perspectives provide a good foundation for exploring differences and simi-
larities in various approaches to the question of research design. Each of the different
approaches has been successful and will be successful in the future to guide research
design. However, none of the approaches exists as the ‘definitive’ work concerning
research design. The authors of this research are not focused on criticizing these research
design perspectives. On the contrary, examination of similarities, differences, and issues
in the different perspectives is instructive. This examination, which could alleviate the
confusion surrounding research design, attempts to help PA researchers gain a better
4 Teaching Public Administration XX(X)

understanding of the possible design options so that they can gain confidence in con-
structing a “rigorous” research design appropriate to answer their research questions. To
achieve this objective, we compared different research design approaches and devel-
opment and examined their applications in the PA field. The following question is
addressed in the research: are the different approaches to research design in the PA
literature sufficiently divergent such that they represent completely different approaches
to the construction of an appropriate research design? To address this question, a
comparison analysis was conducted based on the work of three noted scholars in the area
of research design: Earl R. Babbie (2009), Elizabethann O’Sullivan et al., (2007) and
John W. Creswell (2008). These authors’ cited works and textbooks are considered
prominent contributions to research in the social sciences.
The rationale for selecting these three scholars is presented below, followed by an
overview of the three approaches to research design and a discussion of the divergence and
convergence in research design. The comparison analysis includes the definitions and steps
of research design as well as the perspectives on research methods and methodology
proposed by the selected scholars. Finally, a summary of the three approaches and a
discussion of the implications from this exploration of research design is presented.

Rationale for selecting the three scholars

The inclusion of all the research and works regarding research design is beyond the scope
of this research. However, we are confident that by selecting a subset of “dominant” PA
literature on research design, we will have sufficient representation of the “research
design” literature in order to achieve our present purpose. We developed criteria that
guided our selection of materials to include those scholars that contributed most to the
literature in research design. Stemming from our thorough review of the PA literature,
we settled on the works of Earl R. Babbie (2009), Elizabethann O’Sullivan et al., (2007)
and John W. Creswell (2008) as being sufficiently representative of the wider literature
on research design to support this research paper. These authors, who are among the best-
known contributors to the body of knowledge concerning research design, have had their
works and textbooks frequently cited in the PA field. These scholars were chosen
because of their professional experiences in teaching and their publication of the most
popular textbooks about research design and methods. According to a Google scholar
search, the total citation count for the three selected books is over 100,000 (The Practice
of Social Research by Babbie (2009), Research Methods by O’Sullivan et al. (2007), and
Research Design: Qualitative, Quantitative, and Mixed Methods Approaches by Cres-
well (2008)). These works are used extensively in research design courses to guide
researchers in their considerations when establishing an appropriate research design.

Overview of the three approaches

This section provides an overview of each of the three author’s approaches selected to
examine research design. The overviews are provided to establish a foundation for the
comparison between the approaches. In each case, we have refrained from a comprehensive
Abutabenjeh and Jaradat 5

Figure 1. Babbie’s framework for design.

presentation of the authors’ perspectives and approach to research design but have instead
presented the authors’ approaches that provide the essence of their work while providing
sufficient detail to support a contrast between the approaches.

Approach I: Earl R. Babbie

Babbie’s work sets out to lay the groundwork for social science research, its theory,
research design, and statistics (Forsythe, 1990). Babbie (2004: 90) stated “a wide variety
of research designs are available to social researchers . . . designing a study involves
specifying exactly who or what is to be studied, when, how, and for what purpose.”
Babbie (2004) also considered research design to be a plan that the researcher needs to
determine what to observe and analyze, why, and how. Although the details vary
according to what the researcher wishes to study, Babbie defined two major tasks in any
research design. First, the researcher needs to be as specific as possible about what they
want to find out. Second, the researcher must determine the best way to do it. According
to Babbie, if the researcher handles the first consideration fully, he will be well-
positioned to handle the second in the same process. Babbie recommended that the
researcher begin by assessing three things: researcher interests, researcher abilities, and
available resources. Researchers in PA need to carefully consider the “three things”
proposed by Babbie, especially the available resources. In PA, validating the research
design is a daunting task due to the lack of open data sources available for researchers.
Babbie defined seven steps for designing a research project. The seven steps are sum-
marized below in respect to PA researchers and professionals (Figure 1).

Defining the purpose of the project

In this step, the PA researcher might begin by reading about an issue of interest, for
example, the area of public procurement, and finding out what others have written about
6 Teaching Public Administration XX(X)

the area/topic by reviewing the previous public procurement literature. After that, the
researcher must define the purpose and the focus of the project and what kind of study
she will undertake – for example, explanatory, descriptive, or exploratory research.
According to Babbie (2004), the researcher’s purpose of undertaking research can be
expressed in the form of a report. For example, if the issue of interest is to gain insight
into how public procurement officials perceive the value of professional certifications in
public procurement, the researcher needs to prepare a report that outlines the kind of
study, the purpose of the study, and the related public procurement literature to identify
the themes and gaps in the current body of public procurement.

Conceptualization
Once the PA researcher has a well-defined purpose and a clear description of the kinds
of anticipated outcomes, the researcher can proceed to the next step in designing the
study, conceptualization. Conceptualization is a process through which the researcher
specifies what she means when she uses particular terms in the research. With regards
to the public procurement example, the researcher must define the exact meaning of
public procurement and professional certifications and produce an agreed-upon
meaning for each specific concept or process in the public procurement area for the
purpose of the research.

Choice of research method

In this stage, the PA researcher needs to explore the various observational techniques or
research methods available to social scientists such as experiments, survey research,
qualitative field research, unobtrusive research, and evaluation research. Each research
method has its strengths and weaknesses, and certain concepts are more appropriately
studied by some methods than by others. For example, Babbie (2004) noted that the best
study design uses more than one research method, taking advantage of their different
strengths. Within the public procurement example, survey research is more appropriate
than design of experiments, and thereby the researcher needs to know the advantages of
using survey research over experiments.

Operationalization
Having specified the concepts to be used and the research method to be employed, the
next stage is operationalization. Operationalization is the development of specific
research procedures or operations that will result in empirical observations representing
the concepts in the real world. Part of this process is deciding how the desired data will
be collected: direct observation, review of official documents, questionnaire, or some
other technique. If the PA researcher decides to use survey research as a main research
method, then she has to be consistent with the data collection process. In our example,
data should be collected or obtained from procurement professional organizations
such as The Institute of Public Procurement (NIGP), National Association of State
Abutabenjeh and Jaradat 7

Procurement Officials (NASPO), The Universal Public Procurement Certification

Council (UPCCC), or any other association(s). Any other resources will not provide
meaningful results and would impact the research design of the study.

Population and sampling

In this stage, the researcher needs to decide who or what to study. The population for a
study is that group about whom the researcher wants to draw conclusions. The decisions
about population and sampling are related to decisions about which research method to
use. Within the public procurement example, the population of the study could be the
members of the NIGP.

Observation and data processing

Observation is a way to collect empirical data, and each research method has suitable
observation techniques. Depending on the research method chosen, the researcher will
have amassed a volume of observations in a form that probably is not immediately
interpretable. The researcher’s task is to prepare this mass of observation for analysis. In
the mentioned example, the PA researcher can use data analysis techniques to translate
data into quantitative or qualitative terms.

Analysis and application

In the final stage of research design, the researcher interprets the data to draw conclu-
sions. The researcher then clarifies the application of the research that she has conducted.
In the value of public procurement certifications example, the PA researcher must
present the results and determine whether professional certification is beneficial in
public procurement. She also has to explain the analysis used and the implications and
applications of the research results.

Approach II: O’Sullivan

O’Sullivan and colleagues have provided major contributions to the body of knowledge
for research design. As stated in their work (O’Sullivan et al., 2007), researchers should
begin their research with careful planning and articulation of the purpose of the study.
With the purpose in mind, the researchers can select their elements or variables of
interest and postulate their relationship to each other to build a preliminary model. After
formulating a model, researchers need a research plan, which is called a research design.
O’Sullivan et al. (2007) defined research design as plans that guide decisions about
when and how often to collect data, what data to gather from whom, how to collect data,
and how to analyze data. According to O’Sullivan et al. (2007), there are general and
specific meanings associated with the concept of research design. The general meaning
of research design refers to the plan for the study’s methodology. In the context of this
general meaning, the design specifies the research’s purpose and explains the plan that
will answer the research question(s) and is consistent with the study’s purpose. In
8 Teaching Public Administration XX(X)

Figure 2. O’Sullivan’s framework for design.

addition to the general meaning, O’Sullivan et al. (2007) stated that the specific meaning
of research design refers to both the type of study or design, which involves when and
how often to collect the data as well as how much control the researcher will have over
the research factors.
The following six research methodology steps were proposed by O’Sullivan et al.
(2007) to guide the researcher in her collection and analysis of the data: (a) decide when
and how often to collect data; (b) develop or select measures for each variable for which
data is collected; (c) identify a sample or test population; (d) choose a strategy for
contacting subjects; (e) plan the data analysis; and (f) present the findings (Figure 2).
O’Sullivan et al. (2007) identified two primary categories of research designs: designs
for description and designs for explanation.

Designs for description

Cross-sectional studies, time series studies, and case studies are categorized as designs
for description. These descriptive designs provide a wealth of information that is easy to
understand and interpret, and they may also identify problems and suggest solutions.
Descriptive designs guide studies describing the occurrence of a variable, or the rela-
tionship between variables, and help the researcher decide when to make observations
and how many observations to make. Researchers’ selections may depend on the nature
of the dataset or the problem-solving skills of the practitioner who will be using the
results of the study. For example, PA researchers can perfome designs of description to
show the number of certified public procurement officials.
Abutabenjeh and Jaradat 9

Philosophical
Worldviews

Research Strategies
Methods of Inquiry

Figure 3. Creswell’s framework for design.

Designs for explanation

Experiments and quasi-experimental designs are included in designs for explanation.
Explanation designs provide researchers with useful models of how to conduct research to
establish causal links between independent and dependent variables. These designs are
used not only to describe the change in the value of the dependent variable but also to help
explain those changes. Using our example regarding the issue of professional certifications
in public procurement, the PA researcher should choose to use the design for explanation
because it is the most appropriate research design for the selected research method, the
survey research. In this case, the dependent variable is the value of certification and
independent variables might include work experience, salary, time, gender, etc.

Approach III: John Creswell

Another major contributor to the body of knowledge for research design is Creswell.
According to Creswell (2008: 5), “research designs are plans and the procedures for
research that span the decisions from broad assumptions to detailed methods of data
collection and analysis.” Creswell (2008) identified three types of research designs:
quantitative, qualitative, and mixed methods. Each type of quantitative, qualitative, and
mixed methods consists of three interrelated elements: philosophical worldviews, the
strategies of inquiry, and research methods (see Figure 3). Several research undertakings
have been conducted based on Creswell’s identification of research design such as
Tashakkori and Teddlie’s (2001) study. In their study, they emphasized the importance
of conceptualization of research methods to teach the integration of quantitative and
qualitative methods. The three elements are described below.

Philosophical worldviews
Philosophical worldviews are paradigms that provide information about why researchers
choose a specific research design. Creswell (2008: 3) “sees worldviews as a general
10 Teaching Public Administration XX(X)

philosophical orientation about the world and the nature of research that a researcher
brings to a study.” So, researchers’ beliefs lead to their embracing a qualitative, quan-
titative, or mixed methods approach in their research. Creswell discussed the following
four worldviews. First the postpositivist worldview represents the traditional form of
research and is sometimes called the scientific method. Postpositivists tend to prefer
quantitative research more often than qualitative research. Postpositivists hold a
deterministic philosophy in which cause–effect relationships are anticipated, such as
those found in experiments. Second, the social constructivist worldview is typically
found in approaches based on qualitative research. Social constructivists rely on the
participants’ views of the situation being studied. They take into consideration peo-
ples’ life and work experiences to be able to understand the background of the parti-
cipants. Third, the advocacy and participatory worldview is found in qualitative
research, but it could also be consistent with quantitative research. Participatory
researchers study important social issues and may allow the participants to engage as
active collaborators in their research. In addition, a participatory worldview holds
that research inquiry needs to appreciate politics and a political agenda. Finally, the
pragmatist worldview arises out of actions, situations, and consequences rather than
prior conditions. In this view, instead of focusing on methods, researchers focus on
research problems and use all approaches that meet their needs and purposes. Prag-
matists apply mixed methods research and form quantitative and qualitative assump-
tions when they engage in their research.

The strategies of inquiry

Strategies of inquiry are types of quantitative, qualitative, and mixed method designs that
provide instructions for procedures in a research design. Three strategies of inquiry were
discussed by Creswell (2008). First, quantitative strategies are the strategies of inquiry
that are associated with quantitative research and are used by those holding the post-
positivist worldview. These strategies include experiments, quasi-experiments, and
correlation studies. Second are the qualitative strategies, which encompass various types
of qualitative inquiry approaches such as narrative research, phenomenology, ethno-
graphies, grounded theory studies, and case studies. Finally, the mixed methods strate-
gies are those strategies of inquiry that employ both quantitative and qualitative
approaches. These methods include sequential mixed methods where the researcher aims
to expand the findings of one method with another one, concurrent mixed methods where
the researcher collects quantitative and qualitative data at the same time and then
integrates the information in the interpretation of the results, and transformative mixed
methods where the researcher uses a theoretical lens as an overarching perspective
within a design that contains both quantitative and qualitative data.
In the public procurement example, any of these strategies can be used. However, the
PA researcher needs to examine the purpose of the study through the steps identified in
the approaches of Babbie and O’Sullivan described above. Since the researcher in this
scenario decided to use survey research, she then has to determine the design of the
survey that will be used to collect qualitative or quantitative data. The strategy of inquiry
Abutabenjeh and Jaradat 11

used should be consistent with the research questions and hypothesis of the public
procurement study example.

Research methods
Research methods consist of three elements: (a) the forms of data collection; (b) analysis;
and (c) interpretation that the researchers propose for their studies. The researcher may
use quantitative methods like close-ended questions or numeric data in data collection
and use statistical analysis and interpretation methods to get the overall results. Alter-
natively, a researcher may use qualitative methods like open-ended questions or inter-
views to collect data, and use text and image analysis or themes and patterns
interpretation to get the overall results. Finally, researchers may also mix different
methods to collect, analyze, and interpret data to get their results.
The philosophical worldviews, the strategies of inquiry, and the research methods
together contribute to a research design that tends to be quantitative, qualitative, or
mixed method (Figure 3). According to Creswell (2008), there are different scenarios
that can explain how the three elements make a specific research design. For example,
quantitative design may involve post-positivists worldview, experimental strategy of
inquiry, and pre- and post-test measure of attitudes; or qualitative design may involve
participatory worldview, narrative design, and open-ended interviewing; or mixed
method design may involve pragmatic worldview, and collection of both quantitative
and qualitative data sequentially. Finally, adding to the three factors above, selecting one
design over another depends on the purpose of the study, the researcher’s personal
experiences and the research’s audience. In the public procurement scenario, if the
research hypothesis is to test whether professional certification impacts the job perfor-
mance of public officials, then a pre- and post-test design is needed.

Babbie versus O’Sullivan versus Creswell: A discussion of choices

A discussion of the similarities and differences in research design approaches between
the three scholars is presented in this section. The first part deals with the research
design; the second part addresses research methods and research methodology, and the
last part concerns how the similarities and differences affect research in different fields.
It is important to mention that although the research methods and methodology are part
of the research design process, they nevertheless merit separate discussion to highlight
the manner in which each of the three scholars has dealt with them. However, we would
like to emphasize that setting the boundaries between the three concepts of research
design, research method, and research methodology is challenging because the three
concepts are closely interrelated with one another.

The definitions and stages of research design

As discussed in the previous sections, Babbie (2004), Creswell (2008), and O’Sullivan
et al. (2007) use different terminologies to define research design, but there is significant
12 Teaching Public Administration XX(X)

agreement on what constitutes research design. The following example provides evi-
dence of the convergent state in defining research design. “Research design is a plan that
involves a set of decisions regarding what topic is to be studied among which population
with which research methods for what purpose” (Babbie, 2004: 112), “research designs
are described as blueprints for the final research product” (O’Sullivan, Rassel and
Berner, 2007: 26), and “research design is a plan to conduct research” (Creswell, 2008:
5). Using any of this terminology to define research design for the public procurement
scenario or any other PA research area would be appropriate, and the researcher should
not be concerned about which definition to use for the research design.
A convergence was also found in the three authors’ discussions of the factors that
affect design selection. All of them emphasized the importance of having a well-defined
purpose for the research, adequate resources, and a researcher with the skills to solve
problems. While Creswell (2008) talked explicitly about the influence of philosophical
worldviews in choosing research designs, the other two authors were less straightforward
about the influence of philosophical assumptions although they did not say anything that
contradicts Creswell.
Despite the consistency found in the terminology used to define research design, we
found that there is a divergence in the approaches each author proposed in delineating the
design process. This divergence ranges from the usage of terms to a detailed discussion of
the various stages of the design process, ranging from data collection to analyzing the
results of the study. We also found that there are subtle but substantive differences in the
stages each scholar prescribed in constructing the research design. Follow-up discussion
with a group of new PA doctoral students regarding the concept of research design, we
found that their main confusion is related to the different design process steps discussed in
the literature. For example, Babbie’s research design involved nine steps (as illustrated in
the previous section); O’Sullivan et al. focused on the steps related to data collection and
emphasized the importance of “when, how, what, from whom, and how to analyze data”
(O’Sullivan, Rassel and Berner, 2007: 26) and Creswell offered a general framework of
research design focusing primarily on “the intersection of philosophical worldviews,
strategies of inquiry, and specific methods” (Creswell, 2008: 5). To help the new students
and other PA researchers and professionals, Table 1 shows the common themes in the
definitions and the disparity in the approaches proposed by the three scholars.
Within the public procurement scenario, the PA researcher has to decide which
approach to follow when it comes to the design process. If Babbie’s approach is used,
then the PA researcher has to follow the nine steps discussed above. However, the PA
researcher can still use Creswell’s terminology of research design to conduct the pro-
fessional certification in public procurement research.

Perspectives on research methods and research methodology

The similarities and differences in research methods and research methodology were
found to be less apparent. Babbie used the term “observational techniques” to refer to
research methods such as experiments, survey research, qualitative field research,
unobtrusive research, and evaluation research. He pointed out that each research method
 Table 1. Comparison of the concept of research design.

Compare/ contrast Babbie O’Sullivan et al. Creswell Implication for PA Researchers

Research Design Research design is a plan Research design is a plan Research design is a plan Can use any of the terminology
Concept discussed by the three authors
to define their research design
(convergent perspective)
Factors that affect  The purpose of the study  The purpose of the study  The purpose of the Can use any of the factors
selecting a specific  Resources available to  Resources available to the study discussed by the three
design the researcher researcher  Resources available authors. However, using
 Researcher’s abilities and  Researcher’s abilities and to the researcher Creswell’s approach,
skills skills  Researcher’s researchers need to focus
abilities and skills more on explaining the
 The research’s influence of philosophical
audience worldviews on choosing their
 Philosophical research designs. (somewhat
worldviews convergent perspective)
 Strategies of inquiry
 Research methods
What does research Research design involves all the Research design involves steps Research design involves: Has to select which design
design involve? steps of the project: related to data collection:  Philosophical process to choose since they
 Defining the purpose of  When and how often to worldviews are different among the three
the project. collect data  Strategies of inquiry scholars. There is no best
 Conceptualization  What data to gather  Research methods approach. To avoid confusion,
 Choice of research  From whom practitioners need to
methods  How to collect data understand that each approach
 Operationalization  How to analyze data has different steps. (divergent
 Population and sampling perspective). Design process is
 Observations a main source of confusion
 Data processing mentioned by the group of
 Analysis new PA doctoral students.
 Application

13
PA ¼ Public administration.
14 Teaching Public Administration XX(X)

has an appropriate way to collect empirical data. For example, in the public pro-
curement scenario, if the PA researcher chooses a survey to collect empirical data to
study the value of certification in public procurement, then she could email ques-
tionnaires to the members of NIGP or arrange to have a team of interviewers
conduct the survey by phone. Creswell’s perspective on research methods involves
data collection, analysis and interpretation. Although Babbie expanded his definition
of research methods to cover the research techniques and appropriate ways to collect
data, Creswell’s definition of research method covered just the appropriate way to
collect data. There is an obvious difference in their concepts as far as the definition
of research methods is concerned.
O’Sullivan et al. (2007) did not provide their own definition of research methods (see
Table 2); however, their definition of the types of studies or designs are similar to
Babbie’s definition of research methods and Creswell’s definition of strategies of
inquiry. O’Sullivan et al. (2007) referred to the specific meaning of research design as
the type of study or design which involves when and how often the researcher collects
the data and how much control the researcher will have over the research factors such as
cross-sectional study, case study, and experiments study (Figure 4).
Regarding research methodology, Babbie and Creswell did not offer definitions;
however, O’Sullivan et al. explicitly defined research methodology as the steps
researchers use to collect and analyze data. The steps involve: (a) deciding when and
how often to collect data; (b) developing or selecting measures for each variable;
(c) identifying a sample or test population; (d) choosing a strategy for contacting sub-
jects; (e) planning the data analysis; and (f) presenting the findings. Thus, O’Sullivan
et al. differ from the other two authors. Since Babbie and Creswell did not use the term
research methodology, similarities or differences cannot be determined, but it can be said
that O’Sullivan was more comprehensive in development. Overall, the results from
analyzing the similarities and differences in the definitions regarding research methods
and research methodology are inconclusive. However, the collection of perspectives is
insightful in informing a more robust perspective of research for PA practitioners and
researchers contemplating research design questions. To lessen the confusion with
respect to research methods and methodology, we attempted to construct a comparison
analysis between the three different perspectives as shown in Table 2. The comparison
analysis showed the following.

1. There are different research design stages PA researchers can engage in. How-
ever, the appropriateness for selection of a research design requires an apprecia-
tion of the uniqueness of the nature of the study, its significance, and its purpose.
Specifically, the research question that the study is trying to answer is critical;
“the research question is supposed to have a pivotal role because decisions about
research design and methods are supposed to be made in order to answer research
questions” (Bryman, 2007: 6).
2. The differences and similarities presented in this research will help PA research-
ers in understating which approach to use.
 Table 2. Comparing and contrasting research methods and methodology concepts.

Compare/ Implications for PA

contrast Babbie O’Sullivan Creswell Researchers

Research Research methods are
Method observational techniques that
are available to social
scientists.
Observational techniques
include:
1. Experiments
2. Survey research
3. Qualitative research
Ways to collect empirical data;
each research method has an
appropriate one.
O’Sullivan et al. did not define or
discuss research methods explicitly
in their book, but they mentioned
the type of study involved, when
and how often to collect the data,
and how much control the
researcher can have on research
factors.
Research methods involve the PA researchers have to be
form of data collection, familiar with the different
analysis, interpretation such as: terminology used in
 Open-ended questions research design. Some of
(qualitative) these terminologies have
 Predetermined similar meaning and
(quantitative) purpose. For example
 Emerging methods O’sullivan’s types of studies
(quantitative) terminology is similar to
 Interview data (qualitative) Babbie’s observation
 Both open and closed techniques terminology
ended question (mixed)
Tools to analyze and interpret the
results:
 Statistical analysis and
interpretation (quantitative)
 Text and image analysis
and themes interpretation
(qualitative)
 Statistical and text analysis
(mixed)

(continued)

15
16
Table 2. (continued)

Compare/ Implications for PA

contrast Babbie O’Sullivan Creswell Researchers

Research Babbie did not discuss research Research methodology consists of Research methodology equal to The nine steps in O’Sullivan’s
Methodology methodology. the following steps: strategies of inquiry. methodology are well
 Deciding when and how often defined and organized.
to collect data.
 Developing or selecting
measures for each variable.
 Identifying a sample or test
population.
 Choosing a strategy for
contacting subjects.
 Planning the data analysis.
 Presenting the findings.
Abutabenjeh and Jaradat 17

Research
Methods
Babbie Focused on the particular and
specific approaches to conduct
research

Strategies of
Inquiry
Creswell Emphasis on the high level
strategies pursued for research
design.

Type of Study
O’Sullivan Focused on the particular design
related to the type of study
conducted

Figure 4. Approaches to research design.

3. There is a divergence in the approaches proposed by the scholars in terms of

design process.
4. When selecting the research design, PA researchers have to remain open to
different formulations and not follow one approach.

How the current divergence and convergence affect PA researchers

Despite there being a convergence around the terminology used to define research
design, there is a divergence in the descriptions of approaches used to conduct
research design. While these different approaches are healthy in the development of
research design, at some point we need to have some convergence around the
approaches associated with research design. We believe that the difference in approaches
might be a cause of concern as well as a source of confusion especially for new PA
researchers and graduate students.
PA graduate students and researchers alike might interpret the approaches differently,
which in turn can impede effective communication of both design and the corresponding
results that accrue from execution of the design. Nevertheless, it is obvious from the
discussion that the similarities are more pronounced than the differences. Students with
the right knowledge, skills, and motivations can meet the challenges that might be posed
by the different descriptions and classifications of research design, research methods,
and research methodologies. The inclusion of a broader set of perspectives of research
design can only provide greater confidence in the eventual design selected.
18 Teaching Public Administration XX(X)

Table 3. Questions raised by a group of PA doctoral students.

Question/Concern Response

Which research design definition should they Based on the analysis, there is no best definition
adopt? and PA researchers can use any of the
terminology discussed in in the text to define
their research design (there is a convergence
around the research design definition).
Which approach to research design is most Each approach has different steps and thus, it is
appropriate for their research? recommended to follow one approach for the
design process selection (i.e. Babbie’s approach).
This would eliminate the confusion of
unintentionally mixing these different design
processes. However, researchers can still use
Creswell’s research definition, and Babbie’s
research design process.
What are the common themes among the This research provides a comprehensive answer
current PA approaches to research design? related to the common themes and differences
in research design.

Table 4. Example of research design and high level evaluation.

PA doctoral research areas Criteria for area evaluation/ with their sub-elements

Title of the research Topic within the research discipline.

Introduction of the research
Research problem and purpose
Research questions and/or
hypotheses
Type of research
Research significance
Literature review
A description about the topic area (where research topic
came from).
A brief description about the nature of the problem.
Relevant studies that have addressed similar problems.
Clear and precise research problem.
Relevant concepts and theories to the research problem.
Well-articulated research purpose.
Survey of literature pertinent to the research discipline.
Original topic within the field.
Specific questions or hypotheses related to the nature of the
research problem.
Adherence relationship among questions, purpose, and
research problem.
Specific type of research the researcher will take.
Pragmatic relevance to the research problem.
Shape of the question focused on the research purpose.
Significant original contribution to the field.
Illustration of the theoretical foundations of the research.
Relevant literature review to the nature of research problem.
Identification of major synthesis in the literature.

(continued)
Abutabenjeh and Jaradat 19

Table 4. (continued)

PA doctoral research areas Criteria for area evaluation/ with their sub-elements

Exploration of major themes in the literature.

Identification of the major gaps and issues in the literature
(critique).
Terms of research Clear and well-bounded definitions of the research.
The researcher has to determine which.
Research design and strategies of Determination of the type of research (quantitative,
inquiry qualitative, and mixed methods).
The researcher has to determine which approach to use
(Babbie, O’Sullivan et al., or Creswell) based on the
research problem and significance and the type of research.
Research methods (data collection Specific method to collect data.
and analysis) Alignment between the selected research method and the
type of research design.
Validation of sources of data (primary and secondary).
Consistency between the data and the research purpose.
Again the researcher should follow one approach in the
research design process (Babbie, O’Sullivan et al., or
Creswell)
Population and sample and unit of Clarification of the unit of analysis (individuals, organization,
analysis books, etc.).
Representative sample to population.
Research findings and Validity of the results.
interpretation of data Generalizability and transferability of the research.
Stability reliability of the research method.
Interpretation of the results as supported or refuted.
Validity and reliability The instrument measures what it is supposed to measure.
Repeatability of the instrument.
Theoretical, methodological, and Significant contribution from theoretical aspects.
practical implications Significant contribution from methodological and practical
aspects.
Applicability for future research.
Limitations of the study Limitations in conjunction the research.
Future research Research should be expandable and applicable for future
research.
Fund opportunities Applicability for fund opportunities if available.

Many researchers rely on research findings to identify problems, evaluate solutions,

and make decisions. In pursuit of these “research acts,” they should be aware of the
different descriptions and classifications of research design and should be able to gen-
erate a greater degree of confidence in appropriateness of their eventual research design
choice. After a thorough review of the various concepts, definitions, methods, and
methodologies of research design, we feel comfortable in suggesting that to achieve a
20 Teaching Public Administration XX(X)

rigorous research design, it is essential for graduate students, researchers, and other
practitioners to recognize the differences as well as the similarities between the current
approaches and then choose the one that best fits their research and their research field.

Conclusion
This research has examined three popular approaches (Babbie, O’Sullivan et al., and
Creswell) currently used to discuss research designs and their application. The intent of
the research is to help PA researchers and professionals to better understand the research
design approaches by laying out the similarities and differences so they are more
knowledgeable and confident in the research design selection process. In an attempt to
answer the questions/concerns raised by a group of new PA doctoral students regarding
the concept of research design, a comparative analysis was constructed to study the
definitions and steps involved in research design as well as the perspectives on research
methods and research methodology. Table 3 provides responses to the questions raised
by the students.
Based on the analysis and to better answer the questions, Table 4 provides a
comprehensive framework for the research design process and the evaluation of
doctoral research as an example. The intent is to give guidance and high-level eva-
luation to help PA graduate students and other PA researchers understand the processes
in doctoral research.
Results from the comparative analysis show that the three scholars used differing
terminology to delineate the stages of research design, and they presented a variety of
research methods and methodology concepts. However, results also show that the three
scholars appear to have significant agreement on the definition of research design, all
referring to it as a plan to guide the research process. Further, there is a convergence in
their analyses of the factors that affect design selection. While there is a subtle diver-
gence in the current approaches, there is no substantive disagreement on the basic
meaning of research design that would warrant potential exclusion of any of the
approaches in establishing a research design. In view of this assessment, we feel con-
fident in suggesting that each approach can have applications in conducting research in
the PA field. Likewise, PA researchers and practitioners alike can utilize any one of the
approaches to conduct research in their areas of interest. However, the inclusion of
multiple perspectives can provide greater confidence in the applicability of the research
design eventually settled upon. In conclusion, at the surface level there are considerable
similarities found in describing the concept of research design, but, more importantly,
there are subtle differences in how the approaches to the research design are carried out.
It is essential for PA practitioners, students, and researchers to understand these dif-
ferences when selecting the approach most appropriate for their research.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research,
authorship, and/or publication of this article.
Abutabenjeh and Jaradat 21

Funding
The authors received no financial support for the research, authorship, and/or publication
of this article.

References
Adams GB and White JD (1994) Dissertation research in public administration and cognate fields:
An assessment of methods and quality. Public Administration Review 54(6): 565–576.
Andersen K and Harsell DM (2005) Assessing the impact of a quantitative skills course for
undergraduates. Journal of Political Science Education 1(1): 17–27.
Aguado AN (2009) Teaching research methods: Learning by doing. Journal of Public Affairs
Education. 15(2): 251–260
Babbie E (2004) The Practice of Social Research. Belmont, CA: Thomson/Wadsworth
Brewer GA, Facer RL, O’Toole LJ Jr, et al. (1998) The state of doctoral education in public
administration: developments in the field’s research preparation. Journal of Public Affairs
Education 1998: 123–135.
Brewer GA, Douglas JW, Facer RL II, et al. (1999) Determinants of graduate research productivity
in doctoral programs of public administration. Public Administration Review 59(5): 373–382.
Cleary RE (1992) Revisiting the doctoral dissertation in public administration: An examination of
the dissertations of 1990. Public Administration Review 52(1): 55–61.
Cleary RE (2000) The public administration doctoral dissertation reexamined: An evaluation of
the dissertations of 1998. Public Administration Review 60(5): 446–455.
Creswell JW (2008) Research Design: Qualitative, Quantitative, and Mixed Methods Approaches.
Thousand Oaks, CA: SAGE Publications.
Engbers TA (2016) Comparative research: An approach to teaching research methods in political
science and public administration. Teaching Public Administration 34(3): 270–283.
Fenwick J (1992) Some problems of teaching research method in public policy. Teaching Public
Administration 12(2): 77–81.
Fitzpatrick J (2000) What are our goals in teaching research methods to public administrators?
Journal of Public Affairs Education 6(3): 173–181.
Forrest S (2017) Teaching social science research methods to undergraduate medical students: The
state of the art and opportunities for practice and curriculum development. Teaching Public
Administration 35(3): 280–300.
Forrester JP and Watson SS (1994) An assessment of public administration journals: The perspec-
tive of editors and editorial board members. Public Administration Review 53: 474–482.
Forsythe JM (1990) Review of: Babbie E. The Practice of Social Research: 5th ed. National
Forensic Journal 8(1): 115–117.
Gunn A (2017) Critical debates in teaching research methods in the social sciences. Teaching
Public Administration 35(3): 241–259.
Houston DJ and Delevan SM (1990) Public administration research: An assessment of journal
publications. Public Administration Review 50(6): 674.
Houston DJ and Delevan SM (1994) A comparative assessment of public administration journal
publications. Administration and Society 26(2): 252–271.
McCurdy HE and Cleary RE (1984) Why can’t we resolve the research issue in public adminis-
tration? Public Administration Review 43(1): 49–45.
O’Sullivan E, Rassel GR and Berner M (2007) Research Methods for Public Administrators.
London: Routledge.
22 Teaching Public Administration XX(X)

Mosher FC (1956) Research in public administration: some notes and suggestions. Public Admin-
istration Review 16(3): 169–178.
Mosher FC (ed) (1975) American Public Administration: Past, Present, Future. Tuscaloosa, AL:
The University of Alabama Press.
Roeder PW and Whitaker G (1993) Education for the public service: Policy analysis and admin-
istration in the MPA core curriculum. Administration and Society 24(4): 512–540.
Perry JL and Kraemer KL (1986) Research methodology in the ‘Public Administration Review,’
1975–1984. Public Administration Review 46(3): 215–226.
Stallings RA (1986) Doctoral programs in public administration: An outsider’s perspective. Public
Administration Review 6(3): 235–240.
Stallings RA and Ferris JM (1988) Public administration research: Work in PAR, 1940–1984.
Public Administration Review 48(1): 580–587.
Streib G, Slotkin BJ and Rivera M (2001) Public administration research from a practitioner
perspective. Public Administration Review 61(5): 515–525.
US News and World Report (2016) Best public affairs programs. Available at: http://grad-school
s.usnews.rankingsandreviews.com/best-graduate-schools/top-public-affairsschools/public-
affairs-rankings
White JD (1986) Dissertations and publications in public administration. Public Administration
Review 46(3): 227–234.
Wright BE, Manigault LJ and Black TR (2004) Quantitative research measurement in public
administration: An assessment of journal publications. Administration and Society 35(6):
747–764.