Republic of the Philippines

UNIVERSITY OF THE EASTERN PHILIPPINES

University Town, Catarman, Northern Samar

GRADUATE STUDIES

Name: RASSEL E. COBRANA

MARLIT L. GERARDO
Course: MASTERS IN PUBLIC ADMINISTRATION
Subject: PA 800 – RESEARCH METHODS IN PUBLIC ADMINISTRATION
Topic: CONTENT ANALYSIS
Professor: PROF. IMELDA GELERA

Content Analysis
Objectives:

1. To be able to define Content Analysis comprehensively, discuss its Characteristics

especially those unique ones not found in other methods, be able to learn on the

preparation as to when to use Content Analysis as well as the Steps to take when

undertaking Content Analysis.

2. To be able to discuss on the Applications of Content Analysis of Documentary

Materials, revisit Practical Research Ethics, differentiate between undertaking

Substantive and Structural Content Analysis, and just like any other research

method which has limitation, also discuss on Special Problems in the Use of

Content Analysis.

CONTENT ANALYSIS defined:

- a research technique for the objective systematic, and quantitative description of the

manifest content of communication

- the systematic counting, assessing, and interpreting of the form and substance of

communication.
Content analysis as a research technique deals with documentary materials that are

already existing and available. And since it deals with documentary materials, content

analysis and documentary analysis are one and the same thing

- provides a set of methods by which direct physical evidence of the behaviors of and

relationships among many types of political actors are rigorously examined

Characteristics of Content Analysis

1. It is objective.

Like other research techniques, content analysis must be objective. There should

be no bias involved in the selection and classification of content to be analyzed, in

its sampling design, and in the interpretation of results.

To be objective, as much as possible, all items and aspects in the content to be an

analyzed must be included and all must be placed under appropriate categories.

When there is a large volume of documentary materials to be analyzed and

sampling is necessary, the valid principles of sampling must be applied. Otherwise,

the materials selected for investigation may have some tendencies toward certain

conditions that may adversely affect the true findings of the investigation.

Also interpreting the results, objectivity must be observed as much as possible.

The perceptions of the researcher must not be influenced by factors other than the

true outcomes or the true frequency counts of the variables being inquired into.

2. It is systematic.

Content analysis must be systematic and so it must follow the steps of the scientific

method

Page 2 of 14
 3. It is quantitative.

Content analysis of documentary materials is quantitative because interpretations

and conclusions from the findings have to depend almost entirely on the frequency

counts of the items or aspects studied. The occurrences of the item, aspect, or

phenomenon to be studied have to be tallied one by one and, upon the frequency

of the occurrences depends the analysis and interpretation.

Steps in Content Analysis

1. Recognizing the problem

2. Forming the hypotheses

3. Doing library search

One purpose of library search is to look for some guides in conducting the study.

If other studies of the same nature have already been conducted before, the

procedures used may as well be guides of the researcher in conducting his

research.

The second purpose is to find out if the same findings will result, and if so,

generalizations based on a wider basis can be formulated.

The third purpose of library search is to avoid duplication of other studies. There

is duplication if a study has been made in the same place and using the same

materials. To avoid this, library search is necessary.

4. Designing the study

This is determining the different procedures to be followed in the study. The

instruments for collecting data must be determined. If there is a large volume of

documentary materials, then sampling must be applied in the selection of materials

to be included in the study. Needless to say, sampling must be done properly. The

statistical treatment to be applied must also be decided upon. The method of data

Page 3 of 14
 presentation must also be designed, be it textual, tabular, graphical or all of the

three.

5. Developing the instrument

This is for collecting data. In content analysis, a tally sheet or checklist may be

prepared. The instrument must contain all the items or aspects to be studied.

6. Collecting the data

To reiterate, content analysis is quantitative and so tallies have to be made for

each type of data for collection.

7. Analyzing the data

8. Making conclusions

Generally, conclusions are answers to the questions raised at the beginning of the

investigation. Besides, they are based upon the findings of the study.

9. Making recommendations

Recommendations are made to solve or help solve the problems discovered in the

study. Besides, recommendations are also made for further research on the same

study just finished.

PREPARING TO USE CONTENT ANALYSIS

Content analysis may be used to answer research questions whenever there is a

physical record of communication, as long as the researcher has the access to that

record. Example of such record include books; pamphlets; magazines; newspapers; CDs;

audiotape, videotape or DVD recordings; photographs; Web pages; transcripts of

meetings or proceedings; government documents; memoranda; films; diplomatic

communiques; political posters, cartoons, and advertising; speeches; and even letters

and diaries. Take note that all sources of data for content analysis will involve the

existence of a physical record of communication.

Page 4 of 14
 CHOOSING A POPULATION

The first step in preparing to undertake a content analysis is to define the

population of communications we want to study. In here, the population is about the

certain document not the persons. For example, if we want to study news paper coverage

of a congressional campaign conducted during the presidential campaign, we might

define our population as all campaign-related newspaper article (the type of

communication) of two column inches or more in length (the size of communication)

published in daily newspapers (the frequency of communication) that are home delivered

(the distribution of the communication) in all district of Ohio (the location of the

communication) between September 1 and the first Tuesday after the first Monday in

November of the election year (the time of communication).

In each instance, we defined the population of messages to be studied by

establishing sets of criteria to be met by each item. These criteria may include;

1. The type of Communication

2. The type of communicator

3. The parties to the communication

4. The location of communication

5. The frequency of communication

6. The minimum size of communication

7. The length of communication

8. The distribution of communication

9. The time period of communication

Although other criteria may be used, some or all of those listed here will be found

in most studies that employ content analysis. The first task in preparing for a content

analysis is to choose those criteria that relate most directly to the research question at

hand.

Page 5 of 14
 Once the population is defined, we must decide which particular cases to examine.

Due to large number of cases, the most common sampling procedures used in content

analytic studies are the simple random and systematic random techniques to give every

document a chance to be selected.

Simple random

Stratified random
Probability sampling
Cluster
Sampling
Procedures Systematic random

Non-probability sampling

CHOOSING A UNIT OF ANALYSIS

The unit of analysis for content analysis is the particular element or characteristics

of given communication that we shall choose to observe.

The following are the unit of analysis use in content analysis;

1. The individual word

2. The theme

3. The item

1. The individual word is the most basic element of a communication. It may be used as

the unit of analysis in a straightforward manner.

For example, we might ask, in speeches delivered to the United Nation between

1975 to 2010, which country was most conciliatory on questions of eliminating

conflict in Middle East – Israel, Egypt or Saudi Arabia? We might examine the

record of all such speeches and count references to such word as peace,

brotherhood and compromise. Now, we identify certain words and count the

frequency with which they appear.

Even in so simple procedure, however, we must avoid at least two pitfalls.

Page 6 of 14
 1. We must remember that non-standardized measures can lead to biased result.

2. Relying on raw word count. If we choose to count words, we should count them

in context.

Pitfall #1 “The Israeli representatives uttered a total of 100,000 words, including 50

of the words we wish to count, and the Egyptian have some 200,000 words,

including 100 of the keywords.”

Conclusions;

1. Egyptians have been twice as concerned as the Israel with procuring peace.

(non-standardized measure)

2. Both sides have shared an equal concern about settling their differences

(standardized measure to obtain the proportion of all words)

Pitfall #2 1. “We seek peace, but…….”

2. “There will be no compromise”

Words may have different meaning, depending upon its context in which it is used.

In this example, peace and compromise are considered as negative context. Thus,

these words should not be counted.

2. The theme is particular combination of words, such as a phrase, a sentence, or even

a paragraph. The process is similar to that of counting words and represents an

improvement to the extent that themes incorporate the modifiers (adverbs, adjectives)

and explanatory text that both accompany usage of a particular word and help to establish

its meaning.

3. The item is the commonly used unit of analysis for content analysis wherein the

communication itself is taken as a whole. We examine the item of communication as a

unit and examine its overall characteristics.

Page 7 of 14
COMPUTER-BASED CONTENT ANALYSIS

Diction 5.0 ( www.dictionsoftware.com)

WordStat (www.provalisresearch.com/wordstat/wordstat.html )

General Inquirer ( www.wjh.Harvard.edu/inquirer )

All of these content analysis programs provide basic text analysis function such as word

frequency count and category frequency counts.

Word frequency count – provide a list of all words that occur in a text and the

number of times they occur.

Category frequency count – a set of words or phrases are first group in

categories, then the program shows how many times each category occurs in the

document.

The main advantage of computer-based analysis is the fact that as long as the unit

of content is short, simple, and well defined, it is relatively easy to analyze a large

volume of information. As long as the text is available in digital form, even

documents spanning many thousands of text ages can be analyzed in only one

second. In addition, there is no need to employ, train, and supervise human

coders.

However, a disadvantage of software-aided content analysis is that computers

cannot easily code and analyze latent content, such as meaning or bias.

APPLICATION OF CONTENT ANALYSIS OF DOCUMENTARY MATERIALS

The following are the important uses of content analysis;

1. Analyzing to determine the educational, political, sociological, philosophical,

psychological and other ideas of great authors and thinkers, based upon their

books, essays, poems, diaries, novels, plays, speeches.

Page 8 of 14
 2. Analyzing the contents of textbooks and other instructional material as a basis for

the selection of textbooks and other instructional materials for the different

curricular years of the educational ladder.

3. Analyzing the achievement of the students in the different subjects to determine

the type of errors they committed as a basis for making remedial teaching

programs and for curriculum development.

4. Analyzing the different writing of students such as essays, compositions, poems,

stories, letters, etc. to determine their writing aptitude test, level of their vocabulary

and spelling skills, as bases for designing programs, economical, and useful

school records.

5. Analyzing school records such as class records, report cards, enrolment list,

grading sheets, cumulative records, etc. of different schools and school systems

with the end in view of adopting the common, practical, economical and useful

records.

6. Analyzing the objectives and practices of different schools and school systems

with the end in view of adopting more valid and practical objectives and practice to

attain the same ends.

PRACTICAL RESEARCH ETHICS; No people, no ethical concern?

Although humans are not being directly studied in content analysis, a human

product is being examined. This indirect source of information prevents explanations from

those producing the communications, making accuracy a paramount concern.

Researchers must ensure that communications are being fairly represented by the

sampling, coding, data analysis, and interpretation. Because all humans have biases, you

should utilize a clearly stated and transparent coding process to translate the original

source material into data. The veracity of the result is enhanced by using outside coders

who do not know the purpose of the research.

Page 9 of 14
Once we have settled upon a population, a suitable sample, and appropriate unit of

analysis, we need to choose whether we will study the substance of the communication

or its structure or both. Our theory and hypotheses will guide whether we need to offer

primary attention to the meanings of the words or focus on the structure and quantity of

a given type of content.

Hence, undertaking either Substantive Content Analysis or Structural Content Analysis.

Undertaking a Substantive Content Analysis

Substantive content analysis is based on study of words, themes, and items that focuses

on the content (or substance) of the communication.

Creating a Dictionary

The first step in this type of content analysis is to create a dictionary that defines

each and every observation we might make according to the particular category it

fits.

There is this example in the book about study of all sixth-grade schoolbooks in

Cuba where to identify in them all references to Americans and the United States,

we must define just what constitutes a salient reference. In the said example, if

we look only for the words American and United States, we may miss a great many

salient references using such derogatory terms as Yankee, northern imperialists,

gringos, or invading forces at Guantanamo.

A parallel but more difficult problem arises when the absence of a word or phrase

has substantive meaning and must be captured. An example of this cited in the

book is a twelfth-grade civics text published in 2006 for use in Palestinian schools

where the maps either failed to label Israel or portrayed Palestinian territory as

covering the entire area of Israel. These omissions are meaningful and significant

and a content analysis scheme must capture them.

Page 10 of 14
A more difficult problem arises when we must assign evaluations to salient

references – when we must decide whether a particular reference is favorable or

unfavorable, pro or anti, and so forth – and when a series of such references must

be ranked according to their intensity (which is most favorable, which is next most

favorable, etc.). This requires indicators that are sufficiently refined to tell not only

how the political actor feels, but also how strongly the actor feels that way.

Sample phrases in newspaper editorials endorsing a candidate (random order)

*Best of a bad lot *Best available

*Better than the opponent *Our first choice
*Urge you to vote for *Finest candidate in a crowded
field
*Everything the people of this state *Woman (Man) of the hour
could ask for
*An outstanding leader *Promising
*One of the nation’s best *Lesser of two evils
*Best the selection process could produce *Our perennial favorite
*Acceptable * Most acceptable
*Recommend with reservations *Recommend without
reservations
*Wholeheartedly endorse *Warmly recommend
*Offer our support *Enthusiastically commend to
your attention

In this sample, if our goal is to determine which of several newspapers most

strongly supports that candidate, then our immediate task is to decide how to rank

these statements according to the intensity of support that each reflects.

Several techniques are available to assist us in making these decisions. One of

the most prominent of these ranking techniques is pair-comparison scaling. This

relies upon the decision of a group of judges about the meaning or intensity of a

term. The judges may be drawn from the issuers or the receivers of the

communication, a group of scholars familiar with the general subject area under

study, or the general population. Each item to be evaluated by the judges is paired

with every other item in a series of comparisons, and each judge is asked to decide

Page 11 of 14
 which word or phrase in each pair is more intense. We count the number of times

each statement is so designated by each judge and calculate the average score

the judges as a group have assigned to a particular statement to arrive at a

quantitative ranking of the intensity of each item. The higher its mean score, the

more intense the judges consider a statement.

On the other hand, one of the problems associated with the pair-comparison

procedure is the fact that it relies entirely on the decision of judges whose criteria

for judgment may or may not be appropriate or consistent. Because we are

sampling content and not humans here, there is neither a clear reference

population nor a set of underlying parameters to be approximated. The selections

made by the judges are necessarily arbitrary. Consequently, the reliability of

results derived by depending upon such judges may be less than we would hope

for, and there is no external way to validate this reliability.

Undertaking a Structural Content Analysis

In addition to, or in lieu of words, theme, or other elements that denote the

substantive content of communication, several units of analysis can be used to conduct

a structural content analysis. Here, we are less concerned with what is said than with how

it is said.

We may be concerned, for example, with amount of space or time devoted to a

given subject in a particular source, with how many column or inches of newspaper

coverage, etc.

Alternatively, we might be concerned with other, and perhaps more subtle, aspects of the

communication. Is the particular news item accompanied by an illustration? Those with

an illustration have been found to attract more attention from readers than those without?

How large a headline accompanies a new item? Does coverage of particular subject

Page 12 of 14
receive front page prominence, or is it buried inside? In answering questions like these,

we watch for the presence or absence, the prominence and the extent of treatment of

general theme rather than for substantive meaning.

Special Problems in the Use of Content Analysis

1. Biased Content
Researchers must be aware that communications may be specifically designed for
a purpose, whether it is description, persuasion, exhortation, self-protection, or
even deception. In analyzing communications, therefore, we must attempt to
interpret their content in the context of their apparent purpose. The purpose of a
communication can provide an important context for analyzing its content, so we
should attempt to determine these purposes.

2. Intended Audience
The distribution that is accorded a particular item of communication can have
significant implications for its meaning. Even a newspaper that is generally
available may have a limited or specialized clientele. Very often, in order to assess
properly the significance of a communication, we must know whom it reaches
whether by judgement, by inquiry, by self-evidence, or by reliance on an audience
survey, we must attempt to measure or to estimate how widely a message has
been disseminated and to whom. Having this information enables us to judge the
meaning and importance of the material we analyze.

3. Representative Sample
We must try to gauge the degree of our own access to the items at issue. Have we
been provided with free choice over the materials to be analyzed? Are those
materials available in an unbiased manner, or has some external control been
imposed by someone other than the researcher? The issue here is one of

Page 13 of 14
 generalizability, and the question is whether the research population itself, not to
mention the sample, is truly representative. If it is not, the researcher may either
be misled or manipulated.

The difficulty of overcoming these challenges to a successful content analysis is

that the information we require to make informed judgments may simply be not
available. We may not know, and may be unable to ascertain, the purpose of a
communication, its distribution, or the degree of access to it that we have been
accorded. We must not allow appearances t cloud our judgment but must maintain
a healthy skepticism regarding our data.

4. Intercoder Reliability
With the exception of raw word counts and other content analysis procedures that
have been thoroughly computerized through programs embodying concept
dictionaries and search or count procedures, all content analysis depends on
human judgments about communication content. Messages, after all, must be
evaluated by researchers. Individual researchers may differ from one another in
their understanding of the content of a given communication. Indeed, only when
some degree of consensus can be reached about that meaning that we can have
real confidence in our measurements.

End of Report **********

Page 14 of 14