You are on page 1of 10

A War of Two Worlds – Part 1

In this, Part I of a three part article, the author presents a new method to evaluate
the potential effectiveness of Op Ed or other expository writings. Part 2 considers
the barriers faced by activists as they attempt to influence citizen motivation to
participate in the political process. Part 3 looks at the targets of policy change, the
Politicians, and the potential for changing their behavior.
Introduction
With hundreds of Op Ed articles submitted each week to OEN and many more to
similar sites, there appears to be no mechanism to evaluate the likelihood that
submitted articles will meet their objectives, (persuading readers to adopt the
theses advanced by their authors). While OEN has developed system designed to
assess some perceived qualitative characteristics of submitted articles, it lacks the
clarity of terms, sensitivity and objectivity required to meet this objective.
One element in this process is the level of competition for the eyes and ears of the
audience. OEN is only one of literally thousands of opinion sites, each competing for
their text, audio, or video media offerings. In that competitive environment, an
author seeking attention to his work is required to conform to an increasing
standard of excellence, lest he be quickly discarded.
A central thesis upon which this paper is premised tells us:
Op Ed media, in whatever form presented, have a low
probability of changing attitudes of their audiences, and when
they do, these changes tend to be transient, with audiences
returning to originally held views after a short period of time.
Two opposing articles, “Lesser Evils,” by J.C. Garrett, [JCG] and Kevin Gosztola’s
[KG] Lesser Evils, Psychoses, and the Perils of Voting Your Conscience [Part 1], a
rebuttal to the JCG posting, presented a an opportunity to test this hypothesis, using
a new procedure for rating the potential effectiveness of Op Ed submissions.
Assessing the Impact of the Op Ed Article
In an effort to assess the impact of Op Ed articles on the reader’s voting choices, the
author offered a poll designed to answer this question, with respect to the two
articles being evaluated. While the number of respondents was insufficient to meet
requirements for statistical significance, the trending illustrated in the tables below,
was instructive.
By a ratio of 57:43 respondents indicated an intent to vote for Obama. If this ratio is
at all representative of OEN members, those planning on voting for 3rd Party
candidates, or withholding their votes, is unexpectedly high.
But, is it really that high? We have only to look at the disaffected “professional”
delegates to the Democratic National Convention, the Clinton supporters, some of
whom remain intractable in their decision either to vote for McCain, or not vote.
They, of all people, should have the necessary data in hand to make an informed
choice.
Before attempting to explain this apparent difference between reported and
expected behavior, the effect of the Op Ed postings is assessed. As noted in the row
percentages, those who reported as being influenced by articles accounted for 21%
of respondents. While the total n is small, the difference between those affected
choosing to vote for Obama (13%) vs. those making the irrational choice (30%) is
additional confirmation of higher than expected irrational selections.
Evaluating the Op Ed Submission
The probability that an individual will be persuaded to agree to that advocated by an
Op Ed position if clearly a function of two elements; (a) the readiness or
susceptibility of the individual to persuasion, and (b) the effectiveness of the
presentation of the Op Ed communication. The calculation of (a) is beyond the
scope of this discussion, leaving our focus directed at the development of a method
for objectively assessing the viability of any Op Ed article.

Description of the methodology

While many rating systems are employed to evaluate the effectiveness of articles
(as a class – no discrimination is made between Op Ed and other categories of
postings), all have in common two characteristics. Ratings are entirely
subjective. When terms such as “Importance,” “Value,” “Perspective,” are
employed, none have sufficiently precise definitions to insure raters use the same
criteria for their ratings
Regardless of the clarity of rating terms, the more serious flaw, from the perspective
of analysis, is that each rating criterion is applied to the entirety of the piece
under consideration. Such an application fails to consider intra-article variations,
nor the structure of its elements.
Thus, the author offers a new paradigm, applicable to any expository writing. It
offers opportunity for raters to apply objective criteria to each sentence to
determine, the degree to which it contributes to meeting the overall
objective of the article
Finally, it allows for a comparison of multiple articles, whether by the same or
different authors.
For those interested in the mechanics necessary to carry out this analysis, readers
are referred to in the White Paper prepared by this author last year. Titled, Do it
Yourself Data Mining, it develops in great detail, the theoretical issues and the
methodologies related to this method.
Preparing the Text
To prepare the text of each articles to be rated, first separate each sentence
(defined by a period, question mark, or explanation mark), separated with by a
paragraph. This is quickly accomplished using the Global Replace function in your
word processor.
Following conversion of this format to a table, each sentence is rated, as shown in
the example below, (The “Comment” column is optional).

Sentence Types

Each sentence is classified as being in one of the following categories:


CLASS DESCRIPTION
LEGEND
ANALOG Analogous scenario used to illustrate author’s view
AXIOM Rhetorical or Self Evident statement. Such Statements seldom elicit
disagreement, and are used as the foundation upon which premises are constructed.
Most importantly, they convey no information.
CRITIC A statement questioning validity of fact and/ or conclusion or logic of an assertion
EXPLAIN Explanation, Cause/Effect, or interpretation of a statement. Frequently contains
words like, “because,” “since,” or other cause-effect linkages.
FCON Fact-based and/or logically derived conclusion
PCON Personal Conclusion, representing a personal opinion, a value judgment, and/or
emotionally based statement
PRED Prediction of some future event or consequence
PRES Prescription. Cues come from words like “should,” “must,” “ought to,” etc.
QUOTE A quote from someone other than the writer
RQUES Rhetorical Question
SQUES Slanted Question
ERROR Cannot be classified – either incomprehensible, logically flawed, has unknown
purpose, or is completely off-topic
VFACT Verifiable Fact

Ratings

These numeric ratings are used to answer the question, “To what degree is this
statement helpful to persuading the reader to agree with the author’s intended
message?” These ratings are as shown here:
Rating Contribution to Goal
1 Very counterproductive to
achieving writer’s goal
2 Somewhat
counterproductive to
achieving writer’s goal
3 Neutral
4 Somewhat useful in
achieving author’s goal
5 Very useful in achieving
author’s goal

The Two Articles Evaluated

In order to process the results for multiple articles, they are combined together, with
the author’s initials displayed in the first column. This rather extended set of ratings
may be viewed by clicking this link:
The Results

The table below illustrates the percentage frequency of each of the categories
listed. Immediately evident is the structural difference in the approach the two
authors use to present their views.
JCG KG total
ANALOG 7 0 3
CRITIC 0 6 3
PCON 37 32 35
EXPLAIN 6 12 9
FCON 28 6 17
PRED 8 0 4
PRES 1 0 0
QUOTE 0 5 2
RHET 1 3 2
RQUES 0 4 2
SQUES 3 9 6
ERROR 0 15 8
VFACT 7 8 8
Total 100 100 100

Each of the values shown represents the percentage of each category, within its
respective column. One thing which stands out is the difference in the use of
Factual Conclusions (FCON), between JCG and KG. While both employ similar levels
of Personal Conclusion (PCON) we shall see how these have differing impact on
reaching their goals. There was an unexpectedly high number of statements scored
as ERROR, in the KG article, marked as such because they were either
indecipherable, factually in error, or were seriously off-point.
The question which of whether the difference in structural approach, apparent in the
table above could be reliably expected to occur in additional work by the two
authors. Here, we turn to a test of Statistical Significance. The “p < .0001,” indicates
that there is only a 1 in 10,000 chance a similar rating of two different articles by
the same authors would NOT produce these same differences.
Chi-
square = 65.8 with 12 df
p<
0.0001

It should be noted that KG had the far more challenging task. since even if
successfully accomplished, rebuttal still requires an alternative solution, which while
implied, was never overtly stated.
Even more important than the structural analysis is the Effectiveness Rating given
to each sentence or question.
Shown below, we see that the Effectiveness Rating most frequently appearing is
either a “1,” Very ineffective, or the “4,” Moderately Effective.

When we break these by author, we see some major differences in the ratings
achieved by each:
Here are the number of ratings obtained by JCG:

Compare this with that of KG:

Here, with the sixteen ERROR ratings, which are always scored as a “1,” along with
a number of the PCON statements, he loses the opportunity to make a convincing
argument.
It should be noted that KG had the far more challenging task, since even if
successfully accomplished, rebuttal still requires an alternative solution, which while
implied, was never overtly stated.
Finally, we can look at the Total Effectiveness which is derived by taking the sum of
the ratings and dividing by five x total statements, Thus the effectiveness for JCG is
Calculated as:
392/(5*95) = 83%
and for KG,
221/(5*103) = 43%
This translates into the likelihood, for a reader receptive to persuasion, that the
respective articles will achieve that change in attitude.

The Power of the PCON

Ultimately, persuasion rests with the manipulation of the reader’s emotions. The
statements which have been classified as PCONs are those which most directly
reinforce or attack the presumed existing attitudes of the reader. Low ratings were
attached to those statements which were seen as attacking the reader’s view, while
the converse was true for the high ratings.
To make these differences meaningful, her is an example of two JCG written PCON
statements which were positively rated:
“Barack Obama, while possessing that common trait that lives
in all of us is not evil.
“At least no more so than the rest of us.”
These statements will anger no one, although some may disagree with their
content.
JCG PCON Ratings
Contrast this to these two inflammatory PCONs made by KG, when responding to
JCG’s rhetorical statement that “… people have been choosing the lesser evil since
the very dawn of man.”
“This comment is grossly ignorant of history.”
“People have not been choosing the lesser evil since the
very dawn of man.”
Along with the gratuitous rudeness and lack of respect contained in the first
statement, that this author would chooses to debate a rhetorical comment,
suggests a lack of understanding of the substantive issue under consideration – the
merits of choosing an alternative candidate to Obama.
KG PCON Ratings

Where’s the Beef?

While carefully crafted PCONs can establish receptiveness to a previously


unacceptable position, there still remains the requirement that new actions,
problem resolutions or actions are presented for adoption. Like a political party, an
Op Ed article needs a “platform,” to advocate.
As the table below suggests, the two authors differ widely on this
Information/Emotion Continuum. They are nearly mirror images of each other in the
weight of the categories of sentences which carry information vs. those which are
emotional in content.
Information JCG% KG%
ANALOG 7.4 0.0
AXIOM 11.6 3.7
CRITIC 0.0 0.9
EXPLAIN 15.8 11.1
FCON 13.7 4.6
PRED 10.5 0.9
QUOTE 0.0 4.6
VFACT 2.1 9.3
Total Info 61.1 35.2

Emotion JCG% KG%


ERROR 0.0 19.4
PCON 34.7 31.5
PRES 1.1 0.9
RQUES 0.0 3.7
SQUES 3.2 9.3
Total
Emotional 38.9 64.8
What we see in the KG article is an attack with neither a solution, nor even an
attempt at delineating the consequences of whatever it is he wants the reader to do,
or not do. He makes one limited attempt when he restates a JCG prediction:
“This foundation of his argument makes predictions that
McCain would ruin the Supreme Court, unions, science, ignore
global warming, forget the poor and unemployed, maintain the
gap between rich and poor, poorly address taxes, trash social
security, and more baseless …”
Yet, the JCG article, while in every way, evaluated as being far more persuasive to
the position taken by its author, is the subject of frequent attacks by its readers.
In an attempt to rate the response received by both authors, all the comments
received (minus the author’s own responses) were classified as “Agree,” “Disagree,”
or “Unknown.” The Unknowns were simply that. It was uncertain what, if any point,
the commenter was making. This was the result obtained:
JCG KG Total
Agree 35.7% 11.1% 31.4%
Disagree 38.1% 55.6% 41.2%
Unknown 26.2% 33.3% 27.5%

There is little that can be inferred from this information, except for the evident
dominance of emotion as opposed to thoughtful formulation of ideas by those who
have commented on both articles.

Is it Worth the Work?

If you have reviewed the extended display of the ratings for the combined articles,
your first response is likely to be, “This is just gong to take too much work, and too
much time.” It’s true that the combined articles contain 203 sentences, and that
that configuring the tabular format takes a few minutes. For those lacking formal
statistical tools, the longest part is calculating the many values in your existing
spreadsheet program. For those interested in trying out this process, please contact
me for assistance in preparing spreadsheet templates which can largely automate
these quantitative findings.
For those participating in this rating process, perhaps the greatest advantage is that
the rater is forced to remove all emotion as a result of the intensive consideration
given to each and every statement.
If the author is rating his own work, this is an astonishingly effective tool which
facilitates quality self-editing, as the author is forced to step back and look at the
writing as would an outsider.

You might also like