First-Year Writing Program Course-Embedded Assessment Samples from Dual Enrollment Students

Dr. Clancy Ratliff Director of First-Year Writing

Below is the report from my office's assessment of a sample of 52 papers from dual enrollment English 101 courses. Compared to the courseembedded assessment we do for our on-site English 102 and 115 courses, the numbers are far lower: for example, the mean score for "Overall" for the 102/115 students was 3.36, and the mean for "Overall" for the dual-enrollment students was 2.04. I do not think we can draw any conclusions about dual enrollment based on this assessment for the following reasons: 1. These were students taking English 101 dual enrollment, so the samples are not exactly comparable. It's possible that if we took samples from our on-site English 101 classes and scored them, the number would be lower than the "Overall" mean of 3.36 scored by the 102/115 students; it's also possible that students taking English 102 and 115 through dual enrollment would score higher than the "Overall" mean of 2.04. 2. Moreover, these students were just beginning English 101 dual enrollment, unless I'm misunderstanding the data. The date on these papers is January 19, 2009, so I know they weren't written toward the end of the semester, when we'd typically collect samples from students. If we collected samples from students at the end of our on-site English 90 course and scored them, I would not be surprised if the numbers were similar to these. 3. These papers were written in class -- not under the normal conditions of brainstorming, drafting, revision: the writing process we teach in the firstyear writing program. The samples we scored from English 102/115 were written over a period of weeks with brainstorming, drafting, and revision. 4. There was an assigned topic (prompt) for this set of sample student essays, a topic that I believe was entirely too broad for a short argument essay. The prompt: "Willard Gaylin, Psychologist stated: 'There is so much nonsense out there about feelings. All the pop psychologists are misleading people about things like guilt and conscience. Guilt is a noble emotion; the person without it is a monster.' This brings to mind certain questions: Should people always feel good about themselves? Or is shame or guilt perfectly appropriate for certain actions? What sorts of behavior would occur if people did not feel bad about doing certain actions? Make a logical argument for or against Gaylin's point, 'Guilt is a noble emotion; the person without it is a monster,' giving examples you find in everyday life." I believe that the unsuitability of this assigned topic, which was no fault of the students' own, caused almost all of their papers to fall under "Poor" in the "Content" category, which includes the description, "Topic is too broad and general for a paper of its length."

Results
The readers scored papers in each of the three categories from 1-6, where 1-2 were poor, 3-4 were satisfactory, and 5-6 were outstanding. The FirstYear Writing Program rubric is on page 4 of this report. After a thorough norming exercise, two readers scored each paper.

Results for on-site English 102 and 115 students:

Overall results (n=100) Content
mean: 3.4

Poor 24% 25% 29% 28%

Satisfactory | Outstanding 76% 75% 71% 72%

Organization
mean: 3.31 mean: 3.39

Language Issues Overall

mean: 3.36

Results for dual-enrollment English 101 students:

Overall results (n=100) Content
mean: 1.96 mean: 1.75 mean: 2.43

Poor 89% 95% 71% 89%

Satisfactory | Outstanding 11% 5% 29% 19%

Organization Language Issues Overall

mean: 2.04

First-Year Writing Program Rubric

Poor (1-2) Satisfactory (3-4) Outstanding (5-6)

Content

Topic is too broad and general

for a paper of its length No clear main idea or sustained position in the paper (or argument is incredible to an academic audience) Argument, if present, is unsupported, or evidence is insufficient Paper is overly reliant on cliches or culturally conditioned/ ethnocentric assumptions and bias Sources, if used, are used inappropriately (data dump, awkward integration, unintentional plagiarism, etc.) Introduction does not orient the reader to the concerns of the paper or contextualize the subject of the paper Arrangement of the paper is haphazard and random Paragraphs do not have transitions that guide the reader from one idea to the next Conclusion is absent or abrupt

Topic is manageable for a

paper of its length Position/argument is comprehensible even if not clearly stated Amount of evidence is sufficient Demonstrates critical thinking (ability to recognize complexity, biases, and stereotypical representations; distinguishing fact from opinion) Use of sources is appropriate; no serious problems with integration of other materials

Topic is narrow enough to

allow for a rigorous, nuanced treatment of the subject Thesis is stated clearly Evidence is ample to support position taken Contains some acknowledgment of opposing/divergent views Sources are used purposefully and strategically, integrated seamlessly

Organization

Introduction is
recognizable even if it is not always reader-based Paragraphs generally treat one idea at a time Attempts at transitions between paragraphs are made, even if they are awkward Conclusion provides some closure to the argument, even if only a summary of the main points

Paper contains a
clear, reader-based introduction, development, conclusion Logical, smooth transitions between sections Plan of development stated (forecasting statement, selfannouncing structure to argument) Conclusion does more than just summarize the paper; restates the thesis in a fresh way or

includes a gesture (call for action, unresolved questions, etc.)

Language Issues

Frequency of error (of any

type) seriously detracts from the content of the paper

Grammar, punctuation, and

spelling are mostly correct Student shows command of language (word choice/ vocabulary) Varied sentence structure

Paper is virtually free of error Writer shows an unusual

felicity with regard to word choice, turns of phrase (ex. uses obscure words, bon mots) Sentence structure is complex but not cumbersome