A Test of Stenography: Some Preliminary Results

By HAROLD
G. SEASHORE AND GEORGE
K. BENNETT, The Psy-
chological Corporation

THETEST
IN BRIEF

THE
Seashore-Bennett Stenographic
Proficiency Tests (1) are worksamples for testing the compe-
tence of stenographic applicants in business and industry.
The tests have been devised to provide a standard set of dic-
tated letters which can be presented uniformly to all applicants
and which will approximate real business dictation. The
letters are presented on standard phonograph records. They
are designed to replace informal dictation by emplo.pent in-
terviewers and to relieve executives of having to conduct a
trial dictation period with applicants. It is also planned that
they will be useful for firms which conduct stenographic train-
ing courses. Such firms need a standard and objective set of
materials for determining the level of training needed by the
trainee and for the testing of their progress.
George K . Bennett is President of T h e Psychological Corporation. He joined
the Corporation as Director of the Test Division in 1935 and became President in 1947.
H e i s a Fellow of the A P A , and i s currently President of the Industrial Division. H e
i s also President of the N e w York State Psychological Association and Vice-president 01
the Division of Anthropologg and Psychology of the National Research Council. During
the war he was a member of the Applied Psychology Panel of the Ofice of Scientific
Research and Development. H e received h i s Ph.D. degree from Y a l e University in 1936.

Harold Seashore i s Vice-president of T h e Psychological Corporation and Director

of i t s Test Division. He is a Fellow of the A P A aird active in semeral of i t s divisions.
He i s a diplomate of ABEP i n the field of counseling and guidance. His Ph.D. degree
i s from the University of Iowa in 1933. Prior to the war he was professor of psychology
and director of admissions at Springfield College. H e joined The Psychological Cor-
poration staff in 194.2 and was Associate Director of the N D R C Radio Code Research
Project. I n 1944 he joined the Test Division as Associate Director, becoming Director
in 1946. He i s at present Secretary of the Eastern Psychological Association.
Presented as a paper before the Eastern Psychological Association, April 17,
1948.
197
198 HAROLD G. SEASHORE AND GEORGE K. BENNETT

These recorded Stenographic Proficiency Tests do not solve

all the problems of selecting stenographers, but it is believed
they are a definite step forward in the direction of more ob-
jective evaluation of applicants. They are intended for use in
conjunction with a suitable intelligence test and a compre-
hensive clerical test.
This paper discusses some of the problems connected with
the measurement of stenographic ability, describes the test and
the manner of administration, and some of the difficulties com-
mon to scoring of stenographic tests. Preliminary data on 52
stenographers employed by Standard Oil Company of Ohio and
39 stenographic trainees in business are presented. These ten-
tative results show striking differences between the trainees
(including some about to complete training) and the employed
group. Only one trainee scored above the lowest one-fourth
of the employed group, and only one employed stenographer
scored below the midmost trainee. Moreover, within the em-
ployed stenographers, test scores showed surprisingly close
agreement with supervisor’s ratings on stenographer’s skill on
the job.
T h e Total Job: Shorthand and T y p i n g . In planning this se-
ries of tests, it was decided very early that separate shorthand
and typing tests are not necessary when stenographers are
being employed. What is needed is a comprehensive steno-
graphic test. The job of a stenographer is to take dictation
under the usual office conditions and transcribe her notes into
mailable letters or usable manuscripts.
The subject sits down in a quiet place with pencils and note-
book. A general introduction is given and then five letters,
containing typical business correspondence, are dictated a t
three levels of speed and varying in length and complexity.
Most proper names and all technical words are spelled out.
The dictator speaks from the record in an ordinary business
voice and with a typical manner of spacing words and phrases.
The rate of dictation is not constant but changes from time to
TEST O F STENOGRAPHY 199
time just as occurs when an executive is dictating to a stenog-
rapher as he thinks. No exceptionally difficult ideas or com-
plicated layouts of letters are used, since the purpose is to de-
termine whether the applicant can handle the basic activities
of a stenographer. It is assumed that if she can handle these
letters she can learn on the job any local requirements or special
technical matters which may be expected of her.
The Tests Are Recorded. From their experience in preparing
special recorded tests during the war and from earlier experi-
ence with stenographic recordings, the authors feel that the
only satisfactory way to get reasonable uniformity of dictation
for all applicants is to use phonograph recordings. With mod-
ern reproducers, satisfactory quality and clarity of voice can
be secured. Preliminary tryouts2 with several hundreds of
stenographers have shown that the applicants are not bothered
by the absence of the real speaker. Perhaps the universal ex-
perience of hearing radio speakers has helped in this matter.
The tests are prepared in two forms: B-1 and R-2. Tvvo
sets of five letters each provide alternate forms which permit
retesting. Two forms are desirable for several reasons. A re-
test may be needed when the first testing has been invalidated
by some external interference. Some firms will restrict one
form for use in hiring and the other for use in upgrading.
Others will use one for hiring and the other for graduation from
the training program of the firm. It is anticipated that new
forms will have to be added to the series as the present ones
become more well known.
The scripts of the letters are presented in a, booklet accom-
panying the records. Each letter is shown in typescript as
laid out by competent stenographers. All special variations
in dictation, such as paragraphing, punctuation, and spelled-
out words, are indicated on the scripts. Obviously there are
other ways to arrange the letters on the page; the copies as
2 Mr. Charles R. Langmuir cooperated in the development of the tests and super-

vised these tryouts in a large industrial firm employing hundreds of stenographers.

200 HAROLD G. SEASHORE AND GEORGE K. BENNETT

presented are considered standard and acceptable. Each user

of the test can determine his own standards of aesthetic quality
and layout.
About twenty minutes are needed for dictation and from
thirty to sixty minutes for transcription at the typewriter.
SCORING THE TESTS
The Problem of Rate. In preparing these scripts and record-
ing them, considerable argument was discovered in the matter
of how rate of dictation can be properly measured. We have
concluded that the simple counting of words and dividing this
value by the time is not a satisfactory method. Too many
realistic complications vitiate the significance of the single
number which results. For instance, should the salutation be
included in the count? It can be arbitrarily left out; but in
terms of the stenographer’s own activity, it is part of the con-
tinuous dictation. How does one count the natural “hems and
haws” of the dictator? These moments allow the stenographer
to catch up and thus they are a functional slowing of the rate.
Shall one count one word or many words when a proper name
is pronounced and then spelled out and repronounced? Does
one count a word if one announces a “paragraph” or says “quo-
tat ion’’?
Three ways of computing rate are presented in the manual.
Regardless of which is used it is the authors’ judgment that
any beginner or applicant of little experience who can properly
transcribe the slow letters (1 and 2) is employable in most
positions, and that anyone who can produce mailable letters
a t the medium speed (letters 3 and 4) is stenographically com-
petent for almost any position. Anyone who can transcribe
good letters from the fast dictation (letter 5 ) is eminently com-
petent.
Several methods for scoring the Stenographic Proficiency
Tests were suggested. No one knows at present just how
much each type of error should be weighted. In different
levels of jobs, different degrees of significance are attached to
TEST OF STENOGRAPHY 201
different errors. Any evaluation must include not only the
obvious errors of punctuation and omitted words but also the
additional matters of layout and neatness. It is almost im-
possible to set up any plan for equating the many kinds of
errors so as to yield a single, meaningful numerical score.
The ultimate criterion is mailability. If a letter produced
by the person being tested is completely mailable (or can be
made mailable by neat corrections), it is apparent that the
person is competent at the level of stenographic difficulty rep-
resented by the letter. For this reason in evaluating these
worksamples, some personnel officers may decide that a person
has passed the test at a given speed only when the letters pro-
duced are strictly mailable.
We must note at once, however, that no matter how ideal
the testing situation is, the applicant should be allowed some
leeway from perfection just because her stenography is being
appraised in a test situation with employment as the goal.
Because of emotional stress the applicant usually will not be
as good in performance as when she is settled down to her regu-
lar job. Furthermore, any minor weaknesses can usually be
ironed out on the job if the applicant is essentially employable.
For these reasons, we have attempted to set up a tentative
scoring scheme which will eliminate from consideration those
who are grossly incompetent and at the same time will yield
a range of scores for those who should be considered. The pro-
posed scheme is based upon a careful analysis of hundreds of
letters secured from an earlier edition and of 195 letters secured
on this edition. These came from a sample of 39 stenographers
who were in a training pool of a large insurance company.
A Suggested Scoring.Procedure. Anyone with some experience
in employing stenographers will quickly recognize the following
factors as being important in evaluating the work of a new
stenographer.
1. Typing-Neatness and cleanness.
2. Arrangement of letter on the page, paragraphing, length of lines, etc.
3. Typing-Over-all quality of stroke.
202 HAROLD G. SEASHORE AND GEORGE K. BENNETT

4. Typing-Errors such as erasures, strike overs, uncorrected mistyping, etc.

5. English-Spelling, punctuation, and division of words.
6. Shorthand-Omitted, mutilated, or substituted words, phrases, and sentences.

The precise weighting to be allowed each of these factors, or

to each error within each factor, may vary from job to job.
For instance, one would be more tolerant of shorthand mis-
takes by a beginner than by a person claiming to be experi-
enced; standards of accuracy of spelling or punctuation, how-
ever, should not be much different for these persons.
The authors find that a simple and workable scoring scheme
is possible using the above outline. The main points in the
plan are:
1. Each letter is allowed 20 points of credit.
2. Scoring is by penalties. Each error receives a penalty of -1 or -2 points, as
defined by a “Schedule of Penalties.”
3. The total number of penalties for a letter is limited to 20. That is, if a letter
is very bad, i t is scored as zero (20 points minus 20 penalty points).
4. Letters with penalties between 0 and 20 are considered evidence that the writers
are worthy of more careful scrutiny by the personnel officer. (Any organiza-
tion can, of course, set its own local standards by lowering this limiting value
of 20 points without changing the general scheme.)

A Summary Chart (Figure 1) for recording errors of the ex-

aminee is provided. The Summary Chart not only facilitates
objective scoring of the letters but it also reveals the strengths
and weaknesses of the applicant. For different job require-
ments and for different backgrounds of experience, these dif-
ferences in scores on the several factors may be important. For
example, a person with poor performance on Factor 6, Short-
hand, is not immediately ready for stenographic work no matter
how neat and accurate her typing may be. However, in a firm
which has provisions for training and on-the-job refresher work,
one might consider employing a person with substandard short-
hand if her other skills and abilities are excellent. On the
other hand, a poor speller or an applicant who shows evidence
of simply not being able to understand common words is
probably not employable even as a stenographic trainee.
A Simpler Scoring Procedure. Some users may prefer sim-
TEST OF STENOGRAPHY 203
THE SEASHOREBENNETT STENOCWAPNIC P R O F I C I ~ C YTESTS

SUMMARY C H A W

NAME.................................................................................................................. ,.DATE............................................

Tested with Form El .........

Form 8 2..........

PENALTIES ON EACH FACTOR FOR EACH LETTER

ENGLISH
5. Errors

SHORTHAND
6. Errors
Maximum Score 20 20 20 20 20

Sum of penalties

LETTER XORE (20 minus sum of penalties)

L -
MAKE A CHECK IN THE PROPER BOX IF ANY OF THE POINTS BELOW APPLIES
A. Scoring stopped -over
8. IncompIete -not scored
20 penalties
1
C. Too mixed up to score
D. Letter not attempted
.

Signed........................................................................
26-1895
FIG.1
pler plans of appraisal than the analytical scoring just de-
scribed. In this case, the person making the evaluation should
204 HAROLD G. SEASHORE AND GEORGE K. BENNETT

be thoroughly acquainted with the tests and with the problems

of stenographic performance. This person then would make
an inspection of each letter and mark it acceptable or unac-
ceptable and give an over-all recommendation. One might
use categories like “Mailable,” “Passable,” and “Unaccep-
table” for describing the letters.
SOMEPRELIMINARY DATA
Objectivity of Scoring. A study was made of the objectivity
of scoring. The analytical scoring procedure described above
was applied. In actual practice each error (as defined in the
Schedule of Penalties) is marked on the paper and then entered
on the Summary Chart. However, since two persons were to
score each paper in this research no marks of any kind were
made on the papers themselves.
Two clerks in the research section of the Test Division of The
Psychological Corporation were asked to read the manual and
score about ten letters. These two clerks and the supervisor
were then brought together for a conference to discuss any
problems of scoring. The two clerks then proceeded to score
all of the papers of the 52 subjects independently. The fol-
lowing coefficients of correlation between the two scorers were
obtained: Letter 1, .74; Letter 2, .78; Letter 3, .78; Letter 4,
.97; Letter 5, .94.
It should be noted that for letters 1 and 2 the range of scores
secured was relatively small for this sample of employed ste-
nographers and, consequently, a difference in scoring of one or
two raw score points would greatly affect the correlation co-
efficient of objectivity of scoring. When the scores on each
subject’s five letters were averaged to yield one test score for
each stenographer, the coefficient of correlation beheen the two
scorers was .94,indicating high correspondence between scorers.
We felt that a discrepancy between scorers of four or more
points on a letter should be considered highly undesirable.
The following is a report of the number of cases of discrepancies
of four or more points: Letter 1, none; Letter 2, three; Letter
TEST O F STENOGRAPHY 205
3, one; Letter 4, two; Letter 5 , nine. For the average score
on five letters there were no cases of discrepancies as large as
four points.
On the basis of this study with 52 subjects, it is our belief
that the Stenographic Projiciency Test can be scored with ob-
jectivity by reasonably competent clerical employees. In the
present case the scorers were not stenographers nor did they
know shorthand. In normal use of the test, actual errors
would be marked on the papers rather than tallied on a sheet
of paper a t the side, as was required for this experiment. More
complete location of errors and more careful counting of them
is thus possible under the standard scoring procedure.
iVorms. We are not prepared to publish extensive norms at
the present time, partly because we do not have any “general
sample” of stenographers. Two samples, however, are avail-
able. In the Cleveland Officeof the Standard Oil Company
of Ohio, 52 experienced stenographers were tested. This was
the sample reported above in connection with objectivity of
scoring. We also had a sample of 39 trainee-stenographers
from the “training pool7’of an insurance office in New York;
this group included recent high school graduates and others
with varying amounts of experience who were trying to meet
the company’s minimum requirements for stenographers.
They were temporarily employed in other capacities and spent
one hour a day in the training class; we were told that experi-
enced stenographers sometimes spent as little as three or four
days in the “pool” while others sometimes stayed as long as
five or six months.
Distributions of the scores on each of the five letters of Form
B-1 are given in Table 1. Table 2 presents a distribution of the
sums oj the five letters. It is apparent that the employed ste-
nographers are considerably better on the test than are the
trainees. We have no norms as yet on Form B-2; it is our
belief, however that it is a little harder than Form B-1 and
that norms will not be interchangeable.
Reliability. Three coefficients of reliability have been com-
206 HAROLD G . SEASHORE AND GEORGE K. BENNETT
TABLE I
Distributions of Scores on Each of Five Letters of the Seashore-Bennett Stenographic
Proficiency Test (Form B-1) Made by 62 Employed Stenographers and by 39 Trainee
Stenographers
I LEmB

20
19
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0**

Median
____

* Numbers in italics indicate location of median case in each distribution.

** Persons who made 20 or more errors or did not attempt letter.

puted by correlating performances on letters 1, 3, and 5 with

performances on letters 2 and 4. The coefficients, after cor-
rection for split-half correlation, are as follows :
Trainee-Stenographers (N==39), ................................... .80
Employed Stenographers (N==52). ................................... .83
Combined group (N == 91).. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .91

The three coefficients illustrate the importance of defining the

groups on which reliability reports are based. The two sub-
groups represent relatively restricted ranges of ability. The
TEST OF STENOGRAPHY 207
trainees are restricted in range of competency to the lower
portion of distribution, the experienced group to the upper
portion. The two together are a more representative sampling
of the full range of stenographic competency which one might
TABLE I1
Distributions of Total Scores (gum of 6 Letters) on the Seashore-Bennett Stenographic
Proficiency Test (Form B-1) Made by 6.9 Employed Stenographers and 39 Trainee
Stenographers
TOTAL SCOPE EMPLOYED STENOGPAPHEPS TPAINEE STENOGRAPHERS

95
90
85
80
75
70
65
60
55
50
45
40
35
30
25
20
15
10
5
0

39 (14)*

Median 44

* Number in parentheses indicates the subjects whose total score includes one,
two, or three zero scores for unattempted letters. All subjects tried at least letters
1 and 2.

expect among applicants for work. The reliability coefficient

of .83 is pertinent if one is using the test for merit-rating pur-
poses among successful employees; the r of .91 is pertinent if
one is using the test to screen out the best applicants from
among the wide range of abilities presented by persons at the
employment office.
208 HAROLD G . SEASHORE AND GEORGE K. BENNETT

Validity. We have at present several kinds of evidence

which indicate the Stenographic Proficiency Test is a valid
measure of stenographic competency. First, the test is ob-
viously a test of stenography. It fulfills the requirements of
a worksample with high face validity. Second, the differ-
ences in distribution of scores of the two groups which were
known to be different are evidence of the discriminating power
of the test. One should bear in mind that the trainee group
was not limited to “beginners”; there were actually some per-
sons who would remain in the training class only a few days,
just long enough to pass the minimum requirement tests for
assignment to a stenographic job. If these persons had been
TABLE I11
Validity Coeficients of the Seashore-Bennett Stenographic Projkiency Test (Form B-1)
for 62 Employed Stenographers
CORRELATION WITH
SUPERVISOR’S BATINGP
TESTS

1. Stenography.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .61
2. Typing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41
3. 1 and 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5s
4. Language., . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32
5. General Clerical Ability.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20
6. 1 , 2 , 4 , a n d 5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4s
7. Over-all efficiency.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3s
8. All ratings combined... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49

omitted, even less overlap would have occurred in the distri-

butions in Table 2. As it is, only one member of the employed
group is below the median of the trainees; and only one of the
trainees exceeds the 25th percentile of the employed group.
Third, we present data on the SOH10 group in Table 3 which
show that the performance on the test correlates rather well
with the ratings of these stenographers by their supervisors.3
* The reader may find it interesting to compare these validities with those pre-
sented by Rundquist and Bittner (page 163, this issue). The over-all efficiency
(rating 7) rating of stenographers is considerably less predictabIe than is the simple
raw score sum (rating 6) of the individual trait ratings of which i t is composed. This
finding adds support to the contention of Rundquist and Bittner to the effect that
foremen are generally incapable of assigning the best weights to various job factors
in arriving at an “over-all” rating.-Ed.
TEST OF STENOGRAPHY 209
Considering the restricted range of test scores and the pre-
sumed restricted range of possible ratings, these values are
important.
Fourth, the members of the trainee sample were being taught
in two classes consisting of (A) 23 more experienced trainees,
and (B) 16 less experienced trainees. Classification of trainees
as more or less experienced was made be€ore the test was ad-
ministered. The median score of group A was 52 points and
of group €3, 34 points. Here is again evidence that the test
distinguishes between groups of known difference in compe-
tency.
REFERENCE
1. SEASHORE,HAROLD G., AND BENNETT, GEORGEK. : Stenographic Proficiency Tests.
Manual arid Recordings, 1946, New Pork, The Psychological Corporation.