Evaluation of the Maintenance Management Information Systems of the United States

Postal Service
Author(s): Blair L. Wildermuth and B. L. Foote
Source: Interfaces , Feb., 1979, Vol. 9, No. 2, Part 1 (Feb., 1979), pp. 42-49
Published by: INFORMS

Stable URL: https://www.jstor.org/stable/25059722

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide
range of content in a trusted digital archive. We use information technology and tools to increase productivity and
facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
https://about.jstor.org/terms

INFORMS is collaborating with JSTOR to digitize, preserve and extend access to Interfaces

This content downloaded from

182.255.0.242 on Thu, 17 Sep 2020 07:03:23 UTC
All use subject to https://about.jstor.org/terms
 INTERFACES Copyright ? 1979, The Institute of Management Sciences
Vol. 9, No. 2, Pt. 1, February 1979 0092-2102/79/0902/0042$01.25

EVALUATION OF THE MAINTENANCE MANAGEMENT

INFORMATION SYSTEMS OF THE UNITED STATES
POSTAL SERVICE

Blair L. Wildermuth
U.S. Postal Service Operations Group
Washington, D.C. 20260
B. L. Foote
School of Industrial Engineering
University of Oklahoma
Norman, Oklahoma 73069

Abstract. The methodology used in evaluating a maintenance informa

tion system for the United States Postal Service is described. Maintenance
inspectors filled out the forms and gathered the data while performing their
normal maintenance and inspection duties. This combination of duties con
sumed 100 man-years. The study verified that the use of the computerized
information system had exposed 19 million dollars in excess inventory. Correc
tions made in the system as a result of the study developed an audited savings by
postal authorities of 4.2 million dollars (versus a total study cost of 1.6 million
dollars).

Introduction
The purpose of this paper is to present the methodology of an actual evaluation
study of a large information system. It is hoped that the data and methods presented here
will provide a basis for both practitioners and theoreticians to improve the study of a
Management Information System (MIS).
One hundred and eighteen post offices are part of the National Maintenance
Information and Control System (NMICS) of the United States Postal Service (USPS).
This system controls the maintenance of 44,806 pieces of equipment in 118 offices.
There are 336,917 employees in these offices, of whom 15,592 are maintenance
employees. The budget for the maintenance program is $560,000,000 per year, exclud
ing utilities. The computer system, software, and planning associated with NMICS cost
just over $773,000; the annual cost of operation is more than $150,000. The number of
distinct maintenance-related decisions pertinent to NMICS exceeds 225,000.
The importance of reporting practical experience is highlighted by the results of a
recent workshop summary [3]. Mr. Hoffer reports, "The conferees left the workshop
with no strong agreement on measuring the performance of an MIS" and "the major
disappointment to those in attendance at the workshop was that more experience with
methodology was not reported. ' ' These difficulties were simply due to: ( 1 ) the size of the
system, and (2) the inability of the users to identify revenue generated or costs saved by
the system.

INVENTORY/PRODUCTION ? POLICIES, MAINTENANCE

42 INTERFACES February 1979

This content downloaded from

182.255.0.242 on Thu, 17 Sep 2020 07:03:23 UTC
All use subject to https://about.jstor.org/terms
Evaluation methodology
According to MIS literature, the ideal information system design takes into account
the level of the decision maker in the organization, his personality type, and whether the
decision is clerical, operational, tactical, or strategic. However, in the United States
Postal Service, any information analysis must also contend with the Postal Manual. This
manual is massive; it has weighed from 10 to 30 pounds under various revisions of its six
volumes in the past decade. However, not just its weight makes it unwieldy. Everyone in
the Postal Service, whether mechanic, maintenance supervisor, or postmaster, must ' 'go
by the book," i.e., the Postal Manual. Because of this, the design and evaluation of an
MIS for the USPS was greatly simplified.
The management study reduced to: (1) documenting the decision required; (2)
verifying the appropriateness of the model required to be used as a basis for the decision;
(3) documenting the data and information required by the model; (4) determining the
accuracy and timeliness required; (5) making sure that documents produced by the
computer system were readable by postal personnel.
The resultant information system should display data and information according to
human factors principles. However, there is a paucity of information about the design of
displays from Management Information Systems. In this case, the authors were con
cerned with only two criteria: (1) Is irrelevant information displayed? (2) Can relevant
information be easily located visually and without error?
The evaluation methodology then consists of documenting the five elements in the
National Maintenance Information and Control System of the U.S. Postal Service and
comparing the results with a set of operating standards.

Operating standards
Most of the evaluation process is rather straightforward. The decision makers were
identified by their job description, and their decisions were checked to determine if the
decision rules (applied to their data) did imply the decision they made. The models used
were examined to determine if they were still appropriate. For example, all local post
offices use periodic review systems for their routine maintenance inventory control. The
assumptions of a chosen model were checked to see if they were still valid by looking at
current demand data and cost figures.
The major problem was the determination of standards for accuracy and timeliness.
For example, if the data are in error by 7.3%, is this tolerable or intolerable?
It became clear from the study that many of the operating standards had to be
subjective simply because no real quantitative justification for the standard could be
produced. For example, inventory levels were reviewed monthly or quarterly. No real
reason for the choice of these time periods existed other than they seemed reasonable and
convenient. However, once these decision times were agreed upon, then it was easy to
evaluate the timeliness of the MIS data. In most cases, the data and information were
available at the proper time. The simplest EOQ inventory model possible was used, to
avoid incurring either excessive data collection costs or data storage costs. In addition,
there were insufficient funds to establish unique decision parameters based on the

INTERFACES February 1979 43

This content downloaded from

182.255.0.242 on Thu, 17 Sep 2020 07:03:23 UTC
All use subject to https://about.jstor.org/terms
individual costs associated with the inventory items, since these ranged in the tens of
thousands. Consequently, the decision to include an item in the NMICS model, for the
items sampled, was based on determining if the demand pattern was unusual or if the
item was a high cost item, which really belonged in regional inventory. (Regional
inventory holdings were not a part of the study.) Data on emergency orders were
gathered to validate the observation that the reports were, in fact, timely.
Establishing accuracy standards was more difficult; this usually had to be deter
mined on a case-by-case basis. For example, if the decision required was to determine if
manpower was adequate to maintain the letter sorting machines in a regional post office,
then this decision would be based on the number of breakdowns, number of hours of
mechanic maintenance time, and whether or not adherence to maintenance procedures
by mechanics had been observed. If a 10% error in reported maintenance time would not
cause a change in the work force, then this error was deemed acceptable. Thus, in these
situations, the accuracy required was determined by the percent error which would cause
an unnecessary change in staffing level based on the office's decision rules for staffing.
A percent error in inventory that would cause a 10 percent chance of a stockout would be
unacceptable accuracy for the inventory level values, if a "no stockout" policy is
mandated. Each decision then required a separate analysis for accuracy. Using the above
examples as guides, rules of thumb for the accuracy required were worked out [9].
The following is an example of calculations which were then applied to similar
decisions. One post office has eight Mark II facer cancelers which require 12,785
manhours, according to maintenance standards. Plus or minus five percent accuracy
would be 639 manhours over or under that standard, or about one-third of a manyear
(using 1,768 as hours available per man per year). If we allow up to 10 percent
inaccuracy, rather than five percent, we would show 1,278 extra hours needed (% of a
manyear). Current hiring policy would dictate hiring one more man. At another post
office, the actual hours totalled 15,728 for one year. Ten percent inaccuracy would again
result in one unnecessary man being hired. However, the only office where a five percent
error in manpower hour reporting would cause the hiring of an unnecessary man was that
post office which used 46,902 manhours for six Multi-Position Letter Sorting
Machines.

Data collection

Random samples of the different classes of post offices were taken. In addition,
major post offices of critical importance were added to the sample.
The system was observed over a three year period, with observations continuous
(not random) for the offices selected. This was necessary to insure that the time data
utilized in the system corresponded to complete adherence to the maintenance proce
dures described in the Manual of Operations. Postal maintenance employees are familiar
with routine review and monitoring of the maintenance system, from observations at the
Maintenance Technical Support center (MTSC). Engineers and/or supervisors accom
panied the maintenance personnel, and made sure that the work sheet was followed and
the proper data entry was made into the system. Their observations were then compared

44 INTERFACES February 1979

This content downloaded from

182.255.0.242 on Thu, 17 Sep 2020 07:03:23 UTC
All use subject to https://about.jstor.org/terms
to the computer output of the NMICS. The output displayed the reported manhours,
broken down into the appropriate class of machine and the type of work (routine
preventive maintenance or repair of failed equipment). The period of observation lasted
from one to two days to several weeks. Copies of all reports were obtained over a
one-year period for comparison. (The computer output came from the Postal Data Center
at St. Louis, Missouri, and was monitored by the MTSC office in Norman, Oklahoma.)
A random sample of equipment and decisions was drawn. However, time permitted
evaluating only five out of 118 offices. In these five offices, 165 different pieces of
equipment were reviewed and records pertinent to 380 decisions were compiled. Thus,
the size of the random sample was smaller than originally projected.

Evaluation documentation

The following are examples of data gathered.

Inventory
A. Decision Maker: Inventory clerk, maintenance manager
B. Required Decisions: Level of stock required at local offices
C. Model: periodic review
D. Data Required: previous twelve months usage
E. Accuracy Required: actual level within plus or minus 5%.

Evaluation
Emergency orders from offices were checked. (These indicated whether reordering
was in time to prevent prolonged machine downtime.) The data indicated the system
operated in a timely manner. Observation of clerks indicated they knew the procedures
and used the proper data. They had no problems with the format of the report. The actual
balance on hand, versus the computer balance on hand, for the five offices sampled was
as follows:

Office I +2.3%
Office II -1.9%
Office IB +4.1%
Office IV +3.2%
Office V -1.3%.

Thus the system met the accuracy requirement (plus or minus 5%) in these five
offices.
The adherence of maintenance managers to ordering rules were checked from the
data shown in Table 1.

INTERFACES February 1979 45

This content downloaded from

182.255.0.242 on Thu, 17 Sep 2020 07:03:23 UTC
All use subject to https://about.jstor.org/terms
 Table 1
Inventory line items and costs

Main Actual Needed

Post Office Line Items Value Line Items Value

I 6,276 $ 402,077 5,700 $300,000

II 17,781 $1,800,032 10,575 $810,000
III 6,362 $ 122,170 5,810 $315,000
IV 7,098 $1,317,100 5,695 $415,500
V 5,512 $ 142,136 5,420 $275,950

It is clear from Table 1 that Office II is carrying nearly one million dollars
inventory. The computer control system had been circumvented. This result
checks of other offices for similar situations. The question remains: why, in
where adherence to procedures is so ingrained, would this situation develop? T
lies partly in another unique aspect of the postal service: the postal insp
inspection service is equivalent to an internal police and audit organizati
determines if a post office is operating efficiently and legally. An inspector's a
and recommendations carry a heavy weight with higher level manageme
inspectors descend on you when a disruption of mail service occurs. In the n
mechanized main post offices, the manager is at the mercy of mechanical failu
to make sure, a few extra pieces of equipment may be saved back just in case
bad luck. Actions of this type also reflect a lack of confidence in the postal
promise to guarantee same day delivery from regional inventories. This prob
never be cured by improvements in the computer system, but only by steady per
by the regional inventory managers over the years and the news of this filteri
the informal organization.

Maintenance routine
A. Decision Maker: regional-maintenance manager.
B. Decision Required: yes or no decisions as to whether goals are bein
local post offices in manhour usage and costs.
C. Model: control limit model based on two and one-half standard deviati
the ideal.
D. Data and Information Required: ideal target for manhours and costs, value of
two and one-half standard deviations.
E. Accuracy Required: plus or minus five percent.

Evaluation
Data were collected by manual audit on Mark II Facer Cancelers, Multi-position
Letter Sorter Machines, and Air Handling Units. All functional elements of the systems
met the standards. Table 2 displays data from the manual audit.

46 INTERFACES February 1979

This content downloaded from

182.255.0.242 on Thu, 17 Sep 2020 07:03:23 UTC
All use subject to https://about.jstor.org/terms
 Table 2
Accuracy computer time output vs. manual audit

Main Post Office (Number)

I II III IV

Preventive +2.3% +0.8% -4.4% -4.0% -0.09%

Maint. Hrs.
Corrective -3.5% +4.4% -6.2% +4.7% +0.9%
Maint. Hrs.
Total Hrs. +1.2% +3.1% -1.7% +2.8% -2.2%
Materials +3.1% -2.9% +7.6% -0.4% +1.5%
Costs
Total Cost +1.4% -2.5% +1.9% -4.2% +0.07%

Note that the model here is a control limit model and tha
or minus five percent. The latter figure was obtained by an
limit model would function if the data had an error. It was dete

excess of five percent would cause significant decision erro

model was applied to the data. Office III had two sets of repo
exceeded the five percent limit.
The Postal Service uses QC charts to control performance
out the mean labor costs reported, maintenance downtim
corresponding standard deviations. If a post office falls out
limits, then an investigation is made. The accuracy required
error that would show an office operating at 1.95 or 2.05 sta
mean to be out of control or in control, as the case may be.
levels were arbitrary, as no rational basis similar to the analy
be discerned.)
As offices "out of control" are brought into control, the standard deviation of the
data reported decreases, resulting in "tighter" control limits. The cost implications of
this management process have not been audited by the USPS.

Summary
The methodology used by the United States Postal Service to assess its Maintenance
Management Information System has been presented. The method drew on design
methodology as its basis. Problems in establishing accuracy and timeliness standards are
presented and methods used in computing these have been demonstrated.
The study showed, in general, that NMICS was operating as planned. There were,
however, significant differences between post offices in accuracy and a major flaw in the
system data collection procedure was exposed.

INTERFACES February 1979 47

This content downloaded from

182.255.0.242 on Thu, 17 Sep 2020 07:03:23 UTC
All use subject to https://about.jstor.org/terms
 The study documented that implementation of NMICS resulted in a 2.6 million
dollar present value savings for the USPS. These savings were documented by an
executive summary prepared by the Assistant Postmaster General of the USPS, as shown
in Table 3. In addition, 19 million dollars in inventory savings were identified during the
first year's use, as stated in the letter shown at the end of this article.

Table 3
Executive summary: NMICS savings and cost analysis a
I NMICS SAVINGS
(Manual System)
A. Reduced Workload from MS-10; cost savings FY 76 $2,515,070
B. Cost Avoidances 526,234
C. Interest on the Reduction of Parts Inventory 1,185,789
D. Unquantifiable Benefits
TOTAL SAVINGS FY 76 $4,227,093
II NMICS COSTS
A. Headquarters Development $186,000
B. OMM Staff 189,200
C. Schooling for Clerks 248,600
D. MISD Costs 190,498
E. WASC 288,400
F. MISD Charge-Back 25
G. Local Office Catalog Str
TOTAL COSTS FY 74, FY 75, FY 76 $1,608,591
III NET SAVINGS $2,618,502

a Prepared for J. V. Jellison, USPS, by the Office of Maintenance Management.

References
[1] Eilon, Samuel, "Mathematical Modeling for Management," Interfaces, Vol. 5, No, 2, February 1975.
[2] Glans, Thomas B., Grad, Burton, Holstein, David, Meyers, William E. and Schmidt, Richard Nf,
Management Systems, New York, Holt, Rinehart, and Winston, Inc., 1968.
[3] Hoffer, Jeffrey A., "Information Requirements Analysis: An Experiential Review," MIS Interactions,
TIMS, Vol. 3, No. 1, January 1977.
[4] Kilmann, R.H. and Mitroff, I.I., "Qualitative vs. Quantitative Analysis for Management Science:
Different Forms for Different Psychological Types," Interfaces, Vol. 6, No. 2, February 1976.
[5] Mason, Richard O. and Mitroff, Ian I., "A Program for Research on Management Information
Systems," Management Science, Vol. 19, No. 5, January 1973.
[6] Murdick, Robert G. and Ross, Joel E., Information Systems for Modern Management, Engle wood Cliffs,
New Jersey, Prentice-Hall, Inc., 1971.
[7] Nadler, Gerald, "A Strategy of Designing Management Systems," Journal of Systems Management,
December 1975.
[8] Smith, William A. and Wechsler, Ben L., "Planning Guide for Information System Evaluation
Studies," Publication #4 in the monograph series, Computer and Information Systems Division, AIIE,
25 Technology Park/Atlanta, Norcross, Georgia, 30071.
[9] Wildermuth, Blair L., "Accuracy of Information Gathered for the U.S. Postal Service National
Maintenance Information and Control System," master thesis, University of Oklahoma, College or
Engineering, Norman, Oklahoma, May 1976.

48 INTERFACES February 1979

This content downloaded from

182.255.0.242 on Thu, 17 Sep 2020 07:03:23 UTC
All use subject to https://about.jstor.org/terms
 OPERATIONS GROUP
Washington, OC 20260

July 26, 1977

B. h. Foote, Ph.D.
Associate Professor
The university of Oklahoma
202 West Boyd, Suite 124
Norman, OK 73069
Dear Dr. Foote:

This is in reference to your June 20 letter to Mr. Wildermuth

and my previous April 20 letter attesting to the importance
of the analysis which he made of the National Maintenance
Information and Control System (NMICS).
In my previous letter I attested to the importance of the
need for an accurate management information system in the
maintenance area. The study completed by Mr. Wildermuth and
other Postal Service employees was an effective study. It
shows that information collected and processed is accurate
and timely.
Since NMICS was instituted by the Postal Service, we have
affected various monetary savings. Attached you will find a
recent Executive Summary prepared in response to questions
by our top management and our Postal Service Inspection
Service. It indicates a substantive monetary savings for
FY 1976. Mr. Wildermuth's study gives credence that these
savings are factual. You will also note the cost of insti
tuting and maintaining the information system is for FY 1974,
1975 and 1976. Additional savings which we do not show on
the Executive Summary were that nineteen million dollars of
unneeded machine and repair parts inventory were identified
during the first year's full use of this system. We do not
use that cost figure as a savings since the parts had already
been purchased, however, there would be some savings based on
the fact that additional parts would not have to be purchased
for some period of time. Identification of excess inventory
allows for future savings because of lower inventory
requirements.

This is approval for you to use the information contained

in Mr. Wildermuth's Master's Thesis and additional abstract
information which you have prepared for submittal to
INTERFACES for publication. We reserve the right to review
the final draft before publication in order to assure that
privileged information is not divulged.
Sincerely,

fij. V. Jellison
/Assistant Postmaster General
'jfail Processing Department
Attachment

INTERFACES February 1979 49

This content downloaded from

182.255.0.242 on Thu, 17 Sep 2020 07:03:23 UTC
All use subject to https://about.jstor.org/terms