COMMISSION ON AUDIT MEMORANDUM NO.

84-316-B December 21, 1984

TO : All COA Managers, Heads of Auditing Units Involved in the Research

and Others Concerned.

SUBJECT : Validation of COA's Statistical Sampling Methodology.

A research project is hereby authorized to validate the applicability and

acceptability of a sampling methodology, entitled COA-Sampling-Card in the Post
Audit of Transactions by Auditing Units in selected National and Local Agencies and
government-owned and/or Controlled Corporations which is described in the
attached article. Mr. Peter Maertens of the office of the Auditor General of Canada
and a doctoral candidate f the University of the Philippines, College of Public
administration shall be "researcher" for this project.

In view thereof, Mr. Omar Costibolo and Ms. Cecilia umatay are designated
as team members effective January 2, 1985 until April 30, 1985. Their terms of
reference are attached.

The project team members will coordinate their efforts with he auditing units
in National and Local Agencies and government-owned and/or Controlled
Corporations which have been elected by managers in coordination with the
researcher for the purpose of validating the COA-Sampling-Card Methodology. The
Auditing units shall prepare and submit to the project team the necessary working
papers and reports to be defined by the researcher. The project team will also
coordinate with the manager responsible for the project examining the withdrawal of
Resident auditing units.

Transactions forming part of the backlog may be included in he research but

only to the extent that one month of such transactions may be included for each
month of current transactions.

Deviations from COA-Sampling-Card Methodology described in the

researcher may authorize the attached article. This applies in particular to the
specific parameters and sample Selections methods so as to make them feasible for
the audit Transactions.

SAAC will provide the office space for the project team and Allocate five
thousand (P5, 000.00) to cover transportation costs of the project team to the audit
units. Public utility vehicles shall be utilized.

For compliance.

(SGD.) FRANCISCO S. TANTUICO, JR., Chairman

 A PROPOSED STATISTICAL SAMPLING SCHEME
FOR THE COMMISSION ON AUDIT

INTRODUCTION

During the last decade, there has been an increasing awareness of

statistical sampling as efficient audit tool. The increasing volume of government
transactions, the desire to use available audit resources for task other than
vouching of documents, the introduction of computers and the recent
authoritative statement on audit sampling by the American Institute of Certified
Public Accountants have all contributed to that awareness. Nevertheless, there is a
reluctant among auditors specially in developing countries to use appropriate
techniques that can reduce the workload associated with examining large
volumes of government transactions and their supporting documents. This is
because statistical sampling plans and their techniques remain difficult to
understand and auditors lack the appropriate training therein. The Commission on
Audit (COA) in the Philippines recognized both these constraints as well as the
necessity to reduce audit time and efforts spent on repetitive tasks. Consequently,
they initiated a research project, which to date, has resulted in a policy which
contained a methodology and prescribed procedures to implement statistical
sampling.

The COA-Sampling-Card discussed in this article, was developed in

coordination with and as part of COA's efforts. It is a practical tool to be used in post-
audit examinations that is easy to understand and virtually eliminates statistical
calculations for the auditor in the field. However, some knowledge of statistics
is required to prepare the COA-Sampling-Card in the first place.

USING THE COA-SAMPLING CARD

Post-audit examinations can be of two types. The first type is the

compliance examination which determines whether the amounts paid out are
accurate, valid and properly recorded. Audit conclusions of the first type relate to
the propriety of a transaction and its supporting documents. Conclusions of the
second type relate to the monetary accuracy and the recording of a transaction.
Although compliance and substantive audits result in different conclusions. They are
normally done at the same time and will be treated together in this article.

First, let us follow the steps that a field auditor would have to take after
being given a COA-Sampling-Card such as the one in Exhibit 1-A. Later, we will
see how such a card can be prepared. But one it has been all the auditor has to do
is:

a. Choose the sample size.

b. Select a sample.
c. Test the sample.
d. Evaluate the results.

To illustrate, consider the hypothetical example of post-auditing the

vouchers supporting 3,000 insurance claims and related payments to the holders
of government-backed insurance policies.
Step 1: Choosing the sample size

First, the auditor would list the population of vouchers together with their
monetary amounts and total them, or obtain such a listing from the auditee. Next,
the auditor eliminates voucher amounts that are so high in value that they are
too important to leave out of the audit. For COA, large amounts are defined as a
percentage of the total value of the vouchers. For a population of 3,000 vouchers, the
COA-Sampling-Card instructions recommend 1%. Suppose the total monetary
value of the 3,000 vouchers is 4 million pesos. The auditor would be required to
audit all vouchers over (4,000,000 pesos x 1% =) 40,000 pesos. Let's suppose
there are 5 such vouchers in the population totalling 840,000 pesos.

There may also be vouchers that the auditor wants to check knowing from
past experience they are particularly prone to error. Let's suppose there are 45
smaller payments to dubious construction companies totalling 100,000 pesos.
These might also be eliminated from the population and audited individually.

The auditor is now left with an adjusted population of 2,950 vouchers, places
us somewhere down the third column on the front of the COA-Sampling-Card.
Next, the auditor would check the characteristics of the vouchers, and in this case,
let us assume they are all judged to be current transactions with good controls.
This puts us somewhere along the second row of COA-Sampling-Card. At the
intersection of the row along the column, the auditor can see that the recommended
sample size is 200.

Step 2: Selecting the sample

Before actually choosing the 200 samples, the auditor needs to determine
the sample interval. This defined as the monetary value of the adjusted population
divided by the sample size. In this case, the interval is:

sample interval = P4,000,000 - (P840,000 + P100.00) = P15,300

200

To start the selection process a number between 0 and the sampling interval
is selected from a random number table. If this number is 9570 then, the sample is
selected as follows:

a. Enter the random number (9570) in the adding machine as a negative

amount.

b. Add to the negative amount the value of the first and each succeeding
payment in the order they were provided, until the sub-total reaches
zero or a positive amount.

c. Mark off the payment where the sub-total becomes zero or positive. This
is selected payment.

d. Deduct from the sub-total the sample interval of 15,300; his will cause
the sub-total again to turn negative in most cases. If the figures remains
 positive, deduct the sample interval of 15,300 again. Such payments
and the errors observed during the audit should be recorded each time
the payment was selected.

e. Repeat steps (b) to (d) until the auditor has added through the whole
population. This should result in a sample of 200 vouchers.

A computer program can be written to select the sample if the vouchers

subject to audit are stored in an electronic media. This could be expected to take
one half to two months of a programmer's time depending on the computer
configuration.

Step 3: Testing the sample

The auditor is now ready to test 200 vouchers. For the compliance test,
this might involve reviewing whether or not:

a. Claims had been satisfactorily investigated to establish that the

claimant had incurred a loss.

b. There had been approval of the payment.

c. The payment was properly recorded.

The substantive test could review whether or not:

a. The claim should have been paid.

b. The claim was properly calculated.

In our hypothetical example; suppose the test showed that all transactions
were properly processed except for the following: a) compliance errors - for three
payments there was no evidence that the claim had been investigated, for two
payments no approval was evident and one payment was recorded incorrectly; b)
substantive errors - payments on claims supported by voucher numbers 0057 and
6302 amounting to 3,000 and 500 pesos, respectively should not have been
made because of some recently approved amendments to the insurance policies.
Also, payments on the following claims were incorrectly calculated: Voucher Nos.
1257, 3708, 7401, 9321, 9871 had book values of 2000, 300, 12327, 826 and 625.

Step 4: Evaluating the results

The COA-Sampling-Card assisted the auditor in his evaluation of results. In

terms of the compliance review, the auditor noted that the number of compliance
errors that may be detected in a sample during the audit ( eight from row 2, column
3 of the COA-Sampling-Card). For substantive errors the following audit
worksheet was prepared:
 MONETARY EVALUATION WORKSHEET

A B C D=B-C E=D/B
Voucher Book Audited Absolute Percentage
Number Value Value Error Error

0057 3000 0 3000 1.00

1257 2000 1200 800 .40
3708 325 300 25 .08
6302 500 0 500 1.00
7401 7248 12327 5079 .70
9321 5826 826 5000 .86
9871 627 625 2 .01

Cumulative error * 4.05

Maximum allowable error (from Sampling Card) 7.18

* The cumulative error includes all errors regardless of whether they are over
or under statements. This is done on the assumption that the
government auditor is concerned with the absolute rather than the net
amount of errors.

The 4.05 cumulative error noted in the audit was below the maximum
number of substantive errors that are tolerable in accordance to the COA-
Sampling-Card. On the basis of the 200 payments verified and relating to the
population of 2950 out of 3000 items, the auditor concluded that the error rate in
the population was acceptable to him. The conclusion on the other 50 items was
based on full information and not on the statistical sample. If the result of either the
compliance or the substantive test had shown the cumulative error above the
threshold stated on the COA-Sampling-Card, the auditor would have had to verify
the remaining 2750 vouchers and base his conclusion on full information.

PREPARING THE COA-SAMPLING CARD

The COA-Sampling-Card methodology is based on an approach called

probability-to-size sampling. It uses theories of attribute sampling to express
conclusions in terms of rates of occurrences or monetary amounts. The cards
themselves are references that incorporate statistical concepts of sampling risk,
tolerable error and sample size. The first two concepts need not be familiar to the
auditor using the COA-Sampling-Card, but they are explained now for those who
have the interest and to enable the reader with mathematical background to
construct COA-Sampling-Cards according to his or her own specifications.

Sampling risk is the possibility that the conclusion derived from a test differs
from the conclusion that would have been obtained by examining the whole
population with equal diligence. Sampling can lead to incorrect conclusions of two
types. On one hand, the auditor may conclude from the sample that the error rate
of the population is below the tolerable error rate when, in reality, it is not. On the
other hand, the auditor may consider the inconclusive although the error rate in the
whole population is actually below the tolerable error rate. The auditor is willing
to accept these risk in exchanged for the economy of sampling, provide that the
extent of the risk can be controlled and quantified.

The choice of an acceptable sampling risk is a matter of judgment for the

designer of the COA-Sampling-Card. Twenty, ten, five and one percent are
considered acceptable risks for post audits. The factors that determine the level of
sampling risk include the sensitivity of the audit, the interval accounting control
that exists in the entity subject to audit, and the cost or time it takes to perform the
test. For the COA, 1 percent is recommended for post-audit of old transactions.
This level reflects the fact that the audit results for these transactions are the only
source of information and are, therefore, very sensitive. Ten percent is
recommended for all current transactions except those financial transactions over
which internal controls are deemed inadequate.

Tolerable error is the threshold value for undetected errors that the audit
institution is willing to accept in the population. For a compliance test, this
threshold would be expressed as the maximum percentage of transactions that
have not been completed in conformity with relevant laws, rules and procedures.
For a substantive audit test, the tolerable error would be maximum amount that is
accurate, valid or properly recorded. It can be stated as an absolute amount or a
percentage of the monetary value of all transactions in the population subject to
audit.

The level of tolerable error is matter of judgment and should be viewed

from the perspective of the reader of the resulting audit report. For the
compliance testing, the audit literature suggests a tolerable error of five percent,
counting only significant errors such as unauthorized or undocumented
transactions. In some developing countries there accounting controls are not
well established, the tolerable error rate acceptable to the audit institution could
be as high as 15 percent but not higher. For substantive tests, five percent is also
considered reasonable. Some western countries, however, use a figure as low as
half of one percent. The example in this article uses seven percent for the
compliance tests and six percent for substantive tests.

The COA-Sampling-Card shown in Exhibit 1-A shows the maximum

allowable error that may be observed by the auditor in a particular test rather
than rather than the tolerable error. These were determined by using the formula
N = R/E where N = sample size, E = tolerable error and R = reliability factor,
which is a number using the Poison Distribution. A table for values of R is
contained in Exhibit 3. It shows that R defends on the sampling risk. Once R has
been calculated, the maximum number of errors that may be observed by the auditor
can be read from the first column of Exhibit 3.

As an example, the calculations in Exhibit 2 show the recomputation for

maximum compliance and substantive errors of the third column of the COA-
Sampling-Card in Exhibit 1-A.
 EXHIBIT 2

COMPUTATION OF MAXIMUM NUMBER OF ERRORS SHOWN IN

COLUMN 3 OF THE COA-SAMPLING-CARD

Max. error

Audit rate from

Characteristic Risk N = R/E R Exhibit 3

Compliance:

Old transactions 1% 250=R/.07 17.5 8

Current transactions .10% 200=R/.07 14 8

(good controls)

Current transactions 10% 100=R/.07 7 3

(low error rate)

Current transactions 5% 200=R/.07 14 7

(inadequate controls)

Substantive:

Old Transactions 1% 250=R/.06 15 15-14.58

6 + --------------
16.00-14.58
= 6.29

Current transactions 10% 200=R/.06 12 12-11.78

(good controls) 7+ ---------------
13.00-11.78
= 7.18

Current transactions 10% 100=R/.06 6 6-5.33

2+ ---------------
6.69-5.33
= 2.49

Current transactions 5% 200=R/.06 12 12-11.85

6+ ---------------
13.15-11.85
= 6.12

In compliance tests, there are no partial errors. Therefore, the maximum

error used in an integer equal to or just slightly less than the R calculated in the
formula. For substantive tests, monetary errors are expressed in percentage terms;
therefore, the maximum allowable error is calculated to two decimal points, using
linear extrapolation.

Simple size refers to the number of units selected for testing. It is

dependent on the tolerable error rate, the sampling risk, the size of the population
subject to audit and the number of errors expected in the population. Most
sampling plans go into considerable detail to stipulate an optimal size. While this
improves audit efficiency, it makes the sampling plan more complex. The proposed
sampling plan determines the sample size from the sampling efficiency curve, which
plots the maximum error rate allowable in a sample of a particular size before the
audit results become inconclusive. Such a curve can be plotted by using the same
formula N = R/E and calculating the maximum error allowed for sample sizes from
50 upward using an interval of 50. The sample efficiency curve shown in Exhibit 4,
is for the sampling risk of ten percent and a tolerable rate of seven percent, the
calculations for the maximum error corresponding to sample sizes of 100 and 200
have already been calculated as 3 and 8, i.e., and 4% respectively in Exhibit 2.

EXHIBIT 4

Sampling Efficiency Curve

Audit Risk 10%
Tolerable Error 7%

% Error
Rates
:
:
8 :
:-------------------------------------------------------------- Tolerable Error (7%)
6 :
:
4 : Maximum error rate allowable
: in the sample before audit
2 : results become inconclusive.
:
0 :-------+--------+--------+--------+--------+--------+--------+ Sample
Sizes
100 200 300 400 500 600 700

This curve shows that for small samples (under 100) a few errors would
make a test inconclusive. For larger samples (over 300) this would not be the case.
The maximum percentage of errors that would make the test inconclusive, hardly
changes once the curve flattens out. Based on this curve, the following sampling
sizes are recommended for sampling risk of ten percent and tolerable error of
seven percent:

Transactions
in Populations Point on the curve Sample Sizes

200 - 999 and of step slope 150

1000 - 4999 middle of average slope 200
5000 plus start of flat slope 300
 It is also recommended that the whole population be tested if it contains
less than 200 vouchers, since the additional effort to employ statistical sampling is
not worth the relatively minor savings that sampling would bring.

When a government entity has a history of virtually no errors in the

population, it would be appropriate to cut the sample size in half. This is included
on the Sampling Card. While the resulting smaller sampling size usually is on the
steep part of the curve it is, nevertheless, justified because the expected error rate
remains far below the maximum error rate that may be discovered in the sample
before it becomes inconclusive.

CONCLUSION

This paper has presented a COA-Sampling-Card that can be used by

auditors who are not familiar with statistics. It can be used with very little instruction.
All the auditor needs to know is how to read the card, how to select the samples,
and how to evaluate the sample results. Other topics covered include a brief
explanation of the underlying principles and mechanics for constructing the
COA-Sampling-Card.

Specifically, it is hoped the paper will enable institutions like the Commission
on Audit to significantly reduce their post-audit work without sacrificing quality.
The use of the statistical sampling often improves the quality of audit because the
work is less repetitive, larger values are emphasized, and work papers tend to be
better prepared.

As with any new methodology, there may be problem in adoption the

concept to fit into the practical realities of the financial and operating environment
in government. But COA-Sampling-Cards can be developed and adapted to a wide
variety of situations. For example, COA is testing the COA-Sampling-Card in Exhibit
I-A in organizations like banks and insurance companies. A Sampling Card with
different parameters is being tested for local government transactions. Eventually, it
is envisaged that the COA will have at least one COA-Sampling-Card for each
government entity. Perhaps, audit institutions of other countries will also adopt the
COA-Sampling-Card idea to speed up and simplify their financial audit work.
 EXHIBIT 1-A

COA-SAMPLING-CARD

RECOMMENDED FOR THE COMMISSION ON AUDIT

TO VERIFY FINANCIAL TRANSACTIONS OF
GOVERNMENT FINANCIAL INSTITUTIONS

Number of Items in Population

Characteristic 0-199 200-999 1000-4999 5000+
Old Transactions All
200 5 250 8 300 10
4.26 6.29 8.42
Current Transactions All
Good Controls 150 6 200 8 300 14
4.78 7.18 12.18
Current Transactions 75
Good Controls Low 2 75 2 100 3 150 6
Error Rate 1.42 1.42 2.49 4.78
Current Transactions All
Inadequate Control 150 5 200 7 300 13
3.89 6.12 10.83

INSTRUCTIONS FOR SAMPLING CARD

Parameters:

Audit Risk - Sample Size

Old Transactions 1% Maximum number of

Current Transactions 200 compliance errors
- Good Controls 10% 8 that may be de-
Current Transactions 7.18 tected in sample
- Inadequate Controls 5% during audit.

Tolerable Error Maximum number of substantive

Compliance Test 7% errors that may be detected
Substantive Test 6% in sample during audit.

High value -

Populations 200 - 999 2%

1000 -4999 1%
 COA SAMPLING CARD EXHIBIT 1-B

RECOMMENDED FOR THE COMMISSION ON AUDIT

TO VERIFY TRANSACTIONS OF
NATIONAL GOVERNMENT AGENCIES AND
GOVERNMENT CORPORATIONS

Number of Items in Population

Characteristic 0-199 200-999 1000-4999 5000+
Old Transactions All
200 12 250 17 300 22
5.61 8.07 10.63
Current Transactions All
150 12 200 17 300 21
5.97 8.83 14.74
Current Transactions
Low Error Rate 75 4 75 4 100 7 300 12
1.94 1.94 3.23 5.97

INSTRUCTIONS FOR SAMPLING CARD

Parameters: Sample Size

Audit Risk -
150 Maximum number of
Old Transactions 1% 12 compliance error
Current Transactions 10% 5.97 that may be de-
tected in sample
Tolerable error - during audit.

Compliance Test 12%

Substantive Test 7%
Maximum number of substantive
High Value - errors that may be detected
in sample during audit.
Population 200 - 999 2%
1000 - 4999 1%
5000+ 1/2%
 COA-SAMPLING-CARD EXHIBIT 1-C

RECOMMENDED FOR THE COMMISSION ON AUDIT

TO VERIFY TRANSACTIONS OF
LOCAL GOVERNMENT

Number of Items in Population

Characteristic 0-199 200-999 1000-4999 5000+
Old Transactions All
200 17 250 23 300 29
7.00 9.89 12.89
Current Transactions All
150 16 200 22 300 35
7.18 10.49 17.33
Current Transactions
Low Error Rate 75 6 75 6 100 9 300 16
2.49 2.49 4.00 7.18

INSTRUCTIONS FOR COA-SAMPLING-CARD

Parameters:

Audit Risk - Sampling Card

Old Transactions 1%
Current Transactions 10% 150 Maximum number
of 16
7.18 compliance errors
Tolerable Error - that may be de-
tected in sample
Compliance Test 15% during audit.
Substantive Test 8%

High Value - Maximum number of substantive

error that may be detected
Population 200 - 999 2% in sample during audit.
1000 - 4999 1%
5000+ 1/2%
 Exhibit 3
RELIABILITY FACTORS FOR AUDIT ERRORS
Audit Risk
Number of
Sample Errors 20% 10% 5% 1%
0 1.61 2.31 3.00 4.61
1 3.00 3.89 4.75 6.64
2 4.28 5.33 6.30 8.41
3 5.52 6.69 7.76 10.05
4 6.73 8.00 9..16 11.61
5 7.91 9.28 10.52 13.11
6 9.08 10.54 11.85 14.50
7 10.24 11.78 13.15 16.00
8 11.38 13.00 14.44 17.41
9 12.52 14.21 15.71 18.79
10 13.66 15.41 16.97 20.15
11 1.78 16.60 18.21 21.49
12 15.90 17.79 19.45 22.83
13 17.02 18.96 20.67 24.14
14 18.13 20.13 21.89 25.45
15 19.24 21.30 23.10 26.75
16 20.34 22.46 24.31 28.04
17 21.44 23.61 25.50 29.31
18 22.54 24.76 26.7 30.59
19 23.64 25.91 27.88 31.85
20 24.73 27.05 29.07 33.11
21 25.82 28.19 30.25 34.36
22 26.91 29.32 31.42 35.61
23 28.00 30.46 32.59 36.85
24 29.09 31.59 33.76 38.08
25 30.17 32.72 34.92 39.31
26 31.25 33.84 36.08 40.54
27 32.33 34.96 37.24 41.76
28 33.41 36.08 38.39 42.98
29 34.49 37.20 39.55 44.19
30 35.56 38.32 40.70 45.41
35 40.93 43.88 46.41 51.41
40 46.27 49.39 52.07 57.35
45 51.59 54.89 57.70 63.24
50 56.90 60.34 63.29 69.07
55 62.18 65.79 68.86 74.87
60 67.46 71.20 71.40 80.60
65 72.73 76.61 79.91 86.36
70 77.98 81.99 85.41 92.09
75 83.23 87.37 90.89 97.74
80 88.76 92.73 96.36 103.39
85 93.69 98.09 101.8 109.04
90 98.92 103.42 107.24 114.69
95 104.14 108.76 112.67 120.26
100 109.35 114.07 118.07 125.84
 TERMS OF REFERENCE

Crucial to the success of the validation and the monitoring thereof, is the
availability of an expert with know-how of Statistics for Auditors and a statistical
expert with know-how of Auditing who can provide guidance and revolve
problems encountered by the auditing units as well as serve as the mechanism
to transfer the technology from the researcher to the COA staff. These two expert,
will work in close collaboration with the administrative expert, the project researcher
and the auditing teams involved. These individuals will liaise with COA and other
government officials concerned. Both shall render service from January 1, 1985 to
May 1, 1985.

The principal duties shall include but not be limited to the following:

1. On a continuing basis, monitor and provide technical advice to the

auditing teams involved in the validation to ensure that they are
strictly and correctly implementing instructions, preparing the
relevant reports and all working papers and are highlighting
problems and other information pertinent to the successful
validation.

2. On a continuing basis, apprise the researcher of the progress of the

field work and problems that need to be resolved. Deliver to the
researcher the necessary reports, working papers and other materials
and make appropriate summarize of them.

3. Prepare for the researcher and the COA Chairman the necessary
correspondence and assist in the writing of interim and final reports as
well as the revision, if any, of the COA policy paper on statistical
sampling methodology.