057 - Testing the operating effectiveness of controls

[57.1995] Refer to the section beginning at KAM 9.1455 in the topic, "Audit evidence" for requirements and guidance
regarding testing the completeness and accuracy of the information produced by the entity.

[57.2000] Determining the sample size

[57.2005] We shall determine a sample size sufficient to reduce sampling risk to an acceptably low level.

R
[ISA 530.7]

[57.2010] The level of sampling risk that we are willing to accept affects the sample size required. The lower the risk we are
willing to accept, the greater the sample size will need to be. [ISA 530.A10]

[57.2015] The sample size is determined through the exercise of professional judgment. Various factors typically may
influence determination of sample size, as follows:[Source: ISA 530.A11, Source: ISA 530 Appendix 2]  

• the extent to which our risk assessment takes into account relevant controls [Source: ISA 530 Appendix
2.1]

• the tolerable rate of deviation of the population to be tested [Source: ISA 530 Appendix 2.2] 
• the expected rate of deviation of the population to be tested [Source: ISA 530 Appendix 2.3] 
• the desired level of assurance (complement of risk of overreliance) that the tolerable rate of
deviation is not exceeded by the actual rate of deviation in the population; we may decide the
desired level of assurance based on the extent to which our risk assessment takes into account
relevant controls [Source: ISA 530 Appendix 2.4] 
• the number of sampling units in the population if the population is very small. [Source: ISA 530 Appendix
2.5] 

[57.2020] The extent of testing and therefore the amount of audit evidence to be provided by the sample, in conjunction with
other tests of controls, needs to be sufficient to reduce sampling risk to an acceptably low level. We may consider the
need for more assurance (and therefore a larger sample size) from a test of a single control relating to an assertion than if
we are performing several tests of controls over the same assertion.

[57.2025] Sample size for testing manual controls when we expect no control deviations

[57.2030] We may consider the guidance in the table below related to the frequency of the performance of the control when
planning the extent of tests of the operating effectiveness of manual controls for which we do not expect to find control
deviations. The appropriate number of control occurrences to test is based on the following minimum sample sizes for the
frequency of the control activity dependent on whether the risk of failure of the control is assessed as lower or higher. The
guideline minimum sample sizes included in the table below may be appropriate even if the period under audit is less than
or more than a year (refer to Endnote 1).

© 2014 KPMG International Cooperative ("KPMG International"), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG
International. KPMG International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis­à­vis
third parties, nor does KPMG International have any such authority to obligate or bind any member firm. All rights reserved.

Page 33 / 49
 057 - Testing the operating effectiveness of controls

[57.2035] In some situations, similarly designed controls operate at the same time over many different components of a
significant account or disclosure. In these situations we use our judgment to select the extent of testing both in terms of
the frequency of testing (such as number of days, weeks or months to test) and the number of similar operations to test at
each point in time (such as number of locations or different accounts affected).

For example, when testing a process level control, an entity that has many bank accounts may prepare monthly
bank reconciliations in the same way for each account with the same control processes in place. We may
choose to test the monthly reconciliation control for two months during the period under audit. Rather than
testing the operation of the reconciliation over every bank account at these two points, we use judgment to
select a sample of the many bank accounts at each of the two months. We take account of the aggregate
number of control operations in the period and the guidance on sample sizes above in deciding the minimum
number of control operations to test in aggregate. We test the operating effectiveness of this sample.
[57.2036] In situations where we are testing multiple controls that address a "what could go wrong", we may reduce the
confidence level required for our testing of each control. We may consider the reduction in our required confidence on the
effectiveness of each of these controls when determining the nature, timing and extent of test work. As a result, when we
choose to test one of these controls by reperforming the control, we may reduce our minimum sample sizes when we
assess the risk of failure of the control as lower. For manual recurring controls with a lower risk of failure, our minimum
sample sizes in this situation are 15 instead of 25, for a daily control, the minimum is 10 instead of 15 and for weekly
controls the minimum is 2 instead of 5. The sample size cannot be reduced when the control selected for testing
addresses multiple "what could go wrong" within a process or relates to multiple assertions for one or more significant
accounts, or is the only control identified for a particular "what could go wrong".

[57.2040] Testing controls which operate at more than one homogenous location

[57.2045] Homogeneous locations are locations which operate using common process activities, systems and process
level controls, and are subject to the same entity level controls and/or higher level controls.

For example, branches of the same retailer, bank or restaurant chains are often homogenous locations.
[57.2050] When testing controls which operate at more than one homogenous location, we select a sufficient number of
locations for testing to determine that the operations under consideration are, in fact, homogeneous in nature, and that the
controls over the relevant assertions for the significant accounts and disclosures operating at those locations are effective.

[57.2055] The determination of the number of locations to test and selecting specific locations is a matter of professional
judgment.

[57.2060] In a multi-location environment with homogeneous locations, the results of audit procedures performed at one

© 2014 KPMG International Cooperative ("KPMG International"), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG
International. KPMG International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis­à­vis
third parties, nor does KPMG International have any such authority to obligate or bind any member firm. All rights reserved.

Page 34 / 49
 057 - Testing the operating effectiveness of controls

location are expected to be indicative of results obtained if other locations subject to the same entity level controls and/or
higher level controls were selected and similar tests performed. We consider the effectiveness of the entity level controls
and/or higher level controls, including monitoring of controls, intended to determine the consistency of operations at such
locations when evaluating the extent of homogeneity in operations.

[57.2065] In such circumstances, we normally expect management to have implemented effective monitoring controls, and
our audit approach may include modifying the nature, timing and extent of our audit procedures based on the effective
design and implementation of such monitoring controls.

[57.2070] After determining which locations we will test, we may either:

• set our sample size in accordance with the guidance in the table at KAM 57.2030 in the section
"Sample size for testing manual controls when we expect no control deviations" above and then
allocate this sample size to the locations to be visited on an appropriate basis (i.e. pro-rated on
size or relevant account balance) if we do not plan to tolerate any deviations; or
• select and test a minimum of 10 operations of the recurring manual controls at each location
selected if we choose to tolerate some deviations. We use the table at KAM 57.2170 in the section
"Sample size for testing manual controls when we expect some control deviations" below to
determine the aggregate number of control deviations acceptable for the corresponding total
sample size.
[57.2075] For example, if we test a homogeneous control at 15 locations and choose to tolerate some deviations based on
the guidance in the section in this KAM topic beginning at KAM 57.2155, the minimum sample size is 150 items and,
based on the table at KAM 57.2170, the maximum number of expected control deviations in the sample is 9 for a control
which we assess as having a lower risk of failure.

[57.2080] Refer to the KAM topic, "Multi-location audits - special considerations" for additional information regarding multi-location
audits.

[57.2085] Sampling for general IT controls

[57.2090] We generally obtain evidence that relevant general IT controls are operating effectively throughout the period under
audit because of their pervasive nature and impact on the controls at the process.

[57.2095] When we test the operating effectiveness of general IT controls, the sample size is a matter of professional
judgment.

[57.2100] General IT controls may be manual, manual with an automated component, or automated. Where a general IT
control is manual or manual with an automated component, the guidance above related to the extent of testing of manual
controls may be used to determine the extent of testing of general IT controls. Where a general IT control is automated,
we use our professional judgment, combined with the guidance in this section and the guidance on extent of testing in the
section in this KAM topic beginning at KAM 57.1275.

[57.2105] Audit procedures are designed to determine whether the general IT controls on which we intend to rely to provide
audit evidence and to determine the nature, timing and extent of substantive procedures operated effectively throughout the
period under audit.

[57.2110] An entity may have one change management process to manage changes made to all of its applications. If the
control to authorize, test and approve program changes occurs 12 times during the year and includes multiple changes on
each occurrence, we may use our professional judgment, and choose to test two occurrences of the control during the
period under audit. Rather than test the operation of the control to authorize, test and approve every change at these two
occurrences, we may inspect the documentation to identify changes relevant to financial reporting and use professional
judgment to select a sample of the changes at each of the two occurrences.

[57.2115] Sampling for automated application controls

[57.2116] Where we have tested the operating effectiveness of the relevant general IT controls that support the consistent

© 2014 KPMG International Cooperative ("KPMG International"), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG
International. KPMG International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis­à­vis
third parties, nor does KPMG International have any such authority to obligate or bind any member firm. All rights reserved.

Page 35 / 49
 057 - Testing the operating effectiveness of controls

operation of the application control and concluded that these general IT controls are operating effectively throughout the
period under audit, we may test the application control at one point during the period.

[57.2120] In the situation where we have not tested the operating effectiveness of the relevant general IT controls or where
we have concluded that the relevant general IT controls are not operating effectively throughout the period under audit, we
may perform additional audit procedures to enable us to continue to rely on the automated application control including
testing the application control at more than one point during the period. The number of tests of the application control
depends amongst others on the nature and frequency of the control, the frequency of changes to the application, the
assessment of inherent risk and especially fraud risk.

[57.2121] Refer to the KAM topic, "IT environments and IT controls" for additional information when planning the tests of controls
relating to application controls and general IT controls as part of the financial statement audit.

[57.2125] The number of control deviations which can be accepted

[57.2130] We normally do not expect to find control deviations when we select controls to test. However, there may be
circumstances when testing recurring manual controls where we are prepared to accept a small number of control
deviations but still consider it appropriate to rely on the operating effectiveness of the control to provide audit evidence to
enable us to modify the nature, timing or extent of our substantive procedures.

[57.2135] We may accept control deviations when we:

• test controls which operate at more than one homogeneous location when we choose to test a
specific number of items at each location visited rather than allocate the minimum controls sample
size over the locations, or
• perform a dual-purpose test, where our substantive sample size is larger than the indicated
minimum for the test of operating effectiveness of the control and we apply our controls testing to
the substantive sample.

[57.2140] We shall seek advice from a KPMG sampling specialist when we are planning to accept some

R control deviations in circumstances other than those in KAM 57.2135.

[57.2145] For recurring manual controls which are performed more than daily, we may select a larger sample size and
accept a small number of control deviations. We may not accept any control deviations when it is determined that the
deviation is representative of a systematic or intentional control deviation.

[57.2150] We may accept some control deviations when:

• transactions subject to the control are not individually large, complex or likely to include a material
misstatement
• the operation of the control does not require significant judgment, and
• we are not obtaining a large part of our audit evidence with respect to the relevant assertion from
our test of the operating effectiveness of this control.
[57.2155] Sample size for testing manual controls when we expect some control deviations

[57.2160] If we determine in advance that deviations may exist, we normally would not proceed to test the control. However,
there may be instances (as indicated in KAM 57.2135) where we test a sample that is larger than the minimum control
sample size. In these situations, we may determine that it is appropriate to tolerate some deviations when concluding on
the effectiveness of the control for the purposes of our audit.

[57.2165] In order to avoid an inadvertent rejection of the sample in this circumstance, we may allow for a small number of
deviations.

© 2014 KPMG International Cooperative ("KPMG International"), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG
International. KPMG International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis­à­vis
third parties, nor does KPMG International have any such authority to obligate or bind any member firm. All rights reserved.

Page 36 / 49
 057 - Testing the operating effectiveness of controls

[57.2170] The following table sets out the number of control deviations which can be accepted for a given sample size, for
both a lower or higher risk of failure of the control.

© 2014 KPMG International Cooperative ("KPMG International"), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG
International. KPMG International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis­à­vis
third parties, nor does KPMG International have any such authority to obligate or bind any member firm. All rights reserved.

Page 37 / 49