Professional Documents
Culture Documents
ITAF-Companion-Performance-Guidelines-2208 wpg2208 Res Eng 1020
ITAF-Companion-Performance-Guidelines-2208 wpg2208 Res Eng 1020
Disclaimer
ISACA has designed and created the ITAF™ Companion Performance Guidelines 2208: Information Technology
Audit Sampling (the “Work”) primarily as an educational resource for professionals. ISACA makes no claim that
use of any part of the Work will assure a successful outcome. The Work should not be considered inclusive of all
proper information, procedures and tests or exclusive of other information, procedures and tests that are reasonably
directed to obtaining the same results. In determining the propriety of any specific information, procedure or test,
professionals should apply their own professional judgment to the specific circumstances presented by the
particular systems or information technology environment.
© 2020 ISACA. All rights reserved. No part of this publication may be used, copied, reproduced, modified,
distributed, displayed, stored in a retrieval system or transmitted in any form by any means (electronic, mechanical,
photocopying, recording or otherwise) without the prior written authorization of ISACA.
ISACA
1700 E. Golf Road, Suite 400
Schaumburg, IL 60173, USA
Phone: +1.847.660.5505
Fax: +1.847.253.1755
Contact us: https://support.isaca.org
Website: www.isaca.org
Twitter: http://twitter.com/ISACANews
LinkedIn: www.linkedin.com/company/isaca
Facebook: www.facebook.com/ISACAGlobal
Instagram: www.instagram.com/isacanews/
Acknowledgments
ISACA wishes to recognize:
Expert Reviewers
Glenn Kirke, CISA, Integrated Audit and Compliance, USA
Rafael Pérez Marín, CISA, Venezuela
Board of Directors
Tracey Dedrick, Chair, Former Chief Risk Officer, Hudson City Bancorp, USA
Rolf von Roessing, Vice-Chair, CISA, CISM, CGEIT, CDPSE, CISSP, FBCI, Partner, FORFA Consulting AG,
Switzerland
Gabriela Hernandez-Cardoso, Independent Board Member, Mexico
Pam Nigro, CISA, CRISC, CGEIT, CRMA, Vice President–Information Technology, Security Officer, Home Access
Health, USA
Maureen O’Connell, Board Chair, Acacia Research (NASDAQ), Former Chief Financial Officer and Chief
Administration Officer, Scholastic, Inc., USA
David Samuelson, Chief Executive Officer, ISACA, USA
Gerrard Schmid, President and Chief Executive Officer, Diebold Nixdorf, USA
Gregory Touhill, CISM, CISSP, President, AppGate Federal Group, USA
Asaf Weisberg, CISA, CRISC, CISM, CGEIT, Chief Executive Officer, introSight Ltd., Israel
Anna Yip, Chief Executive Officer, SmarTone Telecommunications Limited, Hong Kong
Brennan P. Baybeck, CISA, CRISC, CISM, CISSP, ISACA Board Chair, 2019-2020, Vice President and Chief
Information Security Officer for Customer Services, Oracle Corporation, USA
Rob Clyde, CISM, ISACA Board Chair, 2018-2019, Independent Director, Titus, and Executive Chair, White Cloud
Security, USA
Chris K. Dimitriadis, Ph.D., CISA, CRISC, CISM, ISACA Board Chair, 2015-2017, Group Chief Executive Officer,
INTRALOT, Greece
Ta b l e o f C o n t e n t s
Introduction .....................................................................................................................................................................................7
Terms and Definitions .........................................................................................................................................................................7
Introduction
ISACA created the Information Technology Audit Sampling guidelines (Guidelines 2208) as a companion to the
ITAF framework. These guidelines support IT audit and assurance practitioners’ use of sampling to draw a
conclusion about a total population when audit procedures are applied to less than 100 percent of that population.
Although these companion guidelines do not have a corresponding ITAF standard, the numbering of these guidelines
aligns with the numbering scheme of ITAF. General, performance and reporting guidance series are numbered 2000,
2200 and 2400 respectively. Number 2208 accommodates numbering of related guidelines before and after it.
Adherence to the guidelines is strongly recommended but not mandatory. Accordingly, IT audit and assurance
practitioners may exercise flexibility in their use of the Information Technology Audit Sampling guidelines. Even so,
practitioners should be prepared to defend and justify any significant deviation from the guidelines or the omission
of relevant sections of the guidance in the performance of IT audit and assurance engagements. The guidelines may
not be applicable in all situations but should always be considered.
Throughout these guidelines, some common words have specific meanings that apply to the most common types of
engagements performed by IT audit and assurance practitioners. For these instances, a definition is provided in
Appendix C to ensure that the meanings of these words, within the context of these guidelines, are understood and
consistently applied.
Where practical, ITAF terms and definitions generally are consistent with commonly used terminology in the
practice of professional auditing and in information technology and security; however, practitioners should consult
the current original source standards relevant to the specific type of engagement to be performed. This will ensure
alignment of terminology with the original source standards that are being followed.
The purpose of these guidelines is to provide guidance to IT audit and assurance practitioners in designing and
selecting an audit sample and evaluating sample results. Appropriate sampling and evaluation help to achieve the
requirements of sufficient and appropriate evidence.
IT audit and assurance practitioners should consider these guidelines when determining how to implement related
standards (see Appendix A) and related guidelines (see Appendix B), use professional judgment in their application,
be prepared to justify any departure, and seek additional guidance if necessary.
2208.1 Introduction The guidelines’ content sections are structured to provide information on the following key
audit sampling topics:
2208.2 Sampling
2208.6 Documentation
2208.2 Sampling
2208.2.1 In forming an opinion or conclusion, practitioners frequently do not examine all the
information available, because doing so may be impractical (e.g., requiring too much time
for the auditee and practitioners to investigate all information). If examination of all the
information is impractical, valid conclusions can be reached using audit sampling.
2208.2.2 When using statistical or nonstatistical sampling methods, practitioners should design and
select an audit sample, perform audit procedures, and evaluate sample results to obtain
sufficient and appropriate evidence to form a conclusion. When using sampling methods to
draw a conclusion on the entire population, practitioners should use statistical sampling.
2208.2.3 Sampling should not be used in some instances. For example, sampling should not be used
for tests of controls if there is no evidence of performance, such as appropriate segregation
of duties.11
2208.3.1 When designing the size and structure of an audit sample, practitioners should consider the
specific IT audit objectives, the audit procedures that are most likely to achieve those
objectives, the nature of the population, the nature of the control (e.g., manual or
automated), relevant subgroups within the population, and the sampling and selection
methods. In addition, when audit sampling is appropriate, consideration should be given to
the nature of the evidence sought, possible error conditions and possible root causes.
1
1
Public Company Accounting Oversight Board (PCAOB), AS 2315: Audit Sampling, www.pcaobus.org/Standards/Auditing/Pages/AS2315.aspx
2208.3.2 When considering the IT audit objectives while designing the sample, IT audit practitioners
should consider the following:
Purpose of the sample
Sampling unit
Population
Sampling risk and sample size
Tolerable error
Underlying expected distribution (e.g., Poisson, binomial, normal or exponential)
Behavior over time (e.g., seasonality and decrease in performance)
Subpopulations or subgroups that occur naturally should be taken into account for
operational relevance
Outliers
Small populations of adverse or rare events
Data from external support tools that are used to confirm or complement the results of
sampling
2208.3.3 The purpose of the sample can be:
Compliance testing/test of controls—An audit procedure designed to obtain audit
evidence on the effectiveness of the controls and their operation during the audit
period. Examples of compliance testing of controls for which sampling can be
considered include user access rights, program change-control procedures, procedure
documentation, program documentation, follow-up exceptions, review of logs and
software license audits.
Substantive testing/test of details—An audit procedure designed to obtain audit evidence
on the completeness, accuracy or existence of activities or transactions during the audit
period. Examples of substantive tests for which sampling can be considered include re-
performance of a complex calculation (e.g., interest) on a sample of accounts, a sample of
transactions to vouch for supporting documentation, etc.
2208.3.4 The sampling unit depends on the purpose of the sample. For compliance testing of
controls in which the sampling unit is an event or transaction (e.g., a control such as
authorization of an invoice), attribute sampling is typically applied, because it determines
the characteristics of a population. For substantive testing in which the sampling unit is
often monetary, variable sampling is frequently applied, because it determines the
monetary or volumetric impact of characteristics of a population.
2208.3.5 The population is the entire set of data from which practitioners wish to sample to reach a
conclusion on the population. Therefore, the population from which the sample is drawn
must be appropriate to test the design and operating effectiveness of the controls and be
verified as complete for the specific IT audit objective and scope.
2208.3.6 To assist in the efficient and effective design of the sample, sampling stratification may be
appropriate. Stratification is the process of dividing a population into subpopulations with
similar characteristics explicitly defined, so that each sampling unit can belong to only one
stratum.
2208.3.7 When determining sample size, practitioners should consider the sampling risk, the amount
of error that is acceptable and the extent to which errors are expected. Sampling risk arises
from the possibility that a practitioner’s conclusion may be different from the conclusion
that is reached if the entire population is subjected to the same audit procedure. The two
types of sampling risk are:
Risk of incorrect acceptance—A material weakness is assessed as unlikely when, in
fact, the population is materially misstated.
Risk of incorrect rejection—A material weakness is assessed as likely, when, in fact,
the population is not materially misstated.
2208.3.8 Sample size is affected by the level of sampling risk that the IT audit and assurance
practitioners are willing to accept. Sampling risk should also be considered in relation to the
audit risk model and its components—inherent risk, control risk and detection risk, as
detailed in ITAF Standard 1201 Risk Assessment in Planning. This standard requires that
practitioners consider subject matter risk, audit risk and related exposure to the enterprise
when planning audit engagements.
2208.3.9 Tolerable error is the maximum error in the population that practitioners are willing to
accept and still conclude that the test objective is achieved. For substantive tests, tolerable
error is related to the practitioner’s judgment about materiality. In compliance tests,
tolerable error is the maximum rate of deviation from a prescribed control procedure that
practitioners are willing to accept.
2208.3.10 Smaller sample sizes are justified when the population is expected to be error-free. If
practitioners expect errors to be present in the population, they must examine a larger
sample to conclude that the actual error in the population is not greater than the expected
tolerable error. When estimating the expected error rate in a population, practitioners should
consider matters such as:
Error levels identified in previous audits
Changes in enterprise procedures
Evidence available from an evaluation of the system of internal control, results from
analytical review procedures and/or results of preliminary tests of the population
2208.3.11 Practitioners should consider, if appropriate, the need to involve specialists in the design
and analysis of complex sampling approaches—such as stratified random samples that
must have statistical validity and sampling that is based on established quality control
methods.
2208.3.12 In some instances, the practitioner may design a sample that can be used as a test of
controls and as a substantive test. See guidance on dual sample tests by the American
Institute of Certified Public Accountants (AICPA).2 2
2208.3.13 If practitioners conclude that sampling does not allow the IT audit objectives to be achieved
and a test of the entire population is required, practitioners should consider applying
continuous assurance, because it allows testing of the entire population in a timely and
cost-effective way.
2208.4.1 Practitioners should ensure that the population is complete and control the selection of the
sample. Practitioners should select sample items to ensure that the sample is
representative of the population regarding the characteristics being tested.
2208.4.2 For a sample to be representative of the entire population, all sampling units in the
population should have an equal or known nonzero probability of being selected. This
suggests that statistical sampling methods should be used, because they use techniques
from which mathematically constructed conclusions about the entire population can be
drawn. Practitioners should validate completeness of the population to ensure that the
sample is selected from an appropriate data set.
2
2
American Institute of Certified Public Accountants (AICPA), AU Section 350 Audit Sampling,
www.aicpa.org/Research/Standards/AuditAttest/DownloadableDocuments/AU-00350.pdf
2208.4.3 Nonstatistical sampling is an approach that is used by practitioners who want to use their
own experience, knowledge and professional judgment to determine a sample. This method
may likely reflect a human bias, because it is not statistically based and does not ensure
that every sampling unit has a known nonzero probability of being selected. Therefore,
results should not be extrapolated over the population, because the sample is unlikely to be
representative of the entire population. Nonstatistical sampling may be used when results
are needed quickly to confirm a proposition, but it should not be used to draw
mathematically constructed conclusions regarding the entire population.
2208.4.4 There are five commonly used sampling methods that are categorized as either statistical
sampling methods or nonstatistical sampling methods:
Statistical sampling methods:
Simple random sampling—Ensures that all combinations of sampling units in the
population have an equal chance of selection.
Systematic sampling—Involves selecting sampling units using a fixed interval between
selections, with the first interval having a random start. Examples include monetary
unit sampling or value-weighted selection in which each individual monetary value
(e.g., $1,000) in the population is given an equal chance of selection. The item that
includes the monetary unit is selected for examination because the individual
monetary unit cannot be examined separately. This method systematically weighs the
selection in favor of the larger amounts. Another example is selecting every nth
sampling unit.
Stratified random sampling—Ensures that all sampling units in each subgroup have a
known chance of selection.
Practitioners should consider using statistical software for calculating standard deviations
and other summary statistics for results of statistical sampling.
Nonstatistical sampling methods:
Haphazard sampling—Practitioners select the sample without following a structured
technique, while avoiding any conscious bias or predictability. Analysis of a haphazard
sample should not be relied on to form a conclusion on the entire population.
Judgmental sampling—Practitioners place a bias on the sample (e.g., all sampling
units over a certain value, all sampling units for a specific type of exception, all
negative sampling units). A judgmental sample is not statistically based, and results
should not be extrapolated over the population, because the sample is unlikely to be
representative of the population as a whole.
2208.5.1 After performing the audit procedures aligned with the particular IT audit objective on each
sample item, practitioners should analyze any possible errors detected in the sample to
determine whether they are actual errors. For possible errors that are determined to be
actual errors, the nature and cause of the errors should be identified. Also, the errors should
be projected as appropriate to the population—but only if a statistically based sampling
method was used.
2208.5.2 When the expected audit evidence regarding a specific sample unit cannot be obtained,
practitioners should consider whether they can obtain sufficient and appropriate audit
evidence by performing alternative procedures on the item selected, or by selecting and
testing a replacement sample unit.
2208.5.3 Practitioners should consider projecting the results of the sample to the population with a
method of projection consistent with the method used to select the sampling unit. The
projection of the sample may involve estimating the probable error in the population, and
estimating any further error that might not have been detected because of the imprecision
of the technique.
2208.5.4 Discussion of nonstatistical sampling (haphazard or judgmental) results should be
restricted to a description of the results of analyzing the sample, in the context of the
population as a whole.
2208.5.5 Practitioners should consider whether errors in the population might exceed the tolerable
error by comparing the projected population error to the estimated or defined tolerable error,
considering the results of other audit procedures relevant to the audit objective. Tolerable
error may be estimated or defined by audit criteria, industry standards, contractual
requirements, software specifications, etc. When the projected population error exceeds the
tolerable error, practitioners should reassess the sampling risk. If that risk is unacceptable,
they should consider extending the audit procedure; recalculating sample size using the
refined tolerable error and testing the additional sample units; or performing alternative
audit procedures.
2208.6 Documentation
2208.6.1 The work papers should include sufficient detail to describe clearly the sampling objective
and the sampling process used. The work papers should include:
Purpose of the sample, including the sample unit
Source of the population, definition of the population, and the relation of the population
to the audit scope
Sampling parameters, e.g., sample size (including any consideration regarding
sampling risk); random start, seed number or method by which random start was
obtained; sampling interval
Sampling method
Items selected and, if nonstatistical sampling is used, justification for the selected
items
Details of audit tests performed, including evaluation of errors and, if applicable,
alternative audit procedures
Conclusions reached
When implementing standards and guidelines, practitioners are encouraged to seek other guidance if necessary. This
guidance can be from IT audit and assurance:
Colleagues within or outside the enterprise, e.g., through professional associations or professional network
groups
Management
Governance bodies of the enterprise (e.g., audit committee)
Professional guidance materials (e.g., books, papers and other guidelines)
Linkages to COBIT® 2019 for Information Technology Audit Sampling Guidelines (Guidelines 2208)
EDM03 Ensured Risk Ensure that I&T-related enterprise risk does not exceed the enterprise’s risk appetite and risk
Optimization tolerance, the impact of I&T risk to enterprise value is identified and managed, and the
potential for compliance failures is minimized.
AP012 Managed Risk Integrate the management of I&T-related enterprise risk with overall enterprise risk
management (ERM) and balance the costs and benefits of managing I&T-related enterprise
risk.
MEA02 Managed System of Obtain transparency for key stakeholders on the adequacy of the system of internal
Internal Control controls and thus provide trust in operations, confidence in the achievement of enterprise
objectives and an adequate understanding of residual risk.
MEA04 Managed Assurance Enable the organization to design and develop efficient and effective assurance initiatives,
providing guidance on planning, scoping, executing and following up on assurance reviews,
using a road map based on well-accepted assurance approaches.
Note: Only those standard statements that are relevant to the Information Technology Audit Sampling Guidelines
(Guidelines 2208) are listed.
1204 Performance and Supervision 1204.4 IT audit and assurance practitioners shall obtain and preserve sufficient and
appropriate evidence to achieve the audit objectives.
1204.5 IT audit and assurance practitioners shall document the audit process and
describe the audit work and the audit evidence that support findings and conclusions.
1204.7 IT audit and assurance practitioners shall provide an appropriate audit opinion
or conclusion and include any scope limitation where required evidence is obtained
through additional test procedures.
1205 Evidence 1205.1 IT audit and assurance practitioners shall obtain sufficient and appropriate
evidence to draw reasonable conclusions.
1205.2 Applying professional skepticism, IT audit and assurance practitioners shall
evaluate the sufficiency of evidence obtained to support conclusions and achieve
engagement objectives.
S
Sampling risk—The probability that an IT auditor has
reached an incorrect conclusion because an audit sample,
rather than the entire population, was tested.