Audit Sampling in SOC Examinations

In completing SOC 1 and SOC 2 examinations (and most other types of audits), there is testing involved to determine the operating effectiveness of controls. There are different types of tests that can be applied to testing controls, and to complete a majority of these tests, a sampling of populations that are required. In this post, we cover what audit sampling is and provide guidance on how to apply audit sampling to get to a confident conclusion on the operating effectiveness of controls. For more information on the five types of tests, refer to our article on testing audit procedures.

What is Audit Sampling?

According to the AICPA, as stated in AU-C Section 530 of SAS No.122, Audit Sampling is defined as “The application of audit procedures to less than 100 percent of items within a population of audit relevance such that all sampling units have a chance of selection in order to provide the auditor with a reasonable basis on which to draw conclusions about the entire population.” The AICPA audit sampling guide was last updated in December 2019.

Why Do Auditors Use Sampling? What is the Purpose?

The definition from the AICPA is a little wordy, but to summarize, as auditors, the purpose of audit sampling is to allow us to do the right amount of testing to confidently determine the operating effectiveness of controls. This does not mean we can always test 100 percent, or even have the capacity to. Therefore, sampling comes into play in testing. But what is the right amount, and how do you figure that out?

As auditors, we need to consider three primary areas when performing audit sampling:

1. Sample method
2. Sample size
3. Tolerable rate of deviation

 

What Are the Different Types of Audit Sampling Methods?

There are four main types of audit sampling methods that are used when completing tests of controls in SOC 1 and SOC 2 examinations. The type of population, how it was generated, and the size of the population can have an impact on the type of audit sampling methodology that is chosen for testing. The four main types include:

• Simple Random Sampling – Every unit has the same probability of being selected. This type of sampling can easily be accomplished by assigning a number to each item in the population and then using a random number generator to randomly select numbers in the range of the population (there are online tools for this, apps, and even Excel formulas can be used to generate random numbers).
• Systematic Sampling – This method selects samples using intervals that are a result of dividing the population of units by the sample size. For example, if there are 250 items in the population and 25 will be selected for testing, 250 is divided by 25 to come up with 10, therefore, every 10th item in the population will be selected for testing.
• Haphazard Sampling – Similar to simple random sampling; however, random number generators or tools are not used, and selections are just made from the population without any bias.
• Block Sampling – Represents contiguous population items; for example, the five most recent transactions in a population or the five most recent events can be selected for testing. Block sampling could include testing 100 percent of a population.

Every SOC examination should follow one or more of these sampling methods for testing of the population. A walkthrough or inquiry only would not be sufficient to test all controls.

 

Statistical vs. Non-statistical Sampling: How They Differ

Statistical sampling requires that samples be selected at random, generally using a tool to generate random numbers. The simple random sampling method above would be considered statistical sampling.

Non-statistical sampling allows an auditor to use professional judgment when selecting samples. Non-statistical methods make a lot of sense when a population is very small and a sample can be selected quickly using judgment, rather than spending the time setting up a statistical sample. While non-statistical sampling allows for auditor judgment, an auditor should always be careful not to include too much bias in selecting samples.

What is the Appropriate Sample Size?

There are a number of factors that need to be considered when determining the sample size.

1. The size of the population being tested.
2. The risk of the control. All the controls that the auditor has selected to test are significant controls, but there is a spectrum that exists regarding the significance of each control. It is important to consider the impact (qualitatively and quantitatively) if a control is not operating effectively.
3. How many deviations/failures would be acceptable in testing the specific control?

The tables below (Table 1 and Table 2) are what we use as guidelines when selecting our sample sizes in our SOC 1 and SOC 2 examinations. These tables align with the guidance set forth in the audit sampling guide from the AICPA.

 

Table 1 is used for larger sample sizes (250 or greater in the population) and shows recommended sample sizes to get to a minimum 90% confidence level. The table includes the sample sizes for up to two deviations and takes into consideration the risk of the control.

 

Table 2 gives further guidance on sampling less frequent operating controls and on smaller populations (transactional).

Audit Sampling Examples

Using the tables above, a couple of examples would include:

Example 1

A population of all employees is provided and consists of 389 people, and you want to test that all employees are attending security awareness training. According to the table, expecting no deviations, the initial sample would be 25, and simple random or haphazard sampling would likely be applied. If it is found that one of the 25 selected did not attend training, the sample would be expanded to 40 people. If another deviation is found, the sample would be expanded to 60. If another deviation is found, sampling would stop, and it would be determined that the control is not operating effectively.

Example 2

The controls being tested state that a monthly reconciliation is completed, and you want to test that it was indeed completed monthly and reviewed by a manager. Using Table 2 above, you would select three months for testing. Using haphazard sampling, you would pick three months from the year and test those months. Because the population is smaller, any deviations would be a failure of the operating effectiveness of the control.

Audit Sampling Highlights

There are a number of factors that go into audit sampling, so here are some quick facts that remember when selecting samples for an audit.

• Where can guidance be found for audit sampling?

• • The AICPA has an audit sampling guide that can be referenced for guidance on sampling in an audit.

• Why is audit sampling important?

• • Audit sampling allows auditors to do the right amount of testing to confidently determine the operating effectiveness of controls.

• What are the methods of audit sampling?

• • There are four main types of audit sampling methods – simple random, systematic, haphazard, and block sampling.

• How do you determine the appropriate sample size?

• • To determine the sample size, the population, risk, and acceptable deviations must be considered. The audit sampling guide provides additional guidance on sample sizes.

Final Thoughts on Audit Sampling

The guidance from the AICPA is pretty extensive around audit sampling. SOC auditors should review their sampling methods to make sure they are aligned with the AICPA guidance when performing their examinations. Please contact us if you would like further information on sampling, testing methods, or any of the services we provide.

This article was originally published on 12/18/2018 and was updated on 4/23/2025.