Type II SOC engagements (for both SOC 1 audits and SOC 2 audits) require walkthroughs and testing of the controls in place at the service organization to be able to opine on the suitability of the design and the operating effectiveness of controls during the period under review. Each control objective or criteria has a number of supporting controls that are walked through and tested, and this is accomplished using a variety of testing methods/procedures.
What Is a Test of Control?
A test of control explains the audit procedures used to determine if a control is designed and operating. The goal of the test of controls in audit procedures is to determine if controls are sufficient to prevent or detect risks that could impact a business.
When performing a SOC examination, we are helping our clients identify the controls that they have, or need to implement. These controls will demonstrate to their clients that the services they are providing for their environment are safe and secure. So once the controls that a client has in place are identified, how does the auditor confirm they are working? That is where a test of control comes in. There are a number of different ways to confirm, or test, that a control is operating. Below we have outlined the five testing methods used for testing controls as part of a SOC examination.
What Are the Five Types of Audit Tests?
There are five main methods to walk through and test each control in place at the service organization which include (listed in order of complexity from lowest to highest):
- inquiry,
- observation,
- examination or inspection of evidence,
- re-performance, and
- computer-assisted audit technique (CAAT).
These methods are used when designing audit procedures.
Inquiry
Simply, the auditor asks appropriate management and staff about the controls in place at the service organization to determine some relevant information. This method is often used in conjunction with other, more reliable methods. For example, an auditor may inquire of management if visitors to the data center are escorted at all times if the auditor is not able to observe this activity while on site. No control objective or audit criteria should ever be supported by controls only tested through inquiry procedures. An inquiry procedure is great to add to another test step using one of the other methods below.
Observation
Activities and operations are tested using observation. This method is useful when there is no documentation of the operation of a control, such as observing that a security camera is in place or observing that a fire suppression system is installed.
Examination or Inspection of Evidence
This method is used to determine whether or not manual controls are being performed. For instance, are backups scheduled to run on a regular basis? Are forms being filled out appropriately? This method often includes reviewing written documentation and records such as employee manuals, visitor logs, and system databases. The evidence is obtained and examined or inspected as part of this method.
Re-performance
Re-performance (sometimes called recalculation) is used when the three above methods combined fail to provide sufficient assurance that a control is operating effectively. This method can also be used to prove by itself that controls are operating effectively. This method of testing (as well as a CAAT) is the strongest type of testing to show the operating effectiveness of a control. Re-performance requires the auditor to manually execute the control, such as re-performing a calculation that a system automatically calculates to confirm that the system performs the control correctly. Re-performance by the auditor is completed and then compared to the systematic output to validate that the result is the same.
Computer-Assisted Audit Technique (CAAT)
This method can be used to analyze large volumes of data, or just be able to analyze every transaction rather than just a sample of all transactions. Software is generally used to perform a CAAT, which can range from using a spreadsheet to using specialized databases or software designed specifically for data analytics (e.g. ACL).
When Do You Use the Different Audit Testing Procedures?
Samples of populations are selected for testing based on the type of test being performed (i.e., a test of one would be completed for an automated control using re-performance, but a sample of the population would be selected for an inspection control). Additional considerations are the population size and the level of precision we want to achieve in the testing. These are all considerations in audit sampling in a SOC examination.
If during testing the auditor encounters an error in a test of controls, they will expand the sample size and conduct further testing, or perform additional tests. Additional types of testing procedures may be required or useful. If additional errors are found, the auditor will consider whether there is a systematic controls problem that renders the controls ineffective, or if the errors appear to be isolated instances that do not reflect upon the overall effectiveness of the control in question.
What Are the Main Procedures for Obtaining Audit Evidence?
When completing the tests of controls, it is very important how audit evidence is obtained. To be able to rely on evidence obtained, the auditor must be comfortable that audit evidence is complete and accurate. This can be accomplished by observing the pulling of audit support directly from the person responsible for the support.
For example, sitting with a system administrator as they pull up and screenshot password restrictions or a population of all system users. Additionally, to confirm the completeness of population, queries can be obtained and reviewed to ensure none of the population has been filtered out.
What Does the AICPA Say About Using the Five Available Audit Techniques?
The American Institute of Certified Public Accountants (AICPA) provides guidance for SOC examinations, and within the SOC guides (both a SOC 1 and SOC 2 guide are available) the AICPA provides some guidance on what methods of testing are acceptable.
Statement on Standards for Attestation Engagements (SSAE) No. 18 (Clarification and Recodification) is the standard governing SOC engagements (AT-C Section 320 for SOC 1 engagements and AT-C Sections 105 and 205 for SOC 2 engagements). Within AT-C Section 320 (Reporting on an Examination of Controls at a Service Organization Relevant to User Entities’ Internal Control Over Financial Reporting) the following is specifically stated about testing methods (bold italics were added to emphasize the key points):
- “.26 The service auditor should determine through inquiries made in combination with other procedures whether the service organization’s system has been implemented.”
The AICPA is saying that inquiry alone should not be used. The standard then provides additional information on the types of audit procedures.
- “.31 When designing and performing tests of controls, the service auditor should
- perform other procedures such as inspection, observation, or reperformance in combination with inquiry to obtain evidence about the following…”
Additionally, specifically within the SOC 1 guide put out by the AICPA, the below paragraph regarding testing is included (bold italics were added to emphasize the key points):
- “4.90 Inquiry alone does not provide sufficient appropriate evidence of the operating effectiveness of controls. Some tests of controls provide more convincing evidence of the operating effectiveness of controls than others.”
The guide then talks about the combination of testing procedures that provides more convincing evidence than inquiry alone and provides examples of combinations of tests.
Where Are the Types of Testing Used & the Results of Testing Listed in a SOC Report?
The methods of testing are listed in section IV of SOC reports where the independent service auditor includes their description of tests of controls and results. Some controls are tested by using only one method, while other controls may use multiple methods. The methods will be listed in section IV along with the results of each of the tests.
Summary
At Linford & Company, we ensure that the audit testing procedures meet the type of controls to confirm design and operating effectiveness, in addition to complying with the guidance set forth by the AICPA.
Please contact us if you have additional questions about testing methods or if you are in need of compliance or advisory services for SOC 1, SOC 2, FedRAMP, HITRUST, and HIPAA audits and assessments.
This article was originally published on 1/25/2018 and was updated on 8/9/2023.
Nicole Hemmer started her career in 2000. She is the co-founder of Linford & Co., LLP. Prior to Linford & Co., Nicole worked for Ernst & Young in Indianapolis, Chicago, and Denver. She specializes in SOC examinations and royalty audits and loves the travel and challenge that comes with clients across all industries. Nicole loves working with her clients to help them through examinations for the first time and then working together closely after that to have successful audits.