Type II SOC engagements (for both SOC 1 audits and SOC 2 audits) require walkthroughs and testing of the controls in place at the service organization to be able to opine on the suitability of the design and the operating effectiveness of controls during the period under review. Each control objective or criteria has a number of supporting controls that are walked through and tested, and this is accomplished using a variety of testing methods/procedures.
What is a Test of Control?
When performing a SOC examination, we are helping our clients identify the controls that they have, or need to implement. These controls will demonstrate to their clients that the services they are providing or their environment is safe and secure. So once the controls are identified how do you confirm they are working? That is where a test of control comes in. There are many different ways to confirm, or test, that a control is working.
What are the Different Types of Testing Methods Used During Audit Procedures?
There are five main methods to walkthrough and test each control in place at the service organization. These methods include (listed in order of complexity from lowest to highest): inquiry, observation, examination or inspection of evidence, re-performance, and computer assisted audit technique (CAAT).
- Inquiry: Simply, the auditor asks appropriate management and staff about the controls in place at the service organization to determine some relevant information. This method is often used in conjunction with other, more reliable methods. For example, an auditor may inquire of management if visitors to the data center are escorted at all times if the auditor is not able to observe this activity while on site. No control objective or criteria should ever be supported by controls only tested through inquiry procedures.
- Observation: Activities and operations are tested using observation. This method is useful when there is no documentation of the operation of a control, such as observing that a security camera is in place or observing that a fire suppression system is installed.
- Examination or Inspection of Evidence: This method is used to determine whether or not manual controls are being performed. For instance, are backups scheduled to run on a regular basis? Are forms being filled out appropriately? This method often includes reviewing written documentation and records such as employee manuals, visitor logs, and system databases.
- Re-performance: This method is used when the other three above methods combined fail to provide sufficient assurance that a control is operating effectively or this method can be used to prove by itself to demonstrate that controls are operating effectively. This method of testing (as well as a CAAT) is the strongest type of testing to show the operating effectiveness of a control. Re-performance requires the auditor to manually execute the control, such as re-performing a calculation that a system automatically calculates.
- CAAT: This method can be used to analyze large volumes of data, or just be able to analyze every transaction rather than just a sample of all transactions. Software is generally used to perform a CAAT, which can range from using a spreadsheet to using specialized databases or software designed specifically for data analytics (e.g. ACL).
When do You Use the Different Audit Testing Procedures?
Samples of populations are selected for testing based on the type of test being performed (i.e., a test of one would be completed for an automated control using re-performance, but a sample of the population would be selected for an inspection control), the population size, and the level of precision we want to achieve in the testing.
If, during testing, the auditor encounters an error in a test of controls, they will expand the sample size and conduct further testing, or perform additional tests. Additional types of testing procedures may be required or useful. If additional errors are found, the auditor will consider whether there is a systematic controls problem that renders the controls ineffective, or if the errors appear to be isolated instances that do not reflect upon the overall effectiveness of the control in question.
What are the Main Procedures for Obtaining Audit Evidence?
When completing the tests of controls, it is very import how audit evidence is obtained. To be able to rely on evidence obtained, the auditor must be comfortable that audit evidence is complete and accurate. This can be accomplished by observing the pulling of audit support directly from the person responsible for the support. For example, sitting with a system administrator as they pull up and screenshot password restrictions or a population of all system users. Additionally, to confirm completeness of population, queries can be obtained and reviewed to ensure none of the population has been filtered out.
What does the AICPA say About Using the Five Available Audit Techniques?
The American Institute of Certified Public Accountants (AICPA) provides the guidance for SOC examinations. Within the SOC guides the AICPA provides some guidance on what methods of testing are acceptable.
Statement on Standards for Attestation Engagements (SSAE) No. 18 (Clarification and Recodification) is the standard governing SOC engagements (AT-C Section 320 for SOC 1 engagements and AT-C Sections 105 and 205 for SOC 2 engagements). Within AT-C Section 320 (Reporting on an Examination of Controls at a Service Organization Relevant to User Entities’ Internal Control Over Financial Reporting) the following is specifically stated about testing methods (bold italics were added to emphasize the key points):
“.26 The service auditor should determine through inquiries made in combination with other procedures whether the service organization’s system has been implemented.” The AICPA is saying that inquiry alone should not be used. The standard then provides additional information on the types of audit procedures.
“.31 When designing and performing tests of controls, the service auditor should
- perform other procedures such as inspection, observation, or reperformance in combination with inquiry to obtain evidence about the following…”
Additionally, specifically within the SOC 1 guide put out by the AICPA, the below paragraph regarding testing is included (bold italics were added to emphasize the key points):
“4.90 Inquiry alone does not provide sufficient appropriate evidence of the operating effectiveness of controls. Some tests of controls provide more convincing evidence of the operating effectiveness of controls than others.” The guide then talks about the combination of testing procedures that provides more convincing evidence than inquiry alone and provides examples of combinations of tests.
Where are the Types of Testing Used and the Results of Testing Listed in a SOC Report?
The methods of testing are listed in section IV of SOC reports where the independent service auditor includes their description of tests of controls and results. Some controls are tested by using only one method, while other controls may use multiple methods. The methods will be listed out in section IV along with the results of each of the tests.
At Linford & Company, we ensure that the audit testing procedures meet the type of controls to confirm design and operating effectiveness, in addition to complying with guidance set forth by the AICPA.
Please contact us if you have additional questions about testing methods or want to learn more about SOC examinations. Also, please contact us if you are in need of compliance or advisory services for SOC 1, SOC 2, FedRAMP, HITRUST, and HIPAA audits and assessments.
Nicole Hemmer started her career in 2000. She is the co-founder of Linford & Co., LLP. Prior to Linford & Co., Nicole worked for Ernst & Young in Indianapolis, Chicago, and Denver. She specializes in SOC examinations and royalty audits and loves the travel and challenge that comes with clients across all industries. Nicole loves working with her clients to help them through examinations for the first time and then working together closely after that to have successful audits.