Cure For SOX-Testing Blues At Royal Ahold

As many Compliance Week readers know, our company—the $44 billion supermarket operator Royal Ahold, listed on the New York Stock Exchange—was accused in 2004 of filing materially false statements to the Securities and Exchange Commission. Since then, Ahold has succesfully completed its road to recovery, and improving our system of internal controls was crucial to doing so. As such, we believe we have a unique perspective on how best to accomplish the objectives of Sarbanes-Oxley. We’ve devised a new approach to SOX Section 404 management testing, and want to share our experience with fellow Compliance Week readers.

Contrary to what you might think while in the depths of an internal controls audit, it is possible to develop an approach to assess the effectiveness of controls that is both highly effective as well as efficient. The approach we developed at Ahold, which we call “embedded testing,” is founded in the most fundamental of internal control principles. External auditors should be able to place a high degree of reliance on embedded testing. Implementation of embedded testing itself can by reduce SOX 404 compliance costs by as much as 50 percent, while at the same time increasing the amount of competent evidence.

The concept of embedded testing is straightforward: testing of the operating effectiveness of a control is performed as an ongoing, natural part of the process to which the control belongs. As such, oftentimes it is executed by the manager or supervisor of the one who is performing the control. Test performance is adequately documented and exceptions are followed up appropriately. Internal audit departments still conduct some testing, but only to verify that managers are executing their assigned tests properly, and not to provide the principal evidence that controls are operating effectively.

With all its simplicity and effectiveness, embedded testing is nevertheless a fundamentally different approach to what almost all Sox-compliant companies do today—an approach we call “add-on testing.” In add-on testing, persons who are not part of the regular process perform the testing. For example, these persons could be internal auditors, other internal control specialists, or persons from other departments (“peer-review testing”).

Embedded testing has several characteristics that make it more appealing than add-on or peer-review testing. Among them:

Embedded testing is far more natural;

the cost of complying with SOX 404 is reduced by as much as 50 percent;

significantly more evidence typically is recorded;

control weaknesses are identified by the persons best positioned to do so;

control weaknesses will usually be identified more quickly;

only value-added testing activities are carried out; and

managers’ control awareness is enhanced.

The Folly Of Add-On Testing

When, say, an accounting clerk performs a reconciliation of a general ledger account, typically this reconciliation is subjected to review by the clerk’s supervisor in the ordinary course of business. Typically, the goal of such a review is to ensure that: the reconciliation was performed and documented in accordance with established guidelines; reconciled items could be adequately explained; and possible exceptions were adequately followed up.

When the supervisor performs the review, in essence he is not adding any new information; he is simply checking—in effect, “testing”—whether the person performing the reconciliation did his job properly, ensuring that the control ( that is, the reconciliation) operated effectively. In contrast, with add-on testing, someone else (for example, an internal auditor) tests the reconciliation. Essentially, that person reconfirms the supervisor’s work.

Currently, many controls designated as “key” for SOX 404 purposes are of a review, monitoring nature. As such, they would be labeled more appropriately as “tests.” Managers routinely test controls because they want to be sure that the persons reporting to them are doing their jobs, that the information coming out of the process they oversee is reliable, that mistakes are caught before they cause problems, and that process improvements can be implemented to avoid future mistakes.

All this is natural; it was done long before Sarbanes-Oxley, and always will be done. It is part of the normal “Plan-Do-Check-Act” management cycle. The “check” in this management cycle is the test, and it should be given appropriate credit in the SOX 404 process.

When looking at the control framework this way, having the key control tested again by an outsider (through add-on testing) is unnecessary. In fact, there is no need to do any add-on testing—so long as management does in fact test the key controls, in accordance with the requirements for proper management testing.

So Why The Add-On Craze?

Almost all companies have management testing performed by persons other than management through add-on testing. And since estimates are that on average, more than half of companies’ SOX 404 compliance costs are spent in executing add-on management testing, it quickly becomes a very costly exercise.

So why, if embedded testing does the trick, do companies still devote so much time and resources to add-on testing?

To answer this question, recall when SOX 404 was implemented. In issuing guidance, the regulators chose to focus on the external auditors, who were tasked with executing their own assessments. One trait specific to external auditors is that they are very ... external. They will have no way of knowing themselves, firsthand, from their own observation, whether controls are operating as described. They must come in and test. This is the clear and fundamental difference between auditors and management: Management is in a position (indeed, the best position) to know about the effective operation of controls because they are there, watching controls operate all day long, every day. They are paid to make sure that the controls operate effectively and to take corrective action in case controls fail.

It is not as if, prior to Sarbanes-Oxley, managers were clueless, only hoping that controls were in fact working. Yet, by executing add-on testing, we are assuming exactly that: that without someone from the outside coming in, management never would know whether controls are operating as intended. Clearly this is not the case. Management has more than a clue—so why not take credit for all of the monitoring-type testing that management is already doing?

Other reasons exist why companies all went to add-on testing, some of them good. For one, without having documented all of the key controls, and having gone through to check whether they actually operated, companies were generally not too sure about where their control weaknesses were, and which managers were doing a good job of verifying this. Everything was implicit rather than explicit. Now that all of the key controls, including those that also qualify as management tests, as well as their operation, have been properly documented, this process has finally become explicit. One of the key requirements for management testing is that it must be documented adequately, since it has to be re-performable by third parties, such as the external auditor. Prior to SOX 404, this was hardly ever the case. So to be able to start taking credit for the testing that management already does in the ordinary course of business, first we had to have the proof that this was actually happening. By the initial implementation of SOX, we now have that proof, managers have grown accustomed to documenting when they perform their controls (including controls that also qualify as tests), and we can start taking credit for those tests.

Another, not so good reason for why companies generally have adopted add-on testing is simply because the external auditor, unaware of a different approach, advised or even required it. From the external auditor’s perspective, it makes perfect sense. To the company, however, it is a costly and inefficient way of getting the required assurance.

Finally, the add-on method is deceptively simple; typically, the approach to implementation was “first we document, then we test.” So, first, all of the controls (whether they were just controls or whether they were tests) were documented. Then testing plans would be drawn up for each control, and off we went—thus missing the point that many of the controls that we documented were already the tests! One positive outcome came out of this: Where managers were inadequately documenting the performance of their tests, this was identified and remediated (in a process often called “evidence gap remediation”).

Being Objective And Competent

Yes, a manager can be both objective and competent; this is the fundamental principle of the segregation of duties. What would be the point of having a supervisory review, if the person performing it is not seen to be independent from the control executor? In fact, if a manager is not objective of the individuals that he hires and fires, and cannot be counted upon to judge his subordinates’ performance objectively, he should not be a manager in that position. The same goes for competence: The direct-line manager should be the person most competent to judge the work of his subordinates (or certainly at least as good as any outsider coming in currently to perform add-on testing). Still, to be sure, the quality of the testing performed by management should be assured through sample tests performed by internal auditors as noted before.

So while external auditors always will have to perform a measure of add-on testing, companies should not. There are two notable exceptions:

Where management testing would be more efficiently carried out by specialist testers. An example of this would be the store-level audit function that operates within larger retail companies. At those retailers, regional managers could be tasked with checking up on the (key) control operators, but it’s just not efficient.

Where the knowledge required to evaluate control execution properly is so highly specialized that the company has decided it is more efficient to not have that expertise in-house, and to leave the checking up to an external party. Examples of this are the insurance company’s in-house actuary, whose work is double-checked from time to time by an outside agency, or the treasury department, where a specialist could be engaging in exotic strategies and products. Some form of external oversight is often employed in this situation as well.

But these are the exceptions to the rule: that managers should perform their own management tests.

Preconditions To Remember

The first important condition is that the company’s internal audit function should verify that management is performing and documenting that all testing is being done properly.

The direct-line manager should be the person most competent to judge the work of his subordinates ... Still, to be sure, the quality of the testing performed by management should be assured through sample tests performed by internal auditors.

The second condition is that managers will need to be supported on an ongoing basis in defining the appropriate testing activities (including the extent of the testing, the documentation required, and so on) and in interpreting and responding to the test results. This support could be provided by the same persons tasked with all of the other required SOX 404 activities, such as scoping and risk assessment, control documentation, evaluation of design effectiveness, and so on.

A third condition is that recording test activities should be made as easy as possible for management. In this regard, an effective software tool, which will also enable the company to monitor the progress and outcome of tests performed by management, may be indispensable. Where companies still can get by without an appropriate tool when using the add-on testing approach—principally because the whole process is executed by relatively few “experts”—getting many managers involved will undoubtedly change that.

The SEC, The PCAOB, And Embedded Testing

With respect to test approaches, a fundamental point that the Securities and Exchange Commission has included in its proposed new guidance for management in its execution of a SOX 404 compliant process is the recognition of the relevance and value of embedded-test activities. As such, the SEC’s proposed guidance provides the first (and strong) official support for embedded testing.

Meanwhile, new guidance from the Public Company Accounting Oversight Board contains one provision that in some ways appears to contradict what the SEC is proposing: namely, that the external auditor cannot make use of tests performed by managers with supervisory responsibility over the area for which the control tested is part. In our opinion, this is an unnecessary provision that could have the (possibly unintended) effect of hampering the efficiency of companies’ SOX 404 compliance processes.

Shifting The Paradigm

Now that the (relatively simple) concept of embedded testing is out there, how does a company go about achieving it? Moving to embedded testing is indeed not easy. It does require the entire control framework to be re-evaluated and viewed in a different perspective. The distinction between mere controls and control tests has to be defined. Controls that are not currently being tested in the ordinary course of business have to be evaluated: Why is a manager not checking that this control is being performed adequately already? New controls will have to be implemented if it turns out the SOX 404 management testing was the first and only assurance we got over important controls.

And it is all worth it!