Ethics and compliance officers, internal auditors, and the like have tried to conduct periodic reviews of their programs, but that has taken on new importance thanks to and updated definition of “effective” compliance programs under the U.S. Sentencing Guidelines.

Those guidelines—whose most recent amendments went into effect Nov. 1—emphasize the importance of assessing compliance and ethics programs following the detection of criminal conduct. Periodic assessment can be critical to ensuring the success of your internal compliance and ethics program and can confirm that it is structured properly to deter and detect actual or potential violations or law.

A survey by the Society of Corporate Compliance and Ethics (SCCE) indicates that organizations have been hearing that message: 66 percent of respondents said they have conducted such audits, and another 15 percent indicated that they intend to do so in the coming year.

So what does that assessment look like? The term “compliance auditing” is often misused or applied loosely and does not appear to be well understood outside the compliance discipline. There are distinctions between compliance auditing, compliance risk assessments, and assessing the compliance program for effectiveness, though these activities often overlap and coincide.

Under the Sentencing Guidelines, a company is to take reasonable steps “to ensure that the organization’s compliance and ethics program is followed, including monitoring and auditing to detect criminal conduct.” What we call compliance auditing is viewed as a substantive and process-oriented audit aimed at evaluating whether employees actually obey the law or corporate procedures (or controls) that have been put in place to foster law-compliant conduct. Such audits should be conducted in areas of high regulatory risk, with large numbers of repetitive transactions that could run afoul of compliance rules.

For example, in healthcare this would entail periodic audits of the billing and coding process to ensure that medical services were billed and paid correctly given the scrutiny of government reimbursement.

The guidelines also provide that an organization “shall periodically assess the risk of criminal conduct.” To determine which areas to audit and monitor, compliance risk assessments are performed. This process involves identifying and evaluating compliance and ethics risks, assessing their significance based on likelihood and consequence, and determining the current and desired level of controls and the acceptable level of risk.

Since amendments in 2004, the guidelines began to further specify that the organization should take reasonable steps “to evaluate periodically the effectiveness of the organization’s compliance and ethics program.” In other words, to have an effective program you need to conduct a program assessment to evaluate whether it works or not, in conjunction with review of the program’s specific features.

An ‘Effective’ Compliance and Ethics Function

The compliance professional and internal auditor should recognize that terminology as defined by the courts or in legislation may be quite different from common usage. The term “effective” for a compliance and ethics program as a practical matter ultimately boils down to what the government deems it to be. If a prosecutor decides that your program is effective when applying the criteria of the Department of Justice (McNulty Memo) and Securities and Exchange Commission (Seaboard policy), then the company may avoid an indictment or get more favorable settlement terms. If the case goes to trial and a judge decides the program is effective under the Sentencing Guidelines, then you get substantial reductions in fines and penalties.

Yet commentators have noted that despite the guidelines’ incentives for companies to implement ethics and compliance programs, examples of how effectiveness was decided are hard to find. A Conference Board study of regulatory enforcement actions suggests there is little evidence to support the view that strong, effective corporate compliance programs help companies secure better treatment when under investigation.

To have an effective program you need to conduct a program assessment to evaluate whether it works or not, in conjunction with review of the program’s specific features.

Effectiveness in this context also does not necessarily equate with the “performance” of the program. The board and management should have assurances that a program is worth its investment and making a difference beyond demonstrating that it has the bare bones features explicitly outlined by the government. An internal auditor can provide more value by examining measures to ensure the organization is optimizing its investments in the program.

Program performance is generally not considered by lawmakers and regulators. For example, regulators do not particularly care if your whistleblower hotline process costs $10,000 per year or $100,000 per year to operate, as long as it is appropriately designed and operating as intended. But as with all enterprise processes, management and stakeholders will expect that the program is not only effective (as defined by the government), but also efficient and responsive and delivering on enterprise objectives.

Who Should Audit?

Program assessments are conducted by professionals with varied skill sets: attorneys knowledgeable of the regulatory environment; auditors, who can apply a systematic approach; and compliance and ethics professionals, who have a deep understanding of program management issues. Each discipline working separately brings strengths and weaknesses to the evaluation process.

It is my experience that program assessments require a multidisciplinary approach so that the most reliable and valid findings can be made.

Lawyers bring technical expertise on compliance requirements, and performing the assessment under the direction of counsel may keep findings and recommendations confidential under privilege (though whether a legal privilege may apply is uncertain). Counsel can advise on disclosure obligations, their implications, and how to characterize the review’s findings in the final report to minimize legal risk in the disclosure. However, reviews by counsel can seem lacking in quantitative detail, and appreciation of operational challenges.

Auditors, on the other hand, can bring a systematic methodology to the review though they need to be trained on the nuances of the compliance and ethics function. While compliance auditing and program assessment can borrow from traditional financial and operational audits, there are key distinctions that the auditor may not realize. They would also need to communicate with counsel if wrongdoing is identified during the assessment. My experience working with auditors is that many need more familiarity with concepts and practices being utilized by compliance professionals.

The company compliance office brings the most expertise to the table, yet they may not have the adequate resources to staff a formal audit of the compliance program. Utilizing internal audit or an outside resource adds a level of independence and objectivity to the program assessment.

In the SCCE survey, when asked who performs the assessment when one is done internally, over half of the respondents (56 percent) stated that members of the compliance and ethics department conduct it. The internal audit department was used by 21 percent, while others utilized members of legal (4 percent), human resources (2 percent), or others (17 percent).

Conducting the Program Assessment

Given the lack of standard measurement techniques, how should an audit of the program be approached? While techniques of compliance program evaluation are still evolving, there are several sound practices that organizations of all sizes can use to consider.

The internal auditor can start with tools utilized when conducting a review of the compliance environment (especially the “soft controls”) under Sarbanes-Oxley Section 404. This includes techniques for evaluating entity-level controls, the control environment, and fraud-control activities. The audit practitioner could apply an internal control reliability model with different levels of effectiveness (including initial, informal, systematic, integrated, and all the way up to optimized). Testing techniques typically include employee surveys, management inquiry, document review, and general computer controls.

A multidisciplinary approach with a mix of the qualitative and quantitative, rigorous science, and artful hunches, is what is needed to best inform whether the compliance and ethics program is truly effective. Since organizations have focused time, energy, and resources on designing, implementing, and improving compliance and ethics programs, it is indeed fair to assess whether all of this work is really working.