The conventional wisdom is that what gets measured gets managed. But apply that wisdom to the question of measuring effectiveness of an ethics and compliance program, and things start to get a bit fuzzy.
Perhaps that's why in the 2011 State of Compliance report published by Compliance Week and PwC last year, a full 38 percent of respondents admitted that they don't measure the effectiveness of their programs at all. And to a certain degree you can see the logic in that. After all, ethics and compliance programs are supposed to prevent problems from happening. How can one quantify that which hasn't happened?
“It's always hard to prove a negative,” says Ted Banks, president with Compliance and Competition Consultants, Chicago.
Moreover, no single indicator can definitively tell that a program is effective or not, says Barbara (Bobby) Kipp, risk assurance services partner with PwC. The ultimate goal is to create a culture where employees do the right thing; achieving that is usually a culmination of many decisions and actions.
Even so, compliance organizations that fail to measure their programs' effectiveness lose out in several ways. For starters, Chapter 8 of the U.S. Sentencing Guidelines includes the following sentence: “The organization shall take reasonable steps to ensure that the organization's compliance and ethics program is followed, including monitoring and auditing to detect criminal conduct.” Admittedly, this isn't the only reason to monitor compliance efforts—but it is one that will resonate with senior executives.
In addition, measuring the effectiveness of a compliance initiative can help an organization better manage its risks and prioritize its goals, says Mitch Avnet, chief ethics and compliance officer with Lincoln Financial Group in Radnor, Penn. “That's the value add.” Lincoln, which is regulated by multiple federal and state agencies has a compliance team of about 35 employees, Avnet says. In addition, workers stationed in each of the business units have responsibility for meeting Lincoln's guidelines for regulatory guidance. In all, about 100 employees throughout the company are involved with compliance somehow.
Avnet points out that to develop a strong compliance and risk-management program, an organization must rigorously assess the internal and external risk environment, distinguish controllable from uncontrollable risks, and identify trends and patterns in activities. “Having this great, quantifiable type of information,” can aid in prioritizing risks and setting goals and objectives, he says.
Many effective, centrally run compliance programs will aggregate volumes of data, adds Rob Biskup, director of forensic and dispute services with Deloitte Services. That information can be used to help senior management better deploy resources, make decisions, and assume risk in an informed manner. “Compliance can be a valuable enabler of corporate strategy.”
Metrics, Objective and Subjective
Many companies start a program evaluation by focusing on objective measures: number of calls to the corporate hotline, or percentage of employees who complete a course on safety. This information usually is easy to gather, and once assembled, can be compared to industry averages. That's helpful, as the government often compares programs against what appears to be standard industry practice, Kipp says. These statistics also can be monitored over time to get a sense of any trends.
“Where compliance sits in the business, that's where measurement of the risk has to happen.”
—Barbara (Bobby) Kipp,
Risk Assurances Service Partner,
PwC
On the other hand, these measures also don't paint the entire picture. For example, a jump in the number of calls to a hotline may mean an increase in problem areas, or simply greater awareness that the hotline exists. The result can be “too narrow a focus on the elements of the program, without taking a holistic view,” Biskup says.
As a result, many organizations also review subjective measures of quality when evaluating the effectiveness of their compliance programs. These tend to be more difficult to ascertain. Rather than, say, tally the number of employees who complete an ethics survey, a company might review surveys where employees state whether they feel that their CEO or immediate supervisors act honestly and ethically, or whether they've felt pressure to cut corners in order to achieve results.
Who Measures?
“Measurement campaigns” typically involve several groups. Obviously the compliance department plays a leading role. At the same time, compliance experts within business units increasingly play a supporting role in measuring effectiveness. “Where compliance sits in the business, that's where measurement of the risk has to happen,” Kipp says.
The internal audit department often is involved as well, Banks says—but while auditors can offer strong analysis and examination skills, many need additional training. For instance, if a contract is recorded as an asset, confirming the numbers is not enough. The validity of the document itself also should be confirmed, he adds.
Technology
Along with the appropriate experts, technology has a role to play in evaluating most compliance programs. That's particularly true as companies get larger, and their compliance efforts cover larger numbers of employees, spread across more regions of the world. In addition, as regulatory enforcement intensifies, companies will face more pressure to track program effectiveness accurately and thoroughly. “Technology is clearly going to play an increasingly important role,” Biskup says.
For instance, some companies use software that can quickly review prices of their products. With this information, they can more easily identify trends that might indicate a violation of anti-trust regulations, Banks says.
Similarly, some tools incorporate predictive analytics to identify parts of the business that may be at greater risk of non-compliance, Kipp says. For instance, the application may analyze the relationships among a location's staffing ratio, hotline calls, and lost work days. “Put together, these give an indication of where you can focus your efforts,” she says.
At Lincoln Financial, Avnet is evaluating several technology solutions with an eye toward achieving “a consistent, cohesive platform,” he says. In the past, many companies would implement a range of point solutions that could separately drill into different areas of their business. In contrast, the goal today is “to create a unified view of the world.”
At the same time, Lincoln also needs systems tailored to specific business units. For instance, within Lincoln's broker-dealer business, transaction data can be analyzed to determine if, for instance, a particular trader seems to be running into problems that warrant a closer look.
Of course, Avnet notes that his department can't simply wait for new systems to get up and running. Instead, he and his team have created ways to get at the data they need, albeit in a more manual and paper-intensive process. Lincoln has both real time reporting mechanisms in place, which Avnet supplements with reports that cover longer time periods and provide insight into trends and any emerging patterns.
Keys to Success
When measuring the effectiveness of a compliance program, approaching the exercise like it's a law school course—say, trying to determine how many employees know the penalty for violating the Foreign Corrupt Practices Act—is not helpful. “The goal is to have people generally committed to behaving ethically,” Banks says. “You're trying to convey values versus rules.”
At the same time, some employees need to know specific rules, Banks adds. If they're going to operate a machine, they need to know the proper safety procedures; if they're going to sell a financial product, they need to know what sorts of claims they can make.
Finally, it's better to start small and acquire quality data, rather than trying to work with inconsistent information, Avnet says. “Measure what you can measure well. Start there and grow the platform.”
No comments yet