Focus Should Be On Ethics Program Outcomes, Not Process

This article is Part Two of a two-part series on ethics programs. Last week, we spoke with ethics leaders at Sprint and other companies, who discussed the importance of "support from the top" in developing successful ethics programs (see box at right).

In implementing their ethics and compliance programs, too many companies are focusing all or most of their attention on process, and little or none on outcome. That's according to Kenneth Johnson, a well-known author and lecturer on business ethics who currently serves as principal consultant of the Ethics Resource Center in Washington, D.C.

Johnson

Indeed, Johnson blatantly answers “almost none” when asked how many companies are doing effective evaluations of their ethics programs.

“Many folks are focusing on the question of, ‘Have we complied with the minimum steps [in the Sentencing Guidelines]? Do we have a training program, etc.?’ But that’s not the point of having a program,” Johnson recently told Compliance Week. “The point is to prevent and detect wrongdoing. Companies don’t know they did that by conducting training.”

The key, he says, is the outcomes. "Are employees aware of issues?" he asks. "Is there misconduct? Will they report it? Are they satisfied with the advice they get?”

According to Johnson, part of the problem derives from the fact that program evaluation was not even mentioned in the Sentencing Guidelines when they were first enacted in 1991; evaluation is now required under the 2004 guideline revisions. Effective evaluation requires attention to outcomes, argues Johnson, and he therefore understands why more companies don’t do it. “It’s very time-consuming, very expensive.”

But it’s also crucial, he says, noting that the Department of Justice considers the effectiveness of a program in determining whether to prosecute, and that it is also an important consideration if a prosecution gets to the point of sentencing before a judge.

“Both the federal government and the Sentencing Commission have recognized that you need to be able to demonstrate that your program is money well spent,” Johnson said. “If you really want to argue it’s effective to a judge, you have to argue that it achieved outcomes and developed organizational culture. Merely focusing on the fact that you do something, demonstrating effective minimal requirements—for example, by merely having a helpline—doesn’t go far enough. You’re focusing on characteristics of the program instead of the outcomes.”

Process vs. Outcomes-Based Evaluations

According to Johnson, who was the principal author of Business Ethics: A Manual on Managing the Responsible Business Enterprise in Emerging Market Economies, there are two basic approaches to evaluating ethics and compliance programs: process evaluations and outcomes-based evaluations.

Process evaluations—the kind that companies generally do—track which program activities were actually performed and measures their outputs. But according to Johnson, program activities—such as ethics training courses—and output measures—like employee satisfaction with training, or retention of information—“have little value, in and of themselves, in offering a measure of effectiveness,” he says.

The more effective type of evaluation is an outcomes-based one, says Johnson. Outcomes-based evaluations are concerned with the extent to which a program achieves its intended results. “The outcomes it measures are changes in the lives, attitudes and conduct of an organization’s employees, agents and other stakeholders, or changes in the organization as a whole.”

The questions that must be answered in such an evaluation include whether there is less exposure to risk for misconduct; whether employees are willing to seek advice and report concerns; and whether the culture of the organization promotes ethical conduct, and discourages misconduct.

But these are tough questions to ask and answer.

“There are very few [companies] doing evaluations that are truly comprehensive—that are ascertaining their culture,” Johnson acknowledges. “The ones we’ve done have been when someone got in trouble and someone [else] ordered it."

And comprehensive evaluations can't be done with just surveys, says Johnson. "A survey is wonderful," he says. "If a company only did a survey, it would be a start—a great filter for getting a profile of ethical culture. But in terms of being comprehensive, doing a survey is not enough.”

There are, says Johnson, five “classic measurements” for a proper evaluation:

One-on-one interviews;

Focus groups;

Surveys;

Document reviews; and

Observation.

Though listed as the third item in Johnson's list, above, he notes that surveys are often a good starting point. “A survey can help profile your culture, help determine any risks in the organization, tell you how many people are taking the training, and reveals certain outcomes. You do that annually and find out where you have problems, and then drill down. That’s where you do your interviews and focus groups.”

And those interviews are critical. “Without exception, every project I’ve been the lead on, we have interviewed the CEO,” he says.

Although time-consuming and expensive, outcomes-based evaluations are not rocket science, says Johnson. “It’s done in the not-for-profit field all the time.” Rather, there is simply “tremendous resistance to being measured” among some in the ethics and compliance field. “If you’ve got a job that you’re happy with," he adds, "you really would rather not have anyone come in and determine whether you’re really effective.”

But the benefits to a company, if it’s ever dragged to court, are worth the financial cost and aggravation, says Johnson. “Companies are getting there. But very few folks have given too much thought to what the purpose of their program is. They need to do some kind of profiling of the organization and begin tracking that, rather than measuring the process. That’s not the point of the program. The point is not necessarily to have a lot of people calling the hotline.”

Companies Still Learning

Jim Brennan, a former compliance officer for Motorola and senior ethics consultant for the Ethical Leadership Group, notes that ethics and compliance programs are still relatively new, and that they are a particular challenge for smaller companies.

Brennan

“The biggest problem with companies that have deficient ethics programs is that it’s new to them—they haven’t really kicked off a program until something went wrong or a member of the board said, ‘We really need to be more systematic,’” says Brennan, who is currently the chief ethics officer and legal counsel of Midi, Inc., a Princeton, N.J.-based provider of online ethics and compliance training.

That “doesn’t mean [these companies] were acting in bad faith,” he adds. “It just means that they lacked expertise in this area, that they failed to understand that there’s a systematic approach [that can be taken]." And while most large companies are aware of the importance of the U.S. Sentencing Commission's federal sentencing guidelines—and how to translate them into their business activities—Brennan notes that smaller companies "haven’t been able to devote their resources to get a grasp as to what this field is all about.”

Brennan agrees with Johnson that focusing on the process rather than the outcome can be problematic. “Enron had a program," says Brennan. "They had a code of ethics. And they had other processes in place. But ethics didn’t permeate the culture sufficient to alter the trajectory of the company.”

While Brennan notes that getting at these "softer issues" can be very expensive, he believes most companies are headed in the right direction. “From my experience, most companies can say they have made a good faith effort to create an effective program," he says. "Have I seen companies that are pretty deficient? Sure. Have I seen some that are trying to improve? Yeah. Are there some that seem to be going through the paces? I believe that there are.”

Brennan believes that much more attention is being paid to effectiveness. “In the past, you saw companies that weren’t really sure what this whole area needs," he says. "If they felt they needed training, they would cobble together some content and send it out." But as the field has evolved, Brennan believes there is a growing recognition that it’s not sufficient to send out a Microsoft PowerPoint presentation without concern about how that content is received. "Companies are paying much more attention to making sure the message is successfully received," he adds. "But there’s a long way to go still. The area is still so young—companies are still learning.”

A number of related resources—including recent coverage, information on the federal sentencing guidelines, guest columns, Q&As, ethics codes, and related commentary and guidance—can be found in the box above, right.