The false promise of "full disclosure".
Hahn, Robert W.
THE ENRON "SCANDAL" has raised several fascinating issues
related to disclosing information and potential conflicts of interest.
For example, an accounting firm that receives consulting fees from a
company it is paid to audit may be less likely to report financial
problems with that company. But contrary to conventional wisdom, simply
not allowing that firm to consult will not necessarily solve the
problem. The accounting firm will still have potential conflicts so long
as it is getting paid by other firms to monitor their performance. In
the past, accounting firms have used professional standards as a way of
helping to ensure their reputation. In addition, they have tried to have
a diverse client base, so the costs of losing one client would not be
overwhelming.
Similarly, academics and pundits receiving monetary compensation
from a company may be more likely to give that company's policy
positions a favorable review. One solution to this problem is to have
these folks come clean by identifying their conflicts of interest.
Unfortunately, as we shall see, this is easier said than done and is
likely to have unintended consequences.
Consider, for example, the problem of conflicts of interest in the
context of funded research and opinions that are disseminated to the
public by journalists, academics, and individuals affiliated with think
tanks. This is a broad topic and one with which I have some personal
experience as a scholar and a consultant to business and government. (1)
My purpose is to evaluate the pros and cons of disclosing potential
"conflicts of interest." "Full disclosure" may be a
laudable goal, but is difficult to define and, therefore, not very
useful. In addition, some disclosure norms imposed by the media are not
likely to be very helpful in promoting useful information for their
audiences, and will likely have unintended adverse consequences. The
problem is not that disclosure is necessarily bad, though it may lead to
bad outcomes in certain situations. Instead, the problem is that too
often, the media and the public use partial disclosure as a substitute
for critical thinking.
The nature of the problem
PAUL KRUGMAN -- ONE of the best known economists in academia --
received $50,000 for serving on an advisory board to Enron. Krugman, of
course, was not alone. For example, Larry Lindsey, President Bush's
chief economic advisor, was reported to have received the same amount.
Defending himself in his New York Times column (January 25, 2002),
Krugman noted that he complied with the Times' conflict of interest
policy. When he agreed to write for the paper, he resigned from the
Enron board. In addition, Krugman noted the potential conflict posed by
his Enron advisory role in a Fortune piece he published three years ago.
Krugman's level of disclosure, however, did not seem to
satisfy Andrew Sullivan -- an excellent journalist. On his website,
andrewsullivan.com (January 25, 2002), Sullivan took Krugman to task for
not noting the amount of money he received. Sullivan noted,
"You'll notice one detail missing from Krugman's apologia -- the amount of money he got. Why won't he mention it? Because
it's the most damning evidence against him." He thinks
"the reading public has a right to know" such information.
Sullivan raises an important question: What does the public have a
right to know about a person's opinion or findings? That is, what
should academics and pundits be required to say about their remarks or
research when presenting it in public?
Sullivan thinks that full disclosure is the key. He made the point
specifically with respect to talking heads: "What this is about is
the enmeshment of some of the pundit class in major corporate money. It
seems to me that an integral part of a journalist's vocation is
independence -- independence from any monetary interests that could even
be perceived as clouding his or her judgment. Disclosure is a must --
and not just when the subject matter comes up a few years down the
line" (January 23, 2002).
Unfortunately, Sullivan's suggestion has serious problems.
There is no obvious place to draw the line on what needs to be
disclosed. In some cases in public life, full disclosure has been
interpreted to go beyond an individual to an individual's
acquaintances or family. For example, when the AOL-Time Warner merger
was approved by the FCC, the impartiality of the current chairman of the
FCC, Michael Powell, was questioned because his father was on the AOL board of directors and owned stock in the company.
Take the Krugman example again. What constitutes all relevant
information for purposes of disclosure? Is it relevant, for example, to
know that Krugman received the Clark Medal -- given every other year to
the "best" economist under 40 or that he has published
numerous pathbreaking books and journal articles? Krugman pointed out in
his January 25 column, in his own defense, that "the compensation
[he] received per day [from Enron] was actually somewhat less than other
companies were paying [him] at the time for speeches on world economic
issues." All of these things are arguably relevant, but some in the
public would really like to know how much credence to put in
Krugman's view on a particular subject. More generally, the public
and interested parties might like to have mechanisms introduced that
would lower the cost of obtaining and evaluating information on a
particular subject.
Firms, non-profits, and individuals have dealt with the problem of
establishing credibility in a number of ways. For example, Consumer
Reports, which evaluates many kinds of consumer products, does not take
money from business for advertising. Indeed, Consumer Reports's
subheading on its website reads: "Our mission since 1936: Test,
Inform, Protect. We accept no ads." The New York Times, the Wall
Street Journal, and many other media outlets place restrictions on what
journalists can do in order to maintain their independence.
The rules governing academicians and think tank types generally
require some form of disclosure. For example, the AEI-Brookings Joint
Center which I direct, requires that authors submitting analyses of
specific regulations do not receive any support from industry for the
regulatory policy under study. We do that in order to preserve our
reputation for impartiality in this area. With regard to other
publications, we ask authors to note sources of financial support. And
research journals are increasingly interested in knowing about sources
of funding for research work.
Universities place restrictions on how professors identify
themselves when doing outside consulting and testimony. Think tanks do
as well. For example, authors are typically required to differentiate
consulting products from university or think tank products by noting the
source of support and not using the institution's logo without
permission.
Many in the press ask for similar kinds of disclosure. When I
receive a call from the press about one of my studies, one of the first
questions asked is who funded it. Indeed, that question often seems to
be more important to the journalist than how I arrived at the results.
That suggests a fundamental problem to me. Many journalists either
don't have the time to, or simply cannot, evaluate the validity of
studies. Instead, they simply take cues from less important aspects,
such as the source of funding or the affiliation of the individual.
Judge Richard Posner, in Public Intellectuals: A Study of Decline
(Harvard University Press, 2001) argues that there is little
accountability for public intellectuals -- most notably, professors --
and that the quality of their public work has declined. A media that
performs "virtually no gate-keeping function" is partially
responsible for an environment with "nobody watching, nobody
keeping score."
Posner attributes the decline in the quality of public intellectual
work to the "growth in the specialization of knowledge." He
states that "the fact that most public intellectuals today are
academics, and thus engaged in public-intellectual work on only a
part-time basis, enables them to exit the public-intellectual market at
a low cost and by doing so has reduced to a trivial level the penalty
for the public intellectual caught selling a defective product."
The problem of assessing quality is not restricted to the press.
Even academic peer review has serious problems, albeit for different
reasons. Peer review has been shown to be an unreliable indicator of a
paper's quality, accuracy, or integrity. A study that examined
several articles from a prominent economics journal found that the
papers almost always contained errors that were not caught by the peer
refereeing process. The errors were sufficiently serious that the
results could not easily be replicated. The authors also found that,
notwithstanding both the general norm that data be available and the
requirement of the National Science Foundation (NSF) that data be
produced on NSF-funded projects, their requests for data were ignored,
denied, or otherwise frustrated in a substantial number of cases. (2)
Moreover, peer review is undermined by the difficulty of actually
procuring the relevant data that are supposedly available.
Finally, peer review cannot necessarily prevent or reveal
dishonesty in academic work. A prominent sociology professor at the
University of Texas recently resigned after acknowledging scientific
misconduct. She had been accused of falsifying data that supported her
research. A historian at Emory University was accused of supporting an
acclaimed book with untrue statistics on gun ownership. And historian
and author Stephen E. Ambrose has been the subject of well-publicized
plagiarism accusations.
The benefits of disclosure -- and the costs
THE BENEFIT OF MORE disclosure is that the media public is given
additional information about possible conflicts of interest. When
disclosure raises a red flag that makes an editor or journalist examine
arguments more closely, this is a benefit. Additionally, disclosure is
also valuable for its potential to deter what Richard Posner calls
"improper and irresponsible moonlighting." More disclosure,
unfortunately, has costs as well. These include difficulties in
monitoring and enforcement, difficulties in defining an appropriate
level of disclosure, and the impact on who provides information and how
they respond to disclosure rules.
Monitoring and enforcement. Some time ago, I was on a radio show
offering my views on the Microsoft case, a firm for which I occasionally
consult. The talk show host correctly identified me as a consultant and
academic, but failed to note that one of the other "experts"
on the show had done a great deal of consulting for another firm on some
key policy issues related to the case. In doing so, she left the
audience with the mistaken impression that my counterpart in the debate
was in some sense "clean" because he had not consulted.
This is an example of providing incomplete information to the
public that tilts the playing field toward the side that is viewed as
clean. I think it is a very serious problem. The problem arises in part
because the disclosure rules are difficult to monitor and in part
because they are not always enforced with the same vigor. Moreover, the
penalties for not disclosing are not that high in most situations.
Take another example. I am aware of many people who write opinion
pieces on a particular subject for direct compensation. Some disclose
that information while many others do not. No one seems to care, except
editors at major newspapers. They are less likely to publish op-eds that
come with some kind of disclosure statement because they do not want to
be viewed as supporting free advertisements for a particular point of
view. This creates an incentive not to disclose. The incentive is just
one factor that determines whether a piece will contain a disclosure
statement. In a study of biomedical articles, 34 percent of the articles
had an author with a financial interest related to the topic of the
article. None of those authors disclosed a conflict of interest. The
study's author explained that even when the publishing journals
have disclosure policies, poor compliance with those policies is
prevalent. (3)
Difficulties in defining an appropriate level of disclosure. The
basic problem is that we are all walking conflicts of interest because
most of us have to work for a living. And in exchange for money, most of
us make compromises. For academics and other professionals, it is not
unusual to work for several companies, either giving speeches or on
other short-term contracts. Over time, it can be easy to develop
connections with businesses in the routine process of making a living.
Not all of these connections pose a conflict of interest. When
considering what to disclose, it is sensible to focus on activities that
could pose a substantial conflict.
Richard Posner suggests a norm in which "academic public
intellectuals disclose their income from all their public intellectual
work." Posner qualifies his position with the caveat that compared
to public officials' disclosure of their income, it is not as
important that academics comply with strict disclosure standards because
they are not as powerful. But he argues that "revelation of the
lucrative character of some of this moonlighting would help the public
to evaluate public intellectual work and would deter some of the most
questionable forms of it."
While some disclosure is justified, it is difficult to know where
to draw the line. How should we deal, for example, with firms that link
pay to visibility in the media? Should I note on the bottom of my op-eds
that a small part of my compensation at AEI and Brookings is related to
the number of op-eds I publish in newspapers? This, indeed, has an
effect on the number of op-eds I write, if not their bias.
In some cases, the linkage between salary and taking a particular
position in the media is more direct. I know several people at so-called
public interest groups who get paid to take particular positions in the
media. I do not doubt that they believe these positions, but why, in
principle, should they be treated any differently from a business
consultant taking money in exchange for writing an op-ed? Yet they are
treated very differently by the media -- with the opinion of the
business consultant being given less credence.
Again, let me return to the Microsoft example to illustrate a
problem with the incompleteness in the definition of disclosure. I wrote
an article for Regulation on the Microsoft case before I became a
consultant for Microsoft. Just prior to the publication of the article,
I was asked by someone representing an opponent of Microsoft if the
article could be withdrawn, and if I would consider a consultancy with
that firm. I said no and, frankly, was offended by the offer.
I use this example because it says lots about the limitations of
actual disclosure policies. You are not asked to disclose clients that
you turn away on principled grounds -- only those for whom you do work.
We might learn more about an individual from how she discriminates among
potential clients if we could observe that, but, unfortunately,
that's difficult to do.
If disclosure policies do not provide much information on whether
an opinion is likely to be biased, we could examine an individual's
incentive to preserve her reputation. Academics have some incentive to
preserve their professional reputation among their colleagues. This can
help to put constraints on their public pronouncements, but may not
solve the problem. Those academics who care less about their reputation
and are more interested in public exposure will be more likely to become
pundits or talking heads. This could decrease the overall quality of
punditry, assuming that were possible.
Let me offer another example from the Joint Center website that
will illustrate some of the problems with "cleanliness" as
perceived by the media. I have signed one statement on liberalizing
spectrum auctions at the Federal Communications Commission and helped
write a Supreme Court brief signed by 39 leading economists on the need
to consider costs, benefits, and other relevant factors in regulatory
decision making. While I did not take any money for either endeavor, I
fully expect that being on both documents will enhance my market
prospects and market value, both as an academician and as a consultant.
So, it would be a bit disingenuous for me to say that I did not have a
direct financial interest in these activities. Frankly, though, that was
not the primary reason I got involved. The main reason I got involved is
because I care deeply about how public policies affect real people
outside the beltway. Should I be required to disclose that?
So, here we have an example of activities that would be viewed as
clean (and therefore, no disclosure is currently required), which give
me substantial monetary and nonmonetary benefits. Should we care more
about my future benefits likely to be derived from an activity than past
payments? I think so, but unfortunately, these benefits are hard to
observe.
The difficulty of observing many features of disclosure taken
together with gaining agreement on a reasonable definition of full
disclosure makes it a difficult goal at best. But I fear that achieving
the goal would actually do more harm than good by reducing the pool of
experts and encouraging people to circumvent the system in ways that do
not aid in the search for "truth."
Impact on who responds and how they respond. Andrew Sullivan has
suggested that an individual who consults for a company should not write
about issues related to that company. He believes that individual's
journalistic independence has been compromised -- no matter how innocent
or transparent the consulting arrangement. "Let's say [Krugman
and Peggy Noonan] just took $50,000 minimum from this company for legit extracurricular work," he wrote. "Haven't these pundits
essentially undermined themselves as independent watchdogs of the
culture?"
Sullivan's position, while extreme, has some empirical
support. When an individual consults for a company, she is more likely
to take on the perspective of that company as a result of continued
interactions with a group of like-minded individuals. Even if a
consultant tries diligently to preserve her impartiality, there is a
likelihood that a company's views will grow on her, and seem more
sensible than they did before the consulting arrangement.
Sullivan's rather extreme policy could reduce such conflicts, but
at the expense of reducing the available pool of experts. Who, after
all, is in a better position to write about a company, or a policy
related to that company, than someone who knows the business firsthand?
Thus, there is a tradeoff between available expertise and the degree of
disclosure required.
Even stopping short of Sullivan's suggestion to disallow any
writing by a consultant, calls for greater disclosure could be
counterproductive. The pool of potential experts on the subject may be
reduced because some individuals will prefer not to disclose and not
participate in the public discussion. Moreover, some may simply evade
the requirements and hope they don't get caught. Still others --
the entrepreneurial types -- will create "fronts" that make
the probability of detection less likely. A front is anything that
obscures the connection between an individual and that person's
sources of support. It could be a business, a non-governmental
organization, or an individual serving as an intermediary.
What kind of fronts might be created? A look around at how the
various think tanks operate can offer some food for thought. Even the
top think tanks, like AEI and Brookings, get much of their money from --
dare I say it? -- business, or foundations whose wealth typically comes
from business. The foundations supporting think tanks run the gamut from
anti-business to pro-business, but a typical foundation will only
provide support if it has a reasonable expectation of the kind of
results that will be produced. And even if think tanks get their money
from government -- read: the taxpayer -- that will create conflicts
because of an interest in pleasing that funder.
The way think tanks deal with potential conflicts is to introduce
mechanisms that help preserve their reputation for doing quality work.
These mechanisms include: hiring scholars who are interested in
preserving their academic reputations, peer-reviewing their major
published works, such as books, and encouraging their scholars to
publish in peer-reviewed journals.
Another important mechanism that think tanks and universities use
to preserve their reputation is to obtain funding from a variety of
sources. Such diversification makes it easier for these institutions to
have their scholars take positions that may be at odds with the views of
their funders. Free trade is a good example. AEI scholars are generally
very supportive of free trade, even though some of AEI's funders
have argued for protectionist policies related to their firm or
industry.
The mechanisms for preserving reputation are not perfect, however.
Scholars may still be subjected to pressures from particular firms on
particular issues. Those places concerned about their academic
reputations tend to be more adept at giving their scholars freer rein.
But the bottom line is that most money, even so-called government money,
comes with some strings related to expected results.
And the competition for funds is fierce, which means there may be
greater emphasis on producing work that increases funding rather than
first-rate scholarship. Still, at the leading think thanks and
universities, I think these mechanisms work reasonably well for
scholarship published in a scholar's area of expertise. Work
published outside of a scholar's area of expertise is another
matter.
Unfortunately, not all think ranks or universities take the same
degree of pride in academic freedom as AEI or Brookings. Posner, for
example, provides an account of the Independent Institute's support
of Microsoft while it was receiving funding from the firm. But if
disclosure requirements were enforced more rigorously, I would expect
more think tanks to emerge that serve as fronts for all sorts of
preferred interest group policies.
The same is true of websites. While I can say that the
AEI-Brookings Joint Center website has nor been influenced one iota by
our funders, I know other websites where that is not the case, and
I'm sure you do too. Moreover, revealing the sources of support
typically provides little, if any, useful information about whether the
work produced or featured by a site represents independent analysis, or
is merely a convenient vehicle for advertising a funder's preferred
policy position. This creates a real problem for people wishing to use
disclosure as a meaningful measure of potential conflicts of interest.
In some cases, disclosure is selective. Posner notes that after the
Valdez oil spill, Exxon paid several academics to write articles on
punitive damages. In the articles, the authors noted that Exxon had paid
them for their work. But when Exxon used the articles as cited sources
in briefs it prepared for its appeal proceedings, it failed to disclose
that it had paid for the articles. Moreover, neither Exxon nor the
commissioned academics would disclose the amount of the received
payment.
The point is that calls for greater disclosure will lead to more
innovative ways to circumvent disclosure and we should keep that in mind
in crafting solutions.
What needs to be done?
CONSIDER DISCLOSURE. A requirement of full disclosure is not
meaningful because it is hard to know how to implement and is likely to
create perverse incentives. The current system of disclosure, for all
its warts, is nor a bad starting point. That system generally requires a
scholar to identify conflicts that would not pass a political
"smell test." That is, if there is a reason to think that an
average reader would be suspicious if a scholar did not disclose
something, then she should disclose it.
The basic problem with the current system of disclosure is that it
is incomplete. The media need to recognize this and do a better job.
Here are five concrete suggestions.
Suggestion 1: Place less reliance on disclosure as a signal.
Disclosure can provide a useful hint about a conflict of interest, but
several other factors should be taken into account. For example, does
the "expert" have a reputation to preserve (e.g., in her field
of expertise)? How do reputable colleagues view her?
Suggestion 2: Apply rules for evaluating experts across the board.
That means doing due diligence on all participants in a debate, not just
those where the conflicts are most obvious. The notion that taking money
from industry should necessarily taint someone is naive. But if it is
going to be treated as a practice that warrants disclosure, the
self-interest of individuals who appear to be clean, such as those from
so-called public interest groups and government, should also be
highlighted.
Suggestion 3: Find out whether the person is really an expert.
I'm an economist -- Ph.D. and all. I can't tell you how many
times I've heard "talking heads" get up there on TV and
radio talk shows and get treated with great respect on the stock market
and forecasting issues, when they actually know next to nothing on the
subject. The press should not give these people a pass, just because
they sound good.
Suggestion 4: The media should think harder. The press needs to be
more critical in an academic sense. There is no substitute for actually
reading some reports to determine their quality. If this skill is in
short supply, as I think it is for many journalists, then leading media
outlets, such as the New York Times, the Wall Street Journal, and CNN,
should hire people to support reporters who can think critically about
technical issues. Some of these skills can be taught and they should be.
Suggestion 5: We all should think harder. One of the things I try
to teach in the classroom is critical thinking. Does the argument the
person is making really hold water? If more people learn to think
critically, this would help.
What's wrong with these recommendations? The media seem to be
happy with the status quo, and for the most part, so is the public. So
there is nothing that is likely to move us in this direction quickly
unless some foundation, or foundations, with serious money decides to
take up the cause.4 One might consider government help, but beyond a
voucher program to stimulate competition and improve quality in
education, I do not see a useful government role.
Posner has suggested that, in addition to disclosing income related
to public intellectual work, academics should provide all of their work
on public intellectual activities in some kind of form that is easily
retrieved, such as posting to a website. He suggests that "one
solution might be for universities to require their faculty members to
post annually, on the university's web page, all the non-academic
writing ... and public speaking that they have done during the preceding
year.... At the end of the year the contents of the web page would be
downloaded and printed out, and copies deposited in the major university
libraries."
Posner's ideas could, of course, be extended to pundits in
general. The question is what good they would do. I'm not so sure.
I am certain that it would decrease the supply of opinion makers on key
public policy issues, but whether it would improve the overall quality
of information is another matter.
Reform?
WE BEGAN WITH A tale of Paul Krugman and a controversy over
disclosure, but I don't think Krugman is the real story here. I
think the real story is that disclosure has serious limitations, there
are lots of major conflicts of interest out there that don't get
reported, and that the press tends to tilt the playing field in ways
that have not been adequately considered.
Full disclosure, far from being a panacea, could make things worse.
My basic suggestions for fixing the problem are a press that thinks more
critically and a public that does the same. Don't hold your breath.
(1.) I rely heavily on personal anecdotes in places to make my
case, in part because I have some inside information that is useful. In
addition, I do not wish to make my colleagues the targets of ad hominem attacks. I would like to thank Christopher C. DeMuth, Harold
Furchtgott-Roth, Thomas W. Hazlett, Scott Hemphill, Charles Jackson,
Clark Judge, Paul Krugman, Robert Litan, Peter Passell, Richard Posner,
and Cass Sunstein for helpful comments, and Mary Beth Muething for
excellent research assistance. This work was supported by the
AEI-Brookings Joint Center for Regulatory Studies. A list of funders can
be found on the Joint Center website: www.aei.brookings.org.
(2.) William G. Dewald, Jerry G. Thursby, and Richard G. Anderson,
American Economic Review 76 (1986).
(3.) See Steven Phillips, "A Conflict that Might Interest the
Ethicists," Times Higher Education Supplement (April 27, 2001).
(4.) Posner notes that "the public-intellectual market does
not appear to exhibit, at least to any marked degree, 'market
failure' in the economic sense" and that the shortcomings of
the market "do not warrant costly methods of correction."
Robert W. Hahn, a resident scholar at the American Enterprise
Institute and a research associate at Harvard University, is director of
the AEI-Brookings Joint Center for Regulatory Studies. The views
expressed in this paper reflect those of the author and do not
necessarily reflect the views of the institutions with which he is
affiliated.