Payoff dominance vs. cognitive transparency in decision making.

Irwin, Julie R. ; McClelland, Gary H. ; McKee, Michael 等

I. INTRODUCTION

There is a general question about how well laboratory markets work in eliciting true preferences given the low opportunity costs of making errors as in Harrison [1989]. Vernon Smith [1982] set out a series of conditions that must be adhered to if a laboratory market investigation is to provide insight into economic behavior in naturally occurring markets. The opportunity cost question, posed above, is covered under Smith's "payoff dominance" and "reward saliency" precepts for laboratory market experiments. If the experimental setting is to elicit true preferences from the subjects the monetary payoffs in the experiment must be salient and dominate potential extraneous influences on subject behavior. This paper reports on three series of experiments which employ the Becker-DeGroot-Marshak (BDM) mechanism to investigate the interaction between payoff dominance/reward saliency and cognitive effort in the decision task. The BDM mechanism is well suited to this task because it involves only individual choice, is theoretically incentive compatible, and offers a high degree of experimenter control over payoffs, initial information, and feedback on the results of decisions.

Becker, DeGroot, and Marschak [1964] introduced the following preference-elicitation mechanism: individuals state the most they would be willing to pay (WTP)(1) for a good. Then, a random price is drawn from a distribution known to the individuals. If the stated value is greater than or equal to the random price, the individual buys the good at the randomly-determined price. Otherwise the individual does not buy the good.

The BDM bidding mechanism can be shown to be incentive-compatible. The intuition is straightforward. It is not in the individual's interest to understate true WTP; if the random buying price falls between the stated WTP and the true WTP, the individual has foregone a beneficial trade. It is also not in an individual's interest to overstate true WTP; if the random buying price is greater than the true value but less than the stated value, the individual will be required to buy the good at a price greater than true WTP.

A more formal proof is easy to construct:(2) Let

V = the true value of one unit of the auction good;

B = the bid submitted to purchase one unit of the good;

R = the price, randomly determined, to be paid if the good is purchased;

p(R) = probability that price R is randomly selected;

[Y.sup.o] = initial income;

U(Y) = utility, a function of money income and the value of the good; and

E = initial monetary balance provided in an experiment.

In the WTP-version of the BDM auction, the good is purchased if B [greater than or equal to] R (with the subject paying only R) and not purchased if B [less than] R. In experiments using the BDM the subject cannot bid more than the experimental balance E so it must be that V [less than] E if the experiment is to be demand revealing. Assuming that subjects maximize expected utility, they will submit bid B that maximizes

EU = [integral of] p (R)U([Y.sup.o] + E + V - R)dR between limits B and 0 + [integral of] p(R)U([Y.sup.o] + E)dR between limits E and B.

The first integral describes the expected payoff for random prices R below the bid B and the second integral describes the expected payoff for random prices between the bid B and the maximum possible bid as limited by the balance E. The maximum over B occurs when the derivative of EU with respect to B is zero; in other words, when

dEU/dB = p(B)[U(Y + E + V - B) - u(Y + E)] = 0

So long as p(B) [greater than] 0 (i.e., there is some probability the offered bid will be the price), the above implies that the maximum occurs when B = V. In other words, it is individually optimal to submit a bid B equal to one's true value V for the commodity. Although the BDM is incentive-compatible in theory, it is an empirical question whether it has this property in practice.

The first experiment reported tests the incentive-compatibility properties of the BDM in a pure induced-value setting. The results from this experiment demonstrate that the BDM is, in fact, incentive-compatible. The second experiment tests the performance of the BDM under different information regimes and payoff schedules. The results suggest that steep payoff schedules are unnecessary to induce optimizing behavior when the subject is able to deduce the optimal strategy from the initial information provided. The BDM would appear to be a transparent decision task and steep payoff schedules have no effect on behavior here. The third experiment tests whether subjects require steep payoff schedules when they must search for, rather than compute, an optimal strategy. The results suggest that steepness of the payoff schedule can make a difference when search is required. We note that, to be useful, feedback must provide the subject with the link between the decision and the payoff and must be provided before the next decision is made.

The following general methods apply to all three experiments reported below. All experiments were conducted in the Laboratory for Economics and Psychology (LEAP) at the University of Colorado, Boulder. The subjects were volunteers from undergraduate economics classes. Subjects sat at individual cubicles containing a display screen and a computer keyboard, and they received both written and verbal instructions.(3) Subjects were permitted to ask questions of the experimenter but they were not allowed to communicate with each other. When recruited, subjects were assured they should expect to earn a minimum of $5 for their participation. In practice, earnings averaged more than $15.00 for sessions lasting less than one hour and the minimum earnings level was non-binding for all subjects.

II. A TEST OF THE BDM MECHANISM: THE INDUCED-VALUE SETTING

The first experiment, Experiment I, is designed to provide a test of the reliability of the BDM mechanism in the form in which it is typically applied. In Experiment I subjects in the WTP setting submit bids for the purchase of a ticket with a known and certain redemption value. In the WTA setting subjects are given a ticket with a known and certain redemption value and they are invited to submit offers to sell the ticket.(4) The random buying price is drawn from a uniform distribution and the subjects are told the shape of this distribution. The empirical question to be addressed is whether the subjects state their true WTP (WTA) values in this setting.

Forty-six subjects participated in Experiment I. There were two between-subject treatments: WTP and WTA. Subjects in the WTP treatment received an initial balance of $10 and were asked to state their bid for the ticket. Subjects in the WTA treatment were given a ticket and asked to state their selling price. There were five practice rounds. After the practice rounds, balances were restored to the initial $10 or $0. The actual experiment consisted of 15 real rounds, although this number was not announced to the subjects to avoid possible end-period effects.

In each round, the computer displayed on each subject's screen a "ticket" value, randomly selected by the computer from the range $0.25 to $5.50 in $0.25 increments. Subjects in the WTP treatment then entered on their keyboards the most they would be willing to pay for their tickets. After all subjects had entered bids, a price was chosen randomly from a cage of 24 bingo balls. Prices ranged from $0 to $6.00, in a uniform distribution with $0.25 increments.(5) The uniform distribution induces a quadratic expected payoff function. All subjects who bid above or equal to the chosen price bought the ticket and the random price was deducted from their balances. The ticket was then redeemed for its displayed value, and this value was added to individual balances. Subjects who bid below the chosen price did not buy the ticket and their balances remained unchanged. All these transactions appeared on the subjects' screens. After viewing the transaction information, subjects pressed a key to move on to the next round.

Subjects in the WTA treatment entered on their keyboards the least they would be willing to accept to relinquish the ticket. After all subjects had entered offers, the price was randomly chosen from the same distribution of bingo balls as that used in the WTP treatment. Those subjects who offered to sell their tickets for a price above the random price kept their tickets. Tickets were then redeemed for their displayed values, and these values were added to balances of those subjects who kept their tickets. For those subjects whose offers to sell were equal to or below the random price, the random BDM price of the ticket was added to their balances in exchange for their tickets.

Results

The analyses reported throughout this paper include only the real rounds of the experiment. Figures 1 and 2 show the distribution of bid-ticket price differences for the WTP and WTA treatments, respectively. Prices were distributed in 25[cents] increments, so a bid-ticket difference between -25[cents] and $0 is individually optimal for the WTP treatment and a bid-ticket difference between $0 and +25[cents] is individually optimal for the WTA treatment. For both treatments, the majority of bid-ticket differences are in the optimal range. In the WTP treatment, 62% of the bids are optimal; in the WTA treatment 67% of the bids are optimal. The mean differences between bid and ticket value for each subject across all fifteen real rounds(6) did not differ significantly from zero (t(27) = 1.21, not significant (n.s.), for WTP and t(19) = .46, n.s., for WTA) There was no difference in mean bid-ticket difference due to WTA versus WTP treatments (t(44) = -0.22, n.s.). Finally, there also was no difference in percentage of optimal bids between the two treatments (t(45) = .47, n.s.).

If subjects were attempting to be strategic, one would expect underbidding for WTP and overbidding for WTA. However, non-optimal bids tended to be overbids in both treatments (66% for WTP and 60% for WTA).

The mean difference between actual and optimal bids (or offers) provides a measure of aggregate bias within an experimental treatment, but it does not assess individual accuracy because it allows overbids and underbids to cancel. The mean absolute difference, plotted in Figure 3 across rounds, does provide a measure of bidding accuracy by comparing the theoretically predicted bidding behavior with the actual behavior. There was no difference between the mean absolute differences for WTP and WTA (t(45) = -0.18, n.s.). However, WTP accuracy improved over time (an average $0.033 accuracy improvement per round) whereas WTA accuracy remained unchanged on average across the rounds (the test of this interaction is t(44) = 3.34, p [less than] .002).

It is interesting to compare our results with the body of literature reporting large differences between stated WTA and WTP values. In contrast to the many studies that have reported large disparities between responses in WTA and WTP conditions (e.g., Gregory [1986]; Kahneman, Knetsch, and Thaler [1990]; Knetsch [1989]; and McClelland and Schulze [1991]), there were essentially no differences between the WTA and WTP conditions in this experiment. Shogren et al. [1994], examined the effect of substitutability between goods as an explanation of the disparity and also finds no difference between WTA and WTP when substitution possibilities exist.

The mean differences between actual and optimal bids in our experiments did not differ between the two conditions and the proportion of optimal bids was about the same in both conditions. The only difference was a tendency for WTP bids to start higher and then improve with experience while WTA offers started closer to optimal and then did not improve. If studies using a BDM mechanism find a large disparity between WTA and WTP responses, then it is not reasonable to suspect that the BDM mechanism itself caused the disparity.

In summary, the results of Experiment I demonstrate that in this simple setting, the BDM mechanism is indeed incentive-compatible. Even for the modest amounts of money offered in this experiment and for the modest expected penalties for deviating from the theoretically predicted individually optimal behavior, most subjects submitted bids that were at or near the optimal bid. The question, addressed in the next experiment, is whether subjects are sensitive to changes in the magnitude of the expected penalties for deviating from optimal behavior.

III. DECISION COSTS, DECISION REWARDS, AND DECISION ERRORS

As a result of Harrison's [1989] critique of experimental findings in sealed bid auctions, a number of researchers have addressed the role of monetary payoffs in economic decision making. Smith and Walker [1993] explicitly introduce decision costs and rewards into the decision task. They draw a distinction between tasks in which the subject is able to identify the optimal decision ex ante the decision and tasks in which the subject must search for an optimum and is informed of success or failure ex post. In the latter, the payoff function must be steep and the feedback timely if the subject is to reach an optimum decision quickly. In the former, the payoff function need not be steep since the subject is able to analytically derive the optimum decision from the information provided by the experimental setting.

As shown above, the BDM mechanism is theoretically and empirically incentive-compatible. In the complete information setting it also appears to confront the subject with a simple task. For example, in the WTP setting the subject merely chooses a bid against a random buying price. This is a simple game against nature and the subject knows the entire distribution of the moves available to nature. The BDM mechanism, as generally applied, is a simple task in which the subject's optimal decision is readily apparent: bid true WTP or ask true WTA. However, the mechanism may be easily confounded by the experimenter so that the subject is faced with decision settings in which the task is made more opaque than in the basic BDM setting. Further, it is possible to manipulate the payoff schedule to make it steeper or flatter and in this way control the costs arising from failing to make optimal decisions. Thus, the BDM mechanism provides a useful means of investigating the role of payoff structure and task transparency in the decision-making behavior of economic agents.

The cost of failing to bid exactly one's true value in typical laboratory settings may be very small and it is possible that bidding behavior may become sloppy. If this occurs, the experimenter has lost control of the experimental setting due to the failure to maintain payoff dominance (Smith [1982, 1991]). Payoff dominance requires that the payoffs to decision-making dominate the cognitive costs of decision-making. Without accurate estimates of decision-making costs, it is necessary to test whether mechanisms such as the BDM are incentive compatible under different payoff conditions. The experiment reported in the next section addresses the effect of payoff steepness.

IV. PAYOFF DOMINANCE AND BIDDING BEHAVIOR IN THE BDM

Experiment II directly investigates the issue of payoff dominance by comparing the behavioral effects of two different distributions of the random buying prices in the BDM mechanism. These distributions produce either a strong or weak penalty, relative to expected value, for non-individually optimal bidding. Formally, note that the slope of the payoff function in utility terms is

p(B)[U(Y + E + V - B) - U(Y + E)];

this reduces, in dollar terms, to

p(R)[V - B]

for risk neutral subjects. Thus, by altering p(R), the probability density function of randomly chosen prices, we can alter the shape of the payoff function and test Harrison's [1989] hypothesis that subjects should perform worse for treatments with relatively fiat payoff functions than for treatments with relatively peaked payoff functions.

A decision task is cognitively transparent when subjects readily understand their optimal strategy from the instructions alone. In such a setting, the non-satiation axiom states that the subjects will make optimal decisions even when the costs of deviating from that strategy are extremely small. Our maintained hypothesis is that the BDM is so transparent and involves so little cognitive effort that subjects will bid optimally, even when the rewards for doing so are small. In contrast, bids ought to be less optimal and payoff structure ought to be more influential if the bidding mechanism were more opaque. In opaque situations subjects must rely on feedback information to search, across repeated decisions, for the optimal bid. In the current experiment, we introduce an opaque version of the BDM which is structurally identical to the basic BDM setting, but about which the subject has little information. The transparency vs. opaqueness treatment is operationalized by giving subjects either full or minimal information about the bidding mechanism. Following Smith and Walker [1993] we predict that payoff structure will be important in the minimal-information, or opaque, condition but not in the regular full-information, or transparent, version of the BDM.

Method

Eighty-seven subjects participated in this experiment. A 2 x 2 factorial design was employed, with two between subjects treatments; type of payoff structure (flat vs. peaked) and amount of information concerning the mechanism (full vs. minimal). All conditions employed the WTP context. Subjects were randomly assigned to each of the four conditions and were each given an initial balance of $10. In each round of the experiment, subjects submitted a bid on the computer terminal to buy a ticket that was redeemable at the end of the round for $3. After all the subjects had submitted a bid for the round, the price of the ticket for that round was determined using the BDM mechanism.

Subjects in the full information conditions were told the value of the ticket ($3), the probability distribution of prices, the actual price of the ticket (as determined by the BDM mechanism), and whether they bought the ticket or not. Subjects in the minimal information conditions were only told that there was an optimal bid between $0.00 and $6.00 and that their task was to find that bid. They were not told that they were bidding on a ticket nor were they told the value of the ticket on which they were bidding. They were not told about the price determination process or the price. They were told only their initial balances, their payoffs and their ending balances for every round. Thus, the outcome feedback about payoff and changes in balances received by the subjects in both the full and minimal information conditions was identical. But all other information was suppressed in the minimal information setting.

Figure 4 shows the distribution of random prices used to generate the two expected payoff functions shown in Figure 5. With the flat payoff structure, which has the majority of prices either much below or much above the optimal, subjects can be sloppy or careless in their bidding behavior and expect to incur only a small penalty so long as their bids are not at the very extremes. That is, as long as their bids are in the general neighborhood of the optimal bid, it will be unlikely that the random BDM price will be between the actual and the optimal bid. In contrast, with the peaked payoff structure, which has a majority of its prices near the optimal, the expected penalty for sloppy bidding, especially in the neighborhood of the true value, is much larger. That is, it is much more likely than the random BDM price will be between the actual bid and the optimal.

As in the baseline Experiment I, each session began with five practice rounds. After the practice rounds, the subjects' balances were reset to the initial amount and all subjects participated in 13 real rounds, but this number was not announced.

Results

Figure 6 displays the mean absolute difference of subjects' bids from the optimal (i.e., [absolute value of B - 3]). The mean absolute difference was larger in the minimal information condition than in the full information condition (t(86) = 5.36, p [less than] .0001). Further, those subjects in the full information condition became more accurate by about $0.025 per round, while those in the minimal information condition became less accurate by about $0.02 per round (t(86) = 3.09, p [less than] .003).

On average, across both information treatments there was no effect due to the payoff structure (t(86) = 0.21, n.s.). However, of more interest is whether there is an interaction between the payoff and the information treatments. Specifically, do payoff structures make a difference when subjects have minimal information, but not when they have full information? As is clear in Figure 7, there is no evidence that payoff structures were differentially effective within the two information conditions (t(86) = 0.40, n.s.).

In summary, the results of this experiment are quite clear: payoff structure had no effect on bidding behavior in the BDM. When the task was transparent, as in the typical application of the BDM mechanism, subjects performed well, relative to the theoretically predicted optimal, and improved over time. However, it made no difference which payoff structure they received; the flat payoff structure (flat relative to the quadratic induced by the uniform distribution) and the peaked payoff structure (peaked relative to the quadratic) produced virtually identical performance to each other and to the performance with the quadratic payoff structure of Experiment I. Hence, when the task is transparent, payoff structure is irrelevant. Even when the penalties for sloppiness are trivial, subjects are not more likely to be sloppy in their bidding.

In sharp contrast to the full information treatment, subjects in the minimal information or opaque treatment, with only the payoff feedback, were unable to find the optimal bid, and, over time, their bids actually became less efficient. This suggests that subjects in the full information treatment were not using the outcome feedback on each round as a clue to optimal behavior because the outcome feedback alone was clearly inadequate. Apparently, the randomness in the feedback provided by BDM payoffs defeats any attempt to search systematically for the optimal bid. For example, the same bid of, say, $4 on two different rounds may receive very different outcomes if the BDM prices are, say, $3.50 (for a net loss of $0.50) and $2.50 (for a net gain of $0.50). With such uncertain feedback, systematic search is difficult without good record-keeping over a large number of rounds. In the next experiment, we address whether systematic search might be feasible and whether payoff structure might have an effect if the outcome feedback were less uncertain.

V. SEARCHING FOR THE OPTIMAL BID: SEARCH UNDER CERTAINTY

The failure to find an effect due to the payoff structure for either the full or minimal information treatments might suggest that the payoff structures, although apparently different, were actually functionally equivalent. That is, the differences in expected payoffs may have been too trivial to produce differential performance. We will have more confidence in our arguments concerning payoff dominance and decision costs if we can show that the payoff structures used in Experiment II are functionally different for at least one task. We attempt to do so in this experiment, Experiment III, by using a minimal information condition that has certain feedback. This is accomplished by changing a subject's balance on each round according to the expected value of his or her bid. That is, the amounts in Figure 5 are the actual payoffs in this experiment. This design eliminates the uncertainty in the feedback and makes it easier for subjects to search systematically for the optimal bid. Thus, in the experiment we hold the feedback mechanism fixed and vary the payoff structure as a treatment.

Even though subjects will have no direct information on the shape of the payoff functions, we expect that subjects in the peaked payoff structure condition will find the optimal bid rather quickly, because as one moves away from the optimal bid of $3.00 in either direction the payoff drops rapidly. For those subjects in the flat payoff condition, we expect the feedback to be much less informative because of the very small differences in payoffs for bids within one dollar of the optimal. Finding a difference due to these payoff structures will demonstrate that the payoff structures used in Experiment II are functionally different and that the differences in expected values could have produced differences in the earlier experiments had subjects chosen to use this feedback information.

Method

Twenty-one subjects participated in this experiment. The procedures used in this condition were the same as those used in the second experiment except for the following changes: At the beginning of the experiment, subjects were each given an initial balance of $6. These subjects were given a smaller initial balance than those in Experiment II to equalize the expected earnings of all subjects across all conditions in both Experiments II and III. The instructions were identical to those for the minimal information condition in Experiment II. In particular, subjects were told that there was an optimal bid between $0.00 and $6.00 and that their task was to find that bid. The only change in procedure was that subjects received in each round an amount exactly equal to the expected value of their payoffs given their stated bids.

Results

The results of this experiment do show an effect of payoff structure on accuracy. The mean absolute difference of bids from optimal was higher for the flat payoffs than for the peaked payoffs (t(19) = 5.87, p [less than] .0001). Figure 8 shows that subjects in the peaked payoff structure condition were able to find the optimum rather quickly whereas those in the flat payoff condition had more difficulty with this task. However, there was improvement across rounds even for those in the flat condition which indicates that subjects are sensitive to even small differences of a penny or two when the feedback information is timely and informative.

Experiment III clearly demonstrates that the two payoff structures are functionally different. That these structures did not produce different behavior in Experiment II suggests that subjects in the transparent setting were able to choose the individually optimal bid on the basis of the information provided and did not rely on the shape of the payoff function to guide their decisions. Sufficient motivation was provided even with a very flat payoff schedule; payoff dominance was achieved.

VI. CONCLUSION

Experiment I demonstrates that the BDM mechanism is incentive compatible within the domain of the payments typically offered to laboratory subjects. Average bids (or offers) were not significantly different from optimal, most bids were near the optimal bid, and accuracy improved with experience.

In its basic form the BDM mechanism provides a task which is cognitively simple but the experimenter may make this task more complex and can also manipulate the payoff schedule easily. The results from Experiment II suggest that subjects make optimal decisions even when payoffs are flat if all of the information necessary for the decision has been presented and the task is transparent. Further, when the task is made opaque a steep payoff schedule is not sufficient to induce optimal decision-making behavior. The failure to find a behavioral effect attributable to payoff structure for the minimal information or opaque condition in Experiment II is instructive. With only outcome feedback equivalent to that received in the full information or transparent conditions, subjects were unable to find the optimal bid and were unable to improve with experience.

The two failures to detect effects of payoff structure in Experiment II might be attributable either to lack of statistical power due to small sample sizes or to the difference between the two payoff structures being too small. However, Experiment III eliminates both these alternative explanations by showing a reliable effect of payoff structures when uncertainty was removed from the BDM feedback. That is, in Experiment III when the feedback was made timely and informative, the two payoff structures were sufficiently different to produce behavioral differences. This result not only increases confidence in the results of Experiment II but also suggests that Harrison's concerns about payoff dominance may be important in experimental settings in which the only feasible player strategy is to search for an optimal bid (e.g., Coursey & Mason [1987]) rather than compute an optimal bid from the information provided in the experimental instructions.

Finally, the experiments undertaken here demonstrate that the BDM mechanism is a reliable preference elicitation mechanism and may be useful in other applications. Incentive compatibility is a desirable property for bidding mechanisms used both in real-word market institutions and in decision-making experiments in economics and psychology. However, theoretical proofs notwithstanding, incentive compatibility must always remain an empirical issue. The cognitive complexity of understanding the bidding mechanism may be an important determinant of whether it is incentive compatible in practice. Also, cognitive complexity may be important in determining whether payoff dominance is a key issue. For the very transparent BDM, payoff dominance does not appear to be an issue; however, Smith and Walker [1993] and Irwin, McClelland, and Schulze [1992] have shown that bidding performance improves with increased experimental incentives for a variety of market-like auction methods, all of which are likely more opaque to subjects than the BDM.

An ideal bidding mechanism should be incentive compatible but also cognitively transparent; further, it should be neutral in the sense of not providing unintentional feedback about what the subject ought to be doing. The results of these experiments suggest that the BDM bidding mechanism satisfies both these criteria.

We would like to thank the Sloan Foundation and the Russell Sage Foundation for financial support for this research. We received helpful comments on this paper from William Neilson and an anonymous referee. All opinions expressed herein and any errors are, of course, the sole responsibility of the authors. Address correspondence to Michael McKee, Department of Economics, University of New Mexico, Albuquerque, NM 87131.

1. The version of the BDM for willingness to accept (WTA) compensation to give up or forego a good is easily obtained by substituting "sell" for "buy" and "less than" for "greater than" in the description in the text.

2. The proof for the WTA case is analogous.

3. The instructions are available on request.

4. The BDM mechanism has been criticized for its reliance on the Independence Axiom of Expected Utility Theory (see Holt [1986] and Keller et al. [1993]). Most applications of the BDM mechanism place subjects in the position of purchasing lottery tickets with obviously uncertain payoffs. That application of the BDM mechanism represents a compound lottery and is inappropriate for testing expected utility theory. In the current application, it is only the preference revelation facet of the BDM mechanism that is being evaluated. There is no risk associated with the good being purchased thus subjects do not face a compound lottery.

5. Bohm et al. [1997], demonstrate the importance of selecting the range of permissible bid values in relation to the range of valuations held by the subjects. They note that allowing bids significantly outside the range of individual values may compromise the mechanism. In our design that is not a problem since the ranges overlap almost completely.

6. To avoid non-independence problems due to repeated bids, a mean was computed for each subject across the 15 rounds so that in the analysis there was only one data value for each subject.

REFERENCES

Becker, G. M., M. H. DeGroot, and J. Marschak. "Measuring Utility By a Single-response Sequential Method." Behavioral Science, July 1964, 226-32.

Bohm, P., J. Linden, and J. Sonnegard. "Eliciting Reservation Prices: Becker-DeGroot-Marschak Mechanisms vs Markets." Economic Journal, forthcoming 1997.

Coursey, D. L., and C. Mason. "Investigations Concerning the Dynamics of Consumer Behavior in Uncertain Environments." Economic Inquiry, October 1987, 549-54.

Gregory, R. "Interpreting Measures of Economic Loss: Evidence from Contingent Valuation and Experimental Studies." Journal of Environmental Economics and Management, December 1986, 325-37.

Grether, D. M., and C. R. Plott. "Economic Theory of Choice and the Preference Reversal Phenomenon." American Economic Review, June 1979, 623-38.

Harrison, G. W. "Theory and Misbehavior of First-price Auctions." American Economic Review, September 1989, 749-62.

-----. "Theory and Misbehavior of First-price Auctions: Reply." American Economic Review, December 1992, 1,426-43.

-----. "Expected Utility Theory and the Experimentalists." Empirical Economics, 19(2) 1994, 223-53.

Holt, C. "Preference Reversals and the Independence Axiom." American Economic Review, June 1986, 508-15.

Irwin, J. R., G. H. McClelland, and W. D. Schulze. "Hypothetical and Real Consequences in Experimental Auctions for Insurance Against Low Probability Risks." Journal of Behavioral Decision Making, 2(1), 1992, 107-16.

Kahneman, D., J. L. Knetsch, and R. H. Thaler. "Experimental Tests of the Endowment Effect and the Coase Theorem." Journal of Political Economy, December 1990, 1,325-48.

Karni, E., and Z. Safra. "Preference Reversals and the Observability of Preferences by Experimental Methods." Econometrica, May 1987, 675-85.

Keller, L. R., U. Segal, and T. Wang. "The Becker-DeGroot-Marschak Mechanism and Generalized Utility Theories: Theoretical Predictions and Empirical Observations." Theory and Decision, March 1993, 83-97.

Knetsch, J. L. "The Endowment Effect and Evidence of Nonreversible Indifference Curves." American Economic Review, December 1989, 1277-84.

Lichtenstein, S, and P. Slovic. "Reversals of Preference Between Bids and Choices in Gambling Decisions." Journal of Experimental Psychology, July-August 1971, 46-55.

McClelland, G. H., and W. D. Schulze. "The Disparity Between Willingness-to-Pay Versus Willingness-To-Accept as a Framing Effect," in Frontiers of Mathematical Psychology: Essays in Honor of Clyde Coombs, edited by D. R. Brown & J. E. K. Smith. New York: Springer-Verlag, 1991, 166-92.

Shogren, J. et al. "Resolving Differences in Willingness to Pay and Willingness to Accept." American Economic Review, March 1994, 255-70.

Smith, V. L. "Microeconomic Systems as an Experimental Science." American Economic Review, December 1982, 923-55.

-----. "Rational Choice: The Contrast Between Economics and Psychology." Journal of Political Economy, August 1991, 877-97.

Smith, V. L., and J. M. Walker. "Monetary Rewards and Decision Cost in Experimental Economics." Economic Inquiry, April 1993, 245-61.

Irwin: Assistant Professor, Stem School of Business, New York University, New York, Phone 1-212-998-0511 Fax 1-212-995-4006 E-mail irwin@fourps.wharton.upenn.edu

McClelland: Professor, Department of Psychology, University of Colorado, Boulder, Phone 1-303-492-8122 Fax 1-303-492-5580 E-mail gary.mcclelland@colorado.edu

McKee: Associate Professor, Department of Economics University of New Mexico, Albuquerque Phone 1-505-277-1960, Fax 1-505-277-9445 E-mail mckee@unm.edu

Schulze: Professor, Department of Agricultural Economics, Cornell University, Ithaca, Phone 1-607-255-9611 Fax 1-607-255-9984, E-mail wds3@cornell.edu

Norden: Consultant, Lodestone Research, Loveland Phone 1-970-663-1055, Fax 1-970-663-2595 E-mail elizabeth._norden@hp.com