Dubious and dubiouser: contingent valuation and the time of day.

Dickinson, David L. ; Whitehead, John C.

I. INTRODUCTION

The contingent valuation method (CVM) is a stated preference approach to the valuation of public goods (Carson 2012a, 2012b). Preferences are "stated" in the sense that survey respondents are asked hypothetical behavior questions that can be used to reveal their home-grown valuations. The CVM may be useful for estimating benefits and costs involving changes in unpriced, nonmarket goods and services for which no revealed preference data exist. Economists traditionally assume that tastes are constant (Stigler and Becker 1977). While many models in economics allow for preference changes, CVM researchers typically either (implicitly) assume stable and unchanging preferences over the course of a day, or they simply ignore how time-of-day may impact willingness to pay. The issue of time-of-day effects on CVM data is the focus of this article.

The Exxon Valdez oil spill focused attention on the CVM and the 1990s was an active period of research that became known as the "CVM debate" (Portney 1994). The high point of the CVM debate may have been Diamond and Hausman's (1994) declaration that they prefer no (willingness to pay) number to some number (developed using the CVM) for policy analysis. During the most recent decade, those interested in CVM research were able to go about their business without much grief from the rest of the economics profession. But now the recent BP Deepwater Horizon oil spill has reawakened the CVM debate within the economics profession (Kling, Phaneuf, and Zhao 2012). The highlight of this reawakening may be Hausman's (2012) declaration that his opinion has gone from "dubious to hopeless" about the ability of the CVM to provide any useful information. (1)

While researchers have considered issues related to time in contingent valuation (e.g., Propper 1990; Whittington et al. 1992; Whitehead and Hoban 1999; Berrens et al. 2002), Diamond and Hausman (1994) and Hausman (2012) will not even give the CVM "the time of day." Therefore, not totally inexplicably, we consider time-of-day effects by requiring survey respondents to answer a willingness to pay question over the entire 24-hour cycle. A guiding principle, consistent with the work of the CVM critics, is that a hypothetical valuation question, especially one used for major policy issues and natural resource damage assessments, should pass the most important theoretical validity test no matter what time of day the survey is administered.

II. THEORY

As the CVM is based on responses to hypothetical valuation questions there have been concerns about the validity of value estimates. Validity is the extent to which a valuation method generates a measure that is unbiased. Theoretical validity is the extent to which a valuation measure changes in response to the changes in conditions under which it is evaluated. Evidence from previous research suggests that willingness to pay estimates may be theoretically valid in certain situations (Carson 2012a, 2012b).

To develop the most basic test of the theoretical validity of the CVM, consider indirect utility, v(p,q,y), where p is a vector of market prices, q is an unpriced public good, and y is income. Willingness to pay for a change in the public good, q >q, can be defined implicitly as v(p, q',y- WTP) = v(p, q, y). Next, consider a dichotomous choice valuation question where respondents are asked whether they would be willing to pay a randomly assigned fee, A, for the change in the public good. The respondent must compare utilities in two situations, with the change in the public good and fee and without: v(p,q',y - A) >/< v (p,q,y). Rearranging terms, the difference in utility caused by the proposal is a function of the assigned fee, [DELTA]v = v(p,q',y - A) - v(p,q,y) >/< 0. When survey respondents are faced with a referendum vote the probability that they will vote for the proposal is the probability that the change in utility is greater than zero, Pr(for) = Pr[[DELTA] v(A)>0]. The probability that a respondent is willing to pay the fee is decreasing in the fee as [partial derivative][DELTA]v/[partial derivative]A < 0.

A. Data

Our data are from an online survey. Names from campus email lists were randomly assigned to a 1-hour response window and subjects were allowed to start and complete the survey once within the assigned response window on Tuesday, Wednesday, or Thursday of the survey week (Dickinson and McElroy 2010). All 24 hours of the day were utilized, and there was an equal chance that any given subject might be assigned to any of the 1-hour response windows. Recruitment emails highlighted the incentives, the survey link, and the randomly assigned time response window. (2) For this article, we focus on subject responses to the following CVM referendum question:

The Renewable Energy Initiative (REI; http://www.rei.appstate.edu) was created by motivated students in an attempt to reduce ASU's carbon footprint. The REI is charged with bringing renewable energy projects to the campus of Appalachian State. Each semester, every student at ASU pays $5 from their student fees into a fund controlled by the REI. One project that the REI is currently considering is the purchase of a medium scale wind turbine. ASU has offered land behind the Broyhill Inn for installation. However, funds for purchase, installation and operation of the wind turbine are unavailable. It is estimated that an additional $A from each ASU student each year would be needed for the purchase, installation and operation of the wind turbine. If this issue was presented as a referendum during the next student election, would you vote for or against the increase in student fees?

* For

* Against

* Don 7 know

* I need more information

The student fee, $A = 4, 8, 16, 24, 40, or 56, was randomly assigned across survey treatments. (3)

A total of 683 subjects completed the survey, with an average 28.5 responses per each 1 -hour time slot of the day. We discard 92 respondents who completed the survey at a time other than their assigned time and 67 nonstudents who would not be bound by the payment vehicle. We use a final data set of 524 subjects. (4) Fifty-seven percent of the sample is female, which is similar to the population of students at our institution (55% female). The average age is 24 (range 18-47) years. Sixty-one percent of respondents voted for the increase in student fees. A total of 11% voted against, 5% did not know, and 23% needed more information. The average student fee is $27.

Time of day is split into four blocks associated with higher versus lower alertness ratings as established in the literature (Dickinson and McElroy 2010). We call these four time blocks as night, morning, afternoon, and evening. Night is between 12:00 am and 6:00 a.m., morning is between 6 a.m. and 12p.m.. afternoon is between 12 p.m. and6 p.m., and evening is between 6 p.m. and 12 p.m. Twenty-four percent of the sample took the survey during the night (n= 128), 23% in the morning (n = 119), 30% in the afternoon (n = 155), and 23% in the evening (n = 122).

III. RESULTS

Considering first the frequency of votes in the student referendum, responses from the night time block were 63% "for" at $4 and 61% "for" at a fee of $56 (Table 1). The "for" votes are highest at fees of $8 and $16 and lowest at fees of $24 and $40. The chi-square statistic indicates that variation in "for" votes is not statistically different across student fees. Results are more typical during the morning time block: all seven respondents vote for the proposal at the $4 fee and only 42% vote yes at the $56 fee, but the chi-square statistic only reveals price variation is statistically significant at the p =. 11 level. The afternoon and evening votes reveal statistically significant variation in the "for" votes at the p~.05 level in the predicted direction. The CVM passes this key validity test only in the afternoon and evening time blocks.

We next estimate the determinants of "for" votes and willingness to pay using the censored logistic regression model (Cameron 1988). During morning, afternoon, and evening time blocks, students vote rationally with "for" votes declining as the student fee rises (Table 2). The model chisquare statistic is significant in the morning, afternoon, and evening time block models but is not statistically significant in the night-time model. During the night time, students are completely insensitive to the student fee, at least in the standard way of thinking; the student fee has no statistically significant effect on "for" votes. (5,6)

Willingness to pay for the renewable energy proposal is $52 during the night time, $38 in the morning, $39 in the afternoon, and $23 in the evening. (7) Given the width of the confidence intervals the willingness to pay estimates are not significantly different from each other. However, the point estimate of the night-time willingness to pay is economically different from the other time periods. For example, with 17,000 students at Appalachian State University, the aggregate willingness to pay during the night time is $884,000 compared to $391,000 during the evening. Nighttime contingent valuation may lead to too many environmental policy proposals passing the benefit cost test.

IV. CONCLUSIONS

In this article, we find that survey respondents do not pass a key validity test during the time of day when most experience the deep circadian trough of sleepiness (i.e., night). Interpreting our results in the least charitable way calls into question the validity of the CVM. Hausman (2012), in the context of another test of validity (i.e., scope) suggests that "as contingent valuation surveys are typically pretested, the survey design can be manipulated to ensure that at least minimal scope effects are present." The same can be said for price effects. Perhaps CVM surveys have been manipulated so that survey respondents participate during times of the day, such as morning, afternoon, and evening, when at least minimal price effects are present.

These results have implications for contingent valuation survey research. Mail, in-person, and phone are the traditional survey modes. Of these, in-person and phone surveys are typically conducted during morning, afternoon, and evening hours. But mail survey respondents are able to complete the questionnaire at any time. Therefore, mail survey respondents could have been answering questions during the night for decades! More recently, online surveys have become ubiquitous. Researchers should implore mail and internet survey respondents with explicit warnings to complete the survey during the morning, afternoon, and evening hours. Given the ability to time stamp survey taking with online survey administration, at the least, time-of-day effects should be routinely reported by CVM researchers. Diligent referees should require such reporting or just assume the worst from the data.

At the risk of encouraging more contingent valuation research, these results suggest future investigation in the area of sleep and contingent valuation. In our survey, the subjects were not forced to respond during the night if that was their assigned time slot. So, our sample includes some self-selection. (8) Presumably those subjects who are less bothered by the prospect of survey-taking during the night are the ones in the sample. This likely reflects the natural mix of individuals who may be awake during the night: some who are awake by voluntary choice (e.g., students who study rather than sleep, the undead, etc.), and others awake for less than completely voluntary reasons (new parents, procrastinating students, the authors as we try to revise this article before it is considered a new submission). Future time of day and CVM research should consider these selection effects.

The night-time bias we find may be a more general reflection of CVM response patterns when cognitive resources are depleted. In other words, when cognitive resources are depleted for any reason, perhaps then CVM responses are insensitive to prices, the scope of the good and willingness to accept diverges from willingness to pay. Future research should explore whether there is cause for concern in CVM studies that might include other forms of cognitively depleted respondents in their data. Examples might include subjects who are multitasking during survey response (thus, cognitive resources are being diverted by other tasks), or whether it has been several hours since the last meal suggesting low blood sugar and limited glucose availability to fuel brain function. Evaluating the validity of CVM response data just got a lot more complicated.

Finally, time-of-day effects should also be explored in other areas of economic research. For example, night-time laboratory and field experiments might also lead to questions about their validity, and time-of-day effects may be important in studies of subjective well-being. Survey respondents may be willing to take a survey during the night time but they may not be at all happy about it. While this research may raise other concerns or ideas for future research, further mention is beyond the scope of this article. In other words, we have decided to just sleep on it.

ABBREVIATIONS

CVM: Contingent Valuation Method

REI: Renewable Energy Initiative

doi: 10.1111/ecin.12161

REFERENCES

Berrens, R. P., H. Jenkins-Smith, A. K. Bohara, and C. L. Silva. "Further Investigation of Voluntary Contribution Contingent Valuation: Fair Share, Time of Contribution, and Respondent Uncertainty." Journal of Environmental Economics and Management, 44(1), 2002, 144-68.

Cameron, T. A. "A New Paradigm for Valuing Non-market Goods Using Referendum Data: Maximum Likelihood Estimation by Censored Logistic Regression." Journal of Environmental Economics and Management, 15(3), 1988, 355-79.

--. "Interval Estimates of Non-market Resource Values from Referendum Contingent Valuation Surveys." Land Economics, 67(4), 1991, 413-21.

Carson, R. T. Contingent Valuation: A Comprehensive Bibliography and History. Cheltenham, UK: Edward Elgar Publishing, 2012a.

--. "Contingent Valuation: A Practical Alternative When Prices Aren't Available." Journal of Economic Perspectives, 26(4), 2012b, 27-42.

Diamond, P. A., and J. A. Hausman. "Contingent Valuation: Is Some Number Better Than No Number?" Journal of Economic Perspectives, 8(4), 1994, 45-64.

Dickinson. D. L., and T. McElroy. "Rationality Around the Clock: Sleep and Time-of-Day Effects on Guessing Game Responses." Economics Letters, 108(2), 2010, 245-48.

Haab, T. C., M. G. Interis, D. R. Petrolia, and J. C. Whitehead. "From Hopeless to Curious? Thoughts on Hausman's 'Dubious to Hopeless' Critique of Contingent Valuation." Applied Economic Policy and Perspectives, 35(4), 2013,593-612.

Hausman, J. "Contingent Valuation: From Dubious to Hopeless." Journal of Economic Perspectives, 26(4), 2012, 43-56.

Kling, C. L., D. J. Phaneuf, and J. Zhao. "From Exxon to BP: Has Some Number Become Better Than No Number?" Journal of Economic Perspectives, 26(4), 2012, 3-26.

Portney, P. R. "The Contingent Valuation Debate: Why Economists Should Care." Journal of Economic Perspectives, 8(4), 1994, 3-17.

Propper, C. "Contingent Valuation of Time Spent on NHS Waiting Lists." The Economic Journal, 100(400), 1990. 193-99.

Stigler, G. J., and G. S. Becker. "De Gustibus Non Est Disputandum." American Economic Review, 67, 1977, 76-90.

Whitehead, J. C., and T. J. Hoban. "Testing for Temporal Reliability in Contingent Valuation with Time for Changes in Factors Affecting Demand." Land Economics, 3, 1999, 453-65.

Whittington, D., V. K. Smith, A. Okorafor, A. Okore, J. L. Liu, and A. McPhail. "Giving Respondents Time to Think in Contingent Valuation Studies: A Developing Country Application." Journal of Environmental Economics and Management, 22(3), 1992, 205-25.

(1.) See Haab et al. (2013) for a cromulent reply to Hausman (2012).

(2.) A random prize drawing of $100 ($300 for midnight to 8 a.m. time slots) was used as an incentive for completing the survey within the assigned response window. The initial recruitment e-mail highlighted that one survey question, unrelated to the CVM question, would also offer the chance to win an additional $50. Responses to that incentivized question were analyzed by Dickinson and McElroy (2010).

(3.) Referendum valuation questions are increasing in incentive compatibility with their consequentiality (Carson, 2012a, 2012b). Questions are consequential if the survey respondent cares about the proposal and feels that it might influence policy. The hypothetical referendum has some degree of consequentiality given that the Broyhill Wind Turbine was a real proposal at the time of the survey.

(4.) All of the data in this study are available on request from the authors. Unlike Bigfoot sightings, we have proof that the data for this study are real and not made up. Our interpretations may be more suspect, but the design and data are real.

(5.) While based on the sleep literature our choice of time blocks is somewhat ad hoc. Note that we obtain similar results for the group of respondents who took the survey between 1 a.m. and 5 a.m. Including respondents at 11 p.m. or 7 a.m. causes the price coefficient to be marginally significant (p =. 10). We should also note that gender, age, and a variable to capture the previous (self-reported) night sleep quantity of the subjects were all statistically insignificant (results available on request). This leaves us with the remaining result: CVM responses do NOT go bump in the night!

(6.) As pointed out by a colleague, given that our subjects are college students, the night-time sample results could be affected by alcohol consumption. This concern is mitigated somewhat by our design choice to not include traditional party nights in the 72-hour data collection period (well, that is unless one considers Thirsty Thursday a party night).

(7.) Willingness to pay is the ratio of the constant and the coefficient on the fee amount. Standard errors for willingness to pay are estimated using the Wald method (Cameron 1991).

(8.) By this we mean that not all subjects recruited from e-mail lists chose to participate. As noted before, if a subject was assigned a particular response time slot but completed the survey during some other time, then we discard the data from that subject.

DAVID L. DICKINSON and JOHN C. WHITEHEAD *

* The authors thank Appalachian State University for funding and Todd Cherry, Tim Haab, Matt Interis. two journal referees, and Yoram Bauman for many helpful comments. While funding from a major oil company has not been received by the authors, on this or any past project, readers still should not take this article too seriously. While the empirical results could be developed into a reasonably serious paper, the authors could only be motivated to write it for yucks and grins. After comments from reviewers such as "I don't think this is humorous" and "I didn't laugh or would I expect others to," we have reconsidered our informal description of this as a "funny paper." In defense of our initial stance, however, we did survey some colleagues and got a strange look or two.

Dickinson: Department of Economics, Appalachian State University, Boone, NC 28607. Phone 828-262-7652, Fax 828-262-6105, E-mail dickinsondl@appstate.edu

Whitehead: Department of Economics, Appalachian State University, Boone, NC 28607. Phone 828-262-6121, Fax 828-262-6105, E-mail whiteheadjc@appstate.edu

TABLE 1
Frequency of "for" Votes

               Night (12     Morning (6   Afternoon      Evening (6
               a.m. to 6     a.m to 12    (12 p.m. to    p.m. to 12
               a.m.)         p.m.)         6 p.m.)       a.m.)

Fee            For   Total   For   Total   For   Total   For   Total

4               5      8      7      7     13     15      6     10
8              23     33     20     34     27     41     18     25
16              9     12      6      8      5      9      2      4
24             14     30     17     32     20     34     15     38
40              9     22      6     12     10     26     10     22
56             14     23     11     26     14     30      6     23

Total          74     128    67     119    89     155    57     122

[chi square]   7.63 (.18)    9.05          11.72         11.89
                             (p = .11)     (p = .04)     (p = .04)

TABLE 2
Logistic Regression

Dependent Variable: Vote ("for" Votes = 1)

                     Night                       Morning
                     (12 a.m. to 6 a.m.)         (6 a.m. to 12 p.m.)

                     Estimate           SE       Estimate

Intercept            0.646 **           0.325    0.828 **

Fee                  -0.012             0.010    -0.022 **

Model [chi square]              1.53                         4.61 **
(2 df)

Pseudo [R.sup.2]                0.009                        0.028

WTP                  52.05 **           25.07    38.42 ***
Cases                           128                     119

                              Afternoon
                              (12 p.m. to 6 p.m.)

                     SE       Estimate                  SE

Intercept            0.334    0.970***                  0.296

Fee                  0.011    -0.025 ***                0.009

Model [chi square]                         8.03 ***
(2 df)

Pseudo [R.sup.2]                           0.038

WTP                  10.31    38.66***                  7.84
Cases                                      155

                     Evening
                     (6 a.m. to 12 p.m.)

                     Estimate               SE

Intercept            0.732**                0.351

Fee                  -0.032***              0.011

Model [chi square]                8.79 **
(2 df)

Pseudo [R.sup.2]                  0.052

WTP                  23.13***               6.07
Cases                             122

Note: ***, **, and * indicates significance at 1%, 5%, and 10%
level, respectively.