Increasing response rate in industrial mail surveys: the effect of respondent involvement in sweepstakes incentive.

Kim, Cue D. ; Dant, Shirish ; Lee, C. Christopher 等

INTRODUCTION

One of the problems facing industrial marketers collecting primary data via mail surveys is that such research procedures suffer from low response rates which affect nonresponse errors. Historically, industrial mail surveys have been criticized for inadequate response rates, which are usually lower than both telephone and face-to-face modes. In spite of lower cost and wider coverage of subjects, low response rates of mail surveys make data analysis difficult and restrict generalization of results (Suchman, 1940; Toops, 1926).

Recently, considerable research has been devoted to developing techniques to improve mail survey response rates. Farrell and Elken have suggested that five main variables of survey design have impact on the response rates: contact, incentive, reward, length, and prose (Farrell, 1994). The difference in color of survey stationary has been also tested in terms of response rates (Greer, 1994).

Sweepstakes as Incentives

Sweepstakes have been popularly used as a form of incentive to respondents. A number of studies (e.g. Balakrishnan, 1992) have tested the effectiveness of sweepstakes, and found that prize giveaway sweepstakes are cost-effective in increasing response rates. When a survey involves a very large sample, sweepstakes are considered more cost effective to increase response rate than other approaches such as pre-notification, follow-up contacts, and monetary incentives (Fox, 1988; Greer, 1994; Kanuk, 1975; Linsky, 1975 & Scott, 1961).

Involvement

According to McKee (McKee, 1992), sample respondents who are involved with the survey topic are more likely to respond. Respondent involvement can be stimulated by appealing to personal interest, perceived importance, or expected benefit. Respondents who have high interest in the survey topic are almost twice as likely to participate, and also are less likely to omit questions (Martin, 1994). Similarly, a high correlation has been reported between subject involvement and their motivation to participate (Lord, 1994). Thus, there exists ample evidence indicating that surveys designed to stimulate respondent involvement will produce higher response rate. This extant research has examined the effect of respondent involvement in subject matter or content of survey on response rate. There has been no extension of these findings to examine the effect on response rate of respondent involvement in the incentives used in industrial mail surveys.

Sweepstakes with involvement

It is very difficult to assure that a survey is designed to appeal to the interests of all potential respondents in a large sample. Limiting the mail survey to those who are interested in the survey topic is not practical, either. One way of increasing respondent involvement is to make the survey more interesting and attractive by using incentives, sweepstakes being one of the popular methods. As mentioned earlier, sweepstakes are cost effective, and their effectiveness can be enhanced if they are combined with ways to increase respondent involvement. Since effectiveness of direct mail has been found to be increased when creativity is combined with other incentives (Offitzer, 1994), it should follow that if a mail survey combines sweepstakes with creativity to increase respondent involvement, its response rate would be higher. Although past studies have confirmed the effectiveness of involvement and sweepstakes separately, the idea of combining involvement with sweepstakes has not been empirically tested.

This paper examines the idea that respondent involvement in sweepstakes increases response rate of mail survey by inducing a high level of initial interest from respondents. A new approach of high-involvement sweepstakes is developed and tested with a control group presented with conventional non-involvement sweepstakes. Our objective is to empirically test whether creative and slightly more complicated sweepstakes produce higher response rate than simple conventional sweepstakes. Section 2 explains the research procedure including survey design and sampling, and it is followed by data analysis in Section 3. Findings are discussed in Section 4, and concluding remarks are in Section 5.

METHODOLOGY

Sampling

A nationwide random sample of 1,870 automobile tire replacement dealers from a yellow page listing service was drawn on an n-th name basis. Since yellow page lists are alphabetically arranged, in order to avoid over-representation of similar names, the list was first sorted by the second letter of dealer names, and then n-th name basis systematic sampling was performed. The whole sample was then divided into half (935 each), and two identical sets of surveys but with different sweepstakes rules were sent to each group. A sample with involvement sweepstakes is the test group, and the other sample with conventional sweepstakes is the control group. From here on, control and test groups will be used to identify each sample.

Sweepstakes

While conventional sweepstakes typically use a random drawing of the winners from the respondents as shown in Exhibit 1, the rules for involvement sweepstakes of this study ask respondents to select a prize of their choice from a list of available prizes.

The respondents of the test group are informed that their chance of winning depends on how many other respondents pick the same prize. This cue is intended to enhance respondents' involvement in the sweepstakes process. This is achieved by making respondents actively go through a decision making process in which they evaluate their chances of winning. By noting that their chances of winning depend on their own decision making, their interest and enthusiasm, and hence involvement, in the sweepstakes is made high. Exhibit 2 shows the sweepstakes with involvement.

Twenty golf clubs of four different kinds were offered as sweepstakes prizes to each sample. The decision to offer golf clubs as prizes was made after conducting a pilot study of 22 managers or owners of tire dealers in a major Midwestern city in the U.S.

Data Collection

One hundred and ninety-six (196) questionnaires were returned in three weeks after the mailing. Four of them were unusable, therefore, 192 responses are used in data analysis.

Exhibit 3 shows the number of responses in each group. Some respondents who completed the survey did not return the cover letter which was required to make them eligible for sweepstakes entry. These respondents are categorized as 'no-entry' group. This no-entry group is excluded from the nonresponse error analysis.

ANALYSIS

Response Rate

To investigate the difference between the two groups with different sweepstakes rules, a simple t-test was performed. The result shown in Table 1 indicates that the test group produced a significantly higher response rate and lower nonresponse error than the control group. Even though the overall response rate was 10.3%, which is about the average for industrial mail surveys, the result of this study is intriguing because the difference in response rates between two groups is statistically significant.

Response Error

Since missing values are a major source of response error, the number of cases that have missing values is used to measure the error for each group. Table 2 shows response errors of each group and the comparison between control and test groups. It is found that the test group has a significantly lower response error than the control group. In other words, respondents with involvement in sweepstakes are less likely to omit questions. This result is consistent with Martin's study (Martin, 1994) of response error and respondent involvement in survey topic.

Response Speed

Since a speedy response is also an important aspect of mail surveys (Brennan, 1992), the speed of responses of the two groups are also compared. Exhibit 4 shows the number of responses received within the first three weeks. The response rates of all three groups (test, control, and no-entry groups) dropped almost to zero after two weeks, but the test group had clearly more early responses that also came in at a higher rate than the other group.

This suggests that the sample with high involvement in the sweepstakes is more likely to respond quickly, which is consistent with the findings of Brennan and Hoek on incentives and response speed (Brennan, 1992).

[GRAPHIC OMITTED]

DISCUSSION

Our analysis suggests that, in industrial mail surveys, respondents' involvement in commonly used incentives like sweepstakes has a positive effect on response rate. The response rate from the test group (treated to high-involvement sweepstakes) is significantly higher than that from the control group (treated to conventional sweepstakes). The response speed is also faster from the first group.

The reason for this higher and faster response in the test group is due to respondents' higher expectation to win, which in turn increases their involvement. The wordings in the cover letter explaining the rules of the sweepstakes are designed to lead respondents to speculate that they have a higher chance of winning if they select a prize item which, in their opinions, fewer other people would pick (see Exhibit 2). Respondents have a stronger incentive to respond when given a chance to choose their prize and influence their likelihood of winning.

The issue of response error is critical in this type of setting because there is a possibility that the sweepstakes incentive could lead respondents to fill out the survey in a careless manner just to enter the sweepstakes. If that is the case, the effort to increase response rate can be undermined by a large number of missing values and thus high response error. Fortunately, the analysis shows that the test group has significantly lower response error, suggesting that the design of sweepstakes used in this study is effective.

CONCLUDING REMARKS

The purpose of the study was to empirically test the extent to which response rate from industrial mail surveys increases when the survey instrument is accompanied by sweepstakes incentive in which potential respondents are made involved. Respondents, presented with high involvement sweepstakes, are more likely to respond, respond promptly, and are less likely to omit questions. As a result, the use of involvement sweepstakes improves response rate and reduces response error, thereby increasing the effectiveness of the survey.

Future studies should focus on other creative methods of increasing respondents' involvement and interest for more cost-effective survey design. This may be extended beyond mail surveys to other types of research settings, such as telephone interviews, face-to-face interviews, on-line surveys, and so on. Future research should also investigate the effectiveness of using different rewards for samples of different characteristics, and the combined impact of multiple incentives on response rate.

In summary, this study has shown how the effectiveness of surveys can be improved by using creative techniques. By increasing the level of involvement and interest of respondents, the cost effectiveness of the survey can be improved. Researchers can enhance the response rate of their surveys by using such creative techniques to involve potential respondents.

REFERENCES

Balakrishnan, P.V., Chawla, S.K., Smith, M.F. & Michalski, B.P. (1992). Mail survey response rates using a lottery prize giveaway incentive. Journal of Direct Marketing, 6(3), 54-59.

Brennan, M. & Hoek, J. (1992). The behavior of respondents, nonrespondents, and refusers across mail surveys. Public Opinion Quarterly, 56(4), 530-535.

Farrell, B. & Elken, T. (1994). Adjust five variables for better mail survey. Marketing News, 28(18), 20.

Fox, R. J., Crask, M. R. & Kim, J. (1988). Mail survey response rate: A meta-analysis of selected techniques for inducing response. Public Opinion Quarterly, 52, 467-491.

Greer, T. V. & Lohtia, R. (1994). Effects of source and paper color on response rates in mail surveys. Industrial Marketing Management, 23(1), 47-54.

Herberlein, T. A. & Baumgartner, R. (1978). Factors affecting response rates to mailed questionnaire: A quantitative analysis of the published literature. American Sociology Review, 43, 447-462.

Kanuk, L. & Berenson, C. (1975). Mail surveys and response rates: A literature review. Journal of Marketing Research, 12(4), 440-453.

Linsky, A. S. (1975). Stimulating responses to mailed questionnaires: A review. Public Opinion Quarterly, 34, 82-101.

Lord, K. R., Lee, Myung-Soo & Sauer, P. L. (1994). Program context antecedents of attitude toward radio commercials. Journal of Academy of Marketing Science, 22(1), 3-15.

Martin, C. L. (1994). The impact of topic interest on mail survey response behavior. Journal of the Market Research Society, 36(4), 327-338.

McKee, D. O. (1992). The effect of using a questionnaire identification code and message about non-response follow-up plans on mail survey response characteristics. Journal of Market Research Society, 34(2), 179-191.

Offitzer, K. (1994). Letter perfect. Incentive, 168(4), 55-62.

Scott, C. (1961) Research on mail Surveys, Journal of The Royal Statistical Society, Series A (General), 124, 143-205.

Suchman, E. A. & McCandless, B. (1940). Who answers questionnaires? Journal of Applied Psychology, 24, 758-769.

Toops, H. A. (1926). Returns from follow-up letters to questionnaires. Journal of Applied Psychology, 10, 92-101.

Cue D. Kim, Drake Beam Morin Korea, Inc., The Republic of Korea

Shirish Dant, The Gallup Organization

C. Christopher Lee, Central Washington University

Yun-Oh Whang, University of Central Florida

TABLE 1
Groupwise Response Rates

 Conventional Involvement
 Sweepstakes Sweepstakes
 (Control (Test
 Group) Group)

Sample Size 935 935
Responses 63 95
Response Rate 6.73% 10.16%
p-value * 0.0039

* one-tail t-test for difference of response rates
between two groups.

TABLE 2
Groupwise Response Errors

 Conventional Involvement
 Sweepstakes Sweepstakes
 (Control (Test
 Group) Group)

Sample Size 63 95
Cases with Missing Values 7 4
Response Error 11.11% 4.21%
p-value * 0.0476

* one-tail t-test for difference of response errors between
two groups.

EXHIBIT 1
Conventional Sweepstakes

In an appreciation for your participation in this study,
we are going to enter your name in a sweepstake to give
away the following golf clubs as prizes. The first 5 picks
will receive the Grand Prize, and the following 3 groups
of 5 winners will receive the 1st to 3rd Prizes.

Place Prize Quantity
 Available

Grand Prize #1 Wood Driver 5
1st Prize #1 Iron 5
2nd Prize Putter 5
3rd Prize Sand Wedge 5

EXHIBIT 2
Sweepstakes with Involvement

In an appreciation for your participation in this study, we are
going to enter your name in a sweepstake to give away the
following golf clubs as prizes. You are asked to choose one of
the prizes in the table below, and your chance of winning
depends on how many other respondents pick the same prize. For
example, if only 5 respondents pick #1 Iron (Choice 2) each
automatically wins the prize without drawing. If more than 5
pick the same prize category, winners will be decided by random
drawing from respondents who pick the category.

Prize Quantity Please mark your choice.
 Available

#1 Wood Driver 5
#1 Iron 5
 Putter 5
 Sand Wedge 5

EXHIBIT 3. Number of Responses

No-Entry to
Sweepstakes 34

Conventional
Sweepstakes
(Control Group) 63

Sweepstakes
with Involvement
(Test Group) 95

Note: Table made from bar graph.