Rules of Non-response and Selectivity: Analysing the Drop-out in the Multi-stage Recruitment Process for the German Mobility Panel
T Kuhnimhof, B Chlond, University of Karlsruhe, DE
The German Mobility Panel (MOP) is a survey of mobility behaviour that elicits information from the probands by means of a seven-day 24-hour diary each year over a time span of three years. As such the MOP not only features a unique dataset but also very high requirements with respect to the probands. This leads to response rates that are much lower than in normal surveys. In this paper we analyse the non-response and selectivity throughout the various stages of the MOP proband recruitment process. Moreover, we show how information about the general rules of non-response and selectivity can be inferred from this analysis.
As do other surveys of mobility behaviour, the MOP aims at a representative image of the mobility of a population. This can only be achieved if all groups of a population are adequately represented in the sample. However, normally a portion of the initially contacted probands does not take part in the survey for various reasons: they might simply forget to participate, or they might as well deliberately boycott the survey because of privacy issues etc.
In most surveys it can be found that certain socio-demographic groups are not represented adequately. This bias is usually corrected by applying appropriate weighting procedures based on secondary statistics. Hence, if individual non-response is not associated with mobility behaviour, the drop-out does not negatively influence the survey results. However, probands could be declining participation because they are mobile below average and therefore not interested in the issue. On the other hand, very active and mobile persons could refuse to provide information because they want to spare themselves the bother of reporting their extensive activity spectrum.
Such systematic non-response can not be rectified because appropriate secondary statistics do not exist. If the surveyed mobility behaviour of the respondents differs from the behaviour of non-participants, the survey results are biased. This is the well known non-response problem of mobility surveys.
The burden for the MOP-respondents who have to report three times over a one week period is considerably high, and dropping out seems an easy option to avoid the hassle. Because of the high burden it is necessary to recruit reliable participants. Therefore the recruitment process is organised in multiple stages:
1.First contact by CATI in order to recruit persons generally willing to provide information
2.If willing to provide information, second CATI in order to raise interest in taking part in the MOP
3.If interested, mailing of detailed MOP information in order to recruit person for the panel
4.If willing to take part, mailing of diary and other survey documents
5.If participating, proband filling in and sending back of documents and diary
6.If successful participation in the first year, re-contacting twice in order to motivate for participation in the second and third year
This shows how high the burden for respondents is, while drop-out opportunities are plenty. Consequently, the non-response in the MOP is much higher than in other surveys where the respondent burden is less. However, this drop out process can be used to obtain more information about selectivity processes: Are persons with certain characteristics more likely to drop out than others? Does this relate to mobility behaviour? What are the specifics of the recruitment process that turn away particular groups? And finally: Do drop-outs differ in behaviour and mobility from the population represented in the sample?
In order to answer these questions a selectivity study is being conducted. In this study the panel participants are compared with those who drop-out in the recruitment process. In order to obtain information about the personal and household context as well as the basic mobility behaviour of non-participants appropriate questions are included in the initial CATI-Interview. These questions predominantly relate to typical or characteristic mobility behaviour and focus less on quantitative information. The CATI-respondents are asked about their roles in the household, the determinants of everyday life such as employment status, the presence of children etc. and the resulting mobility behaviour. Emphasis is on the mobility associated with leisure behaviour since it is not predetermined by a person?s role or status but object of personal decision-making and thus expression of a particular lifestyle.
While a lot of the CATI-respondents drop out during the recruitment process, some do eventually participate in the panel survey. Thus the more qualitative CATI-information will be validated in light of the more quantitative one week panel data. Moreover, this panel data allows for the assessment of the quantitative significance of the information provided in the CATI-Interview.
This information facilitates a comparison of the behaviour and personal characteristics of drop-outs and panel participants. By means of this comparison it is possible to evaluate whether the drop-out is random and does not follow a specific rule or if representatives of particular mobility styles are more likely not to participate. If the latter is the case survey results are likely to be biased. Based on the results of our study this bias can be quantified and appropriate weighting measures can be developed.
Our research differs from other non-response studies in that it does not make a second attempt to elicit information from persons who initially did not respond. Our study traces persons who initially provided information until they either drop-out in the recruitment process or participate in the panel. As always, persons who do not provide this initial information because they decline a CATI or cannot be reached pose a problem. However, the analysis of the dropping-out process allows for conclusions with respect to these initial drop-outs.
The results of our study have implications that go beyond analysing the selectivity of the MOP. If the results indicate that drop-out is random and equally distributed throughout all types of mobility behaviour, then there is no indication that survey results are biased because of selectivity in the drop-out process. But also the contrary could be the case: For example, it could be that persons who are not mobile do not show interest in the survey from the very beginning, while very mobile and active persons initially show interest but are then turned away because they realize the work associated with taking part. If such systematics underlie the dropping-out process this indicates that survey results are biased due to selectivity. The multi-stage set-up of the recruitment of MOP-participants provides ideal data to study this phenomenon.
Keywords: Selectivity, Survey on Mobility Behaviour, German Mobility Panel
Association for European Transport