Potentials and constraints of propensity score adjustments - January 2014

Potentials and constraints of propensity score adjustments. The chapter aims to explore and evaluate in more detail the efficiency of the propensity score adjustment (PSA) and the power of webographic (i.e. behavioral and attitudinal) variables in adjusting biases arising from non-randomized sample selection.

Purchase link:

Steinmetz S., Bianchi A., Tijdens K.G., Biffignandi S. (2014) Improving web survey quality – Potentials and constraints of propensity score adjustments. In Callegaro M., Baker R., Bethlehem J., Goritz A., Krosnick J., Lavrakas P. (eds.) Online Panel Research: A Data Quality Perspective.
Publisher: Chichester: Wiley. ISBN: 978-1-119-94177-4. Book chapter pp 273-298 (ENG)

ABSTRACT:

The chapter aims to explore and evaluate in more detail the efficiency of the propensity score adjustment (PSA) and the power of webographic (i.e. behavioral and attitudinal) variables in adjusting biases arising from non-randomized sample selection. In this context, it is to be considered that evidence for the applicability of PSA in the field of surveys in the scientific community is very limited. The empirical application is based on the Dutch sample of the WageIndicator Survey for 2009 – a multi-country, continuous volunteer web survey devoted to the collection of labor-related variables. In the analysis, the target variable is the monthly gross wage. The sample is compared with a probability-based web sample from the LISS panel (Longitudinal Internet Studies for the Social Sciences) which is also used as a reference survey in the PSA application.

The findings indicate that with the availability of an accurate probability-based reference survey the application of PSA can help reducing biases in volunteer samples. With respect to the inclusion of webographic variables, at least for the target variable wages, the computed propensity weights did not lead to the expected improvements. This was also due to the fact that those propensity weights which effectively reduced the bias between the samples showed a much higher variability impacting on the validity of estimates. Nevertheless, and considering the advantages of volunteer web surveys (like reduced costs, flexibility, worldwide coverage, etc.) and their more and more extensive use, it is important to be aware of how far can weights improve the representativeness of nonprobability panels. The presented adjustment approaches seem to offer improvements with respect to bias correction which also allow for better generalizations of estimates from volunteer samples.