|
Measurement Errors
XIIth course in the ECAS programme |
Preliminary abstracts
Reducing Measurement Errors in Surveys
Edith de Leeuw, Utrecht University
Surveys are one of the major data collection methods today and provide data for a large proportion of social science research and official statistics. The quality of survey data is crucial to today's society. Survey design and implementation determine to a large amount data quality and are of central importance to survey planners and practitioners.
This course focuses on reducing measurement error in surveys and the role of survey design and implementation in improving the efficiency and quality of data collection. The course draws on the recently published International Handbook of Survey Methodology » and introduces the cornerstones of data quality and the role of measurement error in surveys. Topics include the influence of data collection method, modern technologies, and mixed-mode design on survey error. Special attention will be given to sources of measurement error that, in principle, can be reduced by good survey design (questionnaire, interviewer). Of particular interest are the current theories and empirical knowledge related to asking questions, question writing and testing, and optimizing a questionnaire for a specific mode. Participants will obtain a clear overview of recent methods and practical guidelines for quality survey design.
Reducing Measurement Error in Web Surveys
Mick Couper, University of Michigan
The course will focus on the design of Web survey instruments and procedures, based on theories of human-computer interaction, interface design, and research on self-administered questionnaires and computer assisted interviewing. The course will cover various aspects of instrument design for Web surveys, including the appropriate use of widgets (e.g., radio buttons, check boxes, general formatting and layout issues, movement through the instrument (action buttons, navigation, error messages), and so on. The course will draw on empirical results from experiments on alternative design approaches as well as practical experience in the design and implementation of Web surveys.
An Introduction to Latent Class Analysis of Survey Error
Paul Biemer, RTI International, NC
This course presents a statistical framework for modeling and estimating classification error in surveys. It begins by examining some of the early models for survey measurement error and demonstrating their similarities, strengths and weaknesses. Then these models are cast in a general latent class modeling (LCM) framework where the true values of a variable are assumed to be unobserved (latent) and the survey response constitutes a single indicator of this latent variable. The model parameters include the target population proportions for a categorical variable to be estimated in the survey and the misclassification probabilities (for e.g., false positive and false negative, for dichotomous response variables) for measuring the variable. Survey item reliability and construct validity as well as estimator bias are defined and interpreted within this general framework. Methods for estimating the model parameters and issues of model identifiably will be discussed.
One advantage of viewing survey classification error model as a LCM is the availability of general software for estimating the error components. However, the assumptions of the traditional LCM can be somewhat restrictive. An even more general model can be obtained by viewing the LCM as a type of log linear model with latent variables. In doing so, a wide range of error structures and error evaluation designs can easily be discussed and analyzed using log-linear modeling notation and methods. A number of examples and illustrations will be presented to demonstrate the estimation methods and the interpretation of the latent class analysis results. The utility of the models for evaluating and improving survey data quality will also be discuss and demonstrated.
Multilevel analysis for grouped and longitudinal data
Joop Hox, Utrecht University
This course is an introduction to multilevel analysis for researchers who work in the survey field. Multilevel analysis concerns data that have a hierarchical or nested data structure. The most common application is to individual respondents nested within groups or organizations, but the nesting can also be repeated measures within individuals as in longitudinal surveys. Cluster sampling in surveys also produces multilevel data, and multilevel software could be used to analyze such data. What sets multilevel analysis apart is the interest in modeling individual outcomes using a combination of individual level and group level variables, and in the explicit interest in contextual effects.
This course will introduce multilevel modeling for two- and three-level data, including examples from the survey field. Multilevel models for longitudinal data will be treated next, with special attention to the way multilevel modeling copes with the problem of missing data due to panel dropout. In this case, multilevel modeling makes a softer assumption than the standard 'solution' offered by statistical packages, such as SPSS and SAS, that simply delete all incomplete cases. The extension of the model to categorical outcome variables will be discussed with some attention to estimation issues (there are several methods available, with varying accuracy). We discuss methodological and statistical issues, such as estimation methods, testing methods, and required sample sizes. The final part of the course includes two options, depending on the students' interest. One option is an overview of new multilevel modeling options, such as multilevel versions of structural equation analysis and latent class analysis. The other option is to collect questions from the group about multilevel research they are doing or considering, and discuss this with the attendees. This would be the format of a 'master class'. The course is based on J.J. Hox, Multilevel Analysis, Second Edition, which will be published in the first half of 2009.
