Practical Tools for Sampling and Weighting
Prerequisite: Sampling theory (e.g., SURV440 or equivalent) and Applied sampling (e.g., SURV626 or equivalent).
This course will be a combination of hands-on applications and general review of the theory and methods behind different approaches to sampling and weighting. Several survey data sets will be used to illustrate how to design samples, calculate weights, and make estimates from complex surveys. For statisticians, the course is meant to give you some practical experience in applying the theoretical ideas you have learned in this and other classes. For social scientists, the goal is to provide you some insight into the statistical thinking needed and steps taken in actually designing, selecting, and weighting samples. By working together in teams, you should get a taste of how projects are carried out in survey organizations. Project 1. The first project will be to design a sample for a population where single-stage sampling is appropriate. A stratified sample will be designed that achieves specified goals for precision of a series of domain estimates while meeting a budget constraint. Anticipated rates of nonresponse and ineligible units will be accounted for. Multicriteria optimization methods will be used to determine the allocation. Project 2. The second project will be to use a set of data collected from a sample of military personnel and develop survey weights. The weights should account for cases with unknown eligibility, nonrespondents, and uses of auxiliary data to improve estimators. Based on a description of the survey design along with variables that identify cases as respondents and nonrespondents, teams will compute weights, document their methods, discuss pros and cons of different options, and make class presentations on the results. Students will devise quality control checks and will set up the analysis file to allow use of either linearization or replication variance estimation.ion. Project 3. A third application will be an area probability design in which students will use an existing sample of primary units and determine a plan for sampling segments and persons within segments. Rates will be determined to achieve target sample sizes for different demographic groups. Readings for the course will cover the general theory and application of alternative methods of sample allocation, nonresponse adjustment (including weighting cell adjustments and propensity score adjustments), methods of using auxiliary data to reduce coverage biases and improve standard errors (including poststratification and more general applications of regression weighting). Readings will also include alternatives that are used for computing response and coverage rates. Throughout the course the emphasis will be on learning how to apply the methods rather than on learning the theory behind them.