Survey Methods Musings

I've been reading a bit on mixed-mode surveys. I've noticed several discussions of web surveys and coverage error. This is a relatively recent mode, and one of the key issues has been to what extent the population has access to the internet. If someone doesn't have access to the internet, they can't complete a web survey. Everyone agrees upon that. But how do we describe the source of this error? Is it coverage or nonresponse error? In my mind, coverage error is a property of the sampling frame. If the unit is not on the sampling frame, then it is not covered. But many web surveys are general population surveys that don't have a tight association with a frame. That is, since there is not "internet" sampling frame in the way we have RDD or area probability samples. Many surveys start today from ABS sampling and then might do telephone, mail, web, or mixed-mode designs. In this case, a lack of internet access is an impediment to responding and not an imp...

I've been thinking about responsive design and uncertainty. I know that when we teach sample design, we often treat design parameters as if they were known. For example, if I do an optimal allocation for a stratified estimate, I assume that I know the population element variances for each stratum. The same thing could be said about response rates, which relate to the expected final sample size. Many years ago, the uncertainty might have been small about many of these parameters. But responsive design became a "thing" largely because this uncertainty seemed to be growing. The question then becomes, how do we acknowledge and even incorporate this uncertainty into our designs? Especially responsive designs. It seems that the Bayesian approach is a natural fit for this kind of problem. Although I can't find a copy online, I recall a paper that Kristen Olson and Trivellore Raghunathan presented at JSM in 2005. They suggested using a Bayesian approach to update estimate...

I had the opportunity this morning to meet with a medical researcher who runs many clinical trials. He spoke about the problems of explaining randomization when enrolling persons in a trial. It's hard to be sure they understand the concept of randomization. To be sure, it's even more difficult to be sure they understand the consequences of either enrolling or not enrolling in a trial. But the problem of explaining randomization caught my attention. This reminds me of the situation that interviewers find themselves in quite frequently. In implementing random selection of a person from within a household, they often find that the person selected is someone other than the informant who aided with the selection. In these cases, the informant may be disappointed that they weren't selected and ask if they can do the interview instead. It's often difficult to explain why we want to speak to the other person, who is not there or maybe not even willing to do the interview. I...