Survey Methods Musings

The experiment with call scheduling has been running on refusal conversions for a while. It looks as if the experimental method works well for calls prior to the first refusal, but not after that. In retrospect, this makes sense. Refusal conversions are a different problem than establishing contact. When I presented the results to my colleagues at Survey Research Operations, they suggested that it might make more sense to try calling at times other than when the initial refusal came. This would help if we are able to contact someone else in the household who would be more cooperative. I'm thinking about how to set up a different model for this part of the process. A different problem requires a different tool...

Paradata are often quite messy. I guess it shouldn't be that surprising since they are often the by-product of a process (survey interviewing) that can be messy. And, at least initially, they were a means to an end and not the end itself. But there are some issues that run a little deeper than just messy. Brady West had a very interesting paper at AAPOR that looked at measurement error in interviewer observations. On a large face-to-face survey, we ask interviewers to make guesses about key characteristics of selected persons. These guesses are (relatively) highly correlated with survey outcome variables. This is a useful property for many reasons -- monitoring for the risk of bias, adjustment, etc. But, as Brady points out, the measurement or misclassification error reduces their effectiveness. I've been thinking about another kind of error. In talking with interviewers on the same face-to-face survey, they say the visit ever sampled housing unit every time they visit a se...