Survey Methods Musings

A few years ago, I was presenting at a conference of substantive experts. I gave an update on a progress on a survey of interest to this group. I talked about how nonresponse bias can be complex, and that the response rate might not be a good predictor of when this bias occurs -- based on Groves and Peytcheva . I was speaking with one of the researchers after my presentation, and I was surprised to hear her say that she interpreted my comments to mean that "response rates don't matter." Although that interpretation makes sense, it hadn't really occurred to me in that way until she said it. Since then, it seems like we've seen a lot of published papers and conference presentation where lowering the response rate becomes a tactic for improving the survey. Most studies taking this tactic lower the response rates for groups that tend to respond at higher rates. The purported benefit is  response set balance on known characteristics from the sampling frame is improve...