Many surveys work to a sample size/response rate requirement. The contract specified the target response rate. The survey organization works hard to meet that target. In this context, the logical thing for the survey organization to do is to focus on interviewing the easiest cases to interview.
The underlying assumption of this approach is that a higher response rate leads to a lower risk of bias. Theoretically, this need not be true. Empirically, there have been a number of recent studies where this is not true (see Groves and Peytcheva, POQ 2008). So what are we supposed to do?
The search is on for alternative indicators. Bob Groves convened a meeting in Ann Arbor to discuss the issue two years ago. The result was a short list of indicators that might be used to evaluate the quality of survey data (see the October 2007 issue of Survey Practice: http://www.surveypractice.org/).
Now these new measures are starting to appear! Barry Schouten, Fannie Cobben, and Jelke Bethlehem have an article in the June 2009 issue of Survey Methodology. They develop an indicator they call the "R-Indicator." They describe this as an "indicator... for the similarity between the response to a survey and the sample or the population under investigation."
I've been working on using the fraction of missing information, a concept from methods for missing data, as an indicator of the risk of nonresponse bias. I have an article accepted for publication that should be appearing soon.
The underlying assumption of this approach is that a higher response rate leads to a lower risk of bias. Theoretically, this need not be true. Empirically, there have been a number of recent studies where this is not true (see Groves and Peytcheva, POQ 2008). So what are we supposed to do?
The search is on for alternative indicators. Bob Groves convened a meeting in Ann Arbor to discuss the issue two years ago. The result was a short list of indicators that might be used to evaluate the quality of survey data (see the October 2007 issue of Survey Practice: http://www.surveypractice.org/).
Now these new measures are starting to appear! Barry Schouten, Fannie Cobben, and Jelke Bethlehem have an article in the June 2009 issue of Survey Methodology. They develop an indicator they call the "R-Indicator." They describe this as an "indicator... for the similarity between the response to a survey and the sample or the population under investigation."
I've been working on using the fraction of missing information, a concept from methods for missing data, as an indicator of the risk of nonresponse bias. I have an article accepted for publication that should be appearing soon.
Comments
Post a Comment