A big question facing our field is whether it is better to adjust data collection or do post-data collection adjustments to the data in order to reduce nonresponse bias. I blogged about this a few months ago. In my view, we need to do both.
I'm not sure how the argument goes that says we only need to adjust at the end. I'd like to hear more of that. In my mind, it must be an assumption that once you condition on the frame data, the biases disappear and that assumption is valid at all points during the data collection. That must be a caricature -- which is why I'd like to hear more of the argument from a proponent of the view.
In my mind, that assumption may or may not be true. That's an empirical question. But it seems likely that at some point in the process of collecting data, particularly early on, that assumption is not true. That is, the data are NMAR, even when I condition on all my covariates (sampling frame and paradata). Put another way, in a cell adjustment framework, responders and nonresponders within cells have different means.
At some point, however, there may be a shift. As the data accumulate (quantitative change), the mechanism may shift (qualitative change) from NMAR to MAR (or less NMAR, errr, if there is such a thing). I think that must be an empirical question. It would be nice to have some gold standard studies to understand this.
I further speculate that such a shift (from NMAR to MAR) is more likely to occur in a controlled process than in a relatively uncontrolled one. I say that because I have been thinking about adaptive design as an attempt to place control on a process with a lot of variation, much of it coming from interviewers.
I'm not sure how the argument goes that says we only need to adjust at the end. I'd like to hear more of that. In my mind, it must be an assumption that once you condition on the frame data, the biases disappear and that assumption is valid at all points during the data collection. That must be a caricature -- which is why I'd like to hear more of the argument from a proponent of the view.
In my mind, that assumption may or may not be true. That's an empirical question. But it seems likely that at some point in the process of collecting data, particularly early on, that assumption is not true. That is, the data are NMAR, even when I condition on all my covariates (sampling frame and paradata). Put another way, in a cell adjustment framework, responders and nonresponders within cells have different means.
At some point, however, there may be a shift. As the data accumulate (quantitative change), the mechanism may shift (qualitative change) from NMAR to MAR (or less NMAR, errr, if there is such a thing). I think that must be an empirical question. It would be nice to have some gold standard studies to understand this.
I further speculate that such a shift (from NMAR to MAR) is more likely to occur in a controlled process than in a relatively uncontrolled one. I say that because I have been thinking about adaptive design as an attempt to place control on a process with a lot of variation, much of it coming from interviewers.
Comments
Post a Comment