Survey Methods Musings

Another interesting question that I had during the webinar that I recently gave concerned responsive design and surveys with short time frames. I have to say, I mostly work on surveys with relatively long time frames. The shortest data collection that I have worked on in the last few years is about one month. That's not to say that I think responsive design is not relevant for surveys with short field periods. I think it is. If anything, following the prescribed regimen may be more important. A key aspect of responsive design, in my mind, is that the process is pre-planned. The indicators that are monitored, the decision rules for implementing interventions, the interventions, all have to be pre-planned. In a short survey, this is particularly important as their isn't time for developing ad hoc solutions. In a former life, I worked on surveys that had field periods of a day or two. In those studies, there wouldn't have been time to meet, discuss, and decide. Given the s...

I conducted a webinar on responsive design this week. I had several interesting questions. One of these was a question about responsive design and quota sampling.  The question was whether these two approaches are, in fact, different? Of course, there are similarities in that the response process is being controlled -- somewhat -- by the researchers. And this may lead to "allocating" nonresponse to some groups over others. For example, if some group is responding at higher rates, we might allocate resources to the lower responding group. Quota sampling will stop data collection for groups that have reached their quota. There are differences, however. Responsive design attempts to provide balanced response, but doesn't necessarily force that to happen. Further, responsive design is attempting to control the data collection process using a variety of approaches. Quota sampling only has one approach -- stop when the quota is full.  I do worry that there may be a conver...

I've been getting ready to give a webinar on responsive design. I enjoy getting ready for this kind of talk as it gives me an opportunity to think about definitions and concepts. A few years ago, Mick Couper and I had a paper on "responsive vs adaptive" design. My thinking hasn't evolved much since that paper. In preparing the talk, I thought it might be helpful to define responsive design by contrast with ... that which is not responsive design. The contrasts were 1) pre-specified designs, and 2) ad hoc designs. The first category is a design where a pre-specified design is implemented and the results are pretty much as predicted. I personally haven't worked on many surveys like that, but I'm not yet ready to call it a "straw man." The second category is an approach I have seen in action. I sometimes call this approach "shooting from the hip." This is the situation where we start with a pre-specified design, but when it goes off the...