Friday, April 24, 2015

Responsive Design and Surveys with Short Time Frames

Another interesting question that I had during the webinar that I recently gave concerned responsive design and surveys with short time frames. I have to say, I mostly work on surveys with relatively long time frames. The shortest data collection that I have worked on in the last few years is about one month.

That's not to say that I think responsive design is not relevant for surveys with short field periods. I think it is. If anything, following the prescribed regimen may be more important. A key aspect of responsive design, in my mind, is that the process is pre-planned. The indicators that are monitored, the decision rules for implementing interventions, the interventions, all have to be pre-planned. In a short survey, this is particularly important as their isn't time for developing ad hoc solutions.

In a former life, I worked on surveys that had field periods of a day or two. In those studies, there wouldn't have been time to meet, discuss, and decide. Given the short time frame, I could imagine that changes would more likely be made at the case level. Groves and Heeringa defined phases as a consistent set of design features over some period of time. In such a short study, rules might have to be specified for actions that are taken at the case level, possibly actions that are automated by sample management systems in CATI surveys or through various email options for web surveys.

I think it would work. I'd be interested to hear about any experiences with studies like this.

Friday, April 17, 2015

Responsive Design and Quota Sampling

I conducted a webinar on responsive design this week. I had several interesting questions. One of these was a question about responsive design and quota sampling.  The question was whether these two approaches are, in fact, different?

Of course, there are similarities in that the response process is being controlled -- somewhat -- by the researchers. And this may lead to "allocating" nonresponse to some groups over others. For example, if some group is responding at higher rates, we might allocate resources to the lower responding group. Quota sampling will stop data collection for groups that have reached their quota.

There are differences, however. Responsive design attempts to provide balanced response, but doesn't necessarily force that to happen. Further, responsive design is attempting to control the data collection process using a variety of approaches. Quota sampling only has one approach -- stop when the quota is full. 

I do worry that there may be a convergence, where responsive design become quota sampling. To me, the most interesting problems are actually re-allocating resources to improve the balance of respondents while maintaining or increasing response rates. Decreasing response rates in order to improve balance seems too easy. There are some examples where this can be shown to be helpful. I think we need more evidence to be convinced of this.

Friday, April 3, 2015

Responsive Design Definition

I've been getting ready to give a webinar on responsive design. I enjoy getting ready for this kind of talk as it gives me an opportunity to think about definitions and concepts. A few years ago, Mick Couper and I had a paper on "responsive vs adaptive" design. My thinking hasn't evolved much since that paper.

In preparing the talk, I thought it might be helpful to define responsive design by contrast with ... that which is not responsive design. The contrasts were 1) pre-specified designs, and 2) ad hoc designs.

The first category is a design where a pre-specified design is implemented and the results are pretty much as predicted. I personally haven't worked on many surveys like that, but I'm not yet ready to call it a "straw man."

The second category is an approach I have seen in action. I sometimes call this approach "shooting from the hip." This is the situation where we start with a pre-specified design, but when it goes off the rails we don't have a plan. Valuable time is lost coming up with a plan. During the lost time, sub-optimal effort continues.

I thought it was helpful to take the "this is not responsive design approach" to defining responsive design. I suppose we can also cross off the list "responsive design" as used by web designers. This is a kind of web design that adapts to the format of the device and browser. Again, not "responsive design" in the sense we mean here.