Survey Methods Musings

There was another interesting thing that happened in our mixed mode that varied the sequences of modes. As a reminder, the survey was a screening survey. It identified households with an eligible person. An interviewer later returned to eligible households to complete a 'main' interview. We found that eligible households in locked buildings that had received the FtF-Mail treatment on the screening interview responded at higher rates to the main interview. Better even than Mail-FtF. The results were significant, even when accounting for the clustering in the sample. It may be that the FtF-Mail sequence displays our earnestness to the respondent more clearly. It would be nice to replicate these results.

We recently tried to implement a mixed mode approach to a large screening survey that is usually done face-to-face (FtF). We wanted to be sure that modes with lower response rates didn't "contaminate" the sample. If not, there may be cost savings in using a mailed version of the screening survey. We varied the sequence of the mixed mode approach to see if that had any impact. We did FtF-Mail, Mail-FtF, and FtF. We also monitored response rates to the main interview, which is conducted only with eligible persons. The response rates to the screening survey were very similar across the three arms of the trial. But it turns out that one mode combination did better on the main interview response rate with cases in locked buildings. That mode combination was FtF-Mail. This might be fluke, but definitely worth exploring.

I recently went back and read papers written during the early days of the development of CATI. I found this very interesting quote from an article by J. Merrill Shanks from 1983: “Among the procedures that are supported by (or related to) CATI systems, none has proved more difficult to discuss than the algorithms or options available for management of interviewers’ time and the scheduling or assignment of actual calls to specific interviewers. Most observers agree that computer-assisted systems can yield improvements in the efficiency or productivity of interviewer labor by scheduling the calls required to contact respondents in a particular household across an appropriately designed search pattern, and by keeping track of the ‘match’ between staff availability and the schedule of calls to be made” (Shanks, 1983, p. 133). It seems like today people would agree that this is a "difficult to discuss" problem. I'm not sure that there is a sense that there are large gains...

In face-to-face surveys, interviewers play a very important role. They largely determine when they work, at which times they call cases, and how to address the concerns of sampled persons. Several studies have looked at the variability that interviewers have in achieving contact and cooperation. Durrant and Steele (2009) provide a particularly good example of this. It is also the case that interviewers have only a partial view of the data being collected. They cannot detect imbalances that may occur at higher levels of aggregation. For these reasons, it seems like controlling this variability is a useful goal. This may be done through improved training (as suggested by Groves and McGonagle , 2001), or by providing specific recommendations for actions to interviewers. We have had some success in this area. In NSFG Cycle 7, we ran a series of 16 experiments that asked interviewers to prioritize a set of specified cases over other cases in their workload. The results were positive ...