Skip to main content


Showing posts from August, 2011

Experimental Design in Surveys

Sorry for the long layoff! I had a very busy Spring.

I've been working with the results of an experiment we ran on a survey last year. The experimental condition that we wanted to vary was the mode of contact. The results were a bit messy. We didn't have complete control of all the conditions. The main issue was that we couldn't insure that sampled units in each arm of the experiment received the same treatment (equality of effort -- number of calls distributed over windows and across time in equivalent manner).

This is a common problem for experiments that we run. Most of our experiments are 'piggy-backed' onto data collections for which the experiment is a lower priority. They actually need to collect data.

I've been focused on the negatives, the messiness of this situation. But there is a positive. Most of these experiments are embedded in real-world situations. Hence, they should have greater external validity. If we try them again, many of the same essent…