I've been reading papers on email prenotification and reminders. They are very interesting. There are usually several important features for these emails: how many are sent, the lag between messages, the subject line, the content of the email (length etc.), the placement of the URL, etc.
A full factorial design with all these factors is nearly impossible. So folks do the best they can and focus on a few of these features. I've been looking at papers on how many messages were sent, but I find that the lag time between message also varies a lot. It's hard to know which of these dimensions is the "active" component. It could be either, both, and may even be synergies (aka "interactions") between the two (and between other dimensions of the design as well).
Linda Collins and colleagues talk about methods for identifying the "active components" of the treatments in these complex situations. Given the complexity of these designs, with a large number of design features, the fractional factorial designs she describes may be helpful. Further, it might be useful to think of each experiment as a link in a long chain of experimentation (see here). The trick is to design each "link" such that we explore each of the potential design features and any possible interactions with other design features.
A full factorial design with all these factors is nearly impossible. So folks do the best they can and focus on a few of these features. I've been looking at papers on how many messages were sent, but I find that the lag time between message also varies a lot. It's hard to know which of these dimensions is the "active" component. It could be either, both, and may even be synergies (aka "interactions") between the two (and between other dimensions of the design as well).
Linda Collins and colleagues talk about methods for identifying the "active components" of the treatments in these complex situations. Given the complexity of these designs, with a large number of design features, the fractional factorial designs she describes may be helpful. Further, it might be useful to think of each experiment as a link in a long chain of experimentation (see here). The trick is to design each "link" such that we explore each of the potential design features and any possible interactions with other design features.
Hi James, I saw some recent experiments Nancy Bates and colleagues did at the Census Bureau at the nonresponse workshop. She found that none of the things she tested (subject line, content of message) affected response. I think there was one condition were people were more likely to view the survey, but it did not result in more complete responses. We need more experiments on e-mail invitations, as web surveys rely on them. Also, I think that e-mail invitations don't work the same as paper ones. E-mail is much shorter and faster.
ReplyDeleteI'm reviewing this literature because I am writing up the results of an experiment with email reminders. We didn't do anything with the time between reminders. Missed an opportunity. I'm trying to do something on the time between reminders for a study we are beginning next month.
ReplyDeleteI agree, email is different.