Friday, February 3, 2017

Survey Data and Big Data... or is it Big Data and Survey Data

It seems like survey folks have thought about the use of big data  mostly as a problem of linking big data to survey data. This is certainly a very useful thing to do. The model starts from the survey data, and adds big data. This reduces the burden on respondents and may improve the accuracy of data.

But I am also having conversations that start from big data, and then fill the gaps with survey data. For instance, in looking for suitable readings on using big data and survey data, I found several interesting articles that come from folks working with big data who use survey data to validate the logical inferences they make from the data as with this study of travel based upon GPS data, or to understand missing data in electronic health records as with this study.

Now I'm also hearing discussion of how surveys might be triggered by events in the big data. The survey can answer the "why" question. Why the change? This makes for an interesting idea. The big data are the starting point while the survey data are additional to fill the gaps.

Both approaches are valid and useful. As we develop more and more approaches to these uses of data, we may need some new taxonomies to help us think through all the options we have.