If you work in the research and insights industry, have you ever stopped to contemplate who is actually participating in our research projects? In particular, our online surveys conducted through panels and marketplaces (I am not referring to branded community panels as the expereince is generally pretty good). This is something I have been looking into and I have to say that the average online survey experience is not as welcoming as one might expect in a world where ‘experience’ is so important.
Before we even consider the visual appeal or the questions themselves, accepting an invitation to participate in a quantitative survey can more often than not result in a screen out or quota full message. Whilst these are standard industry practices that are convenient for researchers, it isn’t particularly pleasant for those participating. They have no idea about screeners and quotas, they are just responding to an invitation to do a survey. Imaging being invited to an event only to be told upon arrival that you aren’t welcome anymore because you don’t meet the entry criteria but come back next time and try your luck because we really value your opinions.
There are plenty of other things to do.
Is it a big problem? The answer will depend on what you believe the consequences of poor survey taking experiences to be. If you believe that ‘respondents’ are always there eagerly waiting for us to send them something to respond to and will keep responding over and over again regardless of what they are responding to, then you would probably argue there is no problem.
But if you believe that respondents are real people living in this modern world where experience is everything, you might believe that they will stop participating after a few poor experiences. This is certainly a big problem because the people who put up with a poor experience might not be the ‘people’ you were trying to reach?
Many market researchers are telling me they are having to replace up to 40% of their sample (let’s call them people) due to responses being unusable i.e., from bots, click farms and 12-year-olds saving up for their new electric scooter. Before you argue ’Jason calm down, 12-year-olds aren’t doing our surveys’, check out this article about this very topic. They ARE doing our surveys. Is this evidence that we are indeed discouraging real people from participating in online surveys? It’s worth contemplating.
Participation crisis equals data quality crisis?
Over the last 10-15 years, the industry has been talking about data quality from online surveys almost ad nauseum: people doing online surveys aren’t always who they say they are. The challenge, of course, is that online surveys are self-completed; from the ‘profile’ information to the survey answers – it’s always been this way. Whilst online surveys offer speed and cost benefits, the digital experience offers anonymisation that makes it easier to cheat. And the reality is that our industy is being targeted by those who want to cheat the system.
So, we face the prospect of being left with the only 'people’ willing to put up with survey experiences being bots, click farms and those looking to make some extra bucks - this is most definitely a participation crisis, and it is also most definitely a data quality crisis.
It’s not beyond repair.
I recently received a marketing email from one of the industry’s most well-known suppliers telling me I should come to a webinar to find out the benefits of collecting opinions at speed and at scale whilst also saving on my budget. It was all about the convenience to the researcher, but nothing in there about the impact on participants and on the quality of this information. If this is the attitude that reflects the industry, then it may well be beyond repair.
However, the good news is that I have met a growing number of researchers that doesn’t share this attitude. The curtain is finally being lifted and a collective of brave researchers are asking the dififcult questions of their quantitative survey suppliers. At Verve we want to work with those who take this issue seriously, and we are now taking further steps to improve the situation.
Working alongside several partners, we’re developing a solution that allows market researchers to get the answers they need whilst encouraging ‘verified’ people to participate - those who would normally otherwise not be participating in market research. A challenge that is highly rewarding, interesting, and certainly provides us with the opportunity to give our clients better results.
Here’s to re-humanifying market research experiences for the people who really matter – the participants - without whom market research isn’t anywhere near as valuable.
If you want to chew the fat about this important industry issue, I’d love to talk firstname.lastname@example.org.