This article is part of our blog series on panel management. In this blog series, we’ll present the work of our panel managers and share their daily challenges and considerations with you. Please have a look here to get an overview over all topics covered by this series.

Admittedly, it’s relatively easy to get a lot of subscriptions to any panel. Whole industries are specialized in selling traffic, clicks, followers and conversions and there is absolutely no magic behind building a large database of panel members. But as we have seen already, striving for representativeness increases the difficulty and there is another important challenge to consider.

The overall number of registered panel members can be very different to the amount of people who would actually respond to a survey invitation. But, of course, at the end of the day only the amount of people who are motivated to participate in surveys matter.

Therefore, if you are serious about panel quality, you have to remove inactive and unmotivated members from your database on a regular basis. These inactive members will not respond anymore, no matter how many invitations you send. They will just bloat the overall panel numbers, but they don’t increase the feasibility of any research project within that panel. That’s why you shouldn’t care too much about the overall size of a panel. It’s simply not the size that matters, but the amount of active members.

It is against this background that we always try to make transparent how performant our panels actually are and the key concept here is the response rate. The response rate is defined by the amount of survey starts divided by the amount of survey invitations during a given period of time. We feel that the last 3 months is a very reasonable time frame to report the response rate. Most respondents will have received enough invitations during the last 3 months to make the number dependable. At the same time, this period is not reaching too far into the past; hence the number is meaningful when describing the current state of a panel.

Now, how does the response rate help you to estimate the performance of a panel? The general idea is that you can trade panel size for the average response rate. A panel with 100,000 members and a response rate of 30% corresponds to a panel of 60,000 numbers and a response rate of 50%: with both panels you can generate 30,000 survey starts. This number, the product of panel size and the average response rate, is called the net reach and it is a pretty stable indicator for the power of any given panel.

Obviously, nobody will respond to every invitation, the response rate will never be 100%. Some may click on the link of a survey after the study has been closed or they simply don’t have the time to participate at that moment. That doesn’t mean we should content ourselves. The challenge consists in maintaining a high level of motivation for all panelists and get frustrating experiences out of their way. This is why we’d like to encourage you to have a closer look at response rates and the net reach when digging through panel numbers the next time.

P.S. If you’d like to download our current panel book right away, please click here.