Breaking Down Balanced Clicks Studies
A researcher might use a balanced click sampling strategy when researching new products/brands, to understand the market landscape, or to determine the right audience for the brand and its competitors. For established products or brands, a researcher would use the balanced clicks approach to highlight fluctuations in the landscape or audience. This is done typically on tracking projects, on pre/post-marketing campaigns, or after major events in the world that influence purchasing behavior.
Researchers sometimes use phrases like, “I don’t know my target audience – that’s what I’m looking for” or “I need a true natural fallout from the general population to assess my audience.” The more detail the researcher can provide upfront, the better the sample provider understands the research objective.
LET’S BREAK IT DOWN
- – The world of sampling has evolved from offline, to CATI, to double-opt-in panels, river, and aggregated methodologies. When talking about river and aggregated sample, the concept of balancing does not exist anymore since it requires control of invite outgo and knowledge of response rates – neither concept exists with this type of sample, creating a gap in communication.
- – Some sample providers have adapted and are able (via technological innovation) to set quotas on clicks that make it to the main survey. By controlling the invitations from double-opt-in panels and managing the flow of respondents on various demographics from river, some sample providers are able to set demographic quotas and control the respondents that get to the research survey, thus delivering the needed results.
HOW DOES IT WORK IN THE SAMPLING WORLD?
- – The easiest to manage sources on balanced clicks studies is when using double-opt-in panels. With full control over the number of invitations sent at any given time and statistical predictability of response rates, sample providers using this type of sample can control the balancing throughout fieldwork.
- – River sample cannot be balanced. These respondents are not sent invites, they come in when they want, and the concept of response rate does not apply to them — there is nothing to respond to! However, most sample providers that have river capabilities have used them enough to understand the flow capabilities and can control this via quotas on their end. This type of sample greatly expands reach, especially for audiences that are hard to maintain in an online panel.
- Participant Validation: To validate potential research participants’ identity and personal information, they would need to provide a copy of their driver’s license or submit social media verification. This method would help weed out potential fraudsters and narrow down to participants serious about taking part in the research program.
- Content Security: Film previews are a hot item for leaks and theft, a primary concern for the client. Paradigm Sample crafted a participant policy and NDA to address the legal considerations specific to this research type. The start and conclusion of the survey contained reminders to participants of their obligation to maintain the privacy of the study materials. Media assets utilized watermarks, and survey programming featured proprietary security measures to help prevent piracy.
- Custom Profiling and Targeting: In addition to the typical profiling on demographics (e.g., age, gender, ethnicity, location, etc.) required for representative sampling, many of the client’s screening tests required entertainment-related targeting; for example, some studies required participants to have seen a movie in a theater in the past 6 months, others required fans of specific genres of entertainment or to have familiarity with a particular film franchise. To address this need, Paradigm Sample implemented custom profiling for the sub-panel community of validated participants. This profiling allowed the client to target and access the specific segment of the market they were interested in for a given study. Employing this profiling before the project went live meant the client could save time screening participants and helped increase conversion, resulting in lower costs, increased engagement, and ultimately netting higher quality data.
WHY DO SOME SAMPLE PROVIDERS FEAR USING THIS SAMPLING STRATEGY?
- – First, the risk of miscommunication is extremely high. This can result in completely missing the goal of the project. A sample provider should facilitate clear and manage nuanced communication.
- – Second, with double-opt-in panels, feasibility becomes limited, so using multiple panels to complete a survey results in deduplication which lowers the conversion and thus could increase the sample provider’s cost if they lack the experience in managing these types of studies.
- – Third, with different types of sample, a sample provider has a much more labor-intensive task at hand which requires a high level of attention to detail.
BALANCING VS. QUOTAS VS. WEIGHTING
Quotas are typically used for completed surveys. When using quotas on completes, the researcher knows the targeted audience and sets targets according to the research goals to have enough respondents with a specific demographic profile for analysis. “Balancing on completes” means setting hard quotas on completes.
Weighting happens post-data collection when fielding has finished. This is a mathematical algorithm that can offset the lack of respondents in a specific demographic by counting their answers with a higher weight during analysis versus those where we have higher numbers. Each company uses different acceptance rates for weighting, but it is generally considered that a weight of 2 is the maximum acceptable.
Based on the research objective and the sample size needed for analysis, managing a balanced clicks study becomes more of an art than your “average everyday study”. Given all the recent challenges in the world, we can anticipate changes in consumer attitudes, behaviors, and brand perception. In turn, there will be more demand for brand/product penetration studies.
At Paradigm, we are ready to be your go-to resource! We will work with you to ensure that click balancing is the correct approach – and we will be ready to implement to make your project a success.