Cold outreach is how most app teams try to recruit merchants for research, and it's why most of them end up with thin, slow, or biased data. The problem isn't that cold outreach never works. It's that the few merchants who respond to a cold research ask are systematically the wrong ones, and the effort to reach them quietly eats your week. If you're relying on it to validate a product, here's exactly where it breaks, and why the failure is structural, not a matter of writing better emails.

Does cold outreach work for user research?

Occasionally, and expensively. App teams who've run it describe a consistent pattern: low response, low trust, and a brutal time cost for a few calls.

"The response rate was too low, so it was time consuming to do it at scale." — Inez Wihardjo, Gigit.ai

"Low volume and low quality of participants." — Timothy Daniell, Curvature

"Very unpredictable, and we saw low conversions." — Apurva Chawla, ConvertWise

The teams that stuck with cold outreach longest are the clearest about its ceiling: you can grind out a handful of calls a month, but you can't build a reliable research habit on it. And "a few calls a month" is not enough signal to make confident product decisions.

The 4 structural reasons cold outreach fails

This is the core point: cold outreach doesn't fail because your copy is bad. It fails for reasons baked into the channel itself.

1. It reads as a pitch, so trust starts negative

A cold message from an app team lands in a merchant's inbox looking like a sales attempt, because most cold messages from app teams are. That framing poisons the research before it starts.

"When I do outbound, the recipient sees it as a pitch. But we were looking for innovative Shopify sellers willing to work closely with us and shape the product." — Inez Wihardjo, Gigit.ai

"With cold outreach, it was horrible, brands did not trust us." — Soma Toth, Recart

You can't get honest product feedback from someone who thinks they're being sold to. They either ignore you or give guarded, polite answers, which is worse than silence because it looks like data.

2. It's a massive, hidden time sink

The visible cost is the emails. The hidden cost is everything around them, list building, cleaning, copy, follow-up, scheduling, for a tiny yield.

"We were likely spending 5 hours per week just in list cleaning, copy drafting, and cold outreach setup." — Nihal Mandanna CP, MobiLoud

Five hours a week for a few calls is a terrible trade for a team that should be building. And it doesn't compound, next month you start the grind over.

3. You can't guarantee ICP fit

Even when someone responds, you often can't be sure they're the merchant you need. The core challenge teams name again and again is matching the audience to a specific ICP.

"Our biggest challenge was always trying to find an audience that matched our ICP." — Mark Dodgson, Context

A response from the wrong merchant isn't a win, it's noise that can actively mislead your roadmap if you mistake it for signal.

4. Incentive-driven outreach attracts the wrong people

Teams that boost response rates with gift cards or ads solve the volume problem and create a quality problem: they attract people who want the incentive, not people who have the problem.

"Sent email blasts and surveys with the promise of an Amazon gift card. Needed to send a lot of emails to get very few actual calls." — Yishai Segev, Loox

"So many no-shows, and the quality of feedback was not impressive." — Emily Hughes, Supermoon

Paying for volume selects for incentive-seekers and no-shows, the opposite of the engaged, relevant merchant a research call needs.

The deeper cost: biased data, not just slow data

Here's the failure most teams don't see until later. Cold outreach doesn't just give you less data, it gives you skewed data. Because it's so hard to reach fresh prospects, teams fall back on the merchants they can already reach, their own customers.

"We could only talk to customers who'd trialed our products or existing customers already familiar with it. We weren't talking to potential customers in our ICP who'd never been exposed to the product and could provide a fresh, honest take." — John Chao

That's the quiet killer. You validate with people who already like you, conclude the market wants your product, and build on a false positive. The merchants who'd have told you the hard truth never made it onto a call.

What actually works instead

The fix isn't better cold outreach, it's a different model entirely: let vetted merchants in your ICP come to you. Instead of chasing merchants who read you as a pitch, you post a research project and pre-screened Shopify merchants apply to talk to you. The trust problem disappears (both sides know why they're on the call), the ICP problem disappears (you screen on plan, role, and stack up front), and the time sink disappears (calls booked in days, not weeks of grinding).

App teams who switched describe the difference as night and day, "instant trust between the parties," "on calls within days instead of weeks," and, crucially, access to fresh prospects rather than just their own customers. That's what app store research was built to do.

For the full playbook on recruiting the right merchants, see our guide to recruiting Shopify merchants for user research, and if you're weighing platforms, our comparisons of Respondent.io alternatives and User Interviews alternatives.

Building a Shopify app? Stop grinding cold outreach. Post a research project and get matched to vetted Shopify merchants in your exact ICP, who applied because they want to talk. On calls in days, with fresh prospects, not just your own customers. Start a research project →

Quotes are from Shopify app teams and researchers describing their outreach experience, collected via the app store research network, 2026.