Preference tests are a powerful tool for validating design choices, marketing messages, and user flows before launching them to your entire user base. By collecting real user feedback, you can confidently move forward with changes that resonate with your audience, and avoid costly missteps. 

What can you test?

Feel free to test anything you like, from app page headlines to full app designs. 

Here are a few examples:

• App page headline

• Website headlines

• General app layouts

• Branding or logos

• App onboarding screens

• Menu designs

• Button styles

• Color schemes

• Icon designs

• Navigation styles

• Feature prioritization

• User interface (UI) elements

• Notification and alert styles

• Content hierarchy

• Validate copy

• Offers

• User flows

• App designs

• Logos

• Ads

• Loading animations

• Splash screens

• Error messages

• In-app purchase prompts

• User engagement features

• Gamification elements

• Layout variations for different screen sizes

• Personalization features

• Social sharing options

• Feedback mechanisms

• And more

Targeting the right audience:

The key to great research is ensuring that you're getting feedback from the right people. Our platform allows you to reach a highly specific audience from our 2000+ Shopify and Shopify Plus merchants and operators database. This means your tests will be populated with professionals who have relevant expertise and experience.

Filters to help you find your perfect match:

- Business residence
- Role
- Shop category
- Shopify Plan
- Monthly revenue range
- Number of products for sale
- Size of team

How to run a preference test:

1. Create your test: Set up your test by defining the questions, assets, or designs to be tested.

2. Add images for testing: Include up to 5 images for participants to review.

3. Add follow-up questions: Ask up to 5 follow-up questions to delve into participant preferences.

4. Target participants: Select your audience based on specific criteria or use custom screener questions.

5. Receive responses: Get feedback from well-matched participants within hours.

6. Incentivize participation: Offer incentives, starting at $10, to encourage participation.

Paying for Preference testing

The minimum incentive is $10, and at least 10 participants are required.

The service fees are structured as follows: 30% for the Pay-as-you-go plan, 20% for the Plus plan, and 15% for the Pro plan.

Payment is required upfront.

Example of pricing

Suppose you want to offer participants $20 each for taking a test and you're aiming for 40 participants. If you're on a pay-as-you-go plan, the service fee is 30%.

Here's how the calculation works:

Incentive cost: $20 per participant x 40 participants = $800
Service fee: $800 x 30% = $240
Total cost: $800 + $240 = $1040

Thus, the total amount to pay is $1040.

Benefits:

• Fast results: Receive participant matches quickly.

• Customizable targeting: Ensure your test reaches the ideal audience.

• Comprehensive testing: Test multiple images and questions.

• Seamless experience: Manage the process from start to finish on one platform.

• Expert insights: Gain actionable feedback from industry professionals.

Ready to leverage data for informed decision-making? Whether optimizing layouts or validating messaging, AppStoreResearch provides valuable insights quickly. Start preference testing today to enhance your app’s appeal and functionality.

[Create Preference test]

Preference test tips for researchers

1. Avoid mentioning incentives in the title

Keep your project title short and professional.

2. Provide a clear project description

Make sure participants understand the goal of your Preference Test. Clearly explain the project and what they can expect. Keep it simple and to the point.

3. Keep questions simple and direct

Avoid overly complex or leading questions. Use clear language to ensure participants can easily understand and respond accurately.

4. Follow-up questions should not repeat the main question in a preference test.

For example, if you ask someone to choose from 5 logos, your follow-up question should be: "Why did you choose this version?" Avoid asking, "What version did you choose and why?" since their choice will already be clear. You can also ask, "Is there a version you didn't like at all, and why?"

5. Test one variable at a time.

Focus on a single element per test, such as a headline, button style, or color scheme. This ensures you can accurately determine what drives participant preferences.

6. Use high-quality visuals.

Ensure all images, designs, or assets you upload are clear and professional. Poor quality assets can confuse participants and skew results.

7. Avoid jargon.

Use simple language, especially if your participants might not know technical terms.

8. Analyze results objectively.

Avoid confirmation bias by reviewing data objectively, even if it goes against your expectations. For more accurate results, consider asking additional participants to take the test.

9. Balance incentives appropriately.

Offer fair incentives to motivate participants, but avoid overly high rewards that could encourage rushed or insincere responses.

10. Repeat tests when needed.

If the results are unclear or preferences differ greatly, refine your test and try again with a larger or more specific group.