After reading this article on Hacker News I got curious about PickFu. It’s a service that allows you to ask a single A/B question to hundreds of persons, through Amazon Mechanical Turk.

It has several shortcomings though:

• The sample is skewed both geographically (USA/India) and in genre (more females)
• The sample is probably not related to your desired audience
• Your paying them, so expect more sympathetic answers
• Many answer as quickly as they can, without valuable feedback

Having that in mind, we decided to try it anyway. Since we launched an introduction page for Bundlr recently, we asked 50 persons whether they understood the information on the home page.

Overall, the results were 76% positive (sympathetic as I said :P). But the comments were the valuable result. Some examples of negative comments mentioning specific aspects we have to improve:

• Don’t know what a bundle is or where I would share a bundle.
• I have no idea what content curation is. Sounds like you are bundling something together.
• This is some complex stuff. I better description with illustrations is required for me.

And even the positive comments were insightful:

• It was very clear, I just question how it’s different from Stumble Upon etc
• I understood the concept and it was fine for “coming soon,” but some ideas on what the bundles could be used for or why the service exists would be nice.
• Sounds nice to have it all saved in one bundle instead of continuously doing one at a time.
• At least, I think I understand it. But what does a bundle look like? How is it ordered?

I won’t recommende acting soleny upon this kind of feedback, but it’s certainly interesting and gaves us a few toughts on how to better present our concept in the future. For only $5. We’ll probably use it again in following iterations.