Limiting Feedback in Crowdsourced Contests Can Yield Best Outcomes
When you’re conducting a crowdsourcing contest, conventional wisdom says the more feedback you provide, the better. However, a new paper coauthored by Ross School of Business Professor Damian Beil concludes that’s not necessarily the best approach.
The study finds that providing feedback can be highly effective — but when you provide it matters. Providing feedback only during the later stages of the contest can yield the strongest results, the researchers found.
The paper — which is forthcoming at Management Science, and was coauthored by Ross PhD graduate Zoey Jiang and Yan Huang, both of Carnegie Mellon University — studied the effects of feedback on the results of crowdsourced logo design contests. The authors explain that although crowdsourcing platforms typically recommend providing feedback to entrants as often as possible throughout contests, their study shows that approach does not lead to the best outcomes.
Feedback given publicly to contestants can help them best meet the specific design needs of the sponsor, and the study found that it can foster quality submissions. Yet at the same time, this feedback can reveal the level of competition, which can discourage potential entrants.
“Providing performance feedback can generally increase the number of submissions and improve the quality of the submissions, especially the quality of the best submission,” Beil said. “However, it is not true that the more feedback the contest holder provides, the better. Compared with providing feedback throughout the contest, providing feedback only in a later stage of the contest is more cost effective and can lead to even better contest outcomes.”
Offering feedback only in the later stages of a contest can best strike a balance between valuable guidance and discouraging competition, the researchers found. In fact, the study found that following this approach increases the diversity and quality of entries — increasing the number of high-quality submissions by 5 to 10%.
Jiang concluded, “Feedback helps creators effectively choose between exploring new design styles and revising existing designs, but it can have a discouraging effect on new participants’ entries and existing participants’ follow-up actions. The late-feedback policy attains the former benefit while mitigating the latter problem, by only giving feedback after many creators have had a chance to enter.”
