Exciting new research based on thousands of reviews posted on Amazon has revealed a surprising trend. It turns out that when people receive free products in exchange for reviews, their ratings are significantly inflated. Even more interesting, product sales are stronger when reviewers disclose that they received the product for free.
These incentivized reviews, where reviewers receive discounted or free products, are quite common on many online retailers. Although Amazon no longer allows compensated reviews, incentivized reviews continue to proliferate on sites like Walmart, Yelp, and Overstock. Additionally, influencers on Instagram and YouTube frequently hype products in paid reviews. The new study suggests that these disclosures do not protect customers from systematically inflated reviews and that new policies are necessary.
The study, titled “Disclaimer in Incentivized Reviews: Does It Protect Consumers?” was conducted by researchers from the University of Florida and the University of South Carolina. It comes at a time when the Federal Trade Commission is investigating the issue of incentivized reviews. The FTC recently implemented guidance that requires reviewers to disclose any incentives they receive. However, according to the study’s authors, this guidance falls short of providing adequate protection.
“Disclosure is not effective in protecting consumers,” said Jinhong Xie, a professor in UF’s Warrington College of Business. Xie and her co-authors shared their findings with the FTC. The article by Xie, UF’s Woochoel Shin, and Sungisk Park of the University of South Carolina will soon be published in the journal Management Science.
The researchers based their study on every review for over 5,000 products across various categories such as electronics, home and kitchen, toys, and appliances. They analyzed data from both before and after Amazon discontinued their original review system in 2016. This provided a natural experiment for the researchers to test the impact of compensated reviews on products.
Prior to October 2016, businesses selling on Amazon collaborated directly with reviewers to offer free products in exchange for a review. Many reviewers were highly active, writing hundreds of reviews. Some reviewers were even semi-professional, ensuring their future earnings by posting overwhelmingly positive reviews to maintain their relationship with businesses.
“We observed that incentivized reviews are consistently more positive than organic reviews for the exact same product,” said Park, a professor of business at South Carolina. On average, incentivized reviewers rated products nearly 0.5 stars higher on a five-star scale. In conversations with reviewers, Park discovered that some genuinely believed they were providing honest opinions, despite receiving free samples and consistently posting five-star ratings.
“This disclaimer does not hold reviewers accountable at all,” Park explained. “I may state that I received a free sample, but I’m still giving it a five-star rating.”
After Amazon discontinued the program, they began removing existing incentivized reviews. This gradual removal allowed the researchers to observe how product ratings and sales changed when products lost the advantage of inflated reviews.
“The removal of disclosed incentivized reviews led to a significant reduction in sales,” Park noted. Customer satisfaction also increased. “Once Amazon deleted these disclosed reviews, consumers became happier. There were fewer one-star reviews after the policy change.”
Although Amazon ended their business-to-reviewer incentive program, they still offer Vine reviews. Through Vine, Amazon acts as a middleman between reviewers and businesses seeking reviews for new products. Since businesses no longer select their own reviewers, people have less incentive to offer overly positive reviews for mediocre products. The Vine program is still in practice.
The researchers found that these marketplace-led review programs provide enough independence to encourage honest reviews from participants, ultimately protecting consumers.
“If you’re Amazon, you’re not interested in inflating the rating of one specific seller. Amazon has an incentive to keep the marketplace fair, but they also want to provide information to consumers. Reviewers, knowing that the party that hired them does not want to inflate the rating, offer fairer reviews,” said Shin, a UF professor of business. “We showed there is no rating inflation on Vine reviews.”
Other retailers, like Home Depot, also provide this kind of independence by matching reviewers to products instead of allowing sellers to select their own reviewers. However, reviews in exchange for free products or straightforward paid reviews are still very common on social media sites like YouTube and Instagram.
“Our proposal is to change the system by requiring more independence in reviews,” Xie suggested. “It’s difficult to change the individual behavior of reviewers, consumers, or sellers. But we can change the system to make it fairer for everyone.”
