More High-Star Ratings in Product Reviews
Research
27.03.2024
ZEW Study Shows Inflation of Good Ratings
In recent years, the number of stars for products in online shop reviews has seen an upward trend, even as the quality of these products, as indicated by accompanying review texts, has remained unchanged. Consequently, there has been an inflation in the stars awarded. This is the result of a study conducted by researchers from ZEW Mannheim and Stanford University, who analysed over five million online product reviews and compared the content of the text reviews with their corresponding star ratings.
“Both consumers and providers benefit from truthful reviews. Based on ratings, consumers can make online purchasing decisions for suitable products or services. On the other hand, providers can evaluate their offerings on the basis of the feedback they get. In the case of Uber, for example, both drivers and passengers receive ratings,” explains co-author Rebecca Janßen, a researcher in ZEW’s “Digital Economy” Unit. “However, if star ratings consistently improve over time, even when product quality does not increase, online review systems may fail to fulfill their function of providing new customers with helpful experience reports.”
Five-star ratings increase from both experts and non-experts
The study distinguishes between expert and non-expert reviews. It shows that while expert reviews are more factually accurate, their five-star ratings are just as inflationary over time as those of non-experts.
“We would have expected that experts, in particular, would not tend to award more stars. Other users trust them to write reliable reviews,” Janßen says. “We therefore advise both consumers and providers not to rely solely on the number of stars, but also to read the texts in order to get a better idea of the products and services.”
Five million reviews analysed
The study examines how product reviews have changed between 2014 and 2020, using the online wine retailer Vivino as an example. Over five million reviews were analysed based on both star ratings and textual content. The study shows that numerical ratings have improved over time, while the quality assessments in the text-based reviews have changed little. The study distinguishes between experts and non-experts, whereby experts are defined by their follower count, the number of comments per review, or the presence of a website link in their profile. Both groups show an inflation in the number of stars they award.
