Currently, almost all e-commerce websites (Amazon, Taobao, etc.) provide a "user review" feature, aiming to let buyers determine the quality of a product. On the surface, this practice gives the public a sense of fairness and transparency. However, is this actually the case? The eighth issue of Global Science this year contains an article titled "Are User Reviews Reliable?", which discusses how relying solely on "user reviews" to judge a product’s quality is inherently biased. Now, a trial begins: the plaintiff is "User Reviews," the defendant is the article from Global Science, and the judge is Mathematics.
The trial begins... "User Reviews" insists that what it displays reflects reality, while Global Science argues there are inaccuracies. What will the verdict be?
Let us assume the following scenario:
Ratings are divided into three levels, represented by 1, 2, and 3, where 3 is the best.
Readers can be divided into buyers and non-buyers, and we assume each group accounts for 50%. For most current websites, only buyers have the authority to leave a review.
Buyers can be divided into two cases (50% each):
Purchasing because they understand the book ("understand" means having a certain level of knowledge about the specific content before buying, rather than just browsing the table of contents on the website). In this case, most readers like the book; assume this group gives a rating of 3.
Purchasing without understanding the book. In this case, readers might like it very much, feel it is average, or feel it is poor. The probability for each of these three outcomes is equal.
Non-buyers are not further subdivided because they do not have review permissions on the website.
Under these premises, let’s assume 12 people evaluate a book, and they would have rated it 1, 2, and 3 in equal numbers (4 people each). The actual average score of this book should be 2. This is the result obtained through an objective investigation of the entire population, rather than through "user reviews."
If, among these 12 people, only 6 purchased the book: For the 3 people in Buyer Case 1, their total score is 3 \times 3 = 9. For the 3 people in Buyer Case 2, their total score is 1 \times 1 + 1 \times 2 + 1 \times 3 = 6. The total score is 15, so the average score is 15 \div 6 = 2.5, which is higher than the actual score of 2.
The discussion above assumes a very simple situation, but it does not lack representativeness. First, it is unlikely that buyers and non-buyers each account for exactly half of the readers; for most books, it is highly probable that buyers are fewer than non-buyers. This assumption is already quite conservative (in other words, it favors the "User Reviews" side). Furthermore, considering real-world conditions, if netizens have experience with online shopping (especially buying books), they will know that "user reviews" usually happen after receiving the goods. If the package is not damaged, people often give the merchant a positive rating out of convenience (as it seems like a trivial matter), rather than evaluating it after a period of careful "investigation." Thus, the two cases for buyers are also conservative (again favoring "User Reviews"). In reality, buyer ratings tend to be skewed higher, and some websites even set a default positive rating if a buyer does not leave a review.
However, even though we have provided "User Reviews" with plenty of "favorable evidence," the final result remains: "User Reviews" loses the case. Although 2.5 is not much larger than 2, once we scale this up and consider real-world complexities, we find that the final result deviates significantly. Therefore, an old cliché might still be the best advice for consumers: Be careful when shopping!
When reposting, please include the original link: https://kexue.fm/archives/894
For more details regarding reposting, please refer to: Scientific Space FAQ