Project Report on “AI vs. Human Travel Reviews: An Exploratory Study of Perceived Authenticity and Emotional Tone”

Abstract

Over the last few years, the use of artificial intelligence (AI) to create new content has exponentially increased, as big language models such as GPT-4 allow producing realistic and human-feeling text at a scale. The increase of AI-generated reviews on the travel platforms provokes important questions regarding trust and authenticity and the capacity of users to be able to differentiate between the authentic and machine-written review. This exploratory study aims to examine perceived authenticity and emotional tone of AI-generated versus human Airbnb review to determine its implications on consumer trust to digital travel platforms. The research study constituted a mixed method design with a perception-based survey integrated with computation linguistics. As a result, 150 real Airbnb guest reviews and 40 reviews created by AI were analyzed through the natural language processing (NLP) tools including VADER sentiment analysis and Plutchik emotion wheel via NRC lexicon. Textual difference was determined using sentiment scores, emotional distributions, statistical tests, Mann-Whitney U tests, and Chi-square tests. Also, 76 participants were presented with five pairs of reviews (one actual, and one AI-generated) and they were asked which one was “human-like”. The results showed that actual reviews were overall more emotional, with the emotional range that includes joy and fear, whereas AI reviews were more biased toward trust, surprise, and sadness. Participants in 4 out of 5 pairs found the human-rated reviews to be more human-like, which confirms the hypothesis that the reviews written by humans are considered more authentic. However, out of AI-generated reviews, one was significantly chosen, which proves that even the text written by a machine can mislead the reader. This study adds to the rising discussion of AI credibility within digital context and requires an increase in content labeling, user literacy and emotional modeling throughout AI review generation.