News Archives

Shop around if you're making travel plans

June 4, 2015

Full Article
Study examines how many hotel reviews are truthful
By Kim Delker- June 01, 2015

Trueview presentation at the Internatioonal World Wide Web Conference in Florence, Italy.


Vacation season is upon us and with the popularity of online review websites that give travelers a quick snapshot of hotel’s quality, making travel plans is as easy as ever. But can you really trust what you see there?

That’s what a team from the University of New Mexico’s Department of Computer Science sought to find out.

Abdullah Mueen, assistant professor of computer science, said the team was inspired to conduct research on the topic because of what they had personally noticed on various review sites like Hotels.com or TripAdvisor.com. In looking through reviews, Abdullah said they had noticed what they suspected to be fraudulent reviews, such as very similar reviews for the same hotel across different review sites or a lot of very positive or very negative reviews.

The team’s solution was to develop a cross-site analysis system called TrueView that provides a score that consumers can rely upon to be free of fraud. Instead of relying on just one review site, it aggregates the reviews to give a more accurate picture.

“The goal of this research is to have a fraud-free review system,” Abdullah said. “The TrueView system we have developed gives a hotel a trustworthiness score so the user knows a hotel has review discrepancy across websites. Therefore, the consumer can use the trustworthiness score as opposed to the rating given by the review website.”

The researchers compiled data from more than 11 million reviews from Booking.com, more than nine million reviews from Hotels.com and more than three million reviews from TripAdvisor.com from as far back as 2001.

The team began their research by catching the fraudulent reviews on each of the sites, which they say was a challenge.

“This is really hard, because if they are lying, how can you know?” Abdullah said. “We found that it’s not the text that will tell you if it’s fraud or not, it’s the overall behavior of the hotel or restaurant across the Internet, so that’s what we looked at.”

“If a single review is spam, you tend to ignore it, but if you know a hotel has a history of doing that kind of thing, then you probably don’t trust their quality as much,” Amanda Minnich, UNM computer science graduate student.

Another challenge was that not all reviews included text. Some were just a rating. And some reviews didn’t seem to be from verified users.

“Sites like Hotels.com claim that most of their reviews are from verified guests, but they have 25 percent empty reviews, meaning just the rating is there, no title, no text of who is leaving the review. That means nothing. If someone puts a hotel at 4 stars and they are not verifying it, how much trust do you give that rating?” Abdullah asked.

In addition to having the same text issued for the same hotel among different review sites and a large number of positive or negative reviews, another issue is suspicious timing of reviews.

For instance, in looking at a place like Myrtle Beach, which has a huge hotel occupancy difference between summer and winter, if there were a large number of reviews posted in the winter months, that would be suspect, Abdullah said.

The team discovered something that a lot of consumers don’t likely know about: that many reviews are fake, written by people whose job is to write positive or negative reviews on products or services. The research indicated that only about 20 percent of the fake reviews are flagged and removed, which means that the majority of them remain on the site.

This is challenging since the practice of hotels contracting people to write fraudulent reviews is technically a legal practice, Abdullah said. The hotels and review aggregator sites are aware of the issue but are constrained legally from doing much about it.

“Most of these sites want to red flag fraud. But legally, they can do it only if they are 100 percent certain,” he said. “We need a generalized method that will evaluate each and every hotel or each and every product. This is something the websites are not doing right now and what makes TrueView unique.”

Amanda Minnich, a computer science graduate student who worked on the project, said the value of providing an aggregate score across sites is that it allows the consumer to easily recognize a pattern of fraudulent reviews.

“If a single review is spam, you tend to ignore it, but if you know a hotel has a history of doing that kind of thing, then you probably don’t trust their quality as much,” Minnich said.

The team’s goal is to create a website or product that can be used by the public that will provide a TrueView score, or an aggregate score, that can be relied upon to be free of fraud. Until that product is developed, their advice is simple: Use data from multiple review sites before booking a hotel.

The findings from the research were compiled into a paper called, “TrueView: Harnessing the Power of Multiple Review Sites,” which was presented this month at the International World Wide Web Conference in Florence, Italy.

The paper was authored by Abdullah; Minnich; Nikan Chavoshi, a research assistant in computer science; Shuang Luan, an associate professor of computer science; and Michalis Faloutsos, professor and chair of the Department of Computer Science.