Why You So Stupid Netflix?
Monday, August 16, 2010 at 12:51AM 
Which would you rather have for lunch — something a good friend ordered for you, or the most popular lunch among people who also enjoyed such hits as Turkey Sandwich and Fish Taco?
A few years back, Netflix made headlines by offering a prize of one million US dollars to anyone who could build a better recommendation engine for their online movie rental site. Crazy? Not according to Netflix. Their assertion was that if they could get just ten percent better at recommending movies based on users’ ratings (1 to 5 stars) of previously-viewed films, their revenues would increase by much more than a paltry $1M.
What has always boggled my mind about this challenge is that it’s a classic case of struggling to find a technological solution to a distinctly human problem. Google labors to perfect computer algorithms that convert recorded speech to text. For all their research and computational might, they do a pretty poor job. Meanwhile, a small, smart company called CastingWords* uses Amazon’s “Mechanical Turk” service to assign transcription tasks to hundreds of eager, human laborers who work for pennies. The results are near-perfect.
Netflix is sitting on a nearly Wikipedia-sized repository of user-generated movie reviews. These reviews a not just free to Netflix — since only active members can contribute them, people are actually paying for the privilege of reviewing films on Netflix’s site.
Netflix not only ignores these reviews in recommending movies, it also ignores your reaction to the reviews. This is the 100% human answer to their “technological” problem that has been staring them right in the face for years.
When I give Face/Off five stars, am I doing it because I love John Travolta Movies? Or Nick Cage flicks? Or John Woo films? Or hyper-violent 90s action? Or any film that features a speedboat chase? Netflix has no idea why I like or dislike a movie, so how can they predict what I might like or dislike next?
When I read the Netflix reviews of Face/Off, two things are abundantly clear: First, many people like that movie for reasons I don’t agree with. Second, people who don’t like Face/Off, with a few notable exceptions, are people whose cinematic opinions I can live without.
In Real Life I have friends who love Face/Off and friends who hate it. And crazily enough, I respect all of their opinions. Any of these friends are welcome to recommend movies to me, and I will almost always take those recommendations.
That’s Netflix’s second mistake: Thinking that we always only want to watch movies that we’d rate highly. I don’t know about you, but I watch plenty of movies that I know won’t be five-star favorites. A friend’s strong endorsement is often the reason — even that guy who hates Face/Off.
But let’s get back to Netflix’s first mistake: Thinking that what everyone thinks matters. I’m sorry, but everyone is an idiot. Even with the challenge complete and the fancy new algorithm implemented, everyone seems to think that because I liked Mission Impossible III, that I’ll jump for anything starring Tom Cruize. Or that liking the first four Steven Seagal films has anything to do with one’s opinions of his subsequent works.
Read some Netflix reviews. While a few are insightful, most are utter garbage.
If a person’s review is garbage, then what good is their star rating? None whatsoever. So the majority of the data used by these million-dollar algorithms is worse than worthless. No wonder the results plateau despite endless efforts.
Of course, your garbage might be my delicacy. On Netflix, you can rate reviews Helpful or Unhelpful. But Netflix obstinately sticks to it’s “everyone’s opinion” philosophy and uses these ratings to bubble Helpful reviews to the top of the list.
The list. No matter how you rate the reviews, you see the same list as everyone else.
You can probably imagine what I’m going to say next. Unless, of course, you work for Netflix.**
If I read a compelling, insightful review on Netflix, and I mark that review Helpful, then factor that person’s ratings higher in determining what films I might like. Similarly, if I mark a review as Unhelpful, then don’t let that fool’s opinions influence my recommendations.
Let me build a personal network of friends whose opinions I respect, and let their recommendations populate my personalized Netflix experience. It will immediately become obvious that a few, cultivated opinions are worth much more that a watered down average of millions.
Oh, and speaking of millions, you can keep the check Netflix. Better recommendations will be reward enough. In case you hadn’t noticed, I’m hooked on you like strawberry crack.
—
* I found out about CastingWords in Seth Godin’s amazing book Linchpin. If it doesn’t make you want to quit your job, then maybe this will.
** In which case you might sponsor a contest to develop a robot that can predict what I’m about to say based on what everyone else has said.
Reader Comments (17)
Stu, how do you feel about the Amazon recommendations? Either I am way to predictable or their algorithyms are better, since their recommendations hit the mark more ofthen than not. I've ordered some great books on their 'you might also like' options.
Click on any of my Steven Seagal links and you'll see Amazon's recommendations. In each case, they recommend only the other three of those first four (AKA the Golden Period of Seagal). So score one for Amazon right there!
I 100% agree with you here. I think it's a pervasive philosophy in modern life that if we just tap into the collective wisdom with algorithms and pattern recognition we'll explain all of life and everyone's problems will be solved.
I also think there's a larger distrust of people (but not websites...) who try to tell us what is good or bad - thus the loss of critics of a more serious nature in papers. I would love if Netflix pulled in more critical reviews so that I could tell it to give more heed to what they recommend, as I do tend to use certain critics as a judge when deciding what films to see.
But my real question is, how do you find time to watch movies in between making all the cool stuff you make? I'm working a 9 to 5 at an ad agency and trying to build a freelance career outside of that and I just have no time for movies.
in germany, we have moviepilot.de. on this site you can rate all the movies you have watched and then you get pretty good recommendations with a rating 1-10..
also you can see all your friends and how near your movie-taste is an a scale 1-100%
nevertheless, i want netflix in germany with a good pricing!
Here in Spain it is www.filmaffinity.com that most people use
The algorithm works reasonably well... at the beginning. Then, as you see and vote more and more films, it is more difficult to find recommendations, and it makes more mistakes.
I would have loved to take part in that netflix competition, what a pity I didnt get the news on time...
NETFLIX is missing a huge opportunity to "manage" or combine a single user/social networked experience by tie-ing in Twitter/Hashtags/FacebookConnect etc... to allow info from all of these to filter thru into their review system as well as unique "in" Netflix reviews and info.
Agreed. And yet...
Based on the uniformly low star ratings I see on Netflix, I have concluded that people who use Netflix don't actually like motion pictures.
Perhaps that is why the whiz-bang "suggest-a-tron" doesn't work.
Set Might_Watch_Movie=1
IF Rotten_Tomatoes >=50% AND GENRE='action' OR 'thriller'
UNLESS Running_Time >=180 minutes OR DIRECTOR='michael bay'
I can hardly wait for the Recomdista app you're sure to develop now:)
As soon as I become average, Netflix's system will work perfectly for me.
Fair enough. My point is that it tends to work pretty well for most people, I suspect. The data sample is pretty enormous, and I'm sure there are people just like me. The suggestion rate is about 80% positive for me, and the documentary suggestions are nearly 100% (but I'm a doc whore, so that's not surprising). The algorithm is a lot more complex than, "Likes Travolta movies, suggest more of them." It's probably more like, "Likes Travolta, hates Cage, hates Paramount, likes action, sometimes Woo, is a fan of the 90's, so given that criteria, the hundred people with the most similar tastes really liked..."
It's all about the number of inputs, and I suspect that for the vast majority of people, they find the right trends and suggest the right movies. I did my part and gave Face/Off two stars. :)
They should definitely be basing it on Facebook. Although there are movie ratings apps on facebook, most of them are left unused after a while because they are not as integrated with the other activities in the way youtube clips are. They should have paid FB the same amount of money to integrate a short and easy to use movie recommendation status perhaps with some short trailer of movie still.
This is dead on. Not too long ago I had e-mailed them telling them how disgusted I was with their reccomendations, adding that they needed some kind of "network." They also need to delete their whole database of reviews (if that's what you'd call them.) I want to read about the movie, not why an actor should give up acting. All rental areas around me have closed down because of Netflix, so can I at least get the service I'm paying for? Anyway, I'm glad someone like Stu took the time to go in-depth on this problem.
I have to be honest, I find my Netflix recommendations to be almost supernaturally accurate. Maybe it's because I've taken the time to go through and rate hundreds of films, and not just the ones that I watched via Netflix? Almost without exception, their "best guess" for films that I happen to have seen but not rated yet are almost exactly what I would actually rate that particular film, right down to the decimal point.
It was funny; the other day I was strolling through my Instant Queue with my roommate trying to figure out what we wanted to watch. We came across The Messenger (which I had already watched, but not yet rated on Netflix), and he asked me about it. I started going on about how it was one of the best movies I'd seen in recent years, but before I even finished the sentence, he clicked on it, which brought up the "best guess" star rating. It was 5.0. I stopped talking mid-sentence, pointed at the star-rating, and said, "Well, there you go."
This sort of thing happens with shocking regularity. I haven't read anything about their fancy-dancy algorithm, but for all the difference it would make, they might as well just pay a psychic to sit next to me at the computer and tell me how much I'm going to like stuff.
Hah I completely agree with ya Stu! and here is my canny input :)
All netflix had to do was take a look at what the other big boys are doing, like Pandora. Pandora uses your email address to recognize your network of facebook friends, then makes suggestions based on what your friends liked on pandora and what type of music they liked on facebook.
They shoulda put that million towards cutting a deal with facebook for that service and... SLAM DUNK! More compelling movie recommendations because guess what? They now made watching a movie a social networking event.
*** just an idea that popped into my head***. Imagine if they had a quick scrip that updated facebook/twitter (and eveything else social) with what movie you watched(or are watching) via netflix. I am pretty sure that script exists somewhere for free online.
As a matter of fact, netflix doesn't even have a simple share link on any video title... big failure.
and for those who may argue you have to be logged in to see a movie title, that is where netflix will convert the non netflix users. Big win!
Quite an old post, but there's a great recommendation engine at http://filmaster.com
There is also an iPhone app now, and Androind and other platforms on the way. Quite possible it's going to be the best recommendation engine out there pretty soon.