A former Google data scientist explains why Netflix knows you better than you know yourself
But one of the most fascinating parts of the book shows how Big Data can also reveal when people are lying to themselves.
Stephens-Davidowitz, who is a former Google data scientist, points to the evolution of Netflix's algorithm to illustrate his point.
In its early days, the company allowed users to create a queue of movies they wanted to watch in the future. But when users were reminded a few days later of the movies they'd picked out, they rarely clicked on them.
Stephens-Davidowitz writes:
As a result, Stephens-Davidowitz writes, Netflix stopped asking people to choose what they wanted to watch in the future. Instead, Netflix made recommendations based on movies that similar users had watched.
"The result: customers visited Netflix more frequently and watched more movies," he adds.
As Xavier Amatriain, a former Netflix data scientist, told Stephens-Davidowitz, "The algorithms know you better than you know yourself."
That's kind of creepy - and creepier when you realize that Netflix is hardly the only business making bank on people's lack of self-knowledge.
When Stephens-Davidowitz visited the Business Insider office in May, he said that many gyms have learned to harness the power of people's overoptimism. Specifically, he said, "they've figured out you can get people to buy monthly passes or annual passes, even though they're not going to use the gym nearly enough to warrant this purchase."
Most people, he said, would be better off paying per visit to the gym, because they may only go a handful of times.
"But the overwhelming majority of people buy these monthly and yearly passes, thinking they're going to go 30 or 60 times. Gyms have made a lot of money capitalizing on the consumer's overoptimism."