Pascal Wallisch serves as clinical assistant professor of psychology at New York University, where he heads the Fox lab.
Predicting Movie Taste
We were interested in how subjective the movie taste of individuals is. To find out, we asked over 3000 study participants to rate a large and representative sample of major motion pictures on a scale from 0 to 4 stars. When we compared these ratings, we found that individuals agree a little better than one would expect from chance, but not by much. In other words, given this low inter-subjective agreement, disagreements about movie quality should be expected routinely. As a matter of fact, we did not find a single movie – good or bad – that everyone agreed on. So if movie taste is radically idiosyncratic, the question is whether experts – movie critics – can tell you whether or not you will like any given movie. The answer is no – movie critics do no better than non-critics. Even the most renowned movie critics we studied, such as Roger Ebert did no better than a randomly picked person in our sample, suggesting that their renown is not due to prediction accuracy. However, if you aggregate individual information, the picture becomes more nuanced. Pooling ratings from many non-critics such as in the Internet Movie Database allows to achieve a prediction accuracy close to the maximum that is theoretically possible, given the inherent diversity of taste in the population. Conversely, aggregators of critic information such as Rotten Tomatoes predict well what a critic will like, but predict what non critics will like only slightly better than individual non-critics, highlighting the divide between critics and the people who use their reviews as viewing recommendations. This study also illustrates the subjective nature of appraisal. Everyone saw the same movies, but there is very strong disagreement about what is enjoyable, implying a highly complex and multidimensional evaluative landscape inhabited by our minds, only some of which is shared with other individuals.