Comments on: The Wisdom of Mobs

By: Julian Sanchez

Julian Sanchez — Wed, 06 Aug 2008 13:18:12 +0000

Jeff-
Sure, I don’t mean to rag on that particular film, really, just to point out that there’s a bunch of different dimensions that go into these rankings. We often (rightly) regard something as a great work of art because it’s innovative or groundbreaking given the tools and methods available at the time, even if, considered ahistorically, later works make more effective use of those innovations.

By: Jeff

Jeff — Thu, 31 Jul 2008 16:17:51 +0000

All good points, except for your unnecessary slap at Buster Keaton’s Steamboat Bill Jr., which, while not as good at The General, is still a great film.

By: Timon

Timon — Tue, 29 Jul 2008 23:45:27 +0000

There is an inherent problem with no-stakes voting that excludes it from the benefits of the “wisdom of crowds” — that wisdom is only present when the crowd has an interest in being correct. If you asked the same crowd to submit 5 movies that they thought would be on a critic’s list of 100 best, and they were rewarded if their choice ended up on the list, that would be the Surowiecki thesis in action. You could do the same for, say, favorite sci-fi movies of the 80th percentile by income of sysadmins at the top 10 hosting providers as ranked by Netcraft, and with a big sample of people who know a bit about movies, you would probably get better results than any given member of that group.

The wisdom of crowds idea is not some brilliant scheme for how to get uncompensated labor, it is a lesson in Hayekian distributed knowledge — if you ask internet users to name their favorite movies you will get… a list of movies internet users feel the need to share their love of.

By: Tom B.

Tom B. — Tue, 29 Jul 2008 18:59:57 +0000

I think there’s a happy middle to be found between the “elite” and the “mob.” Both lists are basically flawed in my eyes. They’re mostly populated by quality films (moreso the Village Voice than the IMDB list in my mind), but there are some very obvious trend lines in both. The IMDB list seems to be based upon both the visceral appeal of a film (a superficial judgment of whether or not one enjoys a film) and the currency of a film in current popular culture. I’d argue that those are both legitimate grounds to include in judging a film, but they’re very incomplete ones.

The Village Voice list tries to strike deeper at the actual components of filmmaking itself as well as the historical import of a film. At one level such a perspective does lead to a deeper understanding of films, but it does so at the cost of negating the visceral impact of a film (or accessibility, if you will) and ignoring more recent productions (it’s very difficult to put a historical perspective on films made recently, hence there’s an incredibly small number of films post-1980 on the Village Voice list).

I like the thought that Julian brought up of forming such lists based on one-to-one comparisons of films. It would help to control for selection biases and it would balance the valid, if somewhat shallow, values of the casual moviegoer with the values of a more reserved, historically-oriented cinephile elite.

By: lemmy caution

lemmy caution — Tue, 29 Jul 2008 18:18:18 +0000

The critics list is better.

This isn’t to say that IMDB ratings are a bad guide to things. The films on the critics list get good reviews on IMDB. Pather Panchali got an 8.0 for example. People just really love The Shawshank Redemption.

By: Peter Orvetti

Peter Orvetti — Tue, 29 Jul 2008 17:37:20 +0000

Any best/worst lists that have continued for decades tend to be self-perpetuating. “Kane” is one of my favorite films, but it is not so superior to other great films that it should obviously be ranked tops on every list. Then there’s something like “It’s A Wonderful Life” — another fine film, but one that was basically ignored for a quarter-century then embraced as a holiday tradition. It is probably not one of the 100 best U.S. films ever.

Presidential rankings are pretty much the same. Why is James Buchanan the Worst President Ever? Well, uh, because… he was the last time we ranked them! Ditto Grant, etc.

By: That Fuzzy Bastard

That Fuzzy Bastard — Tue, 29 Jul 2008 16:42:52 +0000

There seems to be a real split on what a Top 100 Movies list means, too. If you want a list of 100 most important-influential-thoughtful movies, the Voice’s list is the place to go. If you want 100 most immediately enjoyable to a relatively culturally-uneducated person, the IMDB list would be a much, much better guide. So in that sense, the crowds are working perfectly well at telling you what *they* like, just not at telling you what a cultural elite (and I use that term with the utmost affection) would like.

By: Julian Sanchez

Julian Sanchez — Tue, 29 Jul 2008 16:42:10 +0000

Right, look, obviously part of the problem is that there are lots of different dimensions along which a movie can be “best”: As I suggest above, the best movie to sit and watch over beers with your friends might not be the “best” in terms of elegant narrative and character development or innovative cinematography or whatever.

But first, even bracketing that, there’s an exposure problem. Let’s say you just want the best action movie or the best kung-fu movie or whatever. A handful of people will have seen lots and lots and lots and have a decent basis for comparison; the majority will only be able to vote for the ones that are already well known. If this yields results that aren’t wildly off, it’s going to be primarily because what gets to be well known over time already incorporates a certain amount of expert evaluation.

Second, I want to push back at least somewhat against the idea that the “cultural elite” preference is *just* some kind of niche demographic taste. People who are really really into movies display a degree of convergence about which are the great films, and while this may in *part* be attributable to groupthink, in part it’s because they’re picking up on powerful visual or narrative techniques that anyone could appreciate with a well cultivated eye. Being part of the “cultural elite” *just is* developing the kind of eye that responds to more complex and original (or, if you will, “better”) visual storytelling.

By: Christopher M

Christopher M — Tue, 29 Jul 2008 15:41:31 +0000

I haven’t read Surowiecki, so I don’t know if he gets into this. But the real problem here seems to be that what you want out of a “top 100 movies” list is actually more like a “favorite 100 movies of the cultural elite.” And really, there’s not much reason (even accepting Surowiecki’s wisdom-of-crowds theory) to think that a bunch of non-cultural-elite people are going to be useful at putting that list together.

You can certainly quibble about the definition of “elite” here, and I am very sure that taste in film follows a much finer-grained structure than elite vs. non-elite. The real point, though, is that the “wisdom of crowds” functions very differently — if at all — when you’re talking about matters of taste vs matters of fact, functionality, etc.