Nebula vs. Hugo vs. SFSite

Around the time the Hugo nominees for 2010 were announced, earlier this year, I compiled average Goodreads scores for all novels that were Nebula nominees, Hugo nominees, and SFSite Readers’ Choice selections for the publication years 2001-2010.  I don’t want to read too much into the results, because they don’t seem especially convincing.

Let me just say that my assumptions, irrespective of the data, are that the Nebulas are a nice way for SFWA members to log-roll each other, that the Hugo voters have long held noticeable collective biases that may imply cliquishness (the many nominations for Silverberg and Sawyer leap to mind), and that the SFSite poll seems to suggest more books I actually like, if only because it offers more options.

Anyway, here are the average Goodreads scores for novels that were Nebula nominees, Hugo nominees, and SFSite Readers’ Choice selections over the past ten years.

Graph of average GoodReads scores for three awards over the past 10 years

Board game epidemiology report for 2011Q2

Board games ordered by the number of new people on BoardGameGeek who reported playing the game in the second quarter of 2011–minus a standard penalty(*) of 500 players for any game not seen before:

1705 7 Wonders
1001 Alien Frontiers
 780 Troyes
 684 Lord of the Rings: The Card Game
 609 Forbidden Island
 598 Mansions of Madness
 598 The Resistance
 588 Dominion
 584 Dominant Species
 505 Small World
 498 Carcassonne
 495 Survive: Escape from Atlantis!
 479 Dixit
 473 Agricola
 439 Pandemic
 431 Biblios
 429 Race for the Galaxy
 428 Puerto Rico
 428 Stone Age
 415 London
 410 Settlers of Catan
 398 Sid Meier's Civilization: The Board Game
 383 Power Grid
 377 Citadels
 374 Ticket to Ride
 373 Navegador

*The penalty is mostly an artifact of my data collection method but also a fair way to measure a game’s spread beyond promiscuous consumers who try everything.

IMDb vs. Rotten Tomatoes

IMDb provides text file dumps of most of their public data. Below are this year’s films to date, ranked by the weighted IMDb score. I’ve included the current Rotten Tomatoes score for each.

Computing the correlation coefficient (r) for the raw, unweighted IMDb score vs. the Rotten Tomatoes score, I find it’s a .7. It goes down to .67 for weighted IMDb scores vs. Rotten Tomatoes, and that’s a weak correlation (r^2 = 0.45), indicating Rotten Tomatoes scores are not generally great predictors of what IMDb users will think and vice versa.

But if we consider only IMDb’s top 20 films for this year, the correlation improves greatly: r = 0.83; r^2 = 0.69.  That’s a fairly strong correlation.  Apparently, when IMDb users really like a film, Rotten Tomatoes critics do too.  It’s the unpopular films that are a bit murkier.

Eyeballing the discrepancies at the lower end, it looks like Rotten Tomatoes may be a slightly better source for a neutral viewer trying to prioritize their movie choices. Both sources promote major franchises (Harry Potter 7.5, Fast 5, X-Men, etc.) as well as smaller, quieter films, but Rotten Tomatoes is usually more discriminating. IMDb users often give average ratings to films hated on Rotten Tomatoes (Transformers 3, Cars 2, etc.), but they never really love a film not also loved on Rotten Tomatoes. And the only film unpopular on IMDb that Rotten Tomatoes rates above 70% is Cedar Rapids. Where Rotten Tomatoes really goes wrong, though, is giving a “fresh” rating to films in the 60-69% positive range–those wind up all over the map in the IMDb ratings.

Verdict: to pick a current movie, just use Rotten Tomatoes and common sense. Focus on movies with lots of reviews that score 80%+ overall, and try not to waste time on anything that scores below 70% unless you feel pretty sure going in that it’s the perfect film for the occasion, in which case you may as well not consider any ratings at all.

Histogram      #  Rating / (Weighted) / Rotten  Title
---------- -----  ----------------------------  -----
0000000016  9944  8.7 (8.09740420271945) 97%    Harry Potter and the Deathly Hallows: Part 2 (2011)
0000000007 11636  8.6 (8.08756490844497) 100%   Jodaeiye Nader az Simin (2011)
0000001222 66135  8.0 (7.91755261445)    86%    X-Men: First Class (2011)
0000000016  5277  8.5 (7.63011960855389) 82%    Zeitgeist: Moving Forward (2011)
0000002311 43727  7.7 (7.59727566503308) 91%    Source Code (2011)
0000001212 17889  7.8 (7.55585236248746) 82%    Kung Fu Panda 2 (2011)
0000001211 29002  7.7 (7.55000937441412) 80%    Super 8 (2011)
0000001222  7895  8.1 (7.54928866452506) 92%    Midnight in Paris (2011)
0000000113 11827  7.9 (7.53579955486616) 85%    The Tree of Life (2011)
0000001213 41411  7.6 (7.49867375199839) 78%    Fast Five (2011)
0000002211 28487  7.5 (7.36661161749295) 88%    Rango (2011)
0000001212 15757  7.6 (7.36008956656185) 90%    Bridesmaids (2011)
0000002211 56625  7.4 (7.33459119496856) 77%    Thor (2011)
0000012211 44707  7.3 (7.22453937577296) 70%    Limitless (2011)
0000002311 17596  7.4 (7.21064284327056) 83%    The Lincoln Lawyer (2011)
0000001212  5658  7.7 (7.14559944559951) 72%    Horrible Bosses (2011)
0000012211 33544  7.2 (7.10969789842383) 71%    Paul (2011)
0000012211 39800  7.1 (7.02990654205609) 72%    The Adjustment Bureau (2011)
0000012211 22779  7.1 (6.98362620737812) 71%    Hanna (2011)
0000012211 20744  7.1 (6.97365229110514) 72%    Rio (2011)
0000013201 37913  7.0 (6.9340063060641)  56%    Unknown (2011)
0000011112 22184  7.0 (6.89278907242696) 57%    Scream 4 (2011)
0000012102 49869  6.9 (6.85460477784714) 33%    Pirates of the Caribbean: On Stranger Tides (2011)
0000012111 45917  6.9 (6.85093730196047) 35%    The Hangover Part II (2011)
0000012212 12425  7.0 (6.82495948136146) 60%    Water for Elephants (2011)
0000012211  7667  7.1 (6.8187587887879)  61%    Kill the Irishman (2011)
0000011112 44401  6.7 (6.66202611759247) 36%    Transformers: Dark of the Moon (2011)
0000022100 27577  6.5 (6.46075481571117) 53%    The Mechanic (2011)
0000011102  6664  6.6 (6.44478476821198) 34%    Cars 2 (2011)
0000022100  5989  6.6 (6.43312938035383) 85%    Cedar Rapids (2011)
0000012101  5118  6.6 (6.41522542498159) 28%    Take Me Home Tonight (2011)
0000011101  9170  6.4 (6.32604765817589) 68%    Hobo with a Shotgun (2011)
0000011101 50264  6.3 (6.28873535596276) 22%    Sucker Punch (2011)
0000012101 27481  6.3 (6.28031560644338) 18%    Just Go with It (2011)
0000012101 16191  6.3 (6.26873534469285) 26%    Green Lantern (2011)
0000112101 37451  6.2 (6.19258361968803) 30%    I Am Number Four (2011)
0000122101 27237  6.2 (6.19007838079176) 49%    No Strings Attached (2011)
0000122101 10633  6.2 (6.17799457199447) 40%    The Eagle (2011)
0000112101  8060  6.1 (6.10000000000005) 44%    Bad Teacher (2011)
0000122100 41870  6.1 (6.10000000000001) 44%    The Green Hornet (2011)
0000011101  8004  6.0 (6.02726281352241) 26%    Your Highness (2011)
0000122100 17891  6.0 (6.01436025082574) 34%    Hall Pass (2011)
0000122101 18868  6.0 (6.01371867569053) 18%    The Rite (2011)
0000011102  3143  5.8 (5.94650822073914) 47%    Diary of a Wimpy Kid: Rodrick Rules (2011)
0000111101 46632  5.9 (5.91208897485494) 35%    Battle Los Angeles (2011)
0000111101  8005  5.8 (5.88178100863249) 55%    Gnomeo & Juliet (2011)
2000000003  3198  5.6 (5.84201355275904) ---    5 Days of War (2011)
0000111101 13169  5.7 (5.77421609252276) 31%    Sanctum (2011)
0000121001 14267  5.7 (5.76949672786243) 17%    Priest (2011)
0000111101  3776  5.5 (5.76564344746171) 15%    Something Borrowed (2011)
0000111101  7350  5.6 (5.74492753623194) 26%    Arthur (2011)
0000111000 20588  5.6 (5.66359165677465) 45%    Drive Angry 3D (2011)
1000111002  3637  5.1 (5.55201145095684) 25%    Hop (2011)
0001221000  5611  5.2 (5.51355243293468) 39%    The Resident (2011)
0001121000 20769  5.4 (5.48835037233374)  8%    Season of the Witch (2011)
0001221000 11180  5.2 (5.39040902679835) 21%    The Dilemma (2011)
1000111002  8668  4.9 (5.20853616729521) 16%    Beastly (2011)
0001111001 15773  5.0 (5.17578437117139) 11%    Red Riding Hood (2011)
1001110001  6883  4.5 (4.98568248507544)  4%    The Roommate (2011)
2000000002  5201  3.5 (4.45110352396056)  5%    Big Mommas: Like Father, Like Son (2011)
8000000000  3672  2.1 (3.89856115107922) 17%    The Founding of a Party (2011)
8000000000 42951  1.4 (1.70684859959523) 64%    Justin Bieber: Never Say Never (2011)