«
»

C|net: Just how random is random?

C|Net accuses Apple of favoring iTunes songs over CD-ripped songs in iTunes random playlists. Unfortunately they don’t have the statistical chops to prove anything or do any real analysis:

It’s obviously difficult to tell whether back-room marketing deals or just dumb luck were responsible for the results we saw, but it appears that we can safely lend credence to the suspicions of myriad iPod users around the world. When it comes to choosing songs, ‘random’ clearly is relative.

Actually folks, it’s totally possible to figure out whether your results are random luck or not. For one thing, try repeating the experiment. But what you really need are better statistics. In particular try calculating the chance your results would occur by pure randomness. You haven’t published the raw data, so I can’t do it for you; but this should be well within the reach of anyone whose taken a couple of undergraduate courses in statistics. In fact, it would make a very nice final project for a statistics course. I don’t think it quite rises to the level of an undergraduate thesis though.

Determining statistical significance is the bread and butter of practical statistics in medicine, economics, and a dozen other fields. One would hope basic statistics would be a prerequisite for a journalism degree or a career at a serious news organization. However, if math is not your strong point, call up a statistics prof at your alma mater or hire a professional statistician to help you out. The questions you’re asking are exactly the sorts of questions that statistics was invented to answer. The data you’re working with and the experiments you’re doing are far simpler and more repeatable than most of the systems statisticians analyze. There’s no excuse other than ignorance for not doing the math that’s needed to get the real answer.

«
»

This entry was posted on Friday, March 9th, 2007 at 7:44 AM and is filed under Math. You can follow any responses to this entry through the Atom feed. You can make a comment, Digg this story, or trackback from your own site.

3 Responses to “C|net: Just how random is random?”

  1. Pseudorandom Says:
    March 9th, 2007 at 7:14 PM

    I can think of an excuse other than ignorance: they don’t really want to know, because it might deflate a favored theory that generates lots of chatter, er, I mean web-traffic and page-hits, i.e. revenue.

    Or maybe I’m just being cynical.

  2. Elliotte Rusty Harold Says:
    March 11th, 2007 at 7:31 AM

    It’s not my imagination. Apparently a lot of journalists really are afraid of math. According to Mark Glaser at PBS:

    Clyde Bentley, associate professor at Missouri School of Journalism, told me via email that there’s a disconnect between journalism skills and math skills.

    “A huge number of journalism students select that major because they are math-phobic and they think they will get away from numbers,” Bentley said. “You don’t have to be a mathematician to program, but you can’t be afraid of math.”

    Holovaty, who attended the Missouri School of Journalism, ran into that attitude when he was at school there in the ’90s.

    “When I was in J-school I remember several times, the professor would say something like, ‘Well that would involve math, and that’s why we all went to journalism school — so we wouldn’t have to learn math. Ha, ha, ha.’ And everyone would laugh,” he told me. “C’mon, this is just blatant anti-intellectualism. It’s deeply ingrained in the culture, which is ludicrous.”

    That’s just shocking. I don’t know that every journalist needs to be a math whiz, but most of them do. There are few subjects you can write about sensibly without a solid grasp of statistics and probability.

  3. John Spitzer Says:
    March 15th, 2007 at 5:16 PM

    Computer Science people don’t have an excuse either as there is a magisterial discussion of Randomness in Donald Knuth’s “Art of Computer Programming” – volume 2 – I believe if my memory serves me right.

Leave a Reply