Showing posts with label stats. Show all posts
Showing posts with label stats. Show all posts

03 April 2008

Don't Bet on April 31st...

The Nenana Ice Classic guess submission deadline is day after tomorrow, Saturday April 5. Don't miss it! Guess early and guess often. No, that's voting... Guess late and don't go crazy, that's the best strategy...let me explain.

In my last post I outlined, quite convincingly I feel, what a hopeless guesser I am. How the first year Rationality and Reason prevailed and every year since I picked past winning times kinda willy-nilly and according to birthday's of relatives and pretty girls. Nevertheless I'm going to now segue directly into why I should be convincing you how to guess scientifically... It's back to Reason for me too this year...

First, I established in a comment to the last post that the best guess is April 30, since that is the day on which the most Breakups have occurred. It seems to be the "average" day, and the guess of mine that is almost always closest. But, the question arises, has April 30 always been the average day?

Below is a graph of Breakup date over the years:




OK... "Year" is on the Y-axis, and "Day after Jan 1" on the X-axis is just another way to note the date... What do you see? If you're like me, you see dots... Another way to look at the same exact data is to connect the dots:




OK... Now what do you see? These connected dots show "trends" better, and to me, it looks like the jaggedy line is kind of going down-ish... But IS IT? Well, statistics to the rescue... going back to the original dot view, and plotting a regression line we get...



The red regression line points down. Notice, that the beginning of the line is at about the level of 129 "Days after Jan 1," i.e. about May 8. Notice, that the right end of the line is at about the level of 121 "Days after Jan 1," i.e. about May 1. So, there it is. Breakup is happening *on average* one week early now then when the Ice Classic started. That fact is "statistically significant." All those numbers to the right of the graph tell us that, particularly one that say's "p(uncorr):0.0009916." What that tells us is the chance that this line doesn't actually point down (i.e. have a negative slope, i.e. breakup happening earlier). The chance that this statistical "inference" isn't legitimate is less than 0.0001,meaning a 1/10,000 chance breakup isn't happening earlier, or conversely a 99.999% chance breakup is happening earlier. OK, I'm convinced... but... has it been *gradually* getting earlier. Like, a day a decade, e.g.? Well, look back at the first graph... To me it looks like the jaggedy line starts going south in a hurry about 1970... what do you think? ... If I break up the dataset into two periods 1917-1970 and 1971-2007 and do the same statistical test, on each period individually we get the following. First, 1917-1970:



Well now... turns out that the average date of breakup didn't get statistically signicficantly earlier or later from 1917-1970! Wow... that's interesting... how about 1970-now?



Wow, that's striking. The date of Breakup got way earlier since 1970. What's going on? I haven't a clue.

But, you might fairly ask, great but how is this going to make me rich? Well, all this tells you is the best date to guess, the current "average" date. So, the end of that red line "now" is at about day 119-120 or something, i.e. April 30...

But there has to be something "beneath" that variation or trend in Breakups... Breakup is a combination of things, ice melting because of warmer temperatures in spring, snow melting and flushing into the river and pushing the ice up and out, etc... so, one would imagine that if the ice is thinner, these things would cause Breakup to happen earlier... Is that the case? Well, ice thickness has been measured at the tripod, not since the beginning, but they have since 1989. Below is a graph of the data. On the Y-axis is Breakup date, on the X-axis is not "year," but instead "ice thickness," as measured on about April 1 every year. You don't need to know which dots come from which years, and it doesn't matter (see P.S. if you're interested):



So, here's what this means. Thicker ice in spring tends to result in later Breakups. If ice thickness didn't matter, then the line would be flat. It's not. You can imagine that ice thickness measure on December 1 might result in a flat line, since there's a lot that can happen between then and Breakup, extreme cold snaps, extreme warm snaps, etc... i.e. the predictive power of an ice-measurement on April 1st is much better than the predictive power on Dec 1. Duh... but how "good" a predictor is April 1 ice? Damn good actually. It can explain about 42% (say the stats on the right) of the "variation." If it explained 100% then all the dots would be right on the line, and you'd know exactly when Breakup would be just based on ice thickness. But 42% is darn good for a single "predictor variable." Other things explain the rest. Things like air temperature or how much snow is in the hills contributing to "flushing" the ice out. But we don't have that data. All we have is ice-thickness. So... almost done... what you should do is go to the Ice Classic website, figure out what the measurement for April 1 was this year, find that thickness on the X-axis and see what date to guess... that's what I did in 2004, and I did, meh... OK... Anyway, this is the best reason to guess late, to wait for that April 1 ice measurement, the best predictor. Oh, as for time of day. Throw a dart at a clock... No, just kidding. The early afternoon seems good... Oh, and of course, you can ignore all this and just go with your intuition... In any case, GOOD LUCK!


(P.S. There, apparently is no correlation between year and date or year and thickness in the 1989-2007 dataset... interesting)

Links:
http://www.nenanaakiceclassic.com/

Advanced Reading:

An article in the prestigious peer reviewed journal Science by a Stanford scientist (Wherein Tanana is misspelled Tenana):
http://www.sciencemag.org/cgi/content/full/294/5543/811
And it's interpretation:
http://news-service.stanford.edu/news/2001/october31/alaskabet-1031.html

A caveat (that I haven't addressed) written by the same scientist in the other of the worlds two most prestigious journals, Nature:
http://www.nature.com/nature/journal/v414/n6864/full/414600a.html
And it's interpretaion:
http://news-service.stanford.edu/pr/01/leapyear1212.html

A lengthy retort by some guy on the internet (who has way more graphs than I do, no fair!):
http://www.john-daly.com/nenana.htm

Some journalists have noticed the Nenana Ice Classic over the years
(Thanks Google "News Archive Search"!):



E.g. in 1973, about the time things were heating up... the NYT ran an article:
http://select.nytimes.com/gst/abstract.html?res=F20813F93954137A93CAA9178ED85F478785F9

An article by Sagarin "In Support of Observational Studies":
http://www.esajournals.org/perlserv/?request=get-abstract&doi=10.1890%2F1540-9295(2007)5%5B294%3AWBSOOS%5D2.0.CO%3B2&ct=1

... this may not be my last Ice Classic Post... It's about all I think about this month...

20 March 2008

"I could care less..."

So, though it's been a couple weeks, I should probably not be blogging at this precise moment. I've got some rather pressing things to do, namely I need to put together a talk for a branch meeting this Saturday (American Society for Microbiolgy, AK Branch). It'll be a rather small affair, one invited speaker, a famous microbiologist I've not hear of, and well, a bunch of locals. Despite that dis, I really like this meeting. This year, basically I will have 20 minutes to convince - mostly people from my lab- that I've made progress in the last 6 months. I guess I'm as curious as they are really...

In any case, no, I don't want to work on that right now... what I want to do is tell sundry things to the whole world (or those handful of individuals that will read this in no particular hurry relative to my urge to write it). Aside: The etymology of "Blog" is pretty well established, being a very recently born word... it's a portmanteau of weblog. But I'd argue it's also onomonopoetic. I don't know if anyone has yet to argue that blog is onomonopoetic. I don't want to know, so I won't Google it. I would like to think this afternoon that I'm the first person to be struck that Blog has connotations similar to burp, or belch, barf, other b- words. Like... like.. the sound that verbal diarreah makes when it can't be held back from the keyboard. We all have to blog at times, just let's try not to do it in public or more than we abosolutely have to.

So, despite that introduction, I'd like to announce that I have nothing particularly well thought out or formulated for you today. Don't get your hopes up (ever...), it's just that my fingers get to itching, and I feel like I may suffer consequences if I don't write together the loose threads of thought that seem like they might be braided into something emergent. (I'm seriously swapping/mixing metaphors here, sorry... am I braiding or barfing? not sure)... In any case, today's meandering musings will be on: science training, scientific success, apathy, causation vs. correlation, and beer. Or maybe something else. Oh, and the Gray Lady, the New York Times . She'll make a couple cameos. She's always invited... poor Gray Lady, she probably get's blogged on more than almost anyone else...

So, my title... So, today I sub tought a science course lab for a friend who is out of town. It was a computer lab, and I arrived early to set up the projector. Two students, who having never seen me before, were having no reason not to speak candidly. "What the hell are we doing in lab today?" asks one to the other a few yard away from me (diligently futzing with the projector setup, which I thought might have been a good clue). "I have no idea... I could care less." Right... Actually, I think it might be telling (about me) that the first thing that entered my own private thought baloon was: "Could NOT. You could NOT care less!" A reaction of secondary intensity was the more obvious mild annoyance at the blase attitude. It turned out later these were a couple rather sharp students... but that's immaterial... I've been through enough educational institutions to know that blase is cool someplaces (Turlock High School, I was VERY cool...) and the antithesis of cool elsewhere (At Berkeley, I was, well, sort of cool... I think, but don't quote me on that)... In any case... the first thing I did in lab was have everyone discuss a recent article in the NYT by Amy Harmon. She covers a lot of DNA and genome issues. You can email her at dna@nytimes.com. 'Nuff said. Nick Wade is good too. Read his book: Before the Dawn. Um... OK. So the article we read (http://www.nytimes.com/2008/03/04/health/research/04geno.html)
deals with the story of the first millionaire to pay a commericial outfit to sequence his genome. He forked over $350,000. If you'll allow me to do some math that I believe is original to me and not something Ms. Harmon figured... we have the following:

Equation 1.

three hundred fifty thousand [dollars] divided by six billion [approximate base pairs in a diploid human genome] times 100 [pennies in a dollar]

I kid you not: if you copy and paste the above into Google (sans brackets and included text), it will duly crunch those words and spit out: 0.00583333333. That's right. Folks, by my calculation this guy bought his own "book of life" for 0.005 cents per letter. So, is that a "good deal"? It's a ridiculous question of course. Amy Harmon reports the conceit of comparing the total price tag to a new Bently. Old Jim Watson is then quoted as saying to him it's a toss up between a Bently and one's genome sequence. He of course had his genome sequence a bit back now, for the price tag of $1.5 million... not out of his own pocket though. Jim think's the Bently is a "nice car," but hey, he's already got his genome sequenced... he's looking forward to the day of the "Chevrolet genome." And that's of course the point of the article... the price is dropping... someday... soon... we, me and you, will have the option, or doctor's recommendation...we'd better figure out whether we could care more or less, 'cause soon we'll have to start making some decisions... In fact, there's a whole slate of things to talk about regarding that, but I have two very discrete things I meant to be on track for with this post. First, I wanted to use a slightly different conceit than the luxury car metaphor... how about the more apt metaphor of a book. I'd argue that the story contained in a human genome is the single most interesting story that the universe has ever produced. Woaa, that sounds pretty bold now that I'm looking at it... I'm wanting to hedge now- my natural instinct is to say incredibly declarative brazen things and immediately hedge- but no. I do think this. The human "book of life" says more of interest than any other single record that has ever existed. I believe that. So how does the price compare... Well using the following Google Search

"average book new york times bestseller 'number of words'"

The hit that I liked occured midway down the first page and stated:

"Most nonfiction books average about 50,000 to 80,000 words."

OK, rigorous research done. Let's say 60,000. I'm going to say that $30 is average for hardcover. Please, don't argue... you'll see why...

So, Google, tell me... 30 [dollars] divided by sixty thousand [words] times 100 [pennies per dollar] = [drumroll] 0.05 cents. (finishing touch to the fuzzy math is that the average word has 10 letters, I have no reason other than my own desire to believe that is true... but hey.)

Wow. So I'd say the current rate of a genome is about the same as a new hardcover book on a per letter basis. Or within an order of magnitude one way or another. So, I think IT IS a good deal... And only getting better as the paradigms shift... So, can I ask you "Could you care less?" I know I could. A lot.

Ian

P.S. I just blew an hour on this digital back of the envelope blog and it's math. I'll try to stay away from even my brand of fuzzy math in the future... But, having worn myself out, I'm going to save my musings on the relationship between scientific success and beer and the Times article that goes with that for another day soon. I'll keep you posted. Hey, but I'll put up a couple quotations to end this particularly bloggy blog entry...


"Education is a progressive discovery of our own ignorance."
- Will Durant

... a lot of quotations say about the same thing, to varying degrees of pith, but they're all true...


"Against logic there is no armor like ignorance."
- Laurence J. Peter

... yep... but wearing armor couldn't be bliss... that crap looks uncomfy at best...


"Reminds me of my safari in Africa. Somebody forgot the corkscrew and for several days we had to live on nothing but food and water."
- WC Fields

... this will segue nicely into my next post on scientists and beer and causation vs. correlation...