tag:blogger.com,1999:blog-6211929548240400184.post6349526729695842925..comments2023-05-13T03:35:58.030-07:00Comments on Teddy's Rat Lab: COMMENT: StatisticsSpeakerhttp://www.blogger.com/profile/06067034722507056802noreply@blogger.comBlogger15125tag:blogger.com,1999:blog-6211929548240400184.post-78500519149957833632012-11-03T17:49:26.201-07:002012-11-03T17:49:26.201-07:00No, he will either win the election, or not. Your...No, he will either win the election, or not. Your assessment of the probability (the firmness of your belief) depends upon your model.<br /><br />The whole concept of "probability" means that not all the information is considered (due to what boils down to cost). We may disagree on the probability of a unique event, and neither of us can be proven right -- the event either occurs or it does not. All events are unique, so to use past events to form a model of predicting a future event is to omit information from the sequence of events so that they all may be considered instances of the same class.<br /><br />Take flipping a coin: the outcome is not really "random", at least in theory, because, given detailed knowledge of all the forces that act on the coin during its journey, you could, with 100% accuracy, predict which side shows when it finally comes to rest.<br /><br />Now, the state space is huge: temperature and pressure gradients at the time and at an infinitude of heights, the dynamics of the force of the finger going the flipping (its distribution over the surface of the coin, etc.), even the gravitational influence of large masses in the area all affect the path of the coin. It is safe to say that there are so many possible combinations of conditions that the likelihood that any two flips are identical is effectively zero.<br /><br />Of course, in many cases we make the simplifying assumption that, over enough flips, those influences have a mean bias effect of zero, and that the coin flips can all be placed into a theoretical population from which samples are being drawn.<br /><br />It's the same thing with elections: If we could know he mind of every voter, and compute all the forces on each of the voters (whether, for example, they actually make it to the polls, whether their vote is correctly registered, etc.) we would know with certainty, before the fact, who was to win the election. The cost, however, of obtaining all that information means that nobody could ever do it, and we must be content to express the fact that our knowledge is partial by using probability.<br /><br />Borfhttps://www.blogger.com/profile/01648579535173672965noreply@blogger.comtag:blogger.com,1999:blog-6211929548240400184.post-69697992287136481312012-11-03T11:47:51.824-07:002012-11-03T11:47:51.824-07:00Not to split hairs, but since the election hasn...Not to split hairs, but since the election hasn't happened yet, without question the "probability" of Obama winning is somewhere in the range of values between 0 and 1, not just one of the two extremes.<br /><br />Anyhow -- isn't Silver performing a state-by-state analysis and then (more or less) summing up those results to determine a winner? I realize there's quite a bit of independence issues that crop up (ex. if Romney takes Pennsylvania then his probability of winning Ohio must be increased too) -- and perhaps he's using one gigantic complex model instead of 20-50 smaller ones -- but if it's the state-by-state case then couldn't his methods be examined at much finer detail? I'd always assumed he'd just set up models for 50 states or 20 regions or whatever and then just ran the simulation millions of times.<br /><br />Please forgive me if my impression is wrong; it's safe to say I don't visit the NYT all that often. :)JZnoreply@blogger.comtag:blogger.com,1999:blog-6211929548240400184.post-55046320745081744672012-11-03T08:23:50.649-07:002012-11-03T08:23:50.649-07:00The "probability" says more about Silver...The "probability" says more about Silver's model than anything else. The probability of Obama winning is either 0 or 1.<br /><br />What he's saying is that, 75% of the time, his model can predict the outcome of an election, given the set of parameters he's using, and that, in this case, it is predicting a win for Obama.<br /><br />What he may not be quantifying is the sources of uncertainty: His parameters must have to be measured. How certain are those measurements? How stable is his model in the prediction region (do small changes in parameters result in large changes to the output)?<br /><br />Personally, I'd be willing to place a bet with Mr. Silver using his model to price it.Borfhttps://www.blogger.com/profile/01648579535173672965noreply@blogger.comtag:blogger.com,1999:blog-6211929548240400184.post-33680198246700360522012-11-02T19:04:00.691-07:002012-11-02T19:04:00.691-07:00I do not have a problem with the premise behind th...I do not have a problem with the premise behind the Silver model per se, it's just that he is a captive to data (polls) of unknown province. He has no idea how good or bad the polls are or if the assumptions of partisan turnout are correct. He must take them at face value and run them through the model. I would bet Silver a large sum of money Romney exceeds the electoral vote count he is predicting (234). <br /><br />Also, note that Sean Davis (@seanmdav)very closely replicates Silver's results using a simple monte carlo simulation in excel.sane_voterhttps://www.blogger.com/profile/08122061750041663802noreply@blogger.comtag:blogger.com,1999:blog-6211929548240400184.post-42461235984088161042012-11-02T18:04:46.675-07:002012-11-02T18:04:46.675-07:00OK, I was trying to do combinatorics, but I've...OK, I was trying to do combinatorics, but I've had a few people weigh in, and there's some indication that I was ignoring all valid combinations - while I was thinking that there's no need to duplicate tests which simply change the order of the omitted variables.<br /><br />Still - my calculation simplifies to 1/2 * N * (N+1). The "Test all combinations" resolves to N-squared minus 1 (since the "minus 1" condition represents the test where all coefficients are left out).<br /><br />Now, consider that in fact we are not "leaving out" coefficients, but substituting the 2012 model coeffs with 2008 model coeffs, so the final tally has to be less one, since the final combination is the base 2008 model that we're comparing.<br /><br />Ding, ding, ding! Thanks for playing! <br />Speaker to Lab Animalshttps://www.blogger.com/profile/10060134036743411429noreply@blogger.comtag:blogger.com,1999:blog-6211929548240400184.post-15454688476488266502012-11-02T13:47:20.468-07:002012-11-02T13:47:20.468-07:00That's not entirely true. Bookmakers try to en...That's not entirely true. Bookmakers try to ensure the LARGEST PROFIT, not a balanced book. Imagine an NFL game where the Cowboys should be a true favorite over the Eagles by 6 points, but the betting public overvalues the Cowboys and the bookmaker sets the line at 7 points. 60% of the money may still come in on the Cowboys, and the bookmaker will make out well in the long run. Bookmakers definitely take sides on certain games, they're not always aiming for 50-50.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-6211929548240400184.post-80377499055367765772012-11-02T13:06:41.521-07:002012-11-02T13:06:41.521-07:00Looks like you'll need a correction to the cor...Looks like you'll need a correction to the correction. The number of subsets of the N parameters is 2^N (two to the Nth power) so the number of re-tests to see what subsets of parameters account for the differences would be 2^N-2 (minus two because neither the empty set nor the complete set are meaningful here).<br /><br />The original factorial was a better guess than the sum because at least the factorial function is exponential. But it is wrong because it counts *ordered* subsets and obviously the parameters must have a fixed order.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-6211929548240400184.post-67865662939110907842012-11-02T12:03:09.482-07:002012-11-02T12:03:09.482-07:00It's going to take time and repetition. As a ...It's going to take time and repetition. As a scientist, I am particularly interested in how to use the model to analyze how this election may differ (or not) from previous elections.<br /><br />Thanks for reading!Speaker to Lab Animalshttps://www.blogger.com/profile/10060134036743411429noreply@blogger.comtag:blogger.com,1999:blog-6211929548240400184.post-70782008254660623692012-11-02T12:01:55.931-07:002012-11-02T12:01:55.931-07:00The fact that Silver has successfully used sabreme...The fact that Silver has successfully used sabremetrics and his election models to make predictions that are accurate (within a certain error) validates his *methodology*, but as you clearly state, he doesn't have a whole lot of other tests he can do on the scale of a national presidential election.<br /><br />Validation of the *model* requires repetition and that is very hard to do with something that occurs every 4 years with changing demographics, economy and society.<br /><br />On the other hand, the potential for using the model as an *analytic* tool is quite substantial whether it accurately predicts this election or not.Speaker to Lab Animalshttps://www.blogger.com/profile/10060134036743411429noreply@blogger.comtag:blogger.com,1999:blog-6211929548240400184.post-10550043525385992582012-11-02T11:57:56.608-07:002012-11-02T11:57:56.608-07:00You are correct, and I was in error when I called ...You are correct, and I was in error when I called the function N-factorial. What I wanted was the Sigma - summed- function of N which I wrote out. I've corrected the article.<br /><br />Thanks for reading!Speaker to Lab Animalshttps://www.blogger.com/profile/10060134036743411429noreply@blogger.comtag:blogger.com,1999:blog-6211929548240400184.post-49826615883691656012012-11-02T11:25:08.019-07:002012-11-02T11:25:08.019-07:00You said, "we will need to perform "N!&q...You said, "we will need to perform "N!" (N-factorial = [N]+[N-1]+[N-2]+[N-3]+...+3+2+1) tests to complete the procedure," but that is not what N! means. The factorial is the product of all positive integers <= N, not the sum. So 6! is 720, not 20.Jasonhttps://www.blogger.com/profile/05351091747176176473noreply@blogger.comtag:blogger.com,1999:blog-6211929548240400184.post-16818037974641112452012-11-02T10:53:03.887-07:002012-11-02T10:53:03.887-07:00Vegas odds are driven by the bettors in the end .....Vegas odds are driven by the bettors in the end ... the "odds maker" may start out the betting with a line ... but will adjust that line to ensure that the bets equal out ... they don't try and predict the winner, they try and ensure they have a balanced betting book ... PERIOD ... they are not in business to bet but are in business to act as a middleman for other bettors ... PERIOD ...The Dark Lordhttps://www.blogger.com/profile/02235966436741864018noreply@blogger.comtag:blogger.com,1999:blog-6211929548240400184.post-54409626812826795962012-11-02T10:39:52.632-07:002012-11-02T10:39:52.632-07:00We do have another test of the Silver predictive e...We do have another test of the Silver predictive equations. In 2010, he used his model to predict the 2010 house results. His conclusion prior to the election was that there was a 70%+ chance that the GOP would win less than 60 house seats (they won 63 that year). The house prediction was using different equations, but they are related. So if we say he had a hit with 2008 and a miss with 2010, then you at least have some "breadth and depth" of results to measure and at 50-50 he isn't exactly hitting on all cylinders.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-6211929548240400184.post-45856347995194114702012-11-02T10:37:58.419-07:002012-11-02T10:37:58.419-07:00THe fact is, Silvers "probability" is no...THe fact is, Silvers "probability" is nothing more than what a Las Vegas odds maker does. His claim has no statistical value whatsoever. The Housenoreply@blogger.comtag:blogger.com,1999:blog-6211929548240400184.post-41958931522383947902012-11-02T10:15:25.836-07:002012-11-02T10:15:25.836-07:00Put simply, if Silver's model predicts Obama t...Put simply, if Silver's model predicts Obama to win with a 75% likelihood, but Obama loses anyway, that could mean either that Silver's model is wrong, or that its correct with Obama having experienced an expected 1 in 4 loss. <br /><br />Without dozens or hundreds of other otherwise similar elections to validate Silver's model against, there is no way to know if his model is correct or not. In practice, since every election is unique, Silver's model probably CANNOT be validated this way in the real world. <br /><br />That doesn't make the model wrong, but with such a small sample size to compare the model against, this leaves open the very real possibility that any earlier electoral prediction success using it was due to dumb luck, rather than sophisticated modelling skill. This would be true, even if Obama won. . .were his "true" odds really 75%?<br /><br />Now, all that said, even if we can't apply rigorous mathematical testing to Silver's model for lack of numerous national elections to compare against, there are still real world 'common sense' measures that we could apply. <br /><br />For example, hypothetically, if Romney were to completely blow out Obama, winning a decisive victory with several points of the popular vote, that would, at the very least, suggest that a model predicting a 75% likelihood of an Obama victory was significantly flawed. <br /><br />Anonymousnoreply@blogger.com