1. Small critique of @FiveThirtyEight forecast (polls-only) that is geeky & similar to what Nassim Taleb has said (we both options traders)
88 replies and sub-replies as of Oct 01 2016

Nate's new methods seem to overestimate underdogs, either to prevent embarrassment or spur interest.
I mean, Johnson at 8%? Come on. I'll write a LOT of Johnson contracts at 92 cents on the dollar. Takers?
538 has Johnson at 8% popular vote, not 8% chance of winning.
Johnson only got 2.5% in New Mexico in 2012 (1% US)... 8% is not realistic
Understood. Decimal point's in the wrong place either way.
Is there model just a weighted average of a non-binary event?
Your critique assumes that the models doesn't include some reversion to the mean to forecast (it does).
isn’t that part of the reason for Polls-plus forecast?
2. The election is an all or nothing outcome that is in the future. GOP wins, or not. (Finance folks: Think digital binary option)
No it is not. It is 50 all or nothing outcomes, each with different vol & weights toward outcome. cc/@mattyglesias
That is same thing. The basket nature changes little, especially given the correlation matrix is close to 1
All a matter of degree. Corr close to 1, is not =1. My beef with them all is implied precision, not that they are not 50%.
50 binary options of varying pay-offs that add beyond 270, is, at modeling level, same. Doesn't change spot/forward dynamic
not sure if Trump victory is necessarily GOP 'winning'* *this may not be what you are saying ;)
3. Forecasting it, via models, uses 2 variables. Where race is now (the spot), & what you project uncertainty is going forward (volatility)
4. Changing either variables changes odds State of race as measured now, as seen by 538, is moving around a lot. Dramatically altering odds
huff post pollster uses a kalman filter for this very reason
5. Which means the volatility is high. But if the volatility is very high, then projecting forward should always yield roughly 50% odds .
agree that forecast seems too confident vs nowcast. Polls-only peak was 89% on 8/14 for HRC, vs 91% nowcast
You'd have to either believe the race is very static (i.e. low vol), or predict directional change against HRC
That said, it's not logically inconsistent. You can could have high _realized_ volatility, with low _expected_ volatility
and @ForecasterEnten discuss this in their podcast. The model is closer to 50% than others bc high uncertainty
Could you expand on why this is? It isn't intuitive to us laymen.
that only makes sense if it's volatile around 50%,it should be volatile around the mean in expectation whatever that number is
but in general you're right in that predicting a binary dependent variable is a weird process
I would argue it isn't a weird process! I did it for 20 years on Wall Street. We have models for it!
"volatility around the mean" is just a fancy way of saying "I hardwired my model" and would get you fired on Wall Street
I mean that in a statistical sense. If there's a 70% chance in reality, the model should reflect volatility around 70%
if you flip a coin a bunch you expect 50% heads (the true mean), but if take coin flip samples you get variance around 50%
this seems to me where u are oversimplifying. Even a volatile system can have prob diff from 0.5. It is not random fluc
But some things are not volatile within this. For example: California and Kansas aren't in play so assuming 50:50 isn't real
volatility very very high and not properly pricing time decay like you would with an option
addresses this criticism in one of the recent podcasts. Can't remember which. Little help @NateSilver538?
A coin toss, in other words.
6. Aside for the finance folks. (That is value for a digital binary option with very high volatility, 1/2, regardless of spot)
I don't disagree but does low prob of 3rd pty make it non binary? @Chris_arnade
not sure about the high volatility. Isn't the projection merely reflecting that Trump has a narrow path in electoral college?
right. could be not high vol overall but low vol effect in a couple of states adding up to high vol looking outcome?
could be. Low frequency of quality polls from specific states (NH, ME, NV, ...) leads to higher perception of vol.
But how high is the vol? If we assume HRC is at 50% and will end up between 40% and 60%, delta is still high
Unless you say the distribution is uniform instead of normal then delta is lower
7. Something is off in model. They assume current state of race is rather reflective of future but then show current state moving a lot!
watching his model for few months "volatility" you describe has been change in trend. unlike mkt vol which involves gyrations
The pretense of knowledge: not just for breakfast anymore.
Marking to model rather than (a more fluid than commonly understood) market.
8. I don't want to pick on @FiveThirtyEight. They doing great work (Primarily in aggregating & weighting of polls) on current state of race
- @FiveThirtyEight is the wrong model to complain about. It stays closer than others to 50% bc they are higher on uncertainty
also much like betting on sports futures. Binary outcomes and very little movement until games start
you are misunderstanding likelihood vs posterior probability
& # of outcomes not 2. Many different outcomes of the electoral college, which the model is built to forecast
Maybe the vol rise is exactly the reason why their forecasts have moved closer to 50% in recent months.
Polls show the state of the race is actually moving a lot. Prediction reflects the likelihood it changes
I’ve stopped attention to them - predictwise.com is better and more realistic
They're a media company. 50% predictions wouldn't get enough views to sell ads. Volatility does.
Their model needs to be this volatile, otherwise their story stales and people stop reading.
I suspect a stochastic volatility model would do worse at predicting elections that whatever 538 is doing.
Right, but you either dampen spot vol (via longer time aggregation) or you raise future vol!
Polls plus attempts to dampen volume by assuming some natural state of the race
there's more models under the sun than that.
I think the volatility comes from them aggressively trying to predict trends in the polls
the difference is that elections are inherently more predictable than the stock market
isn't this the point? It's a "rigged" market where we have reason to believe limited vol.
predicting trends, specifically
I don't disagree. But it argues for NY times approach of weighting current momentum in spot movement less
I think, like stock market, there is both momentum & mean reversion. Trump expensive at 50%.
I agree. fwiw I'm not arguing that 538s model is right/good. I just don't buy Chris' argument.
No trader. Binary means all or zero payout not # of candidates? Gary Johnson?
I'd want to see how 538 does at predicting future polls & the election vs alternatives
Wall Street has been pricing this stuff for 20 years! (ps: what don't you buy?)
that intuition from the stochastic volatility model tells us something is wrong with 538's model
maybe, which one has better MSPE?
basically, I'm not convinced that intuition from a model we both agree isn't good here is relevant.
Many big data type ppl seem to believe that this inherent uncertainty doesn't exist.
I suspect there is some inherent uncertainty beyond which we can't enhance pred power.
Not necessarily that the model is "bad" (that's a bigger, more complex q)
If I may interject, the trouble with fin model is that variability is inherently uncertain.
Right. It's not really a forecast. Sam Wang is better for that, so are others.
Right, but that's to some extent a modeling dispute. Nate's model uses trendline adjustment, betting ...
.... that the state of the race changes around and they can pick it up from the polls.
Wang's forecast uses much longer term snapshots.
Right, which provides more stability.
Yeah, but Wang's assumption about the nature of politics isn't necessarily right !
true, it's a weakness in off years. Prez elections have a lot more polling.
Right, but I guess Silver might have done better in 2000/2004 if he was around then, though I am not sure.
Perhaps. Wang admits his error re 2004.
In both there was significant post-convention/Debate time moves towards Bush, IIRC.
Wang claims that had he not subtly put his thumb on the scale in '04, his model would've been accurate.
2014 Senate races were an example where Silver's model did better than Wang's because of late moving races.