Introducing the State of the Senate

This is so awesome.  Really.

It's still pretty early in the 2018 election season.  Only Texas even has official candidates. But it's never too early to make wildly optimistic projections. As a public service to help you figure out how to allocate your resources, I proudly present the State of the Senate.

I described some of the technical details of my projections in a previous post, but the approach involves a few factors:

1. History: If an incumbent is running for re-election, how much did they over- or under-perform the presidential race in 2012? Assume that they're similarly poised to do comparatively better than a generic candidate of the same party.
2. 2016: How did the state go in 2016, and by what margin?
3. Incumbency advantage: There's about a 7.5% incumbency advantage, so if the incumbent is running, include that.
4. The Blue Wave: Specials and the generic ballot suggest anywhere from a 6 to 15 point Democratic advantage.
5. Polling: It's still early, but most of the interesting states have some polling.  I'm assuming that it's not entirely meaningless.

I've only included 15 states as "interesting": the 10 Dem states that Trump won, along with Nevada, Tennessee, Texas, the Mississippi Special Election, and Arizona.  There are others, like Maine or Minnesota, that an argument could be made for, and if polling suggests they might be a close race, I'll add them in.  But for now, my assumption is that if we lose Maine, then things have probably already gone pretty badly.

You can adjust the generic ballot, the incumbency advantage, and how much you want to weight history in your statewide estimate, as well as how strongly you want to weight polls.  The plot will dynamically readjust and show you the most competitive states.  The outline of each point indicates the which party holds the seat at present.  Hovering over each point gives you the state, the candidates (my assumed candidates), polling and more.  You can also see the complete underlying data here.

But you should also consider some serious caveats when using this (or considering anyone's senate projections).  Some important considerations:

1. The models don't have errorbars, and I should include them.

There are sampling errors in polls, natural variability in states, and much else.  Even if we believe a particular model, each component of the model should include some source of error.  Because we average multiple components and some of the errors might be uncorrelated (or even anti-correlated) maybe they partially cancel, but at any rate, it's entirely possible that the plot predicts a candidate winning by 5 and they lose. That's the failure of not include errors.

2 States swing a lot from cycle to cycle

Here's a plot of presidential results from 2012 and 2016 in the 33 states with Senate races this year:

Obama did better than Hillary nationwide, by about 4 points, but even correcting for that, the scatter in these states was large, about 8.5%.  In other words, states are going to change, independent from broader trend, by a fairly large amount.

The individual candidates also matter a lot.  Yes, Democrats like Tester tended to overperform Obama, but the range in potential variation is quite large, about 13%.  This isn't noise per se. Indeed, including past overperformance is likely to give a reasonable estimate as to how much people are willing to cross party lines within a particular state to vote for a Tester or a Manchin.

3. It's very early for polls.

Most of these states don't even have a final matchup, which means that the polls are going to be sketchy at best.  Weight them cautiously.  That said, polls have done pretty well historically and in the 2017 election and the specials.  By and large, well-polled races have hit (by which I mean the average of polls) to within about 5 points.  For now, though, weight them very low.



That's it.  Have fun.