Special Specials and How Well Polls have done since 2016

There are a lot of specials going on today, including a marquee event: the US House Race in AZ.  For all of the competitive races since Nov 2016, check out my Drumpf Effect spreadsheet.

Let's play a game with the predictive power of statistics.  Today is a pretty big day with regards to special elections.  There are 11 (!) races in NY, of which 9 could be considered competitive in any realistic way. But the main event is the AZ08 House race to replace disgraced Republican Trent Franks who resigned due to some super creepy sexual misconduct.  You can get some additional expectations/history in my Drumpf Effect Spreadsheet.  The smart money in AZ is on the Republican,   Unlike the PA18 race last month, AZ08 has a huge Republican advantage in terms of party registration and early voting.  Trump won the district by 21, and the last competitive race here was won by Franks by 28.  It's a very heavy lift.

But it's also a big opportunity for learning.  Ever since November 2016, Republicans have jeered that polls are meaningless because they didn't predict Trump stealing the election.  They are, to a point, correct.  Today's AZ poll represents only the ~6th well-polled race in the last year and change.  So how have pollsters in the other races?  Short answer, pretty damn well.

Unlike previous posts, I'll be updating this, ever so slightly, later in the evening, to see how my predictions turn out.

To measure the outcomes, I'm considering the 5 previous races with at least 2  (non-internet) public polls in the month prior to an election.  I take up to 5 from unique pollsters and and weight them by sample size.  I didn't include, for instance, the MT-AL race for which only 1 pollster (Gravis) polled the contest, and for which Gianforte's assault on a reporter days before the election likely skewed the final outcome (though not by as much as it should have). 

How'd they do?

Date	Race	# of Polls	Formal Error	Weighted Ave	Results	Polling Error
6/20/17	GA06	5	1.7%	+1.0%	-3.8%	-4.8%
11/7/17	NJ Gov	5	1.8%	+14.2%	+13.3%	-0.9%
11/7/17	VA Gov	5	1.5%	+4.4%	+8.9%	+4.5%
12/12/17	AL-Sen	5	1.4%	-1.3%	+1.5%	+2.8%
3/13/18	PA18	3	2.4%	+0.5%	+0.2%	-0.3%
	Average					+0.3%
4/24/18	AZ08	3	2.8%	-5.7%	5.2%	+0.5%

The average is $0.3\pm 3.2\%$.  That is, the typical polling error is unbiased, and a little larger than sampling error, but not by a crazy amount.

Predictions:

Unfortunately, if special election polling really is on track, this means that if the polling misses are consistent, then we'd expect the Republican, Lesko, to win by 2.2 and 9.2 points.

However, I would like to make some positive predictions as well.  Based on the ~15 point swing, I'd expect a flip in the NY Assembly District 10, a hold in AD 142 (which Trump actually carried in 2016), and a possible flip in AD 107.  I don't expect Republicans to flip any Dem seats.

Check in later in the evening for outcomes.


Results:

1 flip, a couple of close races, but not in Arizona.

Overall swing: Around 9.6%. A little less than typical (~15), and less than the national swing needed to win the House, but this is to be expected when the districts are bluer than typical.  As the average lean was about D+3, we'd expect the Dems to perform about a point worse than a neutral district.

As to my particular predictions:

AZ08: Didn't flip (as expected), but holy moley, the polls were dead on.  As of this, the margin was about R5.2%, almost exactly what the polls predicted.

NY AD 10: As expected, both flipped to the Dems, handily, who won by 18.

NY AD 107: A close race, but Dems lost by 4 (defying my predictions).

NY AD142: I was wrong on one particular, because I didn't do my homework. The nominal Republican, Eric Bohen, won by about 6 points, but it was irrelevant, because he pledged to caucus with the Democrats.  He really was one of those RINO's the purists are always kvetching about.  Nevertheless, it doesn't make sense to include a race like this in the analysis.