Regression to the (very) mean

Stuck in the middle with you

Like many of you, I suspect, I've been struggling to move past the election results and determine what it all means. I also have the added benefit of being a professional with ample contacts in affected minority communities, and these folks are understandably concerned with the business and working environment in the U.S. now. Without getting too far into the weeds of partisanship, I'd like to bring my viewpoint as a data analyst, and student of American history, to provide some perspective.

My first love as a numbers guy was baseball statistics, and I'm still fairly current with modern thinking about the sport. Just this last week, I read a study of players with exceptional swings away from the mean; pitchers who were most likely to get fly ball or ground ball outs, and so on. The vast majority of these outlier players regress to the mean the next season, even though their particular style and ways to work makes them likely to, well, continue to produce similar results. And yet, the setting of baseball, and more data, corrects and mediates. Outliers, to a very high percentage, come back to the fold.

I'll pivot now to U.S. political history. After the end of the Civil War, nearly 2,000 African Amercians held public office in the South, from local levels to the U.S. Senate, in a period that came to be known as Radical Reconstruction. This did not last, of course; the rise of the Ku Klux Klan, and the corruption of local law enforcement, led to a backlash and subjugation. Eventually, the pendulum swung back through the civil rights movement of the 1960s, with moments like bussing, a growing tolerance for mixed marriages, and the election of Barack Obama all moving the cause further. You could make the argument that the revocation of the Voting Rights Act, and the election of Donald Trump, as more points to a backlash, and I'm certainly not inclined to disagree with you. But the greater point is that progress is rarely, if ever, a smooth line.

Finally, to my day job in email analytics. One of my favorite statistics to look at is unsubscribe, in that many providers fail to give it strong consideration, due to its relatively low incidence rate, and it's also-ran status against more impactful pieces like open, click and conversion. However, I've always seen unsub as a moment of high importance, since the user has to not just open the message, but hunt for the link and take direct action. Rarely do you get such clear indication of user preference.

This year, we've been able to make some design and coding moves in our better performing templates which have managed to cut our unsub rates, especially in mobile deployment. The effect has been to lower the likelihood that you unsubscribe in error, and for much of 2016, this systematic change has led to lower rates. That sort of thing seems like it would be a longstanding and happy development, but remember... regression to the mean.

I'd love to be wrong about this, but I'd put long odds on this happy trend continuing in the next eight weeks, and probably in the long run as well. The reason why is that we're moving into the heaviest part of the year for email usage, and in high send times, "good" emails (my firm sends marketing messages that also double as de facto education pieces) hit the in-box along with a surge of, well, everything else.

At some point, people start unsubscribing, even to pieces that they are interested in and getting beneifts from, just to get back some control of their inbox. When that happens, the baby gets thrown out with the bath water. In addition, the year over year trend in email is for double-digit increases in the amount of time spent. That's a trend that just can't continue, because people are busy and not getting enough benefit to just keep increasing that time spend... and trends that can't continue, well, don't.

Regression to the mean is like gravity. You can escape it for a while, and there are great benefits from doing so. When you do it, it feels wonderful, and can seem like the new normal. But if you assume you are going to continue to do it, especially without effort, you are likely to catch a painful surprise. Perhaps, very painful.

* * * * *

Feel free to comment, as well as like or share this column, connect with me on LinkedIn, or email me at davidlmountain at gmail dot com, or hit the RFP boxes at top right. RFPs are always free, and we hope to hear from you soon.