#vss365 today - Email infrastructure upgrades

In my previous post about #vss365 today, I wrote in length about the site's growth and my estimated deadline for hitting the Mailjet 200 emails/day limit. (You'll want to read that post for background information, as I won't be repeating it here.) At the time, I theorized that I would hit that limit in late 2019/early 2020. As it turns out, I was wrong, big time. In this post, I want to talk about my plans to remediate that, how that is coming along, and that effects current and future subscribers. Full disclosure: this post may get a bit technical times. 😉

August 2019 witnessed the fastest rate of growth for email notifications yet. 35 new addresses were added to list in 31 days. Recognizing that I was going to hit the limit much sooner than initially expected and, I accelerated my plans to implement a new solution. Being a university student on a budget and wanting something that would last a while, the best route for me was to run my own outgoing-only SMTP email server for use only by my applications running on my web server. I am already paying for it and it is not going anywhere, so I might as well make more use of it and learn new things in the process. Upon finding a method that would work and playing around with it, I felt confident that I would be able to resolve this pending doomsday in good time. My plan was to get things implemented, tested, and running by the end of September. To that end, on September 7, 2019 , I modified the email template and sent out a tweet to give people a head's up.

But, as anyone living longer than yesterday can tell you, life has a way of wrecking even your best plans.

University semester work kicked into high gear. We have lots of projects to do this semester at work before the semester end. I had a job interview and was out of town for a weekend. All this and lots more suddenly delayed my plans for an upgrade. And so, I watched the growth chart increase each day.

For those of you keeping track, July 17 is when I published the last graph. The last graph ran from January 10 to July 17, (6 months, 7 days), in which 4,400 emails were sent. This graph runs from July 17 to October 6 (2 months, 19 days) and saw 9,893 emails sent out (taking into account the campaign emails unrelated to the site and the spike on September 6 from a technical error). That's more than double the emails sent out in roughly a third of the time. September then outpaced August's growth, gaining 60 addresses, bringing the total to 190 emails a day. At the point, knowing that I would not be able to address the issue before reaching the daily message limit (as I was out of town the one weekend I would have had the chance to set everything up), I made the hard decision to temporarily remove the sign up form from the site, capping the number to 190. This was hard to do as the email notifications are, by far, the most popular and useful thing on the site, and running out of resources due to growth is, by far, a good issue to have. Signups are currently still disabled and will be so until the new infrastructure is in place. All existing registrations are continuing to be processed as normal.

Now with that part out of the way, let's take a look at what I'm doing to resolve this opportunity and my timeline for doing so. This will get a bit technical (I am a programmer and this is my blog lol),  so feel free to skip it. 😉

I'm using a Dockerized postfix image as my SMTP server. It works and already seamlessly sends out emails like a charm.  I've implemented new code to send out emails through it, piggy-backing off the existing code. Once I complete the setup and transition to postfix, I'll rework all of it to be simplier.

My biggest issue right now is getting the emails to not land in people's junk box. This is fully possible but new territory to me, which means I'm learning how to set everything up correctly. My current obstacle is getting DKIM working. The postfix image supports DKIM, though it currently has an unresolved issue using the keys. I'm also having trouble correctly generating and supplying the keys, as indicated by dkimvalidator.com. I need to do lots more reading and searching on the topic. I'm wanting to get this resolved before redirecting email traffic though it, but with as long as it is taking me to get this done, that may not be realistic. With email signups disabled since October 1, I am prohibiting lots of people from getting them and I don't want to disrupt delivery as much as possible. I did revise the email message for existing subscribers to give a head's up on what's going on, to keep myself accountable for the upgrades.

Once DKIM is properly configured, I plan on moving 25% of registered emails (selected at random) through postfix, bumping that up in 25% increments until all emails are sent in the new way. I'll also reenable email sign ups once I confirm postfix is reliable and moving 50% at the messages. Doing this effectively creates an A/B test between postfix and Mailjet, allowing me to determine directly see how each method is performing.

I am tracking major tasks on GitHub but my itemized task list is locally stored for ease of access. Here's a screenshot of it so you can get a better idea of what's happening and to come.

As far as a timeline and deadline for getting these upgrades, obviously, the sooner, the better. I have an opportunity to be featured on my university program's website as a student spotlight for being published in the first-ever VSS365 Anthology, and I would like to be able to have them include a link to the site but I don't want to do that while these upgrades are still going on. I also have other changes I want to make, like the admin panel I described in my last post, that can't really be started until this is complete. Right now, I'm aiming for a deadline for the entire transition to be completed by New Years 2020. I'm deliberately being broad and vague because once this current roadblock is resolved, the transition speed can and will increase drastically. It's just a matter of time now.

That's it for the updates. Hopefully next time we talk, I'll have everything running smoothly and there will be new, more interesting things to announce!