BLOG Thursday, March 30, 2017

More unexpected downtime

Slicehost was having network issues again today.  Sorry for the inconvenience.  It’s becoming increasingly clear that since they got bought by Rackspace and (I assume) don’t have much staff left in St. Louis, support & availability there have gotten much worse.  Our servers had already been migrated to Rackspace’s datacenters in Chicago, but because their IP addresses were still being tunneled through St. Louis (!!!) we were affected.

Unexpected downtime

It all began one morning last week when our app server was “emergency migrated” from Slicehost’s datacenter in St. Louis to the new one in Chicago.  It’s been a while since Rackspace acquired Slicehost and Rackspace had been planning to migrate everyone out of St. Louis for a while, but promised to warn customers before migrations were necessary.

We didn’t get much of a warning — just a pre-dawn notice that the machine our server was hosted on was showing signs of failure, and that all customers would be migrated to Chicago.  That was fine, but our database node was still stuck back in St. Louis and couldn’t talk to Chicago.

Now I am happy to report that both of tra.in’s servers have been rebooted, migrated and are back online in Chicago.  Years of uptime, gone in an instant!  Sorry to all for the trouble.

A bit zippier?

If things feel zippier to you, it’s because I’ve been tweaking the tracks.tra.in deployment a bit; the biggest change is that we’re trying out Ruby Enterprise Edition.  I’ve been pretty impressed with the responsiveness of the site, but let me know if this causes any problems for you.  Thanks!

Network issues today

Sorry everyone — our host Slicehost, which has usually been very good wrt uptime, had a whole bunch of trouble with packet loss this morning, and then later this afternoon. They took down the data center where we’re located (STL-A) for “emergency network maintenance” this afternoon, but the network seems to be up and humming smoothly now (you can track Slicehost’s status at status.slicehost.com).

I should make it clear that the server and your data were never at risk, just your connectivity to it. Apologies to all for the downtime: though we’ve been very happy with Slicehost in the past, we’re exploring spreading our system across different providers in the future.

Tracks 1.7 upgrade, SMS working

The ability to send new todos via SMS and email works with the new version of Tracks now; sorry that it was disabled until now.  I’ve also tweaked the CSS slightly and enabled OpenID.  Let me know if there are any issues with the new version!

Upgraded to 1.7

tracks.tra.in has been upgraded to the latest Tracks 1.7 branch, which brings with it the following new features (from the official release notes):

  • Tracks now supports recurring/repeating todos
  • Cleanup of feed page and add feed for starred actions
  • New buttons to quickly defer an action 1 or 7 days
  • Calendar view to review upcoming due actions, includes iCal feed to use in your calendar app (tested with Google Calendar, Evolution, Outlook 2007)
  • You can now sort projects on number of active todos
  • Support for OpenSearch. This means you can add a Tracks as a search provider in your webbrowser (tested on FF3 and IE7)
  • Now allow users again to stay logged in on two devices at the same time

Sorry to everyone for how long it took for this upgrade — lately I have been very busy proposing my thesis. Upgrading the DB for Tracks 1.7 had been giving me some trouble when I’d tried it in the past, but this time I posted a planned outage warning, took the site down for a few hours, and upgraded cleanly without having to worry about out-of-sync data.

I’ll write more about more new features the upgrade provides soon! Please let me know if you have encounter any problems with the upgrade.

Tracks adopts tracks.tra.in patches

[Back-posted to commit date] A number of my patches were accepted on this day by the Tracks team – turns out Github really is a great way to collaborate on open source projects!  I’m glad to have finally made some (admittedly tiny) contributions back to the Tracks project.

Here are the patches, all added on 3/31: add an alternate new_todo API call; enable the much-requested “open signups” feature; clean up the appearance of text and ical feeds; remove the “these preferences should be self-explanatory” message; and some CSS tweaks (a patch to move the right sidebar all the way to the right, another tracks.tra.in CSS tweak, was already added by another developer shortly before my patches).  Thanks to Reiner Balt for commiting them!

Alternate new-todo email addresses

Many of you use the site’s email/SMS feature to quickly jot down and send in your todos when you’re on the run. Unfortunately, in the past these todos may have been lost if your mobile phone service, SMS-to-email bridge, or mail client doesn’t correctly put your secret “add new todo” email address in your message’s “To:” field.

Also, some crafty users have tried in the past to use fancy email address aliasing or forwarding (say, newtodo@myemail.com) to save the trouble of having to remember your secret token.

This didn’t work correctly in the past: without a correct “To:” address, our servers couldn’t add your new action to your account.  Well, now it does work!  Hope this helps all of you with fancy todo-emailing setups out there.

Open signups

Several people have asked me how to set up open signups for their private Tracks deployments, so I added a new FAQ entry for that.  I should put these changes on github, but I haven’t figured out how to push to my fork yet.

When I do, I’ll push another Tracks hack written last year, but never deployed — Javascript-based AES encryption of todos and notes using the Clipperz library.  The trouble with it was: it was hard to see why a suspicious user (typical of one wanting to use encryption) would trust it!

In other news, tracks.tra.in has expanded to using a dedicated MySQL slice at Slicehost, in addition to the 2GB slice that Rails runs on.  I’m considering offering separate, faster database support for users paying more than $1/month — basically like buying a timeshare on a private MySQL slice (let me know if that’s something you’d sign up for).

Email/SMS outage

Hi all — we upgraded the server running tracks.tra.in last week, and in the process disabled the add-todos-via-email feature. Many apologies to those who tried to add new items unsuccessfully during this time; the problem is fixed now. For those who are looking to retrieve an important idea or note mailed during this period, we can replay these lost emails from our queue; just email me with your token. Again, very sorry for the inconvenience.

chris