Friday, the InetDaemon.Com website crashed. HTTP 500 errors all over the place and no WordPress, just blank error pages. Oh, NO! What do I do!?!
If you don’t know already, HTTP errors in the 500 range indicate a server-side problem, usually in the CGI, middleware or database back end. This meant that things had gone seriously wrong with the WordPress installation.
So what does one do when your WordPress won’t work any more and you can’t access the database behind WordPress?
I checked the server and the MySQL instance behind WordPress was locked and wouldn’t respond. Couldn’t log into MySQL or access the database. I checked the server connection logs and MySQL had open hanging/idle connections that were still there after several hours. I killed those and was finally able to log into MySQL. I could see the tables, but not query for data. WordPress still couldn’t access the database either. I ran the built-in MySQL recovery tools, and they seemed to run just fine, but the data was still inaccessible. The size of the database was unchanged, so the data was still there, in some form or another. Manual queries didn’t pull anything up and WordPress was still non-functional.
As I sat there, pondering what to do next, it occurred to me that the time required to fix the MySQL database would be greater than it would take to simply build a brand new instance of WordPress and restore the database data from backup. Fortunately, I made a backup the previous day, so I could afford to take a risk.
You ARE making backups of your own website regularly, aren’t you?
~ Public Service Announcement ~
If you aren’t backing up your WordPress database, you should be. If your provider doesn’t offer phpMyAdmin, you should get it and install it and connect it to your MySQL instance. Using phpMyAdmin, you can export a SQL file with all your WordPress data, users, posts and other data. You should run a backup whenever anything changes, new plugins, configuration changes, or after adding several new posts.
~ END Public Service Announcement ~
First, I blew away the old WordPress installation by deleting the directory containing WordPress (after making sure I had downloaded a backup copy of the entire directory via FTP).
Next, I logged into MySQL via phpMyAdmin and deleted the WordPress database. Then I installed a fresh copy of WordPress, ran the setup which created a fresh database instance and logged into WordPress to make sure WordPress itself was working.
Finally, I pulled out the backup tarball I made the day before, extracted the .sql export for the old WordPress database, logged into phpMyAdmin, selected the WordPress database and used the import option in phpMyAdmin to upload the .sql backup file.
SWISH! Nothing but net!
Everything appears to be back online, AND I took the opportunity to do a little architecture work to speed up the site, fix a couple of problems and add some new features. I’m going to be keeping an eye on things to see if anything is missing, but so far its looking good.
If you had an account here, it should be restored, but if it was lost, just create a new one.
More posts soon regarding the upgrades when I get everything set up.