0xDEADBEEF

Well, the server blewout … a bout a day after my last post, honestly.  Doesn’t that just suck.

So, I spent one day (well, morning, before work for a couple hours) trying to get it back up to limp through the day for the folks who host off my system.  Got it up, but it died again before I got to the office, so obviously THAT wasn’t working.

So over the weekend I divided my time between taking my son around (my wife was out of town and it’s the first time he was away from her by more than a car trip), and migrating everything to the new server I had that I had been slowly working on.  Actually, one of the last things I was waiting for was the 10.04 update, so I was ready to do the migration, at least.  I had just been planning to do it on my own terms

So, I couldn’t copy over the mysql files directly as they’d been corrupted.  My dumps from day of had a bunch of blank tables (oops, guess that’s what happens when you dump AFTER catastrophic failure).  But!  luckily I had a dump from the previous weekend.  And since I had binary logging running, I had a full log of everything that had happened in between.

So after a full restore from the dump, the following command gave me the commands that had been issued in the meantime:

On an unrelated note, I think I’m going to set up a weekly process to dump the db and then archive the binlogs with the dump from the previous week.  Then keep the last, say, month.  Basically “logrotate” for my database

Those of you who are paying attention might have noticed that , well, frankly I haven’t had time to post anything lately.

We had our second child in March, and between the infant and the toddler, home life has no time for using my computer (or even my xbox ). Work ramped up a bunch (yay for responsibility, boo for more work!) and so I’ve lost the chance to make any posts from there either.

So I’ve decided to quit pretending like I’m going to suddenly have time soon. I’ll re-assess come January. I doubt I’ll have much time before that.

You know you’re in trouble when a “Good Day” consists of saying “Well, the database only crashed once, but since we have a caching proxy almost no users will have noticed. ”

Of course, when that’s a Good Day … Bad Days are freakin’ hilarious

For instance, one day the network router connecting the incoming pipe … got turned off?

The main web server … got rebooted?  and didn’t start the web server up?  wtf?

The database starts crashing every twenty minutes … and starts eating data occasionally …

Mind you, the guys who got tasked with beating the test servers within an inch of their lives to figure out the crashes seemed to be enjoying themselves!

In college, one of the tenets I lived by, when I was redeveloping the electronic course management software we were using and in my classes, was that any block of code (function, procedure, etc) should fit on a cocktail napkin.

To be fair, part of this logic was that me and one of the people I hung out with, co-dev on the project, used to go to TGIF for dinner with friends a bunch, or we’d go out for lunch somewhere.  And since we both drank, we tended to go to places that had cocktail napkins.  It got to the point where we would sit down, order drinks, and ask for a stack of napkins.  Because we would both start coding and writing notes.

Most people never much liked the theory, because “a napkin is too small to put any real code onto!”  Most of those complaints were from people like those who wrote v1, with 800-line functions … yeah, that’s useless!

I bring this up because the other day when we were cleaning some, I found some of those old TGIF napkins.  Psuedo-coded, not sure where the real-code is.  These were from when we were still deciding between Perl and Java (in ’99)

You can see in the last image that we actually had a note for typing up the napkins, and a note for explaining to “durga” ABOUT napkins.  IIRC, durga was one of the ones where that went over … not at all.