Kind of reminds me of Arsenic and Old Lace. [Editor’s Note: We again apologize for Dave’s predilection for really old movie references. In his defense who can resist Cary Grant movies during his early, manic phase?]
This is day four of my vacation, and I absolutely refuse to go to work today. I shall have one day of vacation that doesn’t involve work…
…well… maybe a few performace reviews…
Ah, quit whining. You’ve got it easy – at least you’re just at home…
I was at a place (several years ago), where both our CIO and the lead IT tech were both out of town, on vacation. CIO was in Miami, IT tech was in Hawai’i. It was Thanksgiving, so most of the staff were off, too. (I happened to be in the office, having been “volunteered” for bonus holiday pay, otherwise I wouldn’t have known about any of this.)
At 2am Friday (Thanksgiving night), one of our production database servers, a 12-year old beast, crashed. The off-site backup that the CIO had been asking for money to buy for 12 months didn’t exist. The on-site backup, ALSO 12 years old, ALSO used for production purposes (as a web server), ALSO crashed. And there was no backup for the backup.
The CIO, Dev team managers, and (supposedly) the IT tech lead, were all automatically paged by the system. The CIO and my manager answered the call, but the other Dev managers, and the IT tech lead, didn’t. My manager was only 45 minutes away, so he came straight to the office… but didn’t have access to the server cage! It had been KEY locked (usually they rely on the security badge reader), by the IT tech before he went on vacation, and only the CIO had another key. CIO had to fly the red-eye, from Miami to Detroit, get picked up by my manager, get into the cage, and get an even older server online to handle production.
Fortunately, only three clients (out of 45 on that server) noticed the down time on Friday, and our customer admins handled it as “unscheduled maintenance”.
CIO was more pissed at the CEO and company owner than he was at the IT tech… we had new servers in a week, and the off-site location was online by the end of December. And, to this date (5 years later), only 10 of us know what happened.
Awesome… However, it all pales when *I* am supposed to be on vacation. I mean… the world should just adapt, man!