Too much heat!

This post has nothing to do with scoots. This work week started out sucky. Scenario: We have a separate high capacity air conditioner (AC) for our server room (about 30+ servers and various other network/storage equipment). Computers don't like heat, so we have our own separate AC system in addition to the building AC. There is an alert system which will put out a loud noise and flash a strobe light on the outside of our building if the air conditioner isn't working. That way security folks doing their rounds will notice the light and arrange to get the AC problem fixed. That is the ideal situation.

What really happened this weekend instead of the ideal situation: The air conditioning malfunctioned over the weekend, the battery unit for the alert system was bad and thus no alert was sounded and no strobe light. A coworker and I come in to work today and find that the server room was over 120 degrees! It may have been hotter than that, but the thermometer only read up to 120. Yes, 120 freaking degrees farhrenheit. That is almost 49 degrees for those of you in the celcius part of this world. Servers and equipment everywhere malfunctioning with lots of beeping noises saying the equipment is overheated.

Oh crap! Open up the one window in the server room and turn on the huge fan. Get all the doors to the server room open, plus all doors and windows to the other side of the building open to get air circulating and we start shutting off servers. Some were unresponsive, so we had to force power off. Those in IT know that is not a good thing to do, but we had no choice. Try calling the number for maintenance. Get voice mail. Call the main physical plant #. Get voice mail. No one answered the fucking phones! I called security (emergency #) and immediately got someone on the phone. I told them, "Our air conditioner in the server room doesn't work, it is 120 degrees in there, and we need an AC person here NOW because equipment is malfunctioning and some people in physical plant are going to lose their jobs if we don't get someone to get their ass over now ASAP and fix the problem!"

An AC person came over about 10 minutes later. Apparently the AC unit failed because it was low on coolant. WTF? Aren't they supposed to check that shit during their maintenance intervals or whatever? Do they even have scheduled maintenance for the damn thing?

45 minutes later the AC is running again. We waited an hour for the room (and the equipment) to cool before turning them back on. Man, did we ever dodge a bullet! Out of all that equipment, the only casualty was one failed hard drive, and that one was mirrored, so no data was lost and all services were back in action.

Meanwhile, back at the farm, the building AC itself wasn't working today. My office faces windows on the south side of the building. I finally said "to hell with this" and told my boss at 3 "I'm out of here, I can't take working in an office that is over 80 degrees." I went to my American Legion and had a few beers to replace all the water I sweated out today.

An interesting day indeed. I hope the AC is working in the building tomorrow and they get the replacement battery for the alert unit installed ASAP