[SDBUG] San Diego Co-Location Horror Stories

Jeremiah Gowdy jgowdy at cox.net
Wed Aug 17 23:16:39 PDT 2005

It has to do with Joseph's sphere of responsibility.  His responsibility is 
not to recognize that "humans make mistakes."  His responsibility is to find 
the colocation facility with the best track record according to as much 
information as he can collect.  It's hardly dirt digging to perform enough 
due dilligence research to know if a particular facility has had one major 
power outage or four.  One major outage might be within tolerance, but four 
within a certain period of time is not.

It's not "concentrating on the negative" to properly investigate and inquire 
about any known troublesome facilities.  Obviously no colocation facility 
has a perfect track record, but it would be stupid not to ask around and 
make sure you aren't moving into the facility that everyone who cares about 
their business has learned is unreliable.

I took Joseph's report about AC issues at Level 3 San Diego into account 
when I toured their facility, and in concentrating on the AC issue, I found 
several terrible hotspots where there simply was not proper airflow or where 
the airflow entering the cabs front side was hot.

(That combined with the fact that the place looked like a damned ghost town 
since the last time I'd been in there.)

----- Original Message ----- 
From: "Michael J McCafferty" <mike at m5computersecurity.com>
To: "San Diego's BSD Users Group" <sdbug at sdbug.org>
Sent: Wednesday, August 17, 2005 10:35 PM
Subject: Re: [SDBUG] San Diego Co-Location Horror Stories

> Joseph,
>         Now why do you concentrate on the negative ? Data Centers are run 
> by humans, and humans screw up from time to time. Data Center are full of 
> hardware within moving parts. What will distinguish a data center is how 
> they handle things when the people do screw up or the hardware or 
> machinery fails. It sounds to me that Level 3 didn't handle your situation 
> very well. It sounds to me that the humans didn't plan the NOC well if 
> they were not local, didn't handle your complaint. I have 8 years 
> experience building Data Centers... more in the IT industry but I happen 
> to have a great deal of experience building, operating and working in Data 
> Centers, mostly corporate. I have a couple racks at a local data center 
> now too, where I run my hosting business (shared, dedicated and colo)
>         That being said, every data center has had the power go out in 
> spite of redundant systems. Every data center has had the network go down 
> in spite of redundancy there too. Every data center has had a human error 
> that was sufficiently stupid to blow their SLA to one or more customers, 
> caused a security incident or in some way done something that was totally 
> avoidable that pissed someone off.  Yes, every single one. The data center 
> that has only been open for a week will screw up eventually. You show me a 
> data center that claims they haven't gone down, then I will show you a 
> sales person that either wasn't privy to the failures or is straight out 
> lying to you.
>         Now, I am confident that I made the right decision for my business 
> at the time I made the decision. My business has changed since then, and 
> it's still the right place for my business. It's the right combination of 
> price and performance. I speak with customers of other data centers in 
> town all the time and I gather that there are a couple places in town that 
> might have a better support process than my data center does. Of those 
> other places in town, one uses very low grade bandwidth, and the other is 
> far more expensive. I have been at AIS (Formerly ADNC) for more than 2 
> yrs. I recently re-signed for a 3rd year and increased my rack space.
>         This list is hosted on one of my servers in my rack space at AIS 
> (www.americanis.com) My company is www.m5hosting.com
>         If you have specific questions about any of the Tier 2 data 
> centers here in town, I might know something about them. But, I won't give 
> out random horror stories on an archived forum such as this. Maybe next 
> SDBUG Meeting you can get me to divulge. I have a habit of drinking a few 
> pints at the meetings.
> Good luck with your dirt hunt,
> Mike
> At 01:26 PM 8/17/2005 -0700, you wrote:
>>A company I contract for is looking to take the plunge and move their in 
>>house servers to a datacenter.  I'm no stranger to the local datacenters, 
>>but I need to gather as much information on them as possible.  I've got a 
>>nice list of positives for each, and more technical information than any 
>>suit will have interest in.. i'm just really curious to hear what kind of 
>>horror stories you all may have from hosting here in San Diego.
>>Power outages? Water leaks? Connectivity loss? Badge deactivation? Did 
>>their sales team treat you like dirt? Is their closest employee in Los 
>>Angeles?  Go back as far as possible if you can, time frame is of no 
>>concern to me.
>>An example: The week in August of 03', where Level 3's A/C blew hot air 
>>into a cabinet full of my SQL RAID arrays, and their out of city NOC 
>>wouldn't acknowledge that anything was actually wrong.  No apology or 
>>explaination even after we moved out.
>>SDBUG mailing list
>>SDBUG at sdbug.org
> _______________________________________________
> SDBUG mailing list
> SDBUG at sdbug.org
> http://lists.sdbug.org/mailman/listinfo/sdbug 

More information about the SDBUG mailing list