Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Carrier1/Evocative/Prime (1515 Round Table, Dallas, TX) Power Outage - Page 3
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Carrier1/Evocative/Prime (1515 Round Table, Dallas, TX) Power Outage

13»

Comments

  • vpn2024vpn2024 Member
    edited May 2024

    @aqua said:

    @Swiftnode said:

    Your hardware is not safe, anyone with a $15 key from Amazon can open your rack and mess with your hardware. If you have business critical hardware in there, I'd stay away.

    Yep as swiftnode said, isn't that the same pretty much all over the world absent a cage, after check-in Equinix will walk you to your rack then leave you alone, you get the standard warning on entry about not taking pics of other company's racks, and if you accidentally go for a little stroll, on the pretense of checking cooling or taking boxes to trash area and getting lost on the way back, the equinix dude comes out to find you with an attitude... I got warned a good few times because well... I am nosey but I'm pretty sure if I attempted to open or mess with another rack there would of been serious repercussions including police I imagine. It's as safe as you're going to get.

    Thanked by 1Swiftnode
  • aquaaqua Member, Host Rep

    @vpn2024 said:

    @aqua said:

    @Swiftnode said:

    Your hardware is not safe, anyone with a $15 key from Amazon can open your rack and mess with your hardware. If you have business critical hardware in there, I'd stay away.

    Yep as swiftnode said, isn't that the same pretty much all over the world absent a cage, after check-in Equinix will walk you to your rack then leave you alone, you get the standard warning on entry about not taking pics of other company's racks, and if you accidentally go for a little stroll, on the pretense of checking cooling or taking boxes to trash area and getting lost on the way back, the equinix dude comes out to find you with an attitude... I got warned a good few times because well... I am nosey but I'm pretty sure if I attempted to open or mess with another rack there would of been serious repercussions including police I imagine. It's as safer as you're going to get.

    This is what companies expect though as Equinix provides services to a large amount of big companies, and they do not like their "trade secrets" and what hardware they use to be seen or looked at.

    In DA1, Level3 has all of their cages boarded off with "blurred" plastic so nothing within that cage can be seen.

    Thanked by 2vpn2024 Calin
  • vpn2024vpn2024 Member

    This is what companies expect though as Equinix provides services to a large amount of big companies, and they do not like their "trade secrets" and what hardware they use to be seen or looked at.

    Yeah not everyone cares too much about their equipment being semi-visible, there were big players in the shared rack corridors (i.e. non caged areas) including Rackspace and BofA, you could tell what servers and network equipment they were using if you glanced closely.. and you either knew who racks they were by the labelling or their tech was escorted in same time of you..

  • SwiftnodeSwiftnode Member, Host Rep, LIR

    @aqua said:
    The CH1 chillers failed due to below freezing temperatures which ultimately froze over them. These are 2 entirely different instances as the Evocative case was due to improper maintenance to them.

    It's Chicago, it has over 100 days per year below freezing. You make it sound like it's a once every 1000 year event.

    @aqua said:
    These are 2 entirely different instances as the Evocative case was due to improper maintenance to them.

    Do you have a source for that? I haven't heard anything about improper maintenance.

    We were told that power did not failover as intended, HiVelcity explicited stated, "Initially the power load was transferred to battery backup and then generator as designed, However, at some point in time the generators dropped the critical load."

    Subsequently Prime/Evocative was investigating the cause and had electrical engineers onsite working it.

    And Nexeon reported "Prime successfully transitioned to generators last month when there was a loss of utility power. Sometimes, these things just fail and don't go according to plan."

    Indicating that maintenance routines were being followed and the seemless transition from utility to generators previously worked as expected.

    Where did you get the information about "improper maintenance?"

    Also, you give Equinix CHI1 a pass below of freezing temperatures, yet ignore the fact that ONCOR was down all over Dallas due to severe weather and tornados?

    @aqua said:
    We experienced this outage in DC1, but another company Nexril experienced it in DC2 aswell.

    Sounds like that was an isolated event, not something we experienced in DC2.

  • aquaaqua Member, Host Rep
    edited May 2024

    @Swiftnode said:
    It's Chicago, it has over 100 days per year below freezing. You make it sound like it's a once every 1000 year event.

    I believe the area the DC is in was -20 degrees Fahrenheit. It doesn't get that cold all 100 days per year.

    @Swiftnode said:
    Do you have a source for that? I haven't heard anything about improper maintenance.

    We were told that power did not failover as intended, HiVelcity explicited stated, "Initially the power load was transferred to battery backup and then generator as designed, However, at some point in time the generators dropped the critical load."

    Subsequently Prime/Evocative was investigating the cause and had electrical engineers onsite working it.

    And Nexeon reported "Prime successfully transitioned to generators last month when there was a loss of utility power. Sometimes, these things just fail and don't go according to plan."

    Indicating that maintenance routines were being followed and the seemless transition from utility to generators previously worked as expected.

    Where did you get the information about "improper maintenance?"

    UPS's don't just "fail" when you need them if proper maintained. You need to change out the batteries every couple of years and actually perform proper tests to see if it will take the load properly.

    @Swiftnode said:
    Also, you give Equinix CHI1 a pass below of freezing temperatures, yet ignore the fact that ONCOR was down all over Dallas due to severe weather and tornados?

    How come Infomart / Equinix DA1/2/3/6/11 not go down, nor any other major DC?

    @aqua said:
    We experienced this outage in DC1, but another company Nexril experienced it in DC2 aswell.

    Sounds like that was an isolated event, not something we experienced in DC2.

    Hivelocity in DC1, Nexril in DC2 and some other companies experienced this outage aswell that are scattered across. So you might've gotten lucky but the power went out in that facility.

    "Things just fail and don't go accordingly", yeah if you don't keep up with maintenance. Generators are just like car engines, if you don't start it every once in a while, and don't do proper maintenance, it will of course "fail and not go accordingly".

    They're just going to feed you some BS PR information to keep you as a customer, they're not going to tell you flat-out that "oh yeah we didn't service these UPS's and such properly, sorry not sorry, money is tight".

  • SwiftnodeSwiftnode Member, Host Rep, LIR
    edited May 2024

    @aqua said:

    I believe the area the DC is in was -20 degrees Fahrenheit. It doesn't get that cold all 100 days per year.

    Windchill was like -30F, actual temp was like -10F. Historical weather here.

    It wasn't a completely abnormal weather event, it was just a cascading chiller failure that needed to be remedied when it occurred.

    Whereas, the storm hitting Dallas and the large outage on utility power was pretty significant.

    @aqua said:
    How come Infomart / Equinix DA1/2/3/6/11 not go down, nor any other major DC?

    How come DRT ORD11 did not go down but Equinix CHI1 did? How come no other datacenters in Chicago went down except Equinix CHI1?

    @aqua said:
    Hivelocity in DC1, Nexril in DC2 and some other companies experienced this outage aswell that are scattered across. So you might've gotten lucky but the power went out in that facility.

    Perhaps there was a partial outage, but my original post explicitly stated facility wide. We have not had a facility wide outage like we did the other day since 2015.

    "Things just fail and don't go accordingly", yeah if you don't keep up with maintenance.

    ATS failures are pretty common, you can run tests until you're blue in the face, just because you test regularly doesn't mean when shit hits the fan, it won't break.

    @aqua said:
    Generators are just like car engines, if you don't start it every once in a while, and don't do proper maintenance, it will of course "fail and not go accordingly".

    So you are just going to completely ignore the fact that they properly failed over last month when utility power was lost?

    Anyways, this is a waste of both our time. Goodnight!

  • aquaaqua Member, Host Rep

    @Swiftnode said:

    @aqua said:

    I believe the area the DC is in was -20 degrees Fahrenheit. It doesn't get that cold all 100 days per year.

    Windchill was like -30F, actual temp was like -10F. Historical weather here.

    It wasn't a completely abnormal weather event, it was just a cascading chiller failure that needed to be remedied when it occurred.

    Whereas, the storm hitting Dallas and the large outage on utility power was pretty significant.

    @aqua said:
    How come Infomart / Equinix DA1/2/3/6/11 not go down, nor any other major DC?

    How come DRT ORD11 did not go down but Equinix CHI1 did? How come no other datacenters in Chicago went down except Equinix CHI1?

    @aqua said:
    Hivelocity in DC1, Nexril in DC2 and some other companies experienced this outage aswell that are scattered across. So you might've gotten lucky but the power went out in that facility.

    Perhaps there was a partial outage, but my original post explicitly stated facility wide. We have not had a facility wide outage like we did the other day since 2015.

    "Things just fail and don't go accordingly", yeah if you don't keep up with maintenance.

    ATS failures are pretty common, you can run tests until you're blue in the face, just because you test regularly doesn't mean when shit hits the fan, it won't break.

    @aqua said:
    Generators are just like car engines, if you don't start it every once in a while, and don't do proper maintenance, it will of course "fail and not go accordingly".

    So you are just going to completely ignore the fact that they properly failed over last month when utility power was lost?

    Anyways, this is a waste of both our time. Goodnight!

    You really cannot admit at being wrong. You must be fun at parties.

  • NickGNickG Member

    We are a data center here in Albuquerque NM. If there is anything I can assist with as a DR site please let me know.

  • DataIdeas-JoshDataIdeas-Josh Member, Patron Provider

    Honestly the DC market up in Dallas is a wee bit over hyped. (personal opinion)
    Its cheap sure. But with the outages that has happened does raise some flags for me. As in are things being properly maintained??..

    For those that are looking to move but want to stay in Texas. Let me know and I can help you out.

Sign In or Register to comment.