Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Register

Quick Links


Categories

In this Discussion

› › krypt has been down for more than ten hours
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

krypt has been down for more than ten hours

suutsuut Member
in Outages

The source said that the cause of the accident was a fire in the computer room, like ovh.

«1

Comments

  • hkbbdxhkbbdx Member

    my vps too

  • louiejordanlouiejordan Member

    It's not a very important provider, and many people haven't even heard of it.

    Thanked by 7yanling Levi lowendclient dedotatedwam sgno1 Otus9051 shelfchair
  • yanlingyanling Member

    the website is down now https://www.krypt.com/

  • NanjaNanja Member

    @louiejordan said:
    It's not a very important provider, and many people haven't even heard of it.

    You are right, I have never heard of them. They aren't important to me.

    The companies revenue is more than $15 million. The Asia branch is worth more than $5 million.

    I imagine they are important to someone with that kind of cash coming in.

    Interesting thing I discovered is that Asia site is down also...

    They couldn't have burned down both places...

    Thanked by 1seenu
  • suutsuut Member

    @Nanja said:

    @louiejordan said:
    It's not a very important provider, and many people haven't even heard of it.

    You are right, I have never heard of them. They aren't important to me.

    The companies revenue is more than $15 million. The Asia branch is worth more than $5 million.

    I imagine they are important to someone with that kind of cash coming in.

    Interesting thing I discovered is that Asia site is down also...

    They couldn't have burned down both places...

    krypt was created longer than ovh.

  • NanjaNanja Member

    From the company that owns Krypt I guess.

    We are experiencing an outage at our LAX10 facility that is affecting your service.
    Evocative Data Center leadership is on-site and is working to restore service for our Krypt and ION customers.
    We will be providing regular updates and remediation plans as the situation progresses.
    There is currently no ETA for service restoration and we will update you when that is available.
    We appreciate your patience, and if you have immediate concerns or any questions, please contact us at [email protected] or call 888-365-2656.

  • bdlbdl Member
    edited May 2023

    .

    Thanked by 1Robert_baudcom
  • NeoonNeoon Community Contributor, Veteran

    Thanked by 3bdl anubhavhirani CheepCluck
  • h2oh2o Member

    It has been more than 24 hours. :(

  • JabJabJabJab Member

    Pictures?! Any (more specifc) news?

    That LAX10 DC seems to be located in some warehouse/buisness district of LA - there is no way no one have pictures if shit on fire :D

  • malignifymalignify Member

    Krap.

  • WebProjectWebProject Host Rep, Veteran

    @JabJab said:
    Pictures?! Any (more specifc) news?

    That LAX10 DC seems to be located in some warehouse/buisness district of LA - there is no way no one have pictures if shit on fire :D

    Here is pic:

    No issues reported probably just network issue.

    Thanked by 1lowendclient
  • suutsuut Member

    @WebProject said:

    @JabJab said:
    Pictures?! Any (more specifc) news?

    That LAX10 DC seems to be located in some warehouse/buisness district of LA - there is no way no one have pictures if shit on fire :D

    Here is pic:

    No issues reported probably just network issue.

    The customer service claimed that there was a fire, but there was no further fire announcement.

  • MannDudeMannDude Host Rep, Veteran
    edited May 2023

    Fire doesn't automatically mean a building engulfed in an inferno.

    It's LA. They're going to have strict fire codes. Fire could have broken out and been quickly suppressed, but power could and likely is cut, I'd imagine.

  • NeoonNeoon Community Contributor, Veteran

    @MannDude said:
    Fire doesn't automatically mean a building engulfed in an inferno.

    It's LA. They're going to have strict fire codes. Fire could have broken out and been quickly suppressed, but power could and likely is cut, I'd imagine.

    Not if you floor is made out of wood.

    Thanked by 1WebProject
  • NanjaNanja Member

    I tried talking to the them about the fire and asked if everyone lost their data. I was curious about what happened. They won't give me any answers in regard to it and the question got avoided. They just said that I must be a customer and open a ticket to get updates about the outage. Why would they avoid my question about the fire?

    Is it because of a legal reasons that they can't disclose that fact?
    They could've just said hey... sorry to tell you, all our servers burnt up, but instead said open a ticket if you are a customer.

    I don't see any information about it in any social media, why are they so hush hush about this fire.

  • qpsqps Member, Host Rep

    https://www.reddit.com/r/sysadmin/comments/13o44on/digital_realty_el_segundo_fire_5212023/

  • niuwoniuwo Member

    remark , still down now

  • FranciscoFrancisco Top Host, Host Rep, Veteran
    edited May 2023

    @qps said:
    https://www.reddit.com/r/sysadmin/comments/13o44on/digital_realty_el_segundo_fire_5212023/

    Ok, who was overclocking Ryzen 7900x3ds?

    Francisco

    Thanked by 2Calin YIem
  • GanonkGanonk Member

    GWS™

  • febryanvaldofebryanvaldo Member

    @Neoon said: Not if you floor is made out of wood.

    I'm pretty sure you are reffering to someone :D

  • LaurelNotYannyLaurelNotYanny Member

    Still, there has not been any official statement released regarding the outage from iON or Krypt?

  • CheepCluckCheepCluck Member

    The offical statement is to standby patiently for the official statement.

  • GamingAgeGamingAge Member

    Since this forum thread and that subreddit seems to be the only sources of "up-to-date" info for this mess, the latest ticket update from Krypt (have a web server there) we received was...

    As of May 23, 7.00pm PDT, Evocative DC Operations has access to the facility. We
    are coordinating migration planning and hardware replacement in order to restore
    service. Currently no update on ETA for restoration of services.

  • suutsuut Member

    Dear valued customer,

    As previously shared, our Los Angeles LAX10 colocation suite (“Colo 4”) at 2260 W El Segundo Blvd was affected by an event due to causes beyond our reasonable control on Sunday, May 21, 2023, resulting in fire and water damage rendering Colo 4 inoperative. In response, we expedited the installation of new cabinets, power circuits, and network fiber connectivity in an adjacent colocation suite within the same building, reducing the standard installation process by several weeks.

    Our colocation, cloud, and bare metal customers in Colo 4 were directly affected by this incident. As of today, colocation customers have been provided the infrastructure necessary to migrate their environments, cloud customers with offsite disaster recovery services have been restored at a remote facility, and bare metal server environments are being moved to a new location for data access. Unfortunately, our local cloud storage and backup systems in Colo 4 were significantly affected by the incident and cloud customers without offsite disaster recovery services are still affected. We have replaced hardware for both the local storage and backup systems, but several hard drives remain damaged, affecting data recovery efforts. We are actively working with drive manufacturers on data recovery solutions.

    We appreciate your continued patience and will provide further updates to impacted customers as we progress. If have any immediate concerns or additional questions, please contact us at [email protected] or call 888-365-2656.

  • treesmokahtreesmokah Member
    edited June 2023

    @suut said:

    Dear valued customer,

    As previously shared, our Los Angeles LAX10 colocation suite (“Colo 4”) at 2260 W El Segundo Blvd was affected by an event due to causes beyond our reasonable control on Sunday, May 21, 2023, resulting in fire and water damage rendering Colo 4 inoperative. In response, we expedited the installation of new cabinets, power circuits, and network fiber connectivity in an adjacent colocation suite within the same building, reducing the standard installation process by several weeks.

    Our colocation, cloud, and bare metal customers in Colo 4 were directly affected by this incident. As of today, colocation customers have been provided the infrastructure necessary to migrate their environments, cloud customers with offsite disaster recovery services have been restored at a remote facility, and bare metal server environments are being moved to a new location for data access. Unfortunately, our local cloud storage and backup systems in Colo 4 were significantly affected by the incident and cloud customers without offsite disaster recovery services are still affected. We have replaced hardware for both the local storage and backup systems, but several hard drives remain damaged, affecting data recovery efforts. We are actively working with drive manufacturers on data recovery solutions.

    We appreciate your continued patience and will provide further updates to impacted customers as we progress. If have any immediate concerns or additional questions, please contact us at [email protected] or call 888-365-2656.

    Reasonable response, i would not be angry with how they are handling it.
    I wish they told us how it happened, but i do not think its a "must".

  • jbilohjbiloh Administrator, Veteran

    Feel badly for everyone impacted by the fire at that facility. So unfortunate that a single device in a single cabinet has caused such a big inconvenience.

    Thanked by 1Calin
  • LaurelNotYannyLaurelNotYanny Member

    I received this as well, from [email protected]

    @suut said:

    Dear valued customer,

    As previously shared, our Los Angeles LAX10 colocation suite (“Colo 4”) at 2260 W El Segundo Blvd was affected by an event due to causes beyond our reasonable control on Sunday, May 21, 2023, resulting in fire and water damage rendering Colo 4 inoperative. In response, we expedited the installation of new cabinets, power circuits, and network fiber connectivity in an adjacent colocation suite within the same building, reducing the standard installation process by several weeks.

    Our colocation, cloud, and bare metal customers in Colo 4 were directly affected by this incident. As of today, colocation customers have been provided the infrastructure necessary to migrate their environments, cloud customers with offsite disaster recovery services have been restored at a remote facility, and bare metal server environments are being moved to a new location for data access. Unfortunately, our local cloud storage and backup systems in Colo 4 were significantly affected by the incident and cloud customers without offsite disaster recovery services are still affected. We have replaced hardware for both the local storage and backup systems, but several hard drives remain damaged, affecting data recovery efforts. We are actively working with drive manufacturers on data recovery solutions.

    We appreciate your continued patience and will provide further updates to impacted customers as we progress. If have any immediate concerns or additional questions, please contact us at [email protected] or call 888-365-2656.

  • emgemg Veteran

    What fuels those fires in a colocation facility like that? Could there be quantities of lithium batteries there? (I assume they use lithium batteries in UPSs, right?)

  • Robert_baudcomRobert_baudcom Member

    so long time, have not recovered. I worry much about the data. Do not know they have backup or not.

Sign In or Register to comment.