Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Very limited Christmas offers - NVMe & Storage in US, EU and APAC - Page 15
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Very limited Christmas offers - NVMe & Storage in US, EU and APAC

191011121315»

Comments

  • MyHiipMyHiip Member
    edited April 28
  • emghemgh Member, BF Ambassador
  • @hosthatch said:
    To give you an idea about what the past week events were: we had a failure of a storage node (not related to data, but there was no apparent reason since everything looked healthy, could be PSUs, motherboard, CPUs, RAM, etc but no errors at all, so we had to order a new node, overnighting a 4U server is fun and expensive, then move over the drives physically and restore the RAIDs). Everyone is thankfully finally online. Now we can either discard the old server or try to find out the reason it was acting like this while customers are not on it.

    Then there was another carrier related failure in Hong Kong, and another carrier's optic totally died in London and they kept blaming us for our side of the connection being bad. Thankfully we are not singlehomed so we could resolve these issues without much affect to customers, but they are still considered super emergency.

    Then on top some 500+ Chinese customers decided to provision VMs for https://storage.titannet.io/, so we had to suspend all of them since its against our ToS to abuse disk IO like that, especially on storage VMs. We have abuse systems in place, but are not designed for something like that.

    That all happened while we had ongoing scheduled maintenance with Emil being in Tokyo to take care of those node maintenances (hardware upgrades), some network upgrades (installing new routers) + setup our new rack there.

    The team has been doing their best to make sure that the customers feel the least impact possible.

    It's been a rough week overall but things are coming back to normal now :)

    An angry Chinese accusing you are racist. o:)
    https://www.nodeseek.com/post-100948-1

  • @lowendclient said:

    @hosthatch said:
    To give you an idea about what the past week events were: we had a failure of a storage node (not related to data, but there was no apparent reason since everything looked healthy, could be PSUs, motherboard, CPUs, RAM, etc but no errors at all, so we had to order a new node, overnighting a 4U server is fun and expensive, then move over the drives physically and restore the RAIDs). Everyone is thankfully finally online. Now we can either discard the old server or try to find out the reason it was acting like this while customers are not on it.

    Then there was another carrier related failure in Hong Kong, and another carrier's optic totally died in London and they kept blaming us for our side of the connection being bad. Thankfully we are not singlehomed so we could resolve these issues without much affect to customers, but they are still considered super emergency.

    Then on top some 500+ Chinese customers decided to provision VMs for https://storage.titannet.io/, so we had to suspend all of them since its against our ToS to abuse disk IO like that, especially on storage VMs. We have abuse systems in place, but are not designed for something like that.

    That all happened while we had ongoing scheduled maintenance with Emil being in Tokyo to take care of those node maintenances (hardware upgrades), some network upgrades (installing new routers) + setup our new rack there.

    The team has been doing their best to make sure that the customers feel the least impact possible.

    It's been a rough week overall but things are coming back to normal now :)

    An angry Chinese accusing you are racist. o:)
    https://www.nodeseek.com/post-100948-1

    Everyone else in that thread is flaming that angry Chinese dude, schooling him on what TOS means.

    Thanked by 1emgh
  • @hosthatch

    Do you have any proof or screenshots on these 500+ Chinese abusers?

    Your Chinese supporters are being accused as traitors of China and slaves of white people due to their endorsement of your actions in banning abusers.

    They need your help.

Sign In or Register to comment.