Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


what happened with hosthatch's chicago NVMe vps?
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

what happened with hosthatch's chicago NVMe vps?

i/o test got 9mb/s

this has been several days

Comments

  • Context?

    What was your average before? Are these consistent tests? Have you contacted them?

    What answer do you expect?

    Thanked by 1ITLabs
  • NVMe is supposed to be way bettter than ssd, how come io drops until so low

  • SkanderSkander Member
    edited August 2019

    They've "restarted the node" and said they would "keep a close eye on it" a few days ago.
    It seems like the horrible io is back.

    Needless to say I've since moved anything on there and migrated it. Won't renew. :/
    I was planning to keep that one VPS in addition to a dedi but nope.

  • @Mic-hael said:
    Context?

    What was your average before? Are these consistent tests? Have you contacted them?

    What answer do you expect?

  • EHRAEHRA Member

    They said they are aware and would solve the problem within 24 hours. A few days have passed and unfortunately the performance is still poor.

  • harikarkiharikarki Member
    edited August 2019

    Same. Submitted ticket. They said it will be resolved within 24 hours. But its all same. I had to move everything to somewhere else. It was good VPS when it was good :wink: wink:

  • hosthatchhosthatch Patron Provider, Top Host, Veteran

    There is one affected node in Chicago, we will likely be migrating everyone away to another node which will be a few minutes of downtime per VM. Not an easy fix unfortunately.

  • OujiOuji Member

    Abdullah said: There is one affected node in Chicago, we will likely be migrating everyone away to another node which will be a few minutes of downtime per VM. Not an easy fix unfortunately.

    Thanks for the response.

    In another note, that 3TB storage is coming to Chicago as well?

  • @Abdullah said:
    There is one affected node in Chicago, we will likely be migrating everyone away to another node which will be a few minutes of downtime per VM. Not an easy fix unfortunately.

    My VPS is Empty. Is it quick to submit ticket and get another one fresh setup in new node? I have nothing to migrate anything from it.

  • hosthatchhosthatch Patron Provider, Top Host, Veteran

    It should be resolved now. You should be getting an email shortly too if you were affected. It was the kernel acting up, we had to exhaust every other option before we pushed another reboot (we already run kernelcare so kernel issues like this are rare).

    @Ouji said:

    In another note, that 3TB storage is coming to Chicago as well?

    Yes

    Thanked by 3Chocoweb uptime Ouji
  • Email from HostHatch on the Chicago Issue:

    Hello,

    Over the past few days, you will have seen consistent low disk IO and high iowait for multiple hours per day. This disappeared and came back again with random intervals.

    Migrating the whole node to another is what we do as a last resort, and we always try to maintain N+1 capacity in all locations, so we can deal with one node failing completely in a worst case scenario. However this can be an extremely slow process due to the terabytes of data that needs to be migrated, especially when the issue is an array acting slow.

    We have been investigating this issue by trying to find abusers on the node while also testing each disk in the RAID array for errors and/or performance issues. In the end, the issue was resolved with an update of the kernel and related disk drivers, and a reboot of the node.

    Please note that we burn test all new nodes and have been running this kernel for a few months with no issues until a few days ago. We will of course keep an eye on this node to make sure we can prevent any further issues from happening. At the moment, we will consider this to be resolved.

    Thank you for your patience and our apologies for the inconvenience caused here.

    Kindest Regards,
    Your HostHatch team

Sign In or Register to comment.