Crunchbits VDS node outage

Comments

• Don_Keedic Barred

  September 2023

  @rjbl said:
  I already have my backups on E2 object storage

  Just as a heads up, I'd find an alternative. My bucket in Dallas has been down for over a week. It's cheap but you get what you pay for.

  Thanked by 1rjbl
• rjbl Member

  September 2023

  @Don_Keedic said:

  @rjbl said:
  I already have my backups on E2 object storage

  Just as a heads up, I'd find an alternative. My bucket in Dallas has been down for over a week. It's cheap but you get what you pay for.

  Thanks for the heads up. I am still on the fence about this though. As it stands replicating between two E2 locations is still cheaper than using B2. I have no issues in Oregon though.
• cybertech Member

  September 2023

  @rjbl said:

  @cybertech said:

  @rjbl said:
  I did duplicate data after what happened yesterday but recent data is inaccessible until the VDS came up again.

  @cybertech said:

  @rjbl said:
  Sigh, losing many hours of work is not fun.

  you should go with high availability vps.

  Unless you setup a HA cluster, I am not sure what you mean by high availability vps?

  https://www.clouvider.com/cloud-vps/

  https://upcloud.com/products/cloud-servers

  I did not know that these existed thanks! I am bit skeptical though. Are they using something like Ceph to duplicate live data?

  https://upcloud.com/blog/how-seriously-does-your-cloud-hosting-provider-take-redundancy
• louiejordan Member

  September 2023

  Since you have chosen this price range for VDS, you should learn to adapt to outages.
• Don_Keedic Barred

  September 2023

  @rjbl said:

  @Don_Keedic said:

  @rjbl said:
  I already have my backups on E2 object storage

  Just as a heads up, I'd find an alternative. My bucket in Dallas has been down for over a week. It's cheap but you get what you pay for.

  Thanks for the heads up. I am still on the fence about this though. As it stands replicating between two E2 locations is still cheaper than using B2. I have no issues in Oregon though.

  Yeah that's what I've been doing since that went down, getting it spread a bit more geographically. Only thing that worries me is the fact they've let a node sit out of commission for an entire week. If it was a day or a few days, eh, it's cheap, better expect that. A week though with zero communication on an ETA of any kind though? If it was stuff I couldn't live without, I'd have it on B2.
• rjbl Member

  September 2023 edited September 2023

  @louiejordan said:
  Since you have chosen this price range for VDS, you should learn to adapt to outages.

  It is not like I am paying dirt cheap prices for these that I would expect them to go down from time to time like the IDrive E2 service.

  @cybertech said:

  @rjbl said:

  @cybertech said:

  @rjbl said:
  I did duplicate data after what happened yesterday but recent data is inaccessible until the VDS came up again.

  @cybertech said:

  @rjbl said:
  Sigh, losing many hours of work is not fun.

  you should go with high availability vps.

  Unless you setup a HA cluster, I am not sure what you mean by high availability vps?

  https://www.clouvider.com/cloud-vps/

  https://upcloud.com/products/cloud-servers

  I did not know that these existed thanks! I am bit skeptical though. Are they using something like Ceph to duplicate live data?

  https://upcloud.com/blog/how-seriously-does-your-cloud-hosting-provider-take-redundancy

  I read through the article and I think a better word to describe the service would be highly durable instead of highly available as you have described.

  @Don_Keedic said:

  @rjbl said:

  @Don_Keedic said:

  @rjbl said:
  I already have my backups on E2 object storage

  Just as a heads up, I'd find an alternative. My bucket in Dallas has been down for over a week. It's cheap but you get what you pay for.

  Thanks for the heads up. I am still on the fence about this though. As it stands replicating between two E2 locations is still cheaper than using B2. I have no issues in Oregon though.

  Yeah that's what I've been doing since that went down, getting it spread a bit more geographically. Only thing that worries me is the fact they've let a node sit out of commission for an entire week. If it was a day or a few days, eh, it's cheap, better expect that. A week though with zero communication on an ETA of any kind though? If it was stuff I couldn't live without, I'd have it on B2.

  I thought they had learned from their Frankfurt incident but I guess they wouldn't change that easily.
• rjbl Member

  September 2023

  An update:

  Hey,

  You should be good to go. We had to transfer in separate portions since it wouldn’t compress and required the entire disk size to transfer. I was able to confirm booting to Debian GUI login on VNC before I disabled it.

  Credit has also been applied to your account. We’re pushing out an individual node status page sooner than planned to help give quicker insight to customers about their specific node.

  Let me know if I can do anything else, and I’m very sorry for the interruptions and headache.

  Best,
  [Redacted]

  I got credited for a full month for the SLA. No complaints there.

  Thanked by 1Not_Oles