New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
The problem seems solved. No reboots / crashes during transfer
Hope it lasts.
Thanks for your answers
My VM in Stockhold hard-locked again this morning. The last log entry was 03:22:52 UTC. Would be interesting to know if this correlates with anyone else.
This is basically what I encountered the other day, three days in a row, and every day it would shut down and require a manual reboot.
But when I opened a ticket, the staff said they hadn't heard similar feedback.
I tried to catch this via uptime and unfortunately the downtime didn't happen again.
Seeing you guys, I think it's definitely not an isolated case, they just solved it quietly and didn't publicize it.
my SE storage is down since yesterday , when I manual boot up in dashboard, it boots up and will stay alive , but as soon as I mount the storage disk , it will freeze and die ...
It's not solved. It's now happened to me 5 times in a row, and it happens repeatedly within seconds of writing a reasonable amount of data to the attached storage. Just touching a new file and
syncwasn't enough to trigger it butsplit -b1G /dev/random /home/borg/crashtest/is enough to crash it within seconds (where/home/borg/is the mounted/dev/vdb) but doing the same test on the NVMe drive is completely fine.same observation on sweden storage node. Has been freezing every morning. it stays alive if i'm not doing any R/W to the attached storage (for some time).
You're right, it's not resolved at all, just what I thought was resolved. Yesterday it happened again.
I wonder if the @hosthatch guys have any plans to fix this?
I'm sure more than one user has started a ticket stating this.
It's not unacceptable for a host to have a hardware failure, I know, it happens from time to time, it just needs to be fixed and moved forward.
Just make sure your staff is really aware of the issue and let users know you've started to fix it to give some peace of mind.
Interesting. I've had no response to my ticket (from a week ago), but just tested today and I seem to be getting successful writes again. Hopefully whatever the issue was has been properly resolved, rather than just being an intermittent fault.
I'm assuming they're using ceph for the attached storage, and wondering if one of the OSDs failed and the rest were all overloaded trying to rebalance to other nodes.
I confirm, It's not solved.
The last time this happened for me was: 2024-12-05 03:03:14
But I only rebooted manually yesterday. It hasn't happened since. But it will most likely happen again if no action is taken...
It's been several times now that I think the problem has been solved.
I don't think they are using Ceph, just Raid10 or ZFS equivalents. Maybe the array failure is rebuilding? Not sure that's what happened.
Storage in Stockholm has started crashing on write again for me, exactly the same symptoms as before. First noticed the VPS was down on Saturday morning, with another 5 crashes this morning.
I've actually now given up rebooting it and I'm just going to leave this server in its crashed state because the next server to run its automated backup will just crash it again.
@hosthatch - I've raised a new ticket for this: #481763. The ticket from last time this happened was closed automatically after a couple of months, with not a single reply from support.
Same for me - trying to write anything to my storage partition (Stockholm) instantly crashes my server although read is fine. Compute partition is fine though.
Original ticket from NOV-24 #376614 was never resolved although I assume with the problem stopping it was fixed.
Hopefuly @hosthatch can shed some light on whats going on.
I'm on the same boat. Storage server down for now than 24h now.
Ticket raised yesterday morning. No answer.
Had an issue with storage in Los Angeles last month, and it did take a couple of days to get a reply, but got back up in the end without data loss.
Same, multiple crashes.
Had to reboot from panel
This host node with so many incidents is making me change my mind about the quality of hosthatch's service.
Stability was excellent when I had services in Amsterdam for several years.
In this case, problems are really frequent with the same symptoms every time.
And the VM isn't even rebooted automatically...
The HH servers are down, and for them whether the customer has a contingency plan or not is irrelevant. They need to acknowledge and bring their services up independent of whatever setup the customer has. I understand you're trying to help the customer here, but please don't overshadow the very important matter that the customer is highlighting here, HH servers repeatedly having issues and them not acknowledging it on their status pages
This is something HH has to work up and be better and transparent at communication.
I know that.
You know that
Infact even HH knows that
I haven't overshadowed this.
I have always emphasized that if the data/ uptime is critical, customers need to have a plan.
Be it a $1/ year service or $1000/ month service.