New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
HostHatch Chicago data lost ?
This discussion has been closed.
Comments
My bet is your kernel has been updated but your grub config wasn't rewritten. If you reboot into livecd / rescue system and check what kernels are installed, you'll probably see a newer one.
Regenerating initramfs and reinstalling grub should resolve.
I'm curious to see since they say "VPS cannot start for about 12 hours. Now it is able to running". It sounds like they couldn't start it in the WebUI and when they finally could this was the state.
Probably some sort of hardware failure lead to an outage and a boot from cold. This would reveal any issues with grub config.
Ah, true enough. Hadn't considered someone might've left a time-bomb laying about.
I remmember it happening to me in the past with hostsolution storage vps.
I followed these guides but you need to change with your enteries.
https://youtube.com/watch?v=ZpIOteojORE
https://linuxhint.com/grub_rescue_ubuntu_1804/
Hosthatch next Hostsolutioms ?
@zqm840527 - I'm in the same boat at HostHatch Chicago storage. I was showing a bunch of disk I/O errors before it went down. Now same as you.
Ok ... now I'm a little worried. Both of mine are fine so far ...
I think it is issue with his server, but I meant he can follow the same guide.
If you had no downtime over the last 24 hours and are fine now, I would not worry. I expect that HostHatch has more then one storage server in Chicago.
Guess I got lucky. Hope you turn out ok.
I’m in the same situation. Don’t know what’s going on now with Chicago Storage. I’ve submitted an emergency ticket 10 hours ago, when my storage VPS had already been down for 10 hours at that time. Still have got no reply.
Don't expect a quick reply from HostHatch. The average ticket response time for me over the last year is about 3 1/2 weeks.
With storage servers I put my data on a separate partition from the O/S. If I need to reinstall this server the data may still be there. Fingers crossed that I don't need to transfer everything from the backup backup server.
I thought they improved their response time since last year…
My experience has been hit and miss. Sometimes it's literally < 20m ... other times it's significantly longer.
Depends on what the ticket is for. Basic tickets, for example setting reverse DNS when it doesn't work in their control panel, usually get pretty fast replies from Mike. Their support is really good for that!
More advanced tickets take longer, for example I currently have an open ticket about TCP connection drops/timeouts in Los Angeles, with a Wireshark capture attached to the ticket showing the TCP retransmits from the client, that's been open for nearly four weeks. Some just never get answered - I had a ticket open about a one hour outage in Sydney that was open for ~3.5 months before being closed with no response.
If you think you might have lost data, then do not mess with the drive. Boot from a live CD and mount the partition in read-only mode. Check if anything is missing or corrupted. If the data is there then take a full backup if you don't already have one, just in case.
When was the last time you rebooted the system? If it's a long time ago, then maybe you had a failed kernel upgrade and didn't notice until it was rebooted.
can you point to a tutorial which lists steps for the liveCD and mount part? if it helps I usually use CentOS7... I'd like to give it a shot once to see what the process entails.
Thanks
Here is my server from the web console with kernel 5.10.0-11:
And here is kernel 5.10.0-10:
I can still select kernels from the grub menu but they all fail:
I'm not even going to bother with a ticket because what's the point. Ill just watch this thread.
Edit: I loaded the gparted live cd and the partitions are still there. I was able to mount the data partition and it looks like all the data is there. I have to OS on a 10gb partition so its no big deal to me if I have to reinstall.
I have some tickets with an acceptable response time, but one of them to remove a reverse DNS it's opened for + 2w.
I have a few servers with them and I enjoy their service.
Hope they get better on this.
Last time was faulty raid card
@cablepick - After following the tutorial mentioned by @msallak1 above, for the two kernels I have on the system. I get "uncompression error system halted". The original error was something like "failed to boot the default and backup kernels" so some things may be corrupted. I will reinstall the o/s and report back if the data partitions are ok.
EDIT: I can see that files show to be there, but I don't know for sure that they are good yet.
EDIT2: After O/S reinstall the files on the data partitions look good so far.
Mine over a month
What's your experience with virmach storage VPS?
Would you mind sharing your experience? since you're virmach enthusiastic, I'd like to know if it's a wise move to migrate from hosthatch to virmach.
Did it get solved eventually?
@foitin - It is true that I have many VirMach SSD VPS, but I can't comment as I have never used VirMach storage VPS.
I will say I have not been overly impressed with HostHatch Storage VPS and I will be moving my storage to another provider when this VPS expires if not sooner.
EDIT: or maybe it is just HostHatch Chicago servers that I am unimpressed with. To be fair I don't have the kind of issues in the 5 other locations that I have servers with them, as I do in Chicago.
Do you use Netdata? I use the free cloud monitoring and my Chicago ones will spit out half a dozen emails on tcp resets and other tcp related stuff from time to time. It's been annoying me.
I do use Netdata. I get a LOT of alerts about
outbound_packets_dropped- I've been meaning to just disable those since I don't really think there's anything I can do about them (you can disable individual alerts in Netdata's config). I also sometimes get iowait alerts from my storage VPS, but maybe that's not unexpected for HDD-based storage on a virtual server.I've got a 10TB storage VPS in Los Angeles and it's been running very well. I'm 95% sure I'm going to renew it when it comes up for renewal (end of this year I think). My NVMe VPSes with HostHatch are going well too - I'm actually in the process of migrating some stuff from BuyVM to HostHatch.
I originally thought Chicago was a better location as when I was comparing the two locations around Black Friday 2020, the network was better than Los Angeles, and the disks were faster too - YABS showed ~350MB/s reads in Chicago vs ~150MB/s reads in Los Angeles. However I've recently seen posts about network issues there, and now this thread, so maybe LA is better. Hmm.
@Daniel15 - I've had servers with them in Chicago since they opened that location. Then picked up a couple more during BF2020. The storage server blows up every few months for a day or two. I have never lost data and when it is running it is good, but it is too inconsistent IMO. My NVMe there has poor disk speeds of around 35MB/s since November, with the exception of a couple of weeks in Jan when they fixed it. The LAX I have with them has been good with no issues.
In short, I would not recommend moving from LAX to Chicago because the network speed seemed a little better.
I have been having short period of downtime every now and then (sometimes, everyday) for the past 6 months with my Chicago VPS. All other VPS from HH were fine. I have asked for a migration to other location, but was refused because it's part of a "bundle".