All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Aquatis strange VM issue, support tickets also down
- I bought a VM from Aquatis last year during the BF sale, it was located on
SEA-KVM-HR-1. - Yesterday services deployed on it has been down around 1:30 AM (NY time).
- I tried to SSH but connection was alway closed before any handshake would occur.
- So I went to Aquatis Virtualizor panel to see what's happening, and did not find much info. (I should have used the web VNC connection to inspect, but somehow I missed that) So I issued a reboot from the panel.
- After 5 minutes the reboot button is stilling showing 'loading' icon, so I went to status log to check what happened, and found that dispite me only clicking the reboot button once, it actually rebooted twice, and some jobs are stucking:
220690 1501 EMAIL Today at 2:02 am Today at 2:03 am Today at 2:03 am VPS Start Task Completed
220682 1501 EMAIL Today at 2:01 am Today at 2:02 am Today at 2:02 am VPS Restart Restart Done!
220680 1501 EMAIL Today at 2:01 am Today at 2:03 am Today at 2:03 am VPS Restart Completed
220677 1501 EMAIL Today at 2:00 am Today at 2:01 am Today at 2:01 am VPS Start Second attempt to create conf file 64%
220676 1501 EMAIL Today at 2:00 am Today at 2:00 am Today at 2:00 am VPS Restart Restart Done!
220675 1501 EMAIL Today at 2:00 am Today at 2:01 am Today at 2:01 am VPS Restart Completed
220673 1501 EMAIL Today at 2:00 am Today at 2:01 am Today at 2:01 am VPS Start Task Completed
220671 1501 EMAIL Today at 2:00 am Today at 2:01 am Today at 2:01 am VPS Start Setting VPS to autostart 79%
220670 1501 EMAIL Today at 2:00 am Today at 2:00 am Task not Ended VPS Start Performing Startup process 43%
220669 1501 EMAIL Today at 2:00 am Today at 2:00 am Today at 2:00 am VPS Restart Restart Done!
220667 1501 EMAIL Today at 2:00 am Today at 2:01 am Today at 2:01 am VPS Restart Completed
220664 1501 EMAIL Today at 1:59 am Today at 2:00 am Today at 2:00 am VPS Restart Restart Done!
220663 1501 EMAIL Today at 1:59 am Today at 2:00 am Today at 2:00 am VPS Restart Restart Done!
220661 1501 EMAIL Today at 1:59 am Today at 2:01 am Today at 2:01 am VPS Restart Completed
220660 1501 EMAIL Today at 1:59 am Today at 2:59 am Today at 2:59 am VPS Restart Completed
220659 1501 EMAIL Today at 1:59 am Today at 1:59 am Today at 1:59 am VPS Start Task Completed
220658 1501 EMAIL Today at 1:59 am Today at 1:59 am Today at 1:59 am VPS Start Task Completed
220657 1501 EMAIL Today at 1:59 am Today at 1:59 am Today at 1:59 am VPS Start Task Completed
220656 1501 EMAIL Today at 1:58 am Today at 1:59 am Today at 1:59 am VPS Restart Restart Done!
220655 1501 EMAIL Today at 1:58 am Today at 1:59 am Today at 1:59 am VPS Restart Completed
220654 1501 EMAIL Today at 1:58 am Today at 1:59 am Today at 1:59 am VPS Restart Restart Done!
220653 1501 EMAIL Today at 1:58 am Today at 1:59 am Today at 1:59 am VPS Restart Completed
220652 1501 EMAIL Today at 1:57 am Today at 1:59 am Today at 1:59 am VPS Restart Restart Done!
220651 1501 EMAIL Today at 1:57 am Today at 1:59 am Today at 1:59 am VPS Restart Completed
After I clicked the reboot button, the cloudflare captcha page pops up, and I have to click once to get pass it, then the button does not show 'loading' state, so I clicked again. I thought, since the captcha page shows up, it means the first button click did not take effect. But I might be wrong.
I went on to create a support ticket with high prority, and waited for about 45 minutes, the status log does not show any updates. So I tried to put the machine into rescue mode, so I can retrieve files stored on the disk. (I have daily backup, but I'd like to avoid lossing data loss in the past few hours.) The rescue mode did not work at all:
220817 1501 EMAIL Today at 2:52 am Today at 2:52 am Today at 2:52 am Enable Rescue There was an error creating the LVM of the VPS
220815 1501 EMAIL Today at 2:52 am Today at 2:52 am Today at 2:52 am Enable Rescue There were some errors while enabling rescue
220812 1501 EMAIL Today at 2:48 am Today at 2:48 am Today at 2:48 am Enable Rescue There was an error creating the LVM of the VPS
220811 1501 EMAIL Today at 2:48 am Today at 2:48 am Today at 2:48 am Enable Rescue There were some errors while enabling rescue
I then connected to the web VNC, and it shows the machine cannot boot because there is no bootable device: 
I ran out of ideas, and posted all my attempts as comments to the support ticket. And I began to recover my services to VPS of another provider using the latest daily backup. After I recovered my services, I checked again with my Beszel monitoring log, and confirmed the machine has been powered off at around 2:00 AM and cannot boot. I also checked Aquatis status page, and they have a minor server outage incident hanging around from Feb 18, it was still waiting to be confirmed: 
I went to rest, and around 12:43 PM someone (or an automated system) stopped the VPS, and started it at 3:27 PM:
221232 1501 EMAIL Today at 3:27 pm Today at 3:27 pm Today at 3:27 pm VPS Start Task Completed
221020 1501 EMAIL Today at 12:43 pm Today at 12:43 pm Today at 12:43 pm VPS Stop Stop Done!
I then confirmed with SSH that the machine is working again, but I don't need the data anymore, since my services are now running somewhere else.
My support ticket received no reply, and their ticket system seems to be down... I can see the list of my support tickets, but when I click to view a specific ticket it shows An error occurred. The requested ticket could not be found.: 
So, I'm generally satisfied that I only lost some ~9hours of data because my own daily backup worked. But I'm also concered that maybe something is serious wrong with Aquatis. How could you handle customer service if the support ticket system is broken?
Comments
Hello,
Speaking to the Aquatis team today there was a drive issue on a single KVM node in Seattle. I am sorry there may have been delayed replies in support while we fixed the system. Sadly it was down for about 8 hours while our team worked to restore services. All services should be back online and functional and we are following up with tickets.
If you still have an outlying issue feel free to DM me and I will look into it. I am sorry for the minor disruption.
Once again please DM me if you have any outlying issues.
What a monopoly several companies are creating behind PureVoltage. What differences do different companies have since they're all from the same parent company? New Facebook on servers.
Well, I have no particular outlying issues. It's just very upset to see no updates at all on their status page. Status page should be a place where customers can see what went wrong and how the team is handling them, but there is no related incidents, no updates, not to mention that their support ticket system wen down and
Billing Area Websitestill showsworking 100%. I wonder if it's a real status page.The status page is real, the problem from what I have been informed was a drive the VM's were on. (The host OS is kept on a separate environment and drive than the clients). The uptime on the page may reflect the fact the main host was online. I am going to look into this and see if a change can be made to the page. Regardless I agree an update should have been made manually and can only assure you we will do better next time at ensuring customers are properly informed.
Aquatis and LevelOneServers were at one time owned by myself and @Aquatis_Joseph both of us are still working hard on the brands. We sold Aquatis and I sold LevelOneServers to PureVoltage. While PureVoltage is the parent company we both still play a role in the companies future development.
Each brand has their own unique features. I am sure the clients would NOT be happy if the brands were merged suddenly after purchase. For the most part Aquatis and LevelOne operate independently staff wise and hardware wise. The way we did for a decade prior, our clients like how we fit specific niches.
If you or any other brands reading this are interested in selling, please contact [email protected] - We are always looking to add more brands to the family.
Not at all. We're not for sale, nor do we depend on third parties.
I just find it curious because what I see is that PureVoltage is responsible for the money and infrastructure, and the others depend on them. But hey, sometimes it's bad to have 10x as many bad companies as just one good one (same plans, same servers, it's not good).
Capitalism. Big companies buy small. There's literally hundreds of independent companies to choose from, it's not a monopoly. The real monopoly is the datacenters themselves 🤣
It's better for some of these to get bought buy PureVoltage than some other company. Jake @ PV has been doing this stuff for like.. 10-15 years. There's honestly worse ways things could go. But yeah still, an abandoned status page (Edit: Ian says not) and no fast ticket response durring downtime isn't good at all.
That's a good point, but sometimes having the capital to buy several low-level companies is better than expanding my own business than others, knowing what reputation it would have.
It's like buying Calin, EasyVM. Almost all of the businesses they bought were because they left their customers stranded.
I always say Lowendtalk is an excellent community, but it forces providers to register, offering ridiculously unstable prices at $1 or less so you can gain sales. They consider you premium, but if you close your business, they sacrifice you knowing those sales are lost.
I feel they should take more drastic measures, not just paying the tag allows you everything.
Calin, Deluxhost, Charityhost, among others, did bad things, but they still carry on as if nothing happened.
Hi, I received your email notification for the support ticket. But your support ticket system is still broken. Viewing any ticket it would show
An error occurred. The requested ticket could not be found.. This also happens to tickets created before this incident..> @z4r said:
DM me or add me on Discord, if you are having trouble with tickets I will investigate it.
Send me your email.
Hi, I DM-ed you and did not receive any reply. I received an email saying my ticket was changed to
closedbecause I did not reply in 24 hours:But I can't post reply, since visiting the ticket shows an error:
I've said multiple times - both in the original post, and in the following replies, and in the DM - that your support ticket system is broken and I cannot view any ticket. It's been almost two days and you did not fix the problem, instead you accused me for not replying the ticket in 24 hours...
What an customer service disaster.
Tickets automatically close, let me check your DM and get back to you shortly.
OK, I've isolated the problem of ticket viewing to one of my browser extensions. I can view the ticket in a new browser profile.
It's the 2nd time this is happening, When I faced it I immediately opened a ticket and did not receive a response, I had to ping a staff member in their discord server and still had to wait a lot of time, downtime was forever. Getting support has usually been hard with aquatis even if it was normal questions.
They did fix it later but it wasn't a first time issue and neither did I receive any response on ticket regarding it until it was finally fixed, Sure I understand that they were probably busy but because my ticket wasn't acknowledged, it would have of course been frustrating to have kept checking if the VPS was working or not every hour till I received a response from a staff member.
It's common practice to send out a message either on their discord server or email us by acknowledging the downtime even if they detected it late.
But it wasn't a one time thing as this incident happened once previously, I eventually moved what I was hosting elsewhere as I wasn't interested in facing another 15 hour downtime with no notice.
I too moved services away from Aquatis machines. Currently only using it as build machine and other temporary stuff. Such a waste as I purchased additional disk space for it.
Ohhh.