New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
One thing for kvm people: Please remember to unmount the iso after you no longer need it. Many more KVMs would have been working now if this would have been observed.
Hi, my instances on iwstack are running but unreachable, what can I do?
My IWStack instance just got up and running. Thanks!
@squaldi, Same here as well.
Check https://master.iwstack.com/client/
The running status https://www.prometeus.net/billing/ here is fake.
Please do not take the interface in the billing panel at face value, it retains last state from when the contact with the master was still working. Until the API is not working and cannot work until the master is up, it will display last known state, because has no means to verify the individual instances without the master, heck, even we dont know which instance belongs to whom without the database on the master, so giving us the domain and even password(!) in a ticket will not help.
@zakdr Thank you so much! But for me https://master.iwstack.com/client/ is not working.
@Maounique Thank you! Now a couple of vm are reachable, but the really important ones are not!!! Murphy docet!
Yes, but this is what I am saying, until that is not working, we can only start individual nodes and hope most VMs come up, if the nodes are not up when the master comes online, it will go down in flames as the master will try to figure out everything and will presume some nodes down and try to restart vms on anopther node killing everything.
pm51 is still unreachable
Can't help but notice how ultimately fragile that "cloud solution" turned out to be.
While regular VPSes are supposedly already up, the "HA" stuff where people likely hosted their more important stuff, is still floating face down under the bridge. Truly the simplest things are the most reliable. Also a lesson to everyone to not rely on any provider for "HA" -- better roll your own, preferably across different providers.
@Maounique: how would I handle an iwStack Ubuntu node that's not up and running yet? Would I just need to wait patiently or should I file a ticket for that specific instance? My main iwStack instance is still unreachable and I'm not sure whether it's an ISO that's still mounted, GRUB2 bitching, or something else.
Thanks for your help and I honestly appreciate all the hard work you guys put in!
Probably that's the case. I remember @Maounique once said here they're moving cpu/hdd/network abusers to one node.
@Maounique I've managed to start a machine on the xen zone. Is it possible to create new vms on the xen zone with the current state of the system?
pm51 fixed, containers are restarting
Xen zone of iwstack came back immediately as it has a very different design with the experience we got after 1 year of fixing bugs in cloudstack. Still, without the master, it will not work to manage VMs.
The console will not be available without the master authorizing it. Unfortunately, there is nothing you can do now, at most you can tell us the uuid and we can search on all nodes up and start it manually on the node and it will still not work without the virtual router which is a vm on perhaps anopther node and we have no way to know which is related to your VM if you have a virtual router, that is. But as things are now, we would rather fix the big thing, then worry about individual cases.
my pm51 vps still is stuck on status "Offline" (solusvm)
as of 1 second later after i posted the above message, it did show as online, thanks
Totally forgot about it. My OVZ just came up by the way, the KVM is up too now.
Well 700 days of uptime and now this still makes it 99,958% uptime.
@Maounique
Any vague idea when cheap (non cloud) KVM VPS might be back online? An educated guess would suffice.
@Maounique ok, thanks!
It depends on node. Many KVM nodes are plagued by issues with the NFS connection for ISOs, still mounted ISOs prevent the VMs from starting, NFS is fixed for most, but we will not take them down again to allow the VMs to start, you will need to connect and dismount/remount ISO if you need, but better leave it down, in order to remake the link or remove it. If you had the ISo on, you need to manually s tart the VM, in many cases the VM is on, but it booted the ISO...
Prometeus website is down again not sure what that means, I hope nothing bad as my servers are still not up
This incident has triggered below thoughts in me
-Configure a fail over dns
Would this give more freedom for IWSTACK team to work freely without me asking them when the server would be back.
Does it make sense?
I always said that. 2-3 cheap VMs with a few select leb providers will have 100% uptime barring a major catastrophe or bad luck like 1 chance in millions beating any major and expensive providers in uptime, price and resources.
I have used a setup like that for around 4-5 years now. Using rsync to copy the servers, and Navicat MySQL to replicate the databases. and DNS failover service from DNSMadeEasy. This downtime at IwStack, would not had any affect on my servers there, since the DNS failover service had kicked in within seconds, mostly 1-3 minutes.
I have used IwStack for 14 months, but moved my main servers out last weekend to try out a dedicated server. So was not affected at all of this downtime. But still, with a DNS failover setup, backup servers at other providers in other DCs, you would never have any real downtime at all.
@Maounique
Stupid me. You got me. I was one of the idiots who had the CD out of the boot sequence but I didn't expressely unmount the image.
KVM VPS is back on again.
Thanks for the slap; I needed it.
On second thought, we will reboot the badly affected nodes to remake the links and be able to restart the VMs if more than half of them are not starting due to this issue, it is faster and saves some nerves, sorry for the people which did remove the ISO, the others are in majority on your node
The ones set to boot from ISO will still do so, but that is a case of clear user fault.
@Maounique: do you have any update on the pm20 node, please?
After some twelve very patient hours my server have been restored, so if mine works so should yours
We are very sorry for this situation, I do not know what happened and the people there have no time to spare telling stories, we will know more when the rush to fix it is over.
Could you please start the offloaded mysql server, i'm sure alot of people's vps are tied to it