All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
weird shit with my hostcram kvm
I've had a monster 8G from back a while ago. Been through the changes at hostcram over these last few years and never much to complain about..
My KVM is IP whitelist only via custom nftables. Last access by me was around 8 hrs ago before some R&R time.
So I get back to work and can't connect. I find that the KVM is shut down. Startup and check on the VNC, nothing abnormal looking so fire up the ssh session and find a cert mismatch. I didn't continue to auth on SSH and went to the VNC at the VPS CP and can't use my root password. Normal user fine.
Primary activity on this server is working with incus. The couple container instances running Alpine I keep active also had alerts about the host certs not matching.
At this point I just shut the hostcram KVM down and sent off a ticket to see WTF they can tell me..
What does this smell like?
Seems like these conditions are result of root password reset by support/dc people to me.
Comments
Boot the server into a rescue ISO, mount your root drive, and check your logs for anything suspicious.
Unfortunantly ISO booting not available on this via the device manager.
It has netboot.xyz via rebuild but in the past that's been kind of a pain in the ass for me to uncouple from vps and get back to where I was before using rebuild + netboot.xyz -- forget exactly the issue but in the end IIRC I just used it to install a newer debian than was available from the normal debian rebuild templates.
iPXE?
I know little about iPXE, got a prompt via the proxmox boot menu but eth0 closed and can't get an IP @ iPXE>dhcp
Hehe, 100% of my XP with this is right, here: https://ipxe.org/ --> quickstart
Sorry for the inconvenience.
We never log in to our customers' VM/Server without their permission.
The host node "eris" has been online and has not gone down since the last reboot, which occurred 150 days ago.
It's more like your OS/VM crashed and went offline, though I have seen two instability cases recently where the VM dropped offline for no reason, and upgrading Proxmox OS and Kernel might have resolved that issue on the other node.
I will plan and upgrade all nodes soon.
I don't recall ever having the vm crash or go offline spontaneously, but there is always a first time I suppose.
The only recent significant change to this was trixie 13.5 to 13.6 which seemed to go fine.
Can you point me in the right direction to being able to approach the system from the VNC w/o having to provide creds or waking up the OS so I can snag the logs?
..or can you/support do it? as it stands, root PW and certs are not mine/expected and I don't want to volunteer anymore data if this is a breach.
I realize the cause could be a rabbit hole and apologize for suspecting a support PW reset, but I have never had a breach AFAIK or seen odd changes like this.
While I don't mind wiping the VPS and redoing it, I would like to put some effort into finding some reasonable explanation if possible. Crashing in normal circumstances I'd imagine that the specific changes noticed so far would not be..
..welcome anyone's input here, just directing it a bit towards Shakib since HostCram is postured best to investigate unless I can approach the vps from a different way than I see available.
My team doesn't have access to any of our VPS nodes. I kept all access to myself and I didn't change your password or login to your account.
We operate a few other businesses and our team is working on those.
You have nothing to worry about.
I wanted to add some closure to the OP...
@forest, after @Shikib hooked up a live debian ISO I could boot, I paged through the jounal, syslog & kern logs... Thank you both for taking the time to respond here!
I didn't see anything looking like a crash, but then I'm not sure I that includes neon lights and big arrows pointing to it for me to see!
After using the bootable ISO Shakib graciously hooked me up with, I changed the root pw eventually just dove in and looked around. My XP is nothing compared to Shakib with this sort of thing, and given the lack of obvious things I'd expect to find if hacked, I'm convinced Shakib nailed it on the head from the start.
I put the monster back to work a few days ago and wanted to post a new yabs and comment a bit on HostCram and Skakib..
A few years back I asked about something and got ~this: "that's for people who have brains.." LOL nuf said. I used to have one. Forgot it at an after party some years ago, and rot has since set in with age..
However, I have noticed Shakib -has- adjusted some over the time I first hooked up with HostCram. Honestly, he's been a nice guy all along.. I know about the busy life and the value of time.. It's a slippery slope for me I know... PR has never been my strong point.
Anyhow, great products and I aim to continue using what I have at HostCram and perhaps expand on that if I ever figure out what to do with my collection of servers as a tinkerer.