Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

URGENT: Virtualizor Compromised (31st AUG)

12346

Comments

  • @AirnodeHosting do y'all use virtualizor?

    Thanked by 1AirnodeHosting
  • torchbytetorchbyte Member, Patron Provider
    edited September 1

    After a quick look at the java file installed as a service by this hack I have come to think all it was meant to do was participate in a botnet for DDoS attacks, especially against Minecraft servers

    Glad to see the hack was "wasted" on this kind of thing :)

    It also scraped data about the server like CPU count, RAM etc. and perhaps even VM count

  • tentortentor Member, Host Rep

    Why it ends up being Minecraft DDoS related every freaking time?

  • If everybody switches to Virtfusion, wouldn't that mean that almost every cloud would be based on a single system? Virtfusion will have a huge target painted on it. Maybe competition is good, even if sometimes it fails.

  • forestforest Member
    edited September 1

    @torchbyte said: After a quick look at the java file installed as a service by this hack I have come to think all it was meant to do was participate in a botnet for DDoS attacks, especially against Minecraft servers

    No fucking WAY lmfao

  • @neighhouse said:
    @AirnodeHosting do y'all use virtualizor?

  • AlbaHostAlbaHost Member, Patron Provider

    In general, I would be careful with the argument that an upstream firewall alone means a node could not have been compromised.

    If the Virtualizor update infrastructure/IP space was hijacked and controlled by the attacker, the hypervisor itself would still initiate an outbound connection to what it believed was the legitimate Virtualizor update network. Since Virtualizor updates are automatic, an inbound firewall would normally not prevent that.

    The more important point is that once malicious code is delivered through the update mechanism and executed as root, the attacker can potentially execute arbitrary commands, modify files, create users, install services, alter firewall rules, deploy additional payloads, or establish outbound C2 connections. At that stage, SSH is only one of many possible access methods.
    In our case, for example, the attacker even modified the local firewall and allowed their own 193.32.127.0/24 subnet to our non-standard SSH port, but that was simply one post-compromise action.

    The initial problem was already root-level code execution through the compromised update path.

    It is also interesting that @Jamie_DreamIT , and ourselves seem to be among the few publicly confirming infections. Maybe we're the black sheep. 😉 I genuinely hope everyone else avoided it, but given the automatic update mechanism and the nature of the compromise, I would be very cautious about assuming a system was unaffected purely because a particular inbound path was firewalled.

    Thanked by 2zGato borkedascii
  • @AlbaHost said:
    In general, I would be careful with the argument that an upstream firewall alone means a node could not have been compromised.

    If the Virtualizor update infrastructure/IP space was hijacked and controlled by the attacker, the hypervisor itself would still initiate an outbound connection to what it believed was the legitimate Virtualizor update network. Since Virtualizor updates are automatic, an inbound firewall would normally not prevent that.

    The more important point is that once malicious code is delivered through the update mechanism and executed as root, the attacker can potentially execute arbitrary commands, modify files, create users, install services, alter firewall rules, deploy additional payloads, or establish outbound C2 connections. At that stage, SSH is only one of many possible access methods.
    In our case, for example, the attacker even modified the local firewall and allowed their own 193.32.127.0/24 subnet to our non-standard SSH port, but that was simply one post-compromise action.

    The initial problem was already root-level code execution through the compromised update path.

    It is also interesting that @Jamie_DreamIT , and ourselves seem to be among the few publicly confirming infections. Maybe we're the black sheep. 😉 I genuinely hope everyone else avoided it, but given the automatic update mechanism and the nature of the compromise, I would be very cautious about assuming a system was unaffected purely because a particular inbound path was firewalled.

    Certainly worth considering outbound rules in general for certain systems. I do have 1 particular server that has some, let's call it, paranoid lockdown on it

  • AlbaHostAlbaHost Member, Patron Provider

    @Mainfrezzer said:

    @AlbaHost said:
    In general, I would be careful with the argument that an upstream firewall alone means a node could not have been compromised.

    If the Virtualizor update infrastructure/IP space was hijacked and controlled by the attacker, the hypervisor itself would still initiate an outbound connection to what it believed was the legitimate Virtualizor update network. Since Virtualizor updates are automatic, an inbound firewall would normally not prevent that.

    The more important point is that once malicious code is delivered through the update mechanism and executed as root, the attacker can potentially execute arbitrary commands, modify files, create users, install services, alter firewall rules, deploy additional payloads, or establish outbound C2 connections. At that stage, SSH is only one of many possible access methods.
    In our case, for example, the attacker even modified the local firewall and allowed their own 193.32.127.0/24 subnet to our non-standard SSH port, but that was simply one post-compromise action.

    The initial problem was already root-level code execution through the compromised update path.

    It is also interesting that @Jamie_DreamIT , and ourselves seem to be among the few publicly confirming infections. Maybe we're the black sheep. 😉 I genuinely hope everyone else avoided it, but given the automatic update mechanism and the nature of the compromise, I would be very cautious about assuming a system was unaffected purely because a particular inbound path was firewalled.

    Certainly worth considering outbound rules in general for certain systems. I do have 1 particular server that has some, let's call it, paranoid lockdown on it

    Yeah, agreed. I think outbound filtering is something worth looking at more seriously after this.
    That said, if the Virtualizor update IPs themselves were hijacked, the initial update traffic could still have looked completely legitimate. But it definitely could have helped afterwards by blocking C2 traffic or any other unexpected outbound connections.

    Thanked by 2Obelous zGato
  • are those providers using virtualizor:

    @berohost
    @ComputeBox
    @labze
    @MannDude
    @layer7
    @lnx
    @Avoro @dataforest

  • @hyperblast said: are those providers using virtualizor:

    @MannDude uses VirtFusion, at least on the node in Sweden I'm on.

  • MannDudeMannDude Patron Provider, Veteran

    @forest said:

    @hyperblast said: are those providers using virtualizor:

    @MannDude uses VirtFusion, at least on the node in Sweden I'm on.

    Still have a small number of Virtualizor hypervisors, but none of which were impacted by this.

    Thanked by 2forest hyperblast
  • AirnodeHostingAirnodeHosting Member, Patron Provider

    L> @AlbaHost said:

    In general, I would be careful with the argument that an upstream firewall alone means a node could not have been compromised.

    If the Virtualizor update infrastructure/IP space was hijacked and controlled by the attacker, the hypervisor itself would still initiate an outbound connection to what it believed was the legitimate Virtualizor update network. Since Virtualizor updates are automatic, an inbound firewall would normally not prevent that.

    The more important point is that once malicious code is delivered through the update mechanism and executed as root, the attacker can potentially execute arbitrary commands, modify files, create users, install services, alter firewall rules, deploy additional payloads, or establish outbound C2 connections. At that stage, SSH is only one of many possible access methods.
    In our case, for example, the attacker even modified the local firewall and allowed their own 193.32.127.0/24 subnet to our non-standard SSH port, but that was simply one post-compromise action.

    The initial problem was already root-level code execution through the compromised update path.

    It is also interesting that @Jamie_DreamIT , and ourselves seem to be among the few publicly confirming infections. Maybe we're the black sheep. 😉 I genuinely hope everyone else avoided it, but given the automatic update mechanism and the nature of the compromise, I would be very cautious about assuming a system was unaffected purely because a particular inbound path was firewalled.

    Hi,

    I understand what you’re saying about the incoming rules. However, having no one able to connect to the node does provide an additional layer of protection. Those rules also cannot be edited from the server side.

    One of our old nodes, which is more or less used as a backup node without any clients on it, was compromised/entered as described in my own post.

    We acted immediately after discovering this and contacted Virtualizor within approximately 20–30 minutes after the update/cron job on the 29th. At that time, we were told that we were the first ones to report the issue, and that they did not believe it was coming from their side and had not seen anything unusual.

    We also checked with Virtualizor and used the provided commands to verify whether any of our other nodes had been affected. So far, we have found no signs of compromise on any of the other nodes.

    So I’m not saying that we were completely unaffected, but fortunately, none of our active nodes appear to have been compromised. I’m still wondering why this was the case.

    What we noticed is that all of our active nodes seem to have failed to update, while the old node without any incoming/outgoing firewall rules appears to have updated successfully. That may be related, although I obviously can’t say that with certainty.

  • SaahibSaahib Host Rep, Veteran
    edited September 2

    @default said:
    If everybody switches to Virtfusion, wouldn't that mean that almost every cloud would be based on a single system? Virtfusion will have a huge target painted on it. Maybe competition is good, even if sometimes it fails.

    There are other options also, we have solusvm, it still works (racknerd using it @dustinc ), we have virtconsole.io (formely hypervisor by @BharatB ), people can even use proxmox with some billing panel with automation.

  • hostalhostal Member, Host Rep

    @Saahib said:

    @default said:
    If everybody switches to Virtfusion, wouldn't that mean that almost every cloud would be based on a single system? Virtfusion will have a huge target painted on it. Maybe competition is good, even if sometimes it fails.

    There are other options also, we have solusvm, it still works (racknerd using it @dustinc ), we have virtconsole.io (formely hypervisor by @BharatB ), people can even use proxmox with some billing panel with automation.

    I thought your product is hypervisor.io and not virtconsole.io that points to a Namecheap domain holder page.

  • lnxlnx Patron Provider, Veteran

    We do not use Virtualizor either currently or in the past.

  • Arstechnica link: https://arstechnica.com/security/2026/09/well-executed-bgp-attack-uses-hijacked-ips-to-infect-real-networks/

    What happens to Virtfusion now? looks like everyone is looking for replacements or wants to replace.

    Also how's softaculous in general, wouldn't it have been impacted by the same thing as well?

    Any open source alternatives to Virtfusion? I once had made a very small prototype golang backend virtualizor which used gvisor. Maybe proxmox (but I have heard that it recently had some issues as well)

    And what about things which are built on top of Freebsd like https://sylve.io/ or ClonOS.

    It's not as if Open source doesn't have vulnerabilities but rather with this type of vulnerability where it was an update. I feel like Open source could've had more vetting processes rather than trusting the blind source from an IP.

    Another question I have is that I never thought that the bug relied on BGP. From my understanding, the software whitelisted some Hetzner IP used by Softaculous from which the update comes but someone in the BGP world basically spoofed that hetzner IP and then pushed a malicious update. What are the ways that when writing software, you could account for even such type of exploits! Any rule of thumb or things to keep in mind?

  • zedzed Veteran

    what the fuck are you talking about

  • @zed said:
    what the fuck are you talking about

    About Virtfusion, Proxmox, BGP and Hetzner :lol:

  • dangmbdangmb Member
    edited September 2

    @tpoll @MarkSH is ServerHost affected?

  • tpolltpoll Member, Patron Provider

    @dangmb said:
    @tpoll @MarkSH is ServerHost affected?

    No, we were not affected

  • layer7layer7 Member, Host Rep, LIR

    @hyperblast said:
    are those providers using virtualizor:

    @layer7

    Hi,

    have been using it up until ~ 20? years ago ( when they added LXD/LXC support ).

    Unfortunately too many unnecessary restrictions in their software at that time.
    Was switching back then to a self-development based on Arch+libvirt

    Thanked by 1hyperblast
  • berohostberohost Member, Patron Provider

    @hyperblast said:
    are those providers using virtualizor:

    @berohost

    No, we use Proxmox

  • What's more concerning is I have VPS on virtualiser with few providers in this thread, to this date i've received no notification from either of them they suffered potential access.

    Beyond the hack, why aren't providers notifying customers of this?

    Thanked by 1stable_genius
  • @LEBUserJoe said:
    What's more concerning is I have VPS on virtualiser with few providers in this thread, to this date i've received no notification from either of them they suffered potential access.

    Beyond the hack, why aren't providers notifying customers of this?

    hostodo sent notifications to customers. I suppose some providers are more honest than others.

  • @stable_genius said:

    @LEBUserJoe said:
    What's more concerning is I have VPS on virtualiser with few providers in this thread, to this date i've received no notification from either of them they suffered potential access.

    Beyond the hack, why aren't providers notifying customers of this?

    hostodo sent notifications to customers. I suppose some providers are more honest than others.

    Respect to them, personally i've never seen it as bad, yes its awful and I might be upset but such a wide-spread exploit I wouldn't hold it against the host.

    Just weird I found out about this in this thread, root VM manager access, any of my data could of been compromised

    Thanked by 1JohnnySac
  • xHostsxHosts Patron Provider, Veteran

    @LEBUserJoe said:
    What's more concerning is I have VPS on virtualiser with few providers in this thread, to this date i've received no notification from either of them they suffered potential access.

    Beyond the hack, why aren't providers notifying customers of this?

    We sent out a email to clients about the issue a few days ago

  • @xHosts said:

    @LEBUserJoe said:
    What's more concerning is I have VPS on virtualiser with few providers in this thread, to this date i've received no notification from either of them they suffered potential access.

    Beyond the hack, why aren't providers notifying customers of this?

    We sent out a email to clients about the issue a few days ago

    I think I only received an email from you and there are more providers who are using Virtualizor. Sed.

Sign In or Register to comment.