All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Is it normal for the host node to constantly read my SSH info and Fail2ban config?
English is not my native language, so I translated this with the help of AI.
With so many Linux vulnerabilities surfacing recently and lots of people getting compromised, I started learning more about server security. I learned that the QEMU Guest Agent allows a KVM host to run arbitrary commands on the guest VM. I went ahead and checked the qemu-guest-agent logs across all my machines, and noticed that one of them—hosted on ColoCrossing—behaves differently from the rest.
Roughly every hour, it reads my SSH public key, SSH configurations, and Fail2ban configurations. I don't store private keys on it, so I'm not sure if the read behavior would be different if I did.
My technical knowledge is limited, but this looks suspicious to me. I'm just not sure whether this is actually normal. Has the host node been compromised? Could one of the third-party packages I installed have gotten me compromised? Or is this just routine security scanning and completely normal behavior?
I'm reaching out to the community for help: do any of you see similar activity in your logs?
As a side note, I tried submitting a support ticket several times without success. Logging in and filling out the form went smoothly, but at the very last moment when hitting submit, Cloudflare blocked the request.
Command used:
journalctl -u qemu-guest-agent -e
Log output:
info: guest-file-open called, filepath: /proc/meminfo, mode: r
info: guest-file-open, handle: xxxx
info: guest-file-close called, handle: xxxx
info: guest-file-open called, filepath: /root/.ssh/authorized_keys, mode: r
info: guest-file-open called, filepath: /etc/ssh/sshd_confi#g.d/99-admin-access.conf, mode: r
info: guest-file-open called, filepath: /etc/fail2ban/jail.d/99-admin-ignore.conf, mode: r
While editing this post, I also got blocked by LET's Cloudflare. I think I figured out why I couldn't submit the ColoCrossing ticket: Cloudflare inspects the request body and blocks the path /etc/ssh/sshd_confi#g.d, which I had pasted as part of the logs. If you remove the "#" and try to preview the post, Cloudflare will block you.
Comments
Did you create 99-admin-access.conf and 99-admin-ignore.conf yourself? If not, that's more worrying than the reads. The names look like something added an admin login and a fail2ban whitelist for it. I'd check what's inside both files first.
looks normal
Wonderful question, i was expecting some comment about its contents.
Shocking how nonchalant people just take the qemu agent and its powers. Wild to me. But then, considering how many people put shit up publicly available on the internet, which should never be exposed (hello synology guys 👋 ), i really shouldnt be surprised.
I checked all three paths—none of these files existed because they were never created in the first place.
So, I deliberately created
/root/.ssh/authorized_keysas a test, and I noticed a change in its probing behavior:Yep, write failed. It looks like once the path exists, it starts attempting to write unknown data to it. It probably failed due to insufficient permissions. This time, I went ahead and created
99-admin-access.confand99-admin-ignore.confwith write permissions enabled to see if anything changes next time around.I haven't figured out where the issue lies yet—could be something on my end, or something else entirely. I'll keep investigating.
If anyone could run
journalctl -u qemu-guest-agent -eon their own server to see if they get the same behavior, that would be greatly appreciated.i have same behaviour on 2 different providers. I thinks its how that their control vrtulazor panel uses to check some metric of the guest os.
Those CF waf rules are so incredible naive and blunt.
@OP you may be interested in this thread: https://lowendtalk.com/discussion/220023/neutralizing-the-security-risk-of-qemu-guest-agent-without-removing-it
QEMU Guest Agent is one way that providers can offer you "lazy admin" functionality like resetting your root password from within the panel. Does this absolve your provider from anything nefarious? No, it doesn't. But the above link may be interesting to you.
i don't see a grace full shutdown as lazy admin feature. Disabling the qemu agent is asking for file corruption when the vendor needs to bounce the system
And where did I say anything about graceful shutdown or disabling the guest agent?
To quote CS Lewis, "What do they teach people in these schools."
If you believe that you need to have a Trojan horse on your vps, let me sell you a bridge. ACPI ever heard of it?
But then you can call an OS also an trojan horse.
The agent is not for only lazy admin work.
The qemu guest agent is the trojan horse. You dont need to have qemu guest agent running to get a clean shutdown. You send an ACPI signal and that initates the shutdown, just as a normal short press of the power button does on any other computer.
The guest agent is asking for issues. You dont need to get compromised, just your provider and the action can start on your behalf.
Only when the OS support acpi it will be a clean shutdown.
Youre running MS-DOS or what are you trying to say? Which OS doesnt support hardware initiated shutdown?
It really might be Virtualizor. I reinstalled the OS using the provider's default template, but the issue still persists. It looks like some kind of automated health check or inspection.
Really practical advice, already gave it a try.
I am trying to say don't remove virtual guest agents. Many cool functions won't work anymore.
Hence the reason to restrict it instead, so arbitrary commands cannot be executed but the system can still be paused, shut down without ACPI, put into a filesystem freeze for migration/host reboot, etc.
Okay stay with the worst practices.
@PingKing careful leaving those writable on a box you care about. if the write ever goes through, someone just added a key and a fail2ban whitelist on your root. i'd test that on a throwaway vm, or block the guest-file-* calls in qemu-ga (--block-rpcs) and see if the panel complains about anything
Perhaps you don't understand me. You can limit malicious actions while still allowing qemu-ga to perform the "many cool practices" that you want.
Hourly reads through the guest agent are plausible if the panel polls for status, but the write attempt is the part I would not dismiss: check the timestamps against any panel cron or agent task, and treat a successful write as a key compromise until proven otherwise. A useful hardening step is to restrict the guest agent on your side, for example by blacklisting file-write and exec operations in its configuration if you do not need password or key resets from the panel, while keeping shutdown and freeze support. Also tighten permissions on your authorized keys and Fail2ban drop-ins, rotate any key that could have been overwritten, and keep an offline copy of the agent log so you can show the provider the exact first write attempt.
The host can always mount a snapshot copy of your filesystem without your 'software harding' ever noticing it.