New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
Load the ISO via VNC and setup LVM With Encrypted Volume from the benninging
What about Dropbear to unlock LUKS remotely?
I haven't heard of it, might worth take a look at.
luks/clevis with tang server.
Related thread:
https://lowendtalk.com/discussion/215262/do-you-encrypt-luks-disks-on-dedi-server/p1
Another related thread: https://lowendtalk.com/discussion/214238/sane-approach-to-remote-server-encryption
Multiple tang servers for redundancy but other than that spot on. I still have dropbear installed as a fallback just in case I need to do something manually too though.
i'm at the mercy of provider for a fallback, vnc ftw.
You can install any Linux distro using kexec trick the way nixos-anywhere uses it
Is there a method that actually protects against provider technicians with physical access? I mean with GCP, AWS, Azure you get confidential computing, but with "regular" providers like Hetzner and all the ones discussed in this community, we don't.
So is there a fix for this you can recommend?
BTW even with Dropbear, or Clevis/Tang, someone with physical access can still tamper with initramfs etc. Or use IPMI to view the console and enter commands, right?
This is Hetzner specific, but I followed (an older version of) this when I first got my dedi from them and there was no data to lose: https://community.hetzner.com/tutorials/install-debian-with-lvm-encrypted-nvme-software-raid
My OVH and Fibrestate drives are currently not encrypted. I keep meaning to deal with that, but there's always something else more important to do, especially as the machines are both already in use.
Yeah, sure. But probably to alter the initramfs they'd need a reboot, and I'd have noticed an unexpected reboot (because I have to explicitly ssh in to unlock the drive). If that happened, I'd do a bit of forensics before unlocking the drive, but obviously hard to be 100% sure. At that point, I just fall back to the default that at some point you have to just trust the provider. I don't know what they did to the machine before I got it either.
Actually there are some providers here which support confidential computing.
https://lowendtalk.com/discussion/206468/vps-with-tee-capabilities
I'll be making an update on that thread soon.
Right. In theory you could use signed UKI with secure boot to make the initramfs modification a bit harder.
I'm 100% boring: luks + dropbear.
So the reason why I am asking this question and checking what you guys do is that at the moment at work we use Hetzner Cloud instances for our Kubernetes clusters (using my tool https://github.com/vitobotta/hetzner-k3s which some of you might know) and for the databases too since the database is managed by CloudNativePG on regular cluster nodes. Hetzner Cloud instances have limited storage and also we needed to encrypt data at rest for the ISO27001 (besides security in general), so we have been using Hetzner's block storage "volumes" that can grow up to 10 TB per volume. Encrypting these volumes is very easy, because I can just use a custom storage class with LUKS enabled and the keys are stored in Kubernetes secrets.
Problem is, these volumes are not fast in terms of IOPS and we are starting to see some issues so the next step is to move the database nodes to dedicated servers with higher specs and fast NVME drives. I have already implemented support for external nodes in hetzner-k3s so to use dedicated servers as cluster nodes, but I am looking at FDE options to see what would be the best approach.
In the past I have been using Dropbear since it's an easy solution, but I want something more automatic for this because we use Kured in Kubernetes to automatically drain one node per time, upgrade the OS packages, reboot the node and uncordon it again. So for this to work I need the LUKS unlocking to happen automatically.
At the moment the best candidate is to use Clevis with a couple of Tang servers for redundancy with different providers, e.g. one with Netcup since we have something there too already, and one with another provider. I might also add Dropbear as fallback just in case although I have never tried Clevis+Dropbear together.
AFAIK, having the tang servers with other providers makes it more difficult for a rogue Hetzner technician to tamper with initramfs to inject a script that runs after Clevis but before the system fully boots so the script can store the key to file etc. This would require a reboot, right? So we'd notice some unexpected downtime. Or am I missing something?
No Hetzner technician will touch your shit. Many crypto projects and stuff that worth way
more than your database are hosted there and nobody complained.
I know! But with the ISO and stuff, we need to document risks etc.
When it comes to securing hard disk drives within dedicated servers, the best method is full-disk encryption with key management. Some of the best examples include the use of Microsoft BitLocker for Windows-based dedicated servers, while VeraCrypt can be used when going for an open-source route.
No.
https://lowendtalk.com/discussion/comment/4748395#Comment_4748395
Yellowkey
So I ended up with Clevis/Tang + DropBear and it works well. I set up one Tang server in Netcup and the other in Contabo since performance isn't important for this.
That's about the best you can do. You can try to lock things down a little more by adding boot params that prevent a rescue shell. You can also setup scripts to monitor the integrity of your initramfs to hopefully allow you to spot anyone messing with it. Little things like that to give you slightly more peace of mind if you are really paranoid.
As others have pointed out though if someone has physical access and a high level of skill it's almost impossible to prevent them from gaining access to this particular setup but I think what you have done is enough to protect you from most prying eyes of say some corrupt provider trying to spy on their customers.
How did you set up a secure network between your Hetzner servers and Netcup/Contabo?
I'm not worried about tampering. I'm worried that providers may not properly dispose of the disks.