New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
If there is boot in recovery you can install it from there.. but you need to have ram bigger than actual iso size (or 2x). I've managed to install Slackware 15.0 on KS-1 that way which is harder for that specific OS than lets say debian or almalinux.
It seems to me that it's the best way: you don't have to trust the code of the "html console" you'd otherwise have to use. Trusting what's in the repos of the distro is in most cases a better choice. That being said FDE is often overkill for a VPS: if you encrypt the folder where your private data is stored, this can be enough in many cases.
Another one is hardware thrown away. Provider deadpooling, hard drives being sold to a third party, hardware upgrade and everything ends up on the second hand market... all it takes is a clever guy tempted to check what these machines were used for for your 'private' data to become quite public.
But again it depends what data is stored on the VPS. For most uses, encryption at the storage level is not needed, and might even worsen the system's security: with no FDE you can have the server reboot by itself when there is a kernel upgrade... (yeah you can use live patching but the best solutions aren't that cheap!).
Maybe he audited the code before launching the playbook!
All helpful and useful comments above. Thanks for the suggestions.
I have a few new tricks to try: (a) Installing directly from the internet rather than a mounted .iso, (b) SSH connection to authenticate unlocking the full disk encryption, (c) ?...
This is what I do too. There's a decent guide here: https://www.cyberciti.biz/security/how-to-unlock-luks-using-dropbear-ssh-keys-remotely-in-linux/
It runs an SSH server very early in the boot process, before mounting the root file system (which is why it has to be embedded in the initramfs). Essentially all the SSH server does is let you enter the unlock password.
Just make sure you configure it to run on a different port to your usual SSH server, otherwise you'll get errors about the server's identity changing (since it's a completely separate SSH server with different keys).
I recently converted a dedicated server from one SSD to two SSDs in RAID1, adding encryption too, without having to completely reinstall. Essentially:
The unencrypted part can still be compromised: https://en.wikipedia.org/wiki/Evil_maid_attack
E.g. the Dropbear instance could be backdoored to also send your entered password elsewhere. So do not store your Bitcoin wallet worth millions on this.
Sure; and if you chose FDE to limit this risk, /boot is still unencrypted... But well... key is in RAM anyway!
Absolutely!
It isn't hugely complicated.
For the root login notifications I added a line to
.bashrc
to run acurl
call in the background to send me a Pushover notification. I'm going to expand it to send me an email similar to the below. Pretty simple stuff.I do it in
.bashrc
because I figure if someone is gaining root access to one of my servers, I have much larger problems anyway.For reboot notifications I have a simple
systemd
service that calls a bash script that sends me an email usingcurl
and simple SMTP - kind of on topic here.I don't mind at all - but I need to clean them up a little before I publish them.
I've got one role that bootstraps, which I call from a simple bash script to pass the IP address, first time root password etc...
The bootstrap role itself then does apt update+upgrade, installs sudo, creates the ansible management user + keys.
I want to expand the simple bash script to add the bootstrapped hosts to my Ansible inventory - but not sure I can automate this part.
I then run my primary playbook which depending on the "role" of the machine will install what is required (eg, DB, VPN, WEB, Docker, etc...) as well as what I call my "common" role which is effectively what I think all my instances should have installed - snippet below:
All my hosts are Debian or Ubuntu based.
Notifications, firewall, fail2ban are all seperate roles, and so is the monitoring (Hetrix) which I can clean-up and publish on Github soon-ish.
The dev-sec role is a great Ansible Galaxy role that anyone can audit on Github.
I don't really get what you're trying to say - but who said I was paranoid?
Crowdsec behaves very simirlarly to fail2ban but with the added benefit of a crowdsourced threat list.
This is smashing. I will take a look over the weekend and see how it all fits in
I am a centos user. There is going to be lots of learning...
BTW, what does bootstrap mean? Without first install how can you run that script? Maybe I am confused.
Do you need Ansible installed on the server before executing the task? I always use shell script to automate setup in new VPS, I heard Ansible is better suited for deployment task— so I'd like to give it a try.
Textbook definition is:
It's basically running a simple script (or in this case a script that calls an Ansible playbook) that prepares your system in order for it to be managed or adopted into your environment.
In this case, my bootstrap role just creates a management user that the rest of my playbook can then use to manage the host.
No you need to have it installed on your own computer, don't need anything except SSH access on your server.
Ansible is great for "configuration management of an immutable environment" - in that it will always reconfigure things to a particular baseline. Which is why ApisCP uses it.
I used to use shell scripts as well, but Ansible is more flexible, especially when managing more than one host.
If you want to try it, I posted some resources in an older comment - https://lowendtalk.com/discussion/comment/3348095#Comment_3348095
Totally sold on the idea, thanks I'll try it asap. Shouldn't be to hard to convert my current script to playbook.
I find a lot of consoles don't have working paste. It works most often for you?
SSH has working paste.
VNC does not.
Since Debian 11 and Ubuntu 22, pasted commands appear highlighted and I must press ENTER key to execute.
This can be reverted with command:
and then re-login.
This guy fucks.
I've been meaning to switch to using Ansible for setting up servers. I set up new servers so infrequently these days though (I didn't get any new servers during Black Friday for example) that it hasn't been a pressing issue.
I do use Ansible for the DNSTools probes, but the role is super basic: https://github.com/Daniel15/dnstools/blob/master/ansible/roles/dnstools-worker/tasks/main.yml
You literally just need Python and SSH on the server.
Ansible is great because you basically describe the intended final state of the server, and Ansible does the work to get there, avoiding things that have already been done. For example, if you tell it to install a package but that package is already installed, it won't do anything and will just skip that step. Same thing if you ask it to add a line to a file (maybe something in
sysctl.conf
? maybe disabling PasswordAuthentication in sshd_config?) and the line is already there, it won't add it again.This makes it a lot easier to make it idempotent (meaning you can run it multiple times without messing anything up) compared to a shell script. If you add something new, you can run the playbook across all your servers to bring them all up-to-date Shell scripts tend to assume that the server is empty and the script has never been ran.
I have a text file where I keep all my commands which I run one by one. My usual steps are:
I always install from provider template and never use ISO but now I am thinking about full disk encryption after reading OP.
Thanks to @Daniel15 for this great hint. I followed the procedure in the link that Daniel provided, this one:
https://www.cyberciti.biz/security/how-to-unlock-luks-using-dropbear-ssh-keys-remotely-in-linux/
I setup a testbed with an encrypted Debian instance, installed the Dropbear SSH server and followed the configuration steps. For me, the setup was straightforward, but I have experience with SSH public key authentication, something that may be unfamiliar to beginners.
Hint for beginners:
If you have not used public key authentication with SSH before, it might be helpful to try it out first using an ordinary SSH server, rather than trying to troubleshoot it with a newly installed Dropbear SSH server at initial startup/decrypt time.
For the first Dropbear setup, there was only one problem to troubleshoot, and it was easy to find and fix. In this case, the problem was that my chosen port appeared to be closed, but the server was still responding to pings (before the unlock). I ran an nmap scan and determined that port 22 was open, but my chosen port was not. Since the port choice was configured by setting the DROPBEAR_OPTIONS in /etc/dropbear-initramfs/config, it was easy to find the issue. Here were my mistakes, easily corrected:
I hope this helps others.
Thanks to @Daniel15 for posting the links and writing the tutorials on the site. I got the grub loader working as described here:
https://lowendtalk.com/discussion/comment/3556642#Comment_3556642
I followed the tutorial in this link:
https://netboot.xyz/docs/booting/grub/
I am now able to boot installers on my VPS over the internet from the netboot.xyz site. The menu of installers is extensive and easy to follow. I recommend you give it a try and take a look, even if you don't install anything today, it may be useful to you in the future.
A Debian 11 (bullseye) installer is there for me and it works. Here are my new questions:
-> How do I trust the netboot.xyz installers?
Is there a way to run checksums to compare against known installers from the "official" websites?
Is there a digital signature to check?
Are we limited to "trust us"?
A more modern version with more features is forked here: https://github.com/cppla/ServerStatus/
It's in chinese however, but tried running it on a docker container with success, but would need to go through the code and translate it to english. Or does anyone here know of a fork with the update cppla-features but translated to english?
Since posting the above, I found some answers. The answer is: RTFM.
There is a blog entry in the Docs on the netboot.xyz website. It describes how to self-host your own version of netboot.xyz on your own server. You can configure the menus and the boot files for yourself.
(I admit, I am still struggling with the configuration for self-hosting, but it is one step closer.)
The scripts are all plain text files so you can just read them and see what they do.
For Debian, it loads stuff directly from the official Debian servers:
https://github.com/netbootxyz/netboot.xyz/blob/development/roles/netbootxyz/templates/menu/debian.ipxe.j2
https://github.com/netbootxyz/netboot.xyz/blob/9302484e7f522a210edcb091d6faead617850448/roles/netbootxyz/defaults/main.yml#L192
If you go to http://boot.netboot.xyz/debian.ipxe (which is the version used on the production web site), you can verify that it's using the same
archive.debian.org
serverYou can pretty easily make your own iPXE scripts if you want to. You can copy and paste their Debian script, put it on your own server, and it'd work. The netboot.xyz menu is just a script that downloads and runs other scripts depending on which option you pick.
I recently setup KS-LE-1 with zfs native encryption and mirror setup. The intird has sshd and .profile to prompt for key and import pool.
Since the initrd and kernel is unauthenticated(and unencrypted but that's not a concern), theoretically it can be backdoored and i probably won't notice it. So there has to be some trust that the provider won't do such thing .
If you pay a lot maybe it's possible to get a full hardware trust chain going. But not feasible in low-end world .