Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


โ€บ You got a new server... now what?
New on LowEndTalk? Please Register and read our Community Rules.

You got a new server... now what?

Hey everyone, I hope this past BlackFriday and CyberMonday allowed you to fulfill your needs for thr perfect server. I know I got my fair share, but I wasn't lucky to get all what I wanted. Well, next time :)

I am paranoid, so for all servers, I do:

  • Install OS from ISO
  • Setup Full Disk Encryption
  • Try setting up IPv6 if not enabled
  • Setup monitoring (Hetrixtools/ similar)
  • And setup the server for idling

What is it that you guys usually do after receiving the email that the server is setup?
Any particular rituals to harden the server?

Thanks for reading ๐Ÿ“š

ยซ1

Comments

  • My flow is similar to yours but I've got an Ansible playbook that bootstraps my machines, creates relevant users, adds SSH keys, installs necessary (imo) programs, and hardens installations (https://github.com/dev-sec/ssh-baseline) + firewall + crowdsec or fail2ban, and adds monitoring.

    It's all well automated at this point.

    I also have a scripts to notify me if someone logs in as root, or if the server reboots.

    How are you going about full disk encryption?

    Thanked by 3FoodLover ehab abtdw
  • @tjn said:
    My flow is similar to yours but I've got an Ansible playbook that bootstraps my machines, creates relevant users, adds SSH keys, installs necessary (imo) programs, and hardens installations (https://github.com/dev-sec/ssh-baseline) + firewall + crowdsec or fail2ban, and adds monitoring.

    It's all well automated at this point.

    I also have a scripts to notify me if someone logs in as root, or if the server reboots.

    How are you going about full disk encryption?

    Wow, thats so awesome. I would love to hear more about the scripts you have about notifications, if you dont mind.

    FDE, most of the time, I enable it as a part of the initial GUI installer via IPMI/ noVNC console. That allows me to be free of this one piece and forget about it.

    Also, not sounding like a moocher, would you mind sharing some snippets of your playbook?

    Thanks

  • yabs.sh just after setting up os

  • yoursunnyyoursunny Member, IPv6 Advocate

    First, I enable the rescue system and look for secret push-up videos in the hard drive.
    After that, I reinstall with netboot.xyz, so that the system is clean and consistent.

    I keep setup commands of each server in text files.
    They are pasted manually into the console.
    When I acquire a new server, I duplicate the file of a similar server, and make modifications from there.

  • @yoursunny said:
    First, I enable the rescue system and look for secret push-up videos in the hard drive.
    After that, I reinstall with netboot.xyz, so that the system is clean and consistent.

    I keep setup commands of each server in text files.

    What are these commands?

  • @FatGrizzly said:
    yabs.sh just after setting up os

    Completely forgot to add YABS in my post. Yes. I do that too. Albeit manually, sometimes multiple times just for fun

  • add_iTadd_iT Member
    edited December 2022

    Almost same

    for me add some package that i need and then install neofetch (optional)

  • @plumberg said:
    Hey everyone, I hope this past BlackFriday and CyberMonday allowed you to fulfill your needs for thr perfect server. I know I got my fair share, but I wasn't lucky to get all what I wanted. Well, next time :)

    I am paranoid, so for all servers, I do:

    • Install OS from ISO
    • Setup Full Disk Encryption
    • Try setting up IPv6 if not enabled
    • Setup monitoring (Hetrixtools/ similar)
    • And setup the server for idling

    What is it that you guys usually do after receiving the email that the server is setup?
    Any particular rituals to harden the server?

    Thanks for reading ๐Ÿ“š

    whats your process of encryption

  • yoursunnyyoursunny Member, IPv6 Advocate

    @plumberg said:

    @yoursunny said:
    I keep setup commands of each server in text files.

    What are these commands?

    Here's a sample file from 2017~2020, for server vps5-virmach-buf.
    This is not meant to be a tutorial or a reference, but merely gives an idea of what kind of setup commands would be pasted.

    [base]
    Debian 10 CD install
    root password: Mechanic-Weaken-7
    username: sunny
    Partition
      1. 6GB at /
      2. 768MB swap
      3. remaining at /home
    
    echo 'APT::Install-Recommends "no";
    APT::Install-Suggests "no";' >/etc/apt/apt.conf.d/80recommends
    echo 'nameserver 8.8.8.8
    nameserver 1.1.1.1' >/etc/resolv.conf
    chattr +i /etc/resolv.conf
    sed -i -E '/^#?PasswordAuthentication\b/ s|.*|PasswordAuthentication no|' /etc/ssh/sshd_config
    sed -i -E '/^#?SystemMaxUse=/ s|.*|SystemMaxUse=50M|' /etc/systemd/journald.conf
    
    apt purge nano
    apt install curl git htop jq screen sudo tcpdump traceroute ufw vim
    /usr/sbin/usermod -a -G sudo sunny
    
    sudo visudo
      %sudo   ALL=(ALL:ALL) NOPASSWD: ALL
    
    [IPv6]
    https://tunnelbroker.net/tunnel_detail.php?tid=611223
    Example Configurations - Debian/Ubuntu
    
    sudoedit /etc/gai.conf
      precedence ::ffff:0:0/96  100
    
    sudo chattr +i /etc/network/interfaces
    
    [ufw]
    sudo ufw status numbered
    
    sudo ufw allow 22/tcp
    sudo ufw enable
    sudo ufw deny out 25/tcp
    
    # TunnelBroker
    sudo ufw allow from 209.51.161.14 proto ipv6
    
    # websites
    sudo ufw allow to 23.95.234.187 port 443 proto tcp
    sudo ufw allow to 2001:470:1f07:48d::80 port 443 proto tcp
    sudo ufw allow to 23.95.234.187 port 2015 proto tcp
    
    [Caddy]
    sudo apt install -y gnupg
    curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/cfg/gpg/gpg.155B6D79CA56EA34.key' | sudo apt-key add -
    curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/cfg/setup/config.deb.txt?distro=debian&version=any-version' | sudo tee -a /etc/apt/sources.list.d/caddy-stable.list
    sudo apt update
    sudo apt install caddy
    
    sudo mkdir -p /var/log/caddy
    # upload /etc/caddy/Caddyfile
    
    sudo mkdir /home/web
    sudo chown -R sunny:www-data /home/web
    sudo chmod g+ws /home/web
    sudo usermod -a -G www-data sunny
    
  • @socialzzz said:

    @plumberg said:
    Hey everyone, I hope this past BlackFriday and CyberMonday allowed you to fulfill your needs for thr perfect server. I know I got my fair share, but I wasn't lucky to get all what I wanted. Well, next time :)

    I am paranoid, so for all servers, I do:

    • Install OS from ISO
    • Setup Full Disk Encryption
    • Try setting up IPv6 if not enabled
    • Setup monitoring (Hetrixtools/ similar)
    • And setup the server for idling

    What is it that you guys usually do after receiving the email that the server is setup?
    Any particular rituals to harden the server?

    Thanks for reading ๐Ÿ“š

    whats your process of encryption

    I usually use the OS GUI installer to encrypt the partition. So anytime the system restarts it waits for me to enter the passphrase

  • @plumberg said:

    @socialzzz said:

    @plumberg said:
    Hey everyone, I hope this past BlackFriday and CyberMonday allowed you to fulfill your needs for thr perfect server. I know I got my fair share, but I wasn't lucky to get all what I wanted. Well, next time :)

    I am paranoid, so for all servers, I do:

    • Install OS from ISO
    • Setup Full Disk Encryption
    • Try setting up IPv6 if not enabled
    • Setup monitoring (Hetrixtools/ similar)
    • And setup the server for idling

    What is it that you guys usually do after receiving the email that the server is setup?
    Any particular rituals to harden the server?

    Thanks for reading ๐Ÿ“š

    whats your process of encryption

    I usually use the OS GUI installer to encrypt the partition. So anytime the system restarts it waits for me to enter the passphrase

    i find this interesting as once your server is unlocked and in a running state you can still potentially access it.

    Thanked by 1Tony40
  • @yoursunny said:

    @plumberg said:

    @yoursunny said:
    I keep setup commands of each server in text files.

    What are these commands?

    Here's a sample file from 2017~2020, for server vps5-virmach-buf.
    This is not meant to be a tutorial or a reference, but merely gives an idea of what kind of setup commands would be pasted.

    [base]
    Debian 10 CD install
    root password: Mechanic-Weaken-7
    username: sunny
    Partition
      1. 6GB at /
      2. 768MB swap
      3. remaining at /home
    
    echo 'APT::Install-Recommends "no";
    APT::Install-Suggests "no";' >/etc/apt/apt.conf.d/80recommends
    echo 'nameserver 8.8.8.8
    nameserver 1.1.1.1' >/etc/resolv.conf
    chattr +i /etc/resolv.conf
    sed -i -E '/^#?PasswordAuthentication\b/ s|.*|PasswordAuthentication no|' /etc/ssh/sshd_config
    sed -i -E '/^#?SystemMaxUse=/ s|.*|SystemMaxUse=50M|' /etc/systemd/journald.conf
    
    apt purge nano
    apt install curl git htop jq screen sudo tcpdump traceroute ufw vim
    /usr/sbin/usermod -a -G sudo sunny
    
    sudo visudo
      %sudo   ALL=(ALL:ALL) NOPASSWD: ALL
    
    [IPv6]
    https://tunnelbroker.net/tunnel_detail.php?tid=611223
    Example Configurations - Debian/Ubuntu
    
    sudoedit /etc/gai.conf
      precedence ::ffff:0:0/96  100
    
    sudo chattr +i /etc/network/interfaces
    
    [ufw]
    sudo ufw status numbered
    
    sudo ufw allow 22/tcp
    sudo ufw enable
    sudo ufw deny out 25/tcp
    
    # TunnelBroker
    sudo ufw allow from 209.51.161.14 proto ipv6
    
    # websites
    sudo ufw allow to 23.95.234.187 port 443 proto tcp
    sudo ufw allow to 2001:470:1f07:48d::80 port 443 proto tcp
    sudo ufw allow to 23.95.234.187 port 2015 proto tcp
    
    [Caddy]
    sudo apt install -y gnupg
    curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/cfg/gpg/gpg.155B6D79CA56EA34.key' | sudo apt-key add -
    curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/cfg/setup/config.deb.txt?distro=debian&version=any-version' | sudo tee -a /etc/apt/sources.list.d/caddy-stable.list
    sudo apt update
    sudo apt install caddy
    
    sudo mkdir -p /var/log/caddy
    # upload /etc/caddy/Caddyfile
    
    sudo mkdir /home/web
    sudo chown -R sunny:www-data /home/web
    sudo chmod g+ws /home/web
    sudo usermod -a -G www-data sunny
    

    This is actually neat way of keeping all commands handy to setup ๐Ÿ‘Œ

    I have started curating something similar in the last few weeks.

    Thanks

  • @socialzzz said:

    @plumberg said:

    @socialzzz said:

    @plumberg said:
    Hey everyone, I hope this past BlackFriday and CyberMonday allowed you to fulfill your needs for thr perfect server. I know I got my fair share, but I wasn't lucky to get all what I wanted. Well, next time :)

    I am paranoid, so for all servers, I do:

    • Install OS from ISO
    • Setup Full Disk Encryption
    • Try setting up IPv6 if not enabled
    • Setup monitoring (Hetrixtools/ similar)
    • And setup the server for idling

    What is it that you guys usually do after receiving the email that the server is setup?
    Any particular rituals to harden the server?

    Thanks for reading ๐Ÿ“š

    whats your process of encryption

    I usually use the OS GUI installer to encrypt the partition. So anytime the system restarts it waits for me to enter the passphrase

    i find this interesting as once your server is unlocked and in a running state you can still potentially access it.

    What do you mean?

    Yes once unlocked it's accessible. So?

  • @plumberg said:

    @socialzzz said:

    @plumberg said:

    @socialzzz said:

    @plumberg said:
    Hey everyone, I hope this past BlackFriday and CyberMonday allowed you to fulfill your needs for thr perfect server. I know I got my fair share, but I wasn't lucky to get all what I wanted. Well, next time :)

    I am paranoid, so for all servers, I do:

    • Install OS from ISO
    • Setup Full Disk Encryption
    • Try setting up IPv6 if not enabled
    • Setup monitoring (Hetrixtools/ similar)
    • And setup the server for idling

    What is it that you guys usually do after receiving the email that the server is setup?
    Any particular rituals to harden the server?

    Thanks for reading ๐Ÿ“š

    whats your process of encryption

    I usually use the OS GUI installer to encrypt the partition. So anytime the system restarts it waits for me to enter the passphrase

    i find this interesting as once your server is unlocked and in a running state you can still potentially access it.

    What do you mean?

    Yes once unlocked it's accessible. So?

    so generally you have you vps running all the time right....so the point of encryption is to encrypt so unless you turn of your vps every night it's pointless. Unless you go one step further and do client side encryption on deployment

    Thanked by 1plumberg
  • @socialzzz said:

    @plumberg said:

    @socialzzz said:

    @plumberg said:

    @socialzzz said:

    @plumberg said:
    Hey everyone, I hope this past BlackFriday and CyberMonday allowed you to fulfill your needs for thr perfect server. I know I got my fair share, but I wasn't lucky to get all what I wanted. Well, next time :)

    I am paranoid, so for all servers, I do:

    • Install OS from ISO
    • Setup Full Disk Encryption
    • Try setting up IPv6 if not enabled
    • Setup monitoring (Hetrixtools/ similar)
    • And setup the server for idling

    What is it that you guys usually do after receiving the email that the server is setup?
    Any particular rituals to harden the server?

    Thanks for reading ๐Ÿ“š

    whats your process of encryption

    I usually use the OS GUI installer to encrypt the partition. So anytime the system restarts it waits for me to enter the passphrase

    i find this interesting as once your server is unlocked and in a running state you can still potentially access it.

    What do you mean?

    Yes once unlocked it's accessible. So?

    so generally you have you vps running all the time right....so the point of encryption is to encrypt so unless you turn of your vps every night it's pointless. Unless you go one step further and do client side encryption on deployment

    Depends on the use case. For storage boxes, the data stored is encrypted client side before persisting on the vps

    Plus FDE offers protection at rest. And if for some reason my credentials are compromised and server is restarted, the attacker will be out of luck.

    Also with modern technology, there is native support for hardware encryption so the performance hut us minimal

    I know the key is probably accessible from RAM in the host. So this act is not where I don't trust the hosts... if that was the case, I'd have no business with them.

  • emgemg Member
    edited December 2022

    What I would like to know is:

    • How do you decrypt / unlock it? Are you using the provider's HTML VNC console (in the control panel) from your browser? Something else?
    • When is it running?
    • When do you leave it shutdown (encrypted)?

    @socialzzz said:
    whats your process of encryption

    If you are booting from the .iso and installing the operating system yourself, full disk encryption is offered during that installation. In the Debian installer, choose "Configure encrypted volumes" when you reach the "Partition disks" panel and follow the prompts. Be sure you understand how you plan to use the console to unlock your encrypted VPS when it reboots. When your encrypted VPS boots, it just sits there, inaccessible, doing nothing, waiting for you to unlock the encryption so it can boot. You probably have the console figured out already, as you probably used it to install your operating system from the mounted .iso installer.

    Thanked by 1plumberg
  • ralfralf Member
    edited December 2022

    I set up wireguard and then add iptables rules to limit ssh to only wireguard and a single other IP that I trust.

    Then I run netstat -alnp to see what services the default install has left around that I might want to tidy up. And finally, as a sanity check, I run nmap from another machine to verify that no ports are open and/or what ports the provider filters.

    But your checklist is good - I didn't think about full disk-encryption when I got my new dedi, so I might rework things a little before I fill it up too much...

    Thanked by 2plumberg melp57
  • @nullnothere said:
    I'm usually stuck here: https://xkcd.com/910/

    I have decided to opt for random characters or Defaults provided

    Choosing the hostname is something I have been carried away for weeks.

    But on a serious nite I use a list of common unique names I found posted by someone here

    It's much easier

    I'll post that line once I find it

    Thanked by 1nullnothere
  • ralfralf Member
    edited December 2022

    @plumberg said:
    Choosing the hostname is something I have been carried away for weeks.

    I have 3 domains that I generally use: 1 for my personal servers, 1 for my company's contract work and 1 for the company's (unrelated) product. In each case, I had a theme for these names based on the domain name.

    On all of these, I had VM instances that were more logically named, such as test-api-nl (for API test in NL), but the problem was I could never remember what the underlying host was called when I wanted to change anything on it.

    Especially, by the time I got to about 10 hosts (one dedi and many biggish VPS), I found it was just impossible to remember anything apart from the first 3 I'd set up, so I then renamed all of the later ones as: provider-location-1, e.g. adv-nl-1, hc-tx-1, gc-sg-1, etc

    It's a bit more typing, and the -1 feels redundant, but it does make it more obvious that it's the host, and also I got a -2 for the first time this BF!

    In a total abuse of RFCs, I also bought a cheap 4 character .uk domain and only populate it the private addresses of hosts from within my wireguard network. But that means, I don't have to fiddle with the DNS resolver but can still look up everything easily as soon as wireguard is set up.

    Thanked by 1plumberg
  • @emg said:
    What I would like to know is:

    • How do you decrypt / unlock it? Are you using the provider's HTML VNC console (in the control panel) from your browser? Something else?

    Debian has a package that lets you unlock it over SSH dropbear-initramfs

    Thanked by 1emg
  • @darkimmortal said:

    @emg said:
    What I would like to know is:

    • How do you decrypt / unlock it? Are you using the provider's HTML VNC console (in the control panel) from your browser? Something else?

    Debian has a package that lets you unlock it over SSH dropbear-initramfs

    Infact Centos and it's like have Tang/ Clevis which offer running a dedicated service to allow remote unlock. I tried both before. But I feel the thrill to get alerts that my system is actually down and rush to unlock it

    Thanked by 2emg webcraft
  • emgemg Member
    edited December 2022

    I have been looking at VPSs and whole disk encryption from the perspective of the threat model. Others commented about it above.

    Keep in mind that whole disk encryption is the solution, not the requirement. The threat model drives the requirements. Think about what is required for a successful attack on an encrypted VPS.

    In my opinion, the primary value of whole disk encryption in a VPS is that it protects your data from "random" attackers who target your provider. Such attackers are unlikely to be sniffing unlock operations or capturing encryption keys from the RAM of running VPSs, but they may scoop up your VPS's encrypted container along with other stored data belonging to the provider. If the attackers are looking at the contents of VPSs, where yours is encrypted and the vast majority of the others are not, your data is secure.

    Another threat is the provider itself. Your only protection against the provider's employees comes when the VPS is not running. If the VPS is running, then you must trust the provider and its employees not to look. It may come down to the technical skills of a curious individual.

    If your VPS becomes the target of a well-funded adversary, then you have much bigger problems. That adversary may have legal authority and/or lots of resources to bribe or coerce employees to provide access to your running VPS. They may also have the patience to wait until you unlock your VPS or use automated tools to watch it for them. In such threat models, whole disk encryption is not going to keep your data secure.

    I have my own procedure to install and configure a "tightly secured" VPS, including whole disk encryption. It was an interesting exercise, because it exposed assumptions about the security and associated threats for each step. Think about who can observe and record your actions when you setup your "secure" VPS.

    This is a rabbit hole with no bottom.

    P.S. Adding: When I wrote the posts above, I did not know about remote unlocking via SSH as mentioned by @darkimmortal and @plumberg until after I saved this post. I will look into it. Thanks!

    Thanked by 1plumberg
  • 1/ Set the root password to something easy to remember (e.g. Blah123)
    2/ Idle for a year till next BF

  • @plumberg said: Install OS from ISO

    Quick question, how do you install from ISO if the provider does not officially support it ?

    Thanked by 1plumberg
  • @Liso said:

    @plumberg said: Install OS from ISO

    Quick question, how do you install from ISO if the provider does not officially support it ?

    sometimes via netboot.xyz if that is offered...

    else try what @Daniel15 suggested in another thread: https://lowendtalk.com/discussion/comment/3556642#Comment_3556642

    HTH

    Thanked by 2abtdw webcraft
  • emperoremperor Member
    edited December 2022

    Install OS from ISO
    apt update & apt upgrade
    apt install wget curl nano htop iftop tcl tcl-dev vnstat python2 fortune-mod fortunes cowsay build-essential
    setup fancy MOTD
    change ssh port, disallow root login and setup AllowUsers
    add user with ssh keys
    setup ipv6 if its not already installed
    install serverstatus client
    run yabs
    install openvpn and (or) wireguard
    idle ofc for 364 days

    Fancy MOTD :

    This is for debian, slackware and alma are slightly different

    Thanked by 1plumberg
  • @emperor said:
    Install OS from ISO
    apt update & apt upgrade
    apt install wget curl nano htop iftop tcl tcl-dev vnstat python2 fortune-mod fortunes cowsay build-essential
    setup fancy MOTD
    change ssh port, disallow root login and setup AllowUsers
    add user with ssh keys
    setup ipv6 if its not already installed
    install serverstatus client
    run yabs
    install openvpn and (or) wireguard
    idle ofc for 364 days

    Fancy MOTD :

    This is for debian, slackware and alma are slightly different

    This is slick...

    What's this serverstatus client?

    Thanked by 1emperor
  • @tjn said:
    My flow is similar to yours but I've got an Ansible playbook that bootstraps my machines, creates relevant users, adds SSH keys, installs necessary (imo) programs, and hardens installations (https://github.com/dev-sec/ssh-baseline) + firewall + crowdsec or fail2ban, and adds monitoring.

    It's all well automated at this point.

    I also have a scripts to notify me if someone logs in as root, or if the server reboots.

    How are you going about full disk encryption?

    Sooo you are also paranoid but you use a company that creates honeypot?
    https://www.crowdsec.net/product/threat-intelligence

    Great way to be "paranoid", only god knows if your machine is also a "honeypot collecting data" or not.

    Thanked by 1pbx
  • @plumberg said: What's this serverstatus client?

    Its BoToX script for monitoring : https://github.com/BotoX/ServerStatus
    Sadly script its in python2 so wont work on alma9 and other os where python2 is not in repos. It will work in bash but in bash shows wrong values.

    Thanked by 1plumberg
Sign In or Register to comment.