What's best practice to migrate a setup from one VPS to another?

Ympker

Currently setting up a private gameserver environment (gameserver, website, mysql/php/nginx; Ubuntu 20.04 LTS) on Lunanode which I want to migrate to an affordable long-term vps at a later point.
I will run it for like a month on Lunanode in order to see which kind of performance/specs I require and then want to migrate it to another cheapo yearly vps.

What's the best practice to move such a setup?
Just rsync the homedirectory to the new vps and setup nginx/mysql again, import database and go?
Or can I somehow export the whole setup without having to re-configure mysql/nginx?

Needlessly to say, some files of course would need to be altered anyway (IP change..) during the process, but just wondering.

Ympker

cpsd

If you have novnc and you can boot from a custom iso, the best is using clonezilla to make an image of your disk. Or you can load first netboot.xyz and run it from there. You can use hetzner cloud to try this image.

szarka

I'm a big fan of Ansible. There's investment up front in writing your configuration in a way that's reproducible. But, after that, installing on a new server is a cinch, as is reinstalling when something goes wrong.

One of my dedicated servers went bad a week or two ago and, using Ansible, it took about 3 minutes to go from a virgin install of AlmaLinux to a working name server with all my zones loaded.

Abd

I believe this can work https://lowendspirit.com/discussion/3840/migrate-a-kvm-vps-from-one-host-to-another-easy-mode

Ympker

@cpsd Will have a look at clonezilla. I remember hearing good things about it. Netboot is also a good idea

@szarka Ansible is something I should have a look at. Heard the name way too often to just ignore it. Same with Docker.. Is it similar to Docker or what is the main difference here?

@Abd Instrucitons from the legendary Anth.. This could work Thanks!

jmgcaguicla

@Ympker said:
What's the best practice to move such a setup?

Not best, but if the speeds between the servers are decent and the disks aren't too big you can just copy the disk to the new one, all you need is ssh.

Both running a rescue system, initiated from the new one:

ssh user@old_server "sudo dd if=/dev/vda bs=1M" | dd of=/dev/vda oflag=sync status=progress

Of course this has a small requirement that both providers have similar KVM/QEMU setups (e.g. hard disk as vda/sda, network interface names, etc.) then the only thing you'd need to change here would be network config.

Or can I somehow export the whole setup without having to re-configure mysql/nginx?

I am an Ansible evangelist and would like to ask you if you believe in our lord and savior Ansible? If this was a playbook, you'd simply change a single line of config then run a single command.

Or even better

szarka

@Ympker said:
@szarka Ansible is something I should have a look at. Heard the name way too often to just ignore it. Same with Docker.. Is it similar to Docker or what is the main difference here?

There are certainly use cases when Ansible or Docker might address the same needs, but they're really two very different creatures. Ansible is essentially a way of scripting configuration of all your servers (and other stuff, like routers) remotely via SSH. You can use Ansible to install, set up, configure, etc., your Docker containers, so they are complementary technologies.

I've only played around a bit with Docker. As I understand it, Docker is a big win when you have multiple applications that you want to run on the same hardware in a way that keeps them isolated from one another. It's relevant to your use case in that you could build a container with Docker that contains all your applications and then easily deploy that container on another machine. But then something like Ansible would still be useful to actually deploy it, including copying/changing configuration files, etc.

bulbasaur

@Ympker said: @szarka Ansible is something I should have a look at. Heard the name way too often to just ignore it. Same with Docker.. Is it similar to Docker or what is the main difference here?

It allows you to describe the state of your servers in a declarative way. For example, what packages should be installed, which files should be created on the system, and similar. The advantage of doing it this way, as opposed to writing a shell script is for the most part you don't have to write if statements to detect the current state of your system and do things conditionally. So, whenever you want to install, upgrade or migrate systems, you just run the same playbook over and over.

szarka

@stevewatson301 said:
It allows you to describe the state of your servers in an imperative way. For example, what packages should be installed, which files should be created on the system, and similar. The advantage of doing it this way, as opposed to writing a shell script is for the most part you don't have to write if statements to detect the current state of your system and do things conditionally. So, whenever you want to install, upgrade or migrate systems, you just run the same playbook over and over.

Exactly. For example, the first thing I do when I get a new VPS or dedi to play with is make sure I have a few tools that I need to check things out and can log in with SSH keys (and then the next step will be to turn off password authentication for SSH). The little Ansible "playbook" that does that looks like this:

- hosts: all
  tasks:
  - name: install common packages
    yum: name={{ item }} state=latest
    with_items:
    - traceroute
    - wget
  - name: add ssh keys
    authorized_key:
      user: root
      state: present
      key: '{{item}}'
    with_file:
    - pubkeys/ansible

This translates as "make sure the latest versions of traceroute and wget are installed and then add this SSH public key to the authorized users for the root account."

Ympker

It seems like I'll check out some Ansible guides/tutorials later. This sounds great on so many levels Can't hurt learning smth new along the way, eh.

I also disable pw auth for ssh by default and use keys instesd so this seems nice.

This already reads exiting: https://www.guru99.com/ansible-tutorial.html#5

Thanks for all the helpful advice, everyone!

szarka

Yeah, it's nice because you don't have to go all or nothing. You can use it to automate your most common tasks and then keep automating more as you go.

rm_

Ansible sounds like the way to go for you.

Personally though I never got into it, and use "Ansible the manual way", record all steps doing the setup as a shell script, then re-run that wherever else. It is also possible to change or add lines to config files from scripts, via the likes of grep and sed. Or prepare a complete ready-to-use config and just upload it to any new host.

Another tool that I use is a shell script to "pack up" a specific service together with all its configs and data, for migration to a new machine.

Ympker

@rm_ said:
Ansible sounds like the way to go for you.

Personally though I never got into it, and use "Ansible the manual way", record all steps doing the setup as a shell script, then re-run that wherever else. It is also possible to change or add lines to config files from scripts, via the likes of grep and sed. Or prepare a complete ready-to-use config and just upload it to any new host.

Another tool that I use is a shell script to "pack up" a specific service together with all its configs and data, for migration to a new machine.

Thanks for chiming in, mate! Best to record all commands for setup would be "history", right? Along with noting down what (config)files are changed or adapted.

Yeah, grep and sed are also quite useful.
I still kind of need to read a bit into Ansible to understand what the advantages of Ansible are compared to a self-scripted bash script with several options to install all services like e.g. nyr ovpn script has "select 1) to install all services 2) to install this 3) to do that...
But I guess I'll find out once I give ansible a try

Arkas

Clonezilla isn't as reliable for this, unless you have the exact same hardware and specs, virtualization, IP...etc... In this case, I would avoid it.

TimboJones

Because this is related to tuning and changing config files, I'd use github gists for copy/pasting commands and repos for configuration files. That way, your changes are tracked and can correlate with performance.

If you had to setup many servers, ansible without question.

yoursunny

1. Wipe the old server.
2. Restore your backup to the new server.

_MS_

@Ympker, the first two steps of above mentioned process.
-1. Create a backup.
0. Check your backup.

Then go from step 2. to 1.

@yoursunny said:
2. Restore your backup to the new server.

Lastly.

@yoursunny said:
1. Wipe the old server.

Ympker

While the backup idea is nice, I think I'll give Ansible a try to learn along the way. Also, there's nothing to backup yet, because the setup isn't complete so far (still missing some steps/config). Shelving the Lunanode vm and powering it on whenever I need is really convenient, I must say.

danblaze

I would say that it is a good practice to use docker as much as possible to make the migration easier. Or choose a provider that can export server backups and import them again, and you'll have a completely seamless migration.

jackb

If you think Ansible is a good idea but prefer code to config I'd recommend checking out pyinfra.

Donkey

@Abd said:
I believe this can work https://lowendspirit.com/discussion/3840/migrate-a-kvm-vps-from-one-host-to-another-easy-mode

Will it work on all provider?

Abd

@Donkey said:

@Abd said:
I believe this can work https://lowendspirit.com/discussion/3840/migrate-a-kvm-vps-from-one-host-to-another-easy-mode

Will it work on all provider?

Yes it should.
Disk needs to be same size or more.

kuzmenkovlad

"Yes, technically, I agree with everything that was said, my tips are more about the organization, so that everything goes as smoothly as possible.
Don’t migrate your data before a campaign – always assume things can go wrong during a migration. Even if the chances are low to zero – it’s never a smart idea to schedule a data transfer before a marketing campaign, product reveal, or any other mission-critical event.
Don’t do the migration during office hours – in tune with the advice above, you should avoid starting the transfer in your busiest hours of operation. If you are running a 24/7 business, try to schedule the migration in your least active hours.
Don’t forget to test – always make extended checks after a data migration, even if it was handled by your hosting support team. Go through all your website sections, pages, and links, ensuring all images and URLs display properly. Only make domain DNS changes after you’re sure everything is operational on the new VPS server."

Kousaka

Shut both servers down, boot both into rescue environment (any linux distro basically), create filesystem on the new server, move the files. Don’t forget to keep all permissions and symlinks. This method always work for me.