New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Server Syncing
TinyTunnel_Tom
Member
in Help
Hello,
Today I was setting up a load balanced website and noticed that I need a cron job I can run to keep all servers in sync. MySQL database is hosted on another server just files need to be synced. I would prefer if the script connected via SSH.
Thanks
Thanked by 1hawkjohn7
Comments
rsync every x minutes or so via cron?
I use Syncthing on my LES cluster, they have a Tutorial on DO, i think its way better then RSYNC on some point over SSH.
Let's do this the cool way...
https://www.digitalocean.com/community/tutorials/how-to-set-up-master-slave-replication-in-mysql
Just files not DB
rSync
http://www.tecmint.com/sync-two-apache-websites-using-rsync/
You can set how often u want it to check sync via a cronjob. A good number is 5 mins
rsync + cron
https://code.google.com/p/lsyncd/
Syncthing
Are you syncing code or uploaded files?
Just the entire of /var/www
If you're syncing a webserver with file uploads, DONT just use plain rSync, use something like unison instead (it was built for this purpose exactly).
Unision can handle both directions.
http://www.cis.upenn.edu/~bcpierce/unison/
Are you syncing more then two sites then Unison will be a pain to setup.
Tried it with the LES wiki (four locations) and it gave me tons of random php errors because files were being synced by Unison.
I ended up with having on 'master' site and syncing changed files between that site and the other sites with rsync.
Works very nice.
Got multiple servers (around 5+) just one site however
using rsync right now. It can also be used for two-way sync, right? (with -u, which skips files with newer date?) Byebye Dropbox.
Then Unison will be a bitch setting up since it syncs both ways and is designed to sync with only two servers. It uses hidden files to keep info on the status on files and when I tried I had enough problems to decide not to use it.
Rsync with one master and the other servers as slaves/leafs.
Do you know if anyone has a cron job for this. Like a shell script to help out?
I go with staged sync on boxes that have bandwidth limits as...
rsync master to slave a/b
slave a > slave c
slave b > slave d
or
rsync master > slave a,b,c,d after each other in a .sh file
For the rsync setup?
I would point you to wiki.lowendguide.com with the setup guide there but I guess the latest update broke all leafs.
I'll get to it
Put it this way there is 12 servers to sync together and possibly more
I'd go master to one in each region then region master to all local slaves.
this is basicly what I did: http://wiki.lowendguide.com/doku.php?id=wiki_setup (if you get a CF error its because th US server is being derpy).
for 12 servers it could be done like:
2 master (Unison sync)
5 slaves per master that either pull or each master could push to 5 slaves.
2nd suggestion; 1 master that is only used internally, not as a member in the oad balanced cluster. Master pushes updated files to slaves via rsync/cron.
This way your master won't "waste" bandwidth as this is not public facing and you should always be able to work with your site.
Bandwidth isn't a huge issue I have anything from 250GB and above apart from one server but it is weighted so the other servers serve double the amount of users. And thank you so much for the tutorial