All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Realtime File SYNCHRO betw 2 servers (separated datacenters) in master/master mode
Hello guys,
i would to glad do following:
website Domain points (www A records) to Anycast IP (for example w active 2 POPs only, #1=US, and #2=EU)
on each mentioned POPs i have assigned VPS (directly, w/o proxy forwarding): VPS#1=US and VPS#2=EU
and need have each VPS's apache html_public folders real time synchronized w second one (and in MASTER/MASTER mode):
1) if is in html_public DIR file-write, update or deletion on VPS1, this must start immediately synchronization in the same DIR (html_public) on VPS2, connecting through second (another) public IPv4 assigned on VPS#2 (i do not have internal (private LAN) between servers)
2) if is in html_public DIR file-write, update or deletion on VPS2,this must start immediately synchronization in the same DIR (html_public) on VPS1, connecting through second (another) public IPv4 assigned on VPS#1
example:
if VPS1 haves file write, then VPS1 w anycast public IP 10.20.30.51, starts synchro update where target is: second public (no anycast) IP of VPS2, for example 215.40.80.99
if VPS2 haves file write, then VPS2 w anycast public IP 10.20.30.51, starts synchro update where target is: second public (no anycast) IP of VPS1, for example 5.46.124.32
in other words, to makes content of both VPS html_public automatically synchronized on any changes over internet (not private LAN), in MASTER/MASTER mode (not in Master/Slave only)
i know this will slow down IOPS on each one at each write, and will have any latency caused by distance of VPS1 to VPS2...
But here will be no intensive file writing often (no every second, but for example each 20, 30 sec)
rsync with cron is not what i want (no real time, + everytime is checking whole dir for changes what can cause strong IO degrading and resources consume, also long time at every run when html_public dir haves large amount of data - subdirs and files)
GlusterFS is also not very want i'm searching because require to have separate partition or disk for GlusterFS, and i want use existing home/user/ html_public folder
SAN or FibreChannel storage is not available for me in this case (only local node storage), NFS also i exclude as not right option for me
NOTE: i cannot solve all writes in database only and need same files write directly in html_public folders, so only use replicated MySQL severs is not solution, i need also replicate in realtime files in "/home" folders
- i apologize for bad English but still hope u will understand me enough to know what i want, thanks
my questions are:
1) Any recommendation for best way as do it?? software to use
2) I'm thinking currently about lsyncd :
but here is one issue: is real-time, but does not officially support two-way, but any people mentions on net that haves used it as 2 way (master/master)
so i would to glad ask:
HAVES ANYBODY HERE EXPERIENCES w lsyncd configured as MASTER/MASTER (or multiple masters, more than 2..) how it WORKS?? and it's really usable and stable working in master/master mode?
many thanks for all suggestions, opinions and experiences share w me
regards miu
Comments
No idea.
lsyncd or syncthing?
Edit: seems you already found lsyncd. I haven't ever configured it with two masters, but in theory as the sync is triggered by filesystem inotify there's no reason it wouldn't work... But expect trouble if you are modifying a file with no locking system in place.
Did anyone mention hetzner?
here is nice looking solution w lsyncd:
https://unix.stackexchange.com/questions/307046/real-time-file-synchronization
third comment by Sourav
but i'm not sure how it will stable work...
because lsyncd does not officially support master/master mode, and only master/slave
so i want ask if anyone here haves any experiences w lsyncd how work in master/master mode as Sourav is mentioned
(if not, then i can only try it and after any time gain own experience)
@jackb:
thanks for your opinion
i also expect and think it can maybe work also in master/master mode till is there no conflict in write into simultaneously opened files...
Unison?
https://www.cis.upenn.edu/~bcpierce/unison/
I don't think anyone mentioned Hetzner yet.
@miu
What's stopping you from mounting the entire /home on the glusterfs drive?
did you get a dedicated first.
a dedicated always works.
its good - i know.
Dedicated.
The more dedicated the better.
YES - very good question
https://www.cyberciti.biz/faq/howto-glusterfs-replicated-high-availability-storage-volume-on-ubuntu-linux/
he mentione:
Each server needs a separate partition or disk for GlusterFS
so im not sure can i also configure existing /home subfolder with data for GlusterFS use?
and is not necessary create extra new partition and folder for it??
@CrossBox
for concrete example, i'm not sure how will looks Set up a GlusterFS volume
for /home/userXYZ/public_html??
Step 8 – Set up a GlusterFS volume:
gluster volume create public_html replica 2 VPS1:/home/userXYZ/public_html VPS2:/home/userXYZ/public_htmlbecause he mentione (https://www.cyberciti.biz/faq/howto-glusterfs-replicated-high-availability-storage-volume-on-ubuntu-linux/):
Warning: Do not edit or write files directly to a /nodirectwritedata/brick1/ or /nodirectwritedata/brick2/ brick on each server. A direct write will corrupt your volume.
so in my case it's: do not edit or write files directly to a /home/userXYZ (what is in my case "brick" on both VPS, 1 and 2), because it can corrupt your volume
and just dir /home/userXYZ is used also for emails, SSL certs, config and by several server services writes
any idea how resolve it?
thanks
and you can cluster many dedicates---- for this you must.
i feel good now.
dedi Oh yeahhh
thank you for spamming my thread with useless replies, opacifying this thread
Unison or syncthing, both should do the job!
@miu , relax, be happy, its holidays.. u make me think too hard and its late and my brains says what you read ....
@ehab is right though.
Dedicated is love, is life, is happiness.
Now i think this set up should / would be right and working:
1) create bricks on new partitions:
VPS1:
echo '/dev/vda5 /nodirectwritedata/brick1 ext4 defaults 1 2' >> /etc/fstab ; mount -a ; mkdir /nodirectwritedata/brick1/gvol0VPS2:
echo '/dev/vda5 /nodirectwritedata/brick2 ext4 defaults 1 2' >> /etc/fstab ; mount -a ; mkdir /nodirectwritedata/brick2/gvol02) Set up a GlusterFS volume (on VPS1):
gluster volume create gvol0 replica 2 VPS1:/nodirectwritedata/brick1/gvol0 VPS2:/nodirectwritedata/brick2/gvol03) mount bricks as /home/userXYZ/public_html folder on each VPSes:
VPS1:
echo 'VPS1:/gvol0 /home/userXYZ/public_htm glusterfs defaults,_netdev 0 0' >> /etc/fstabVPS2:
echo 'VPS2:/gvol0 /home/userXYZ/public_htm glusterfs defaults,_netdev 0 0' >> /etc/fstabNo, you need 3 servers at a minimum and 4 is preferred. 1 client and two storage (or even better 3 storage servers to avoid split brain).
there is mentioned also as possible "Client mount on node itself"
so if i good understood it, then also is possible to have for it only 2 storage servers (w redundant storage / glusterfs "bricks" partitions) as necessary minimum amount of servers, where each one (gluster storage server) can be simultaneously as client mount on itself
/etc/fstab on node1:
node1:/glusterfs_volume /home glusterfs defaults,_netdev 0 0where node1 mount as client itself own glusterfs_volume to dir /home
and this probably can much improve write and read time for webAps running on itself node, because they are read and write on localhost HDD directly, instead from remote server through network (what in my case means a quite latency between servers from different datacenters and networks)
but i understand for data safety is probably better as u did recommend haves 3 "bricks" ( data redundant on 3 different storage servers), and ideally in the same private LAN with minimum latency between their physical locations
Note that any kind of master/master file sync over WAN is extremely difficult to pull off.
You'll either suffer from severe performance issues, data durability issues or reliability issues. From my experience, Gluster master/master replication works like shit over high-latency WAN.
The fundamental problem you're facing is the type of synchronization used. You are always going to have some trade-offs. You can either do synchronous or asynchronous replication.
Synchronous means that on write, the system will wait until the data is safely written to the other node. While very safe, this kind of replication absolutely cannot be done over high-latency links as it provides abysmal performance (directly dependent on link latency).
Asynchronous means that on write, the system waits until the data is written to the local node. It will then sync stuff on the background to the remote node... This kind of synchronization provides a higher risk of data loss - there is no guarantee that the write IO will be correctly sent to the other node, file overwrites won't happen... But it is the only choice you have for syncing over WAN.
The problem I have with lsyncd and similar pieces of software is that they don't track file state nor do any locking. For example, what happens when the software running on both nodes decides it's a good idea to modify a file at the exact same time? The answer is data corruption.
What exactly is your application? If you just need to sync files for file upload handling and whatnot, either
(a) make the app itself aware. When someone uploads some stuff, make the app push it to the other server.
(b) forget about multiple geo-redundant servers and just use a CDN
@SkylonHost
great, thank you for very useful experiences and opinions!
Do you also write novels in english?
i realized it now, so i send PM to him rather
all my apologies
i realized i should go to sleep. 9'th all .. eol - HIGH FIVE
HIGH FIVE.
GN.
i wrote that his ideas are great and i agree w him, because my theoretical imagines would be not working in real (in WAN environment) from several reasons.. as he mentioned
and one of the best solutions are as he mentioned such as: use internal LAN w very low latency + synchro mode (for HA, failover nodes), modify webapps to work synchronous at file writes in accordance w other asynchronous nodes, or simple use CDN (or anycast proxies) in case if only purpose is load distribution and speed up pages load for ends clients in several locations
If you just need to sync your public_html folders, just use btsync. It's by far the simplest solution for the syncing. Near real time and sync both ways. Clients for all platforms.
@FHR is right. The only proper way to do that is to have the application itself (in your case the web server) do it.
That said and looking at your knowledge level (no insult whatsoever intended) the above seems to be out of reach for you. So I'll quickly go through what has been proposed and comment for you:
GlusterFS and similar. - forget it. That introduces lots of complication and is the wrong approach for you anyway. Reason: Think of it as a "Raid drives over the network", so its purpose is quite different from your need.
Lsyncd - Principally a nice and useful tool I strongly suggest to stay away from it in your case. Yes, one almost certainly can "abuse" it and get it to work in master-master but then you'll end up with having to do plenty of rsync trickery. (One main) Reason: Once a file has been updated from V1 to V2 the software on V2 will take that as a change (or new file) and update V1 and so on. Ping Pong.
Unison - looks like the best option in your case. Think of it as "Lsyncd for master-master". The big disadvantage is that unison has been designed to be (run as) a user program that is quite talkative (which is good for its intended purpose and helps unexperienced users to avoid creating havoc) and normally expects a user sitting in front of the screen. I'm by no means an expert in unison and have used it only for occasional mirroring but iirc one can configure unison to act kind of daemon like.
Scripting a solution yourself - IF you happen to have the knowledge and experience. The reason I bring this option up at all is that all the above tools are generalistic; they have been designed for a general solution for some type of problem - and unfortunately your exact problem is addressed as a somewhat exotic use case at best. Plus being generalistic almost always also translates to being quite complex. But what you want is actually quite simple: occasionally sync a couple of (well known and defined) files between two servers.(Note: on servers anything that happens less frequently than say ten times per second is "occasionally").
The building blocks are available. inotify and similar are supported by quite some programming languages and hashing (for change checks), compression (if desired), and even some simple (to use) crypto are available too.
Unfortunately I don't have much free time at my hands now so I can't help you by writing such a solution but I'm willing to help you with advice if you decide to do it yourself. Just contact me by PM. For the sake of fairness: This offer is only valid if you happen to be at least a somewhat experienced developer say with Python or Lua; I simply don't have the time right now for a dev. newbie.
thank u very much for sharing your experiences, skills and opinions here.
(i'm only PHP+MySQL webdeveloper, but i'll try find usable solution in accordance of all useful opinions here already shared)
but if i can ask u also for your opinions about MySQL server DB replication in master-master mode (if u have too experiences with it already), what u think:
this also can enough well work (is recommended) only between servers (baremetal - dedi) within one datacenter through internal network (private LAN... for example OVH internal network)?
many thanks in advance for your opinion on this