"My own cloud"

netomx

I think I have seen this topic before, but I didn't get it on the results.

I want to put my web server on a "fail-safe" mode. This is, is my primary server does down, a backup server will be online.

The approach is like this:

1. Set the DNS servers to have to A records with different TTL (is this possible?), so if the primary goes down, it will redirect to the backup
2. rsync the html directory every hour/half hour/whatever.
3. Make the main server's MySQL to "master" and the backup as "slave" and make replications.

Any suggestion? Thank you! =)

Wintereise

Use those dns based solutions that redirect based on availability, usually not free though.

DNSMadeEasy offers something like that.

netomx

@Wintereise said: Use those dns based solutions that redirect based on availability, usually not free though.

maybe a dns server with a script to verify availability (pinging or connecting to a port) and change the IP if needed?

gsrdgrdghd

rage4 can do what you need in combination with the UptimeRobot API

jhadley

Have done this before with DNSMadeEasy exactly as you mentioned and it worked fine.

Wintereise

maybe a dns server with a script to verify availability (pinging or connecting to a port) and change the IP if needed?

Sure, why not? Just set an extremely low ttl and you'll be good to go.

Patrick

+1 for DNSMadeEasy, it's only $29.95 + 1 Failover Record $4.95

ajit

I am using Cloudflare. It has APIs (not used) and a Free Plan.

Fliphost

If i remember correctly, Rage4 has support for uptimeRobot API. You may be able to do it with that.

FRCorey

Linode + LinodeBalancer

gbshouse

@netomx - we have few customers which use similar configuration using our failover and GeoDNS. Regarding point #3 - I would rather use master-master setup

netomx

@gbshouse said: master-master setup

why not master-slave ?

gbshouse

@netomx - for better performance - setup round robin DNS with cross failover - master-master MySQL setup (remember to configure primary keys) - deploy static files from external git repo with autosyncing - use shared cache for better performance

pubcrawler

Good recommendations there @gbshouse.

Round robin DNS is blah. It's DNS solution 101. Has limitations and inefficiencies.

Instead of round robin DNS use something like Rage4 DNS to accomplish failover via DNS and geographic direction of traffic to nearest server in your cloud pool.

gbshouse

Round robin is not so bad if you have 2 or more nodes in let's say common area -if you put two VMs in for example LA with two different providers (so latency for distributed cache and MySQL replication will be small) you can use round robin to achieve balanced load on all nodes.

DNSbed

@netomx said: Set the DNS servers to have to A records with different TTL (is this possible?), so if the primary goes down, it will redirect to the backup

The same subdomain with multi-A records and each one with different TTL? it's impossible at least for BIND, BIND will take the lowest one. What you need is a dns product like GSLB if you deploy the servers acrossing the IDCs. If your servers are in the same location, thus something like LVS or even Nginx can do this HA job.

pubcrawler

I've done exactly what is proposed here with DNSMADEEASY. $50 a year roughly for our use and DNS total lookups.

If you don't need geographic distribution, DNSMADEEASY is probably the easiest, least costly and most mature solution out there. Great DNS company with rock solid uptime.

The rsync mirroring is fairly alright. We use it at 5 minute schedule interval via cron. It runs against a file set that is around 500k-1 million files. Even with that number of files, could run it more frequently, but in the case that big files need mirrored I don't want another rsync going off and trying to do the same thing while the first one is still running.

There are mirroring solutions at the OS level, but bound not to work with VPSes. So not a portable and able to duplicated always working solution.

MySQL is still the trouble spot. Lots out there about mirroring that. Probably best to run a master-master setup. Still dragging my feet on this. We do regular full dumps of major tables/databases and mirror those via rsync to remote server. In case of total failure, I have the master data as of last export and mirror job. I detest MySQL complexity :)

The other thing to think about that we do is front ending both cloud servers with NGINX. We cache static files to avoid any backend load. More important to me though is the use of autossh to maintain a SSH channel that stays open between the two servers. On both servers, we use NGINX in proxy mode to first try to get files and dynamic elements from the local server, but in case of failure, error, etc. we reattempt the request that request over the SSH tunnel to the other server.

BronzeByte

@netomx said: maybe a dns server with a script to verify availability (pinging or connecting to a port) and change the IP if needed?

Just specify multiple DNS entries and the one who does the actual DNS lookup while look which one pings back fastest and uses that.

pubcrawler

DNS order of A records is a bit problematic.

Not sure what the spec says and what the adherence is.

You can return multiple A records for say www.whatever.com like: 199.1.1.1 200.2.2.2

And you could reorder those based on whatever your cobble together in your own DNS backend.

Problem is if the intermediary server will pass the records along unchanged and if the end device keeps the order.

Prudent to return multiple A records that are on different servers, geographically, etc. Never know when connectivity to the end user is fouled up, even though your servers might be functioning just fine from your perspective.

joepie91

@netomx said: Make the main server's MySQL to "master" and the backup as "slave" and make replications.

I think you'll want master-master replication or your site will basically end up in read-only mode when the main server goes down. (Mind you, I'm not anywhere near an expert on MySQL replication.)

djvdorp

@ajit cloudflare dnses had al lotta ddos lately. I stopped using them because of it.

netomx

Any good guides for master-master? I'm using:

http://www.howtoforge.com/mysql_master_master_replication

but the slave can't start. If I remove the slave part in the config, and add it in the mysql client, it works:

change master to MASTER_HOST='host', MASTER_USER='user', MASTER_PASSWORD='pass', MASTER_LOG_FILE='mysqld-bin.000005', MASTER_LOG_POS=9379;

thanks!

gbshouse

@netomx - which version? in > 5.1 do no declare this part

master-host = 192.168.16.4 master-user = replication master-password = slave master-port = 3306

just run "change master ..." query

netomx

Ok got it. Thanks!

fresher_06

@netomx -- so which solution you are going for "Rage4 with uptimeRobot API" , DNSMADEEASY ,Cloudflare or anything else which is Free or very cheap .. It would be really helpful if you can share what steps you have taken to configure the whole setup..

netomx

@fresher_06 said: Cloudflare

this. yes I will, when I finished with the f$&%"ing replication :@

fresher_06

@netomx -- have you got this sorted ??

netomx

I dont know, will tedt tomorrow

ElliotJ

@Netomx I too am looking for a way to remove all points of failure, including MySQL.

I've not set it up yet, but I'm having a peek at NuoDB who seem to have developed an alternative MySQL server, a bit like MariaDB, but designed to be scaled upwards with redundancy.

Might be worth looking into :)

MrOwen

@ElliotJ said: I've not set it up yet, but I'm having a peek at NuoDB who seem to have developed an alternative MySQL server, a bit like MariaDB, but designed to be scaled upwards with redundancy.

Possibly interesting, but as of right now, it looks like it only supports 64 bit *nix OSes.

And the setup looks like a bear. And it looks like it's written in Java? (or at least part of it is) *shudder*

gbshouse

If you need stable MySQL alternative try Percona Cluster - NuoDB looks good but putting something in beta stage into production environment is not best choice