Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Geolocation for cheapskates like me
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Geolocation for cheapskates like me

Biggest expense so far has been the Namesilo domain name... :)

http://kate-cms.com/geolocation-with-php-part-i/

«1

Comments

  • Might give this a go. Just maybe a lot more servers and lots more traffic.

  • sleddogsleddog Member

    You're welcome.

    Thanked by 1perennate
  • cassacassa Member

    Saw it some days ago, will try it out soon :-)

  • perennateperennate Member, Host Rep
    edited May 2015

    Interesting with the redirects. I used to have something that used the great circle distance (only for large file downloads though, not the entire website), it saves some code but manually tweaking it based on continent probably reduces ping:

    //distance function is from http://www.geodatasource.com/developers/php
    function distance($lat1, $lon1, $lat2, $lon2) {
        $theta = $lon1 - $lon2;
        $dist = sin(deg2rad($lat1)) * sin(deg2rad($lat2)) +  cos(deg2rad($lat1)) * cos(deg2rad($lat2)) * cos(deg2rad($theta));
        $dist = acos($dist);
        $dist = rad2deg($dist);
        return $dist;
    }
    Thanked by 1sleddog
  • nitro85nitro85 Member

    Great tutorial, you could complete it with some Cloudflare config screenshots

    Thanked by 1inthecloudblog
  • sleddogsleddog Member

    nitro85 said: Great tutorial, you could complete it with some Cloudflare config screenshots

    Thanks. Not much to do in Cloudflare. Add A records. Click to make the clouds grey :)

    A Records:

    one.mydomain.com   => 192.168.0.1
    two.mydomain.com   => 192.168.0.2
    three.mydomain.com => 192.168.0.3
    four.mydomain.com  => 192.168.0.4
    
    mydomain.com =>  192.168.0.1
    mydomain.com =>  192.168.0.2
    mydomain.com =>  192.168.0.3
    mydomain.com =>  192.168.0.4
    

    May want to throw in a 'www' CNAME record pointing to mydomain.com.

    Thanked by 1inthecloudblog
  • sleddogsleddog Member

    @perennate said:
    Interesting with the redirects. I used to have something that used the great circle distance (only for large file downloads though, not the entire website), it saves some code but manually tweaking it based on continent probably reduces ping:

    //distance function is from http://www.geodatasource.com/developers/php
    > function distance($lat1, $lon1, $lat2, $lon2) {
    >   $theta = $lon1 - $lon2;
    >   $dist = sin(deg2rad($lat1)) * sin(deg2rad($lat2)) +  cos(deg2rad($lat1)) * cos(deg2rad($lat2)) * cos(deg2rad($theta));
    >   $dist = acos($dist);
    >   $dist = rad2deg($dist);
    >   return $dist;
    > }

    Interesting, thanks. That function makes my brain hurt :)

  • 4n0nx4n0nx Member

    :O I want to meet kate

    Thanked by 1Scottsman
  • nitro85nitro85 Member

    Just found this, do you use a cloud host or some random VPS's from diferent providers?

    http://www.lowendguide.com/3/monitoring/setup-a-ha-round-robin-blog-on-debian-7-with-cloudflare/

    I think this Round-robin DNS approch would only make sense using cheap or free shared hosting, using a master server to sync files and DB, otherwise anyone could just use some Cloud shared host / VPS provider

    The "To Do - Part II" from your article is still the most important and that gives sense to overall High Availability Round-robin DNS thing

  • sleddogsleddog Member
    edited May 2015

    4n0nx said: :O I want to meet kate

    She's very shy, but I'm hoping to coax her out to meet some people later this summer.

    image
    image

    Still lots to do...

    Thanked by 1JustAMacUser
  • sleddogsleddog Member
    edited May 2015

    @nitro85 said:
    Just found this, do you use a cloud host or some random VPS's from diferent providers?

    http://www.lowendguide.com/3/monitoring/setup-a-ha-round-robin-blog-on-debian-7-with-cloudflare/

    Yes, I've seen that. I'm using random VMs from three different providers. Which provides HA from a business perspective :)

    I think this Round-robin DNS approch would only make sense using cheap or free shared hosting, using a master server to sync files and DB, otherwise anyone could just use some Cloud shared host / VPS provider

    My 'master server' is at home. A bash script sync to the four mirrors. There's no MySQL so the sync is simply rsync.

    There are many definitions of "cloud". A "cloud" provider isn't necessarily mirroring your site at four different geographic locations, or providing HA. And if they are, it'll cost much more than the ~ $18/year I'm paying for this :)

    The "To Do - Part II" from your article is still the most important and that gives sense to overall High Availability Round-robin DNS thing

    Absolutely agree. I've intentionally not called this High Availability yet, until Part II is sorted. I haven't had time to look at it yet. If anyone has experience using the Cloudflare api, tips are appreciated!

  • jrsmithjrsmith Member

    The last time I had to implement something like this, I found that appengine started doing free geolocation for all inbound requests a couple of years ago. It made it ridiculously easy to run a simple frontend, very similar to what's described in this post, but in about 10 lines of code. It didn't use 90% of appengine's features, either, so it's been running under the free quota for about a year.

    Some details on how it works can be found here. You just read those request headers and do what you need to do.

  • sleddogsleddog Member

    jrsmith said: appengine

    I looked at that some time ago. Fact is, I like to try to keep reliance on 3rd-party services to a minimum. That's just me.

  • jrsmithjrsmith Member

    @sleddog said:
    I looked at that some time ago. Fact is, I like to try to keep reliance on 3rd-party services to a minimum. That's just me.

    As do I, but for geolocation, it's practically impossible to do with zero 3rd party dependencies. Even in the linked article, you're depending on maxmind for the geolocation database. You're not subject to them going down, sure, but you are dependent on them for updates.

  • sleddogsleddog Member

    jrsmith said: You're not subject to them going down, sure, but you are dependent on them for updates.

    That's true. But at the very coarse scale that I'm doing geolocation I don't see database updates as a big issue. For example, geolocating my home dynamic IP returns a half-dozen different results, all of which are incorrect by 200-300 miles. It doesn't matter, that's close enough. And if geolocation fails (say the database doesn't contain info for a particular IP) then the site simply loads from the current mirror. Maybe not ideal, but at least nothing breaks.

  • jrsmithjrsmith Member

    @sleddog said:

    I'm not disagreeing at all, it's a fine approach. My use case wasn't the same as yours, and Google was the client, so depending on their service wasn't an issue :)

  • sleddogsleddog Member

    jrsmith said: I'm not disagreeing at all, it's a fine approach. My use case wasn't the same as yours, and Google was the client, so depending on their service wasn't an issue :)

    I'm not disagreeing with your approach either. And I'll happily change if Google wants to hire me :)

  • @sleddog said:
    I'm not disagreeing with your approach either. And I'll happily change if Google wants to hire me :)

    Can you build it so it goes to the nearest location based on IP location. I have many servers and can't be arsed finding longs and lats

  • ScottsmanScottsman Member
    edited May 2015

    @TinyTunnel_Tom said:
    Can you build it so it goes to the nearest location based on IP location. I have many servers and can't be arsed finding longs and lats

    +1 for this.

  • sleddogsleddog Member

    TinyTunnel_Tom said: Can you build it so it goes to the nearest location based on IP location. I have many servers and can't be arsed finding longs and lats

    Scottsman said: +1 for this.

    http://www.lowendtalk.com/discussion/comment/1060464#Comment_1060464

  • @sleddog said:

    How would that be added. Not got much logic in me today.

  • sleddogsleddog Member

    TinyTunnel_Tom said: How would that be added. Not got much logic in me today.

    Something like:

    • Append the distance function to my script.
    • Remove my conditions that assign $redirect by continent & longitude.
    • Add a new condition that loops through your mirrors, calculates the distance to each, and assigns $redirect to the mirror with the smallest (shortest) distance.
  • sleddogsleddog Member

    nitro85 said: The "To Do - Part II" from your article is still the most important and that gives sense to overall High Availability Round-robin DNS thing

    I've got this scripted and working now. Will let it run and sleep on it for a day/night before posting :)

  • sleddogsleddog Member

    Looking for some advice here regarding monitoring and manipulating DNS records....

    The basic setup is:

    one.mydomain.com   => 192.168.0.1
    two.mydomain.com   => 192.168.0.2
    three.mydomain.com => 192.168.0.3
    four.mydomain.com  => 192.168.0.4
    
    mydomain.com =>  192.168.0.1
    mydomain.com =>  192.168.0.2
    mydomain.com =>  192.168.0.3
    mydomain.com =>  192.168.0.4

    Let's say mirror "three" goes down. We remove the 'A' record mydomain.com => 192.168.0.3. No problem.

    But what to do with the 'A' record three.mydomain.com => 192.168.0.3?

    The choices are:

    1. Remove it. But that'll break any links to "http://three.mydomain.com" that people may have posted around: 'Server not found" messages.
    2. Leave it. But links are still broken, with a different error message ("Unable to connect").
    3. Modify it. For example, if "one" has tested OK (is up) we could use it's IP as a temporary failover for any down mirror. So the IP for "three" is edited to 192.168.0.1.

    Option #3 has two issues, both resolvable I think....

    • Changing the IP breaks my geolocation-by-continent logic. "three" is no longer in the expected geographic location. This can be fixed by getting rid of geolocation-by-continent and using the closest-mirror approach suggested by @perennate
    • If the A record for "three" now points to the IP for "one", then nginx on "one" needs to be configured to serve requests for "three". Essentially, nginx on each mirror needs to be configured to accept requests for all mirror hostnames, e.g.
    server_name    one.mydomain.com two.mydomain.com three.mydomain.com four.mydomain.com mydomain.com;

    But the order needs to be different. The first server_name must always be the actual hostname of the mirror. For example, "two" would have:

    server_name    two.mydomain.com one.mydomain.com three.mydomain.com four.mydomain.com mydomain.com;

    Make sense? Any better ideas? Thanks for reading....

  • nitro85nitro85 Member
    edited May 2015

    @sleddog said:
    Looking for some advice here regarding monitoring and manipulating DNS records....

    Why does it redirect to a different subdomain of each VPS in the first place?

    There's no way of manipulating the destination server using PHP and Cloudflare API, and using all servers pointing to the same subdomain? Or in this way you have no control of the round-robin process and the the selection get done randomly by Cloudflare?

    The point is, for example I have a website, where most visitors are from a single country, so most traffic would be redirected to the closest server without a balanced distribution?

    The lowendguide tutorial used IP's

  • sleddogsleddog Member

    If I'm understanding it correctly, that gives you HA but not geolocation.

    To have geolocation (putting the visitor directly onto the closest mirror) it's necessary to redirect. Or use a DNS service that provides geolocation services for http, which isn't available for free as far as I know.

  • nitro85nitro85 Member
    edited May 2015

    @sleddog said:
    If I'm understanding it correctly, that gives you HA but not geolocation.

    To have geolocation (putting the visitor directly onto the closest mirror) it's necessary to redirect. Or use a DNS service that provides geolocation services for http, which isn't available for free as far as I know.

    Ok
    I edited my last message I dont know if you noticed my last question

    I've never been much concerned about latency, this world isn't that big so would have a need to think about that, for me about HA and distribution of load

    For example if I have a VPS in France and other in US West cost and 90% of my traffic is from Japan, my French VPS will barely doing anything and many times distance does not equal to latency

  • @sleddog said:
    If I'm understanding it correctly, that gives you HA but not geolocation.

    To have geolocation (putting the visitor directly onto the closest mirror) it's necessary to redirect. Or use a DNS service that provides geolocation services for http, which isn't available for free as far as I know.

    For three.xxx.com just set it to goto a diff server. Remember to have ultra low TTLs 60 is about it.

  • yomeroyomero Member
    edited May 2015

    sleddog said: which isn't available for free as far as I know.

    Rage4 it's free until certain quota. Also prometeus offers DNS which afaik is almost (if not) the same as Rage4, but I am not sure if their panel allows to customize these things.

    Also there are these ones:

    https://www.zerigo.com/managed-dns (just found it, no idea if their free plan has all the features).

    https://nsone.net/pricing/ (I want to try it since long time ago, they put it like a wonderful thing hehe)

    http://www.gslb.me/users/ Apparently just two servers on the free plan

  • sleddogsleddog Member

    yomero said: Rage4 it's free until certain quota

    I though Rage4 entry level was €1 per month?

Sign In or Register to comment.