New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
If there's a way to select the probes with more specific location then you would need less of them. For the initial round of finding out the continent, two per each would be enough. Then if you land in North America, the next step would be to pick north, south, west, east and central probes, again 2 each could be enough, then follow up with more granular choice around the best result. If you want to speed things up then pick one probe per each state and see what you get.
Of course for exotic location you will need more rounds and cast wider net to get any useful result. On the other hand most of the time you can use set of probes in the most likely locations and go from there.
There are no half-baked quick-proof-of-concept projects on LET, you will always get a feedback
On that topic - you could incorporate whois/geofeed and if it claims the IP is German and you run 3 pings from German and they are optimal - you can skip first two steps and just directly jump to step 3 (with 3 measurments already done). If sub-same-continent range jump to step 2. If shit crazy jump start normal flow from step 1.
--
AlphaVPS @AlexBarakov flex time? :-D
Having the full table (nC2 = 0.5*n^2) or in current probe is (~4000^2)/2 might offer better calculation. But, it's not efficient from storage and calculation. With n = 200 countries, that's 20000 rows. Then, it could be augmented by having region/country level lookup table.
For example, let's say there's 2 probe in antipodal position. Assuming the probe is always on equatorial position, at the worst case, there's 2 range of longitude where the target IP might be located. Then, the 3rd probe (nearest to one of the longitude range) is used. If it has higher ping, then the longitude guess is wrong. Assuming it's wrong, the fourth probe must be closer to the target IP, then the fifth probe could make guess at latitude range.
But, that's oversimplified since there may be congestion, routing policy, etc which is far from ideal.
It will certainly reduce the amount request to probe, in exchange of longer time to determine the nearest probe. However, does 10-20s of extra time and local compute matters when probing bulk amount of IP?
@rpqu It does sound its possible to lower the amount of probe usage this way. Probably this code can be used as basis. We would need to stop using Globalping's API for location and instead pull the full list of probes with coordinates and build our own implementation to support regions and radiuses.
But to ensure the same or better accuracy we would still need plenty of probes to dig down to the city level. Not as many, but plenty. The difference does not feel like it justifies the complexity in the code and ongoing maintenance of the DB.
But if you're interested you could send me a PR, we could add a parameter like
--methodto switch between algorithms.That would help most likely.
Can you suggest a specific source, that is public and has no limiting license.
I was actually vibe coding something similar a few weeks back, the below DBs are free to use if citation is mentioned. You could compare each of the country/city with from the data sources below, and then run a global ping on the data fetched (this is what I had in mind for mine - but looks like it's the same thing you have achieved with this implementation)
Using an existing DB feels like cheating, I would prefer a raw source. I'll look into using geofeeds somehow
Sorry, I didn't know you weren't associated with globalping.
But there's
curl "https://api.globalping.io/v1/probes" \It includes lon/lat. Therefore, you could make better estimation even without lookup table.
However, if you want to, you could have discover their IP by pinging at the server you can control, then use it as the target IP.
Assuming 200 countries, 20k row. At worst you're going to need 3 credits per row. So, it would take 3*20k/500=120 days to build the table. At best. 20000/(500-200)=66.66 ~67 days
Fuck, I sucks at math.
Day 1: 200 + 199 + 101
Day 2: 199 + 97 + 197 + 7
Day 3: 197 + 189 + 114
Day 4: 196 + 81 + 194 + 29
Day 5: 194 + 164 + 142
Day 6: 193 + 50 + 191 + 66
Day 7: 191 + 124 + 185
Day 8: 190 + 4 + 188 + 118
Day 9: 188 + 69 + 186 + 57
Day 10: 186 + 128 + 184 + 2
Day 11: 184 + 181 + 135
Day 12: 183 + 47 + 181 + 89
Day 13: 181 + 91 + 179 + 49
Day 14: 179 + 129 + 177 + 15
Day 15: 177 + 161 + 163
Day 16: 175 + 11 + 173 + 141
Day 17: 173 + 31 + 171 + 125
Day 18: 171 + 45 + 169 + 115
Day 19: 169 + 53 + 167 + 111
Day 20: 167 + 55 + 165 + 113
Day 21: 165 + 51 + 163 + 121
Day 22: 163 + 41 + 161 + 135
Day 23: 161 + 25 + 159 + 155
Day 24: 159 + 3 + 157 + 156 + 25
Day 25: 156 + 130 + 154 + 60
Day 26: 154 + 93 + 152 + 101
Day 27: 152 + 50 + 150 + 148
Day 28: 150 + 1 + 148 + 147 + 54
Day 29: 147 + 92 + 145 + 116
Day 30: 145 + 28 + 143 + 142 + 42
Day 31: 142 + 99 + 140 + 119
Day 32: 140 + 20 + 138 + 137 + 65
Day 33: 137 + 71 + 135 + 134 + 23
Day 34: 134 + 110 + 132 + 124
Day 35: 132 + 7 + 130 + 129 + 102
Day 36: 129 + 26 + 127 + 126 + 92
Day 37: 126 + 33 + 124 + 123 + 94
Day 38: 123 + 28 + 121 + 120 + 108
Day 39: 120 + 11 + 118 + 117 + 116 + 18
Day 40: 116 + 97 + 114 + 113 + 60
Day 41: 113 + 52 + 111 + 110 + 109 + 5
Day 42: 109 + 103 + 107 + 106 + 75
Day 43: 106 + 30 + 104 + 103 + 102 + 55
Day 44: 102 + 56 + 100 + 99 + 98 + 45
Day 45: 98 + 52 + 96 + 95 + 94 + 65
Day 46: 94 + 28 + 92 + 91 + 90 + 89 + 16
Day 47: 89 + 72 + 87 + 86 + 85 + 81
Day 48: 85 + 3 + 83 + 82 + 81 + 80 + 79 + 7
Day 49: 79 + 71 + 77 + 76 + 75 + 74 + 48
Day 50: 74 + 25 + 72 + 71 + 70 + 69 + 68 + 51
Day 51: 68 + 16 + 66 + 65 + 64 + 63 + 62 + 61 + 35
Day 52: 61 + 25 + 59 + 58 + 57 + 56 + 55 + 54 + 53 + 22
Day 53: 53 + 30 + 51 + 50 + 49 + 48 + 47 + 46 + 45 + 44 + 37
Day 54: 44 + 6 + 42 + 41 + 40 + 39 + 38 + 37 + 36 + 35 + 34 + 33 + 32 + 31 + 12
Day 55: 31 + 18 + 29 + 28 + 27 + 26 + 25 + 24 + 23 + 22 + 21 + 20 + 19 + 18 + 17 + 16 + 15 + 14 + 13 + 12 + 11 + 10 + 9 + 8 + 7 + 6 + 5 + 4 + 3 + 2 + 1
So, it would take 55 days Day 3: 197 + 189 + 114
This is gay, for multiple reasons.
I am associated with Globalping. And I know how to pull all probes
I just meant its a lot of work to:
1. To actually build and maintain the DB
2. Figure out and implement the geo triangulation parts. Drawing triangles, radiuses...
3. Actually debug this whole thing
And at the end you will still need to use a bunch of probes as the last step. So even if the savings are 50% the complexity does not sound like it's worth it
So I tried pulling geofeeds from https://geolocatemuch.com/ and using them as the phase 0 like @JabJab said, but its kind of pointless since it only contains 572,009 IP ranges....
The next version may be suboptimal because you probably hadn't figured out the optimal solution, but that's better than draining free credits.
I don't know the structure of your database. In my vision, it would be like this inside client side
|country_code_1| country_code_2| ping (float)|Then, separate table for triangulation (lon,lat). The triangulation itself doesn't need to be perfect because cable doesn't go straight
If there is some serious interest in the project and people actually start using it I will consider spending the time to figure out triangulation based on coordinates and pre-built latency DB. But I think for now it's a bit too much
@Operable4829
Jin Maek, you are trying to do the same as ipinfo.io
. So it is possible! Don’t give up. Your service is useful and brings value.
Just to be clear I dont plan on building an ipinfo competitor. Just a useful tool that showcases what Globalping is capable of
But after lots of testing I'm seriously considering rewriting the whole thing with something smarter. Maybe rpqu's idea but I really dont want to build and maintain a latency DB just for this.
For now it works well but only at a high probe limit. The default limit is really underwhelming. The main problem is ICMP being blocked on too many network levels.
e.g.
The solution is easy for me with millions of credits, just bruteforce with a shitload of probes. And it works.
But how do I make it work for <200 total used probes
I don't mean to be the Debbie downer of this, but wanted to be blunt. This will fail and will show incorrect locations. GlobalPing had many false locations on it. So pings will show to be near something that it isn't. Everyone can publish a Geofeed and GlobalPing uses database that accept those.
This isn't the proper way to do this. Your need a VM confirmed to be in each location, which is really hard to do. And ping from all at once. Even that isn't guaranteed. For example for Albania, a lot of things route to Italy, over Bulgaria, Macedonia, etc and our IXP here is barely used. An ISP here can have 100ms to another host or ISP in the same country if they don't share a PoP.
Really, all you'd be proving is if 2 are in the same pop, and only if the ms are less than 2. Over 2ms could be in another DC that's decently connected even. For example I get 8ms to Bulgaria or 7ms to Serbia from Albania on different ISPs.
This is all very interesting and the blog posts were great, I didn't know that "virtual locations" was a thing at all, the more you know.
@avsisp it's only an issue if there aren't enough probes. In my high credit tests it was incredibly accurate and any false positives were only because I was trying to optimize for minimal credit usage instead of best accuracy
What you described about verified locations is what ipinfo does and I would also do if I was building a geo IP product. This isn't that. It's a very simple brute force method and a little demo of what Globalping is capable of. I'm having fun working on it.
The wrong locations of probes is something I'm aware of and it's being worked on. And over time it's getting better. We have discussed using latency as a metric as well. You can check the current logic here https://github.com/jsdelivr/globalping/blob/master/docs/geoip.md + user corrections
But even with some wrongly located probes it's again a question of using multiple probes to double check.
Incredible work, Dmitriy. This is truly awesome!
The latency DB doesn't need to be updated frequently, as our world has been using cable for global telecommunication purposes for almost 2 centuries.
While this isn't my specialty, I'll try tinkering this Christmas & New year.
Congrats everyone, the tool hit the HN frontpage https://news.ycombinator.com/item?id=46834953
Just wanted to humble brag about it, it doesn't happen everyday
Jin Maek landed on hackernews! Congrats.
https://news.ycombinator.com/item?id=46834953
Should I keep a probe online if its reported geolocation is wrong? I have two servers that claim to be in South Africa but are really in Romania. I've submitted a correction report, but until it's corrected, should I leave them running?
Keep it. Software can be refactored, but hardware location is immutable.
But while it's running, won't it skew the results?
The question of software. Don't bother, unless you are trying to conserve energy.
I don't understand what you're saying.
I have nearly 40 probes. Two of them are sending false information through the API, providing fictitious location data. What does it being software have to do with anything?
The hardware and location it-self is immutable. Software interprets data points those probes send. Or does the network is so twisted that data sent is totally innacurate?
The data is indeed totally inaccurate. With those two probes, any results they come up with will be recorded as coming from South Africa. I doubt the false information would be devastating, considering correctly-marked probes outnumber incorrectly-marked ones, but I am struggling to see any benefit to keeping the probes online, since their measurements will be, to my understanding, strictly worse than nothing.
Try to fix the location in the dashboard. If it's not possible then report it here https://geodebug.globalping.dev/
Do not take it down, over time they will all be fixed