@kait said:
icmp flood the world Interesting project hope it goes well, will you do a comparison against maxmind and other ip2location services?
I already wrote benchmark scripts, to keep track on the builds.
Basically I grab a routing table dump and throw ip's against selected .mmdb's including this one.
Right now the .mmdb has a 74% hit rate, if you take any IP from the internet and look it up.
For the rest, I just don't have any data.
However, no idea about the accuracy, I would need a 3rd party source, that is nearly 100% certain where the IP's originate from, to benchmark that.
And yes please don't try to ping the whole internet using Globalping
Well, to be fair, correcting a city, within a country, is going to be very hard.
However, the biggest problem I think rather is, continents.
There are a lot of entries, that just give you the entire wrong continent.
In extreme cases, you can ask 3 different databases and they return you different values.
Then I would go down to countries, cities etc...
For example things my tool found:
Thanks to some people that followed my github repo, they actually gave me the idea, to make an mtr only geo database.
I did code it in less than 24 hours, however, the hit rates where to low and my brain did not manage to figure out yet where the fuck up was.
However, today I found the mapping error.
db/mtr.mmdb {'fail': 126502, 'success': 849072, 'percentage': 87.03306976200678}
This database is only 4.2MB in size, only contains geo coordinates, right now.
I will add the usual info in a later build, such as country, continent etc.
Plus I will add a combined build later, with geo.mmdb and mtr.mmdb which first uses latency, then mtr for better accuracy.
Just a thought but would it not be reasonable to write up a daemon for the probe which can be publicly hosted by anyone? this way you could really turn this into a community effort and in turn this could result in more valuable data overall
@pinguin said:
Just a thought but would it not be reasonable to write up a daemon for the probe which can be publicly hosted by anyone? this way you could really turn this into a community effort and in turn this could result in more valuable data overall
Possible, however it would have to be written and field tested.
The current System doesn't support any public probes.
Donations/Sponsors are welcome though.
Practically, meh.
Right now, it uses a preset of probes, not random, means one build is carried out on the same probes.
These probes are reliable and have a high uptime.
There is error correction build in, if a probe goes offline during a build, the system can reschedule these tests to a certain point.
If this would be build and integrated, I still would select probes by hand, means not every probe added would be ever used. Same as before.
It does not make sense to use every probe/server you have.
From my point of view, it does not change much, its more work.
Comments
I already wrote benchmark scripts, to keep track on the builds.
Basically I grab a routing table dump and throw ip's against selected .mmdb's including this one.
Right now the .mmdb has a 74% hit rate, if you take any IP from the internet and look it up.
For the rest, I just don't have any data.
However, no idea about the accuracy, I would need a 3rd party source, that is nearly 100% certain where the IP's originate from, to benchmark that.
Which I haven't found yet.
If you do please let me know cause I have the same problem https://github.com/jsdelivr/globalping/issues/264. No reliable dataset to benchmark against.
And yes please don't try to ping the whole internet using Globalping
Well, to be fair, correcting a city, within a country, is going to be very hard.
However, the biggest problem I think rather is, continents.
There are a lot of entries, that just give you the entire wrong continent.
In extreme cases, you can ask 3 different databases and they return you different values.
Then I would go down to countries, cities etc...
For example things my tool found:
Corrected AS to EU (223.120.188.0, 8.0)
https://db-ip.com/223.120.188.0
https://ping.sx/ping?t=223.120.188.1
No, its not china.
Thanks to some people that followed my github repo, they actually gave me the idea, to make an mtr only geo database.
I did code it in less than 24 hours, however, the hit rates where to low and my brain did not manage to figure out yet where the fuck up was.
However, today I found the mapping error.
db/mtr.mmdb {'fail': 126502, 'success': 849072, 'percentage': 87.03306976200678}
From 64% to 74% now 87% hitrate, not bad.
I put the .mmdb as usual on https://yammdb.serv.app/mtr.mmdb
This database is only 4.2MB in size, only contains geo coordinates, right now.
I will add the usual info in a later build, such as country, continent etc.
Plus I will add a combined build later, with geo.mmdb and mtr.mmdb which first uses latency, then mtr for better accuracy.
As per request, the files are also now available as csv.
https://yammdb.serv.app/geo.csv
https://yammdb.serv.app/mtr.csv
any future updates planned?
The buildserver broke, I did upload the wrong config file, hence no update.
I fixed it, the next fresh build should be up in roughly 22 hours.
However it does not include Paris and Zurich, will run another one for these.
It seems like entire continents are set to the same country and location, so is this correctly set up or am I misunderstanding something?
The Country and the City is set, where the closest server is at.
Including the latency measured from it.
Just a thought but would it not be reasonable to write up a daemon for the probe which can be publicly hosted by anyone? this way you could really turn this into a community effort and in turn this could result in more valuable data overall
Possible, however it would have to be written and field tested.
The current System doesn't support any public probes.
Donations/Sponsors are welcome though.
Practically, meh.
Right now, it uses a preset of probes, not random, means one build is carried out on the same probes.
These probes are reliable and have a high uptime.
There is error correction build in, if a probe goes offline during a build, the system can reschedule these tests to a certain point.
If this would be build and integrated, I still would select probes by hand, means not every probe added would be ever used. Same as before.
It does not make sense to use every probe/server you have.
From my point of view, it does not change much, its more work.