Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


yammdb - just another .mmdb - Page 2
New on LowEndTalk? Please Register and read our Community Rules.

yammdb - just another .mmdb

2»

Comments

  • NeoonNeoon Community Contributor, Veteran
    edited May 19

    @kait said:
    icmp flood the world :) Interesting project hope it goes well, will you do a comparison against maxmind and other ip2location services?

    I already wrote benchmark scripts, to keep track on the builds.
    Basically I grab a routing table dump and throw ip's against selected .mmdb's including this one.

    Right now the .mmdb has a 74% hit rate, if you take any IP from the internet and look it up.
    For the rest, I just don't have any data.

    However, no idea about the accuracy, I would need a 3rd party source, that is nearly 100% certain where the IP's originate from, to benchmark that.

    Which I haven't found yet.

  • jimaekjimaek Member

    Which I haven't found yet.

    If you do please let me know cause I have the same problem https://github.com/jsdelivr/globalping/issues/264. No reliable dataset to benchmark against.

    And yes please don't try to ping the whole internet using Globalping :smiley:

  • NeoonNeoon Community Contributor, Veteran

    @jimaek said:

    Which I haven't found yet.

    If you do please let me know cause I have the same problem https://github.com/jsdelivr/globalping/issues/264. No reliable dataset to benchmark against.

    And yes please don't try to ping the whole internet using Globalping :smiley:

    Well, to be fair, correcting a city, within a country, is going to be very hard.
    However, the biggest problem I think rather is, continents.

    There are a lot of entries, that just give you the entire wrong continent.
    In extreme cases, you can ask 3 different databases and they return you different values.

    Then I would go down to countries, cities etc...
    For example things my tool found:

    Corrected AS to EU (223.120.188.0, 8.0)
    https://db-ip.com/223.120.188.0
    https://ping.sx/ping?t=223.120.188.1

    No, its not china.

  • NeoonNeoon Community Contributor, Veteran

    Thanks to some people that followed my github repo, they actually gave me the idea, to make an mtr only geo database.
    I did code it in less than 24 hours, however, the hit rates where to low and my brain did not manage to figure out yet where the fuck up was.

    However, today I found the mapping error.
    db/mtr.mmdb {'fail': 126502, 'success': 849072, 'percentage': 87.03306976200678}

    From 64% to 74% now 87% hitrate, not bad.
    I put the .mmdb as usual on https://yammdb.serv.app/mtr.mmdb

    This database is only 4.2MB in size, only contains geo coordinates, right now.
    I will add the usual info in a later build, such as country, continent etc.

    Plus I will add a combined build later, with geo.mmdb and mtr.mmdb which first uses latency, then mtr for better accuracy.

    Thanked by 1BasToTheMax
  • NeoonNeoon Community Contributor, Veteran

    As per request, the files are also now available as csv.
    https://yammdb.serv.app/geo.csv
    https://yammdb.serv.app/mtr.csv

    Thanked by 2BasToTheMax tuc
  • mrTommrTom Member

    any future updates planned? :)

  • NeoonNeoon Community Contributor, Veteran
    edited June 19

    @mrTom said:
    any future updates planned? :)

    The buildserver broke, I did upload the wrong config file, hence no update.
    I fixed it, the next fresh build should be up in roughly 22 hours.

    However it does not include Paris and Zurich, will run another one for these.

    Thanked by 1mrTom
  • fartfart Member

    It seems like entire continents are set to the same country and location, so is this correctly set up or am I misunderstanding something?

  • NeoonNeoon Community Contributor, Veteran

    @fart said:
    It seems like entire continents are set to the same country and location, so is this correctly set up or am I misunderstanding something?

    The Country and the City is set, where the closest server is at.
    Including the latency measured from it.

  • Just a thought but would it not be reasonable to write up a daemon for the probe which can be publicly hosted by anyone? this way you could really turn this into a community effort and in turn this could result in more valuable data overall

  • NeoonNeoon Community Contributor, Veteran

    @pinguin said:
    Just a thought but would it not be reasonable to write up a daemon for the probe which can be publicly hosted by anyone? this way you could really turn this into a community effort and in turn this could result in more valuable data overall

    Possible, however it would have to be written and field tested.
    The current System doesn't support any public probes.
    Donations/Sponsors are welcome though.

    Practically, meh.
    Right now, it uses a preset of probes, not random, means one build is carried out on the same probes.

    These probes are reliable and have a high uptime.
    There is error correction build in, if a probe goes offline during a build, the system can reschedule these tests to a certain point.

    If this would be build and integrated, I still would select probes by hand, means not every probe added would be ever used. Same as before.

    It does not make sense to use every probe/server you have.
    From my point of view, it does not change much, its more work.

Sign In or Register to comment.