Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Register

Quick Links


Categories

In this Discussion

› › how can i download a large amount quickly and stably from the tor network?
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

how can i download a large amount quickly and stably from the tor network?

hyperblasthyperblast Member
in Help

i would like to download a large amount of data (~27 GB) from the tor network. it is very tedious and above all slow using the standard method. are there any ways of speeding this up?

Comments

  • tentortentor Member, Patron Provider
    edited February 19

    You can try downloading in multiple chunks simultaneously by using HTTP header Range. Probably effect will be even more noticeable if you will be downloading from different devices.

    Thanked by 1gwnd1989
  • BruhGamer12BruhGamer12 Member

    You can also use your own guard node to speed up as much of the connection that's within your control.

  • qbit15qbit15 Member

    @BruhGamer12 said:
    You can also use your own guard node to speed up as much of the connection that's within your control.

    Is that possible? I don't think you control that

  • rm_rm_ IPv6 Advocate, Veteran
    edited February 19

    @qbit15 said: Is that possible? I don't think you control that

    You can specify the entry node to use by its fingerprint via EntryNodes in torrc.

    Another option is to limit it down to only European countries with fast connection and good peerings, not venture out into the 3rd world or across the ocean:

    EntryNodes {at},{be},{bg},{hr},{cy},{cz},{dk},{ee},{fi},{fr},{de},{gr},{hu},{ie},{it},{lv},{lt},{lu},{mt},{nl},{pl},{pt},{ro},{sk},{si},{es},{se},{gb}
MiddleNodes {at},{be},{bg},{hr},{cy},{cz},{dk},{ee},{fi},{fr},{de},{gr},{hu},{ie},{it},{lv},{lt},{lu},{mt},{nl},{pl},{pt},{ro},{sk},{si},{es},{se},{gb}
ExitNodes  {at},{be},{bg},{hr},{cy},{cz},{dk},{ee},{fi},{fr},{de},{gr},{hu},{ie},{it},{lv},{lt},{lu},{mt},{nl},{pl},{pt},{ro},{sk},{si},{es},{se},{gb}

    Or if you are in Asia, then adjust this to only use Asian nodes:

    {af},{am},{az},{bh},{bd},{bt},{bn},{kh},{cn},{cy},{ge},{in},{id},{ir},{iq},{il},{jp},{jo},{kz},{kw},{kg},{la},{lb},{my},{mv},{mn},{mm},{np},{kp},{om},{pk},{ps},{ph},{qa},{sa},{sg},{kr},{lk},{sy},{tw},{tj},{th},{tr},{tm},{ae},{uz},{vn},{ye}

    Probably want to remove most of these to only leave countries with good Internet and nice peerings, like Japan or SG (and maybe that's it, lol).

    ChatGPT is helpful in building these lists, can tell it which set of countries you want and specify the exact format (curly braces, commas and lowercase).

    Thanked by 2BruhGamer12 0xC7
  • farsighterfarsighter Member

    @rm_ said:

    @qbit15 said: Is that possible? I don't think you control that

    You can specify the entry node to use by its fingerprint via EntryNodes in torrc.

    but how much will that help given that subsequent relay nodes and exit node can be from any country?

  • rm_rm_ IPv6 Advocate, Veteran
    edited February 19

    @farsighter said: but how much will that help given that subsequent relay nodes and exit node can be from any country?

    It will help about 33%, since each Tor circuit consists of 3 nodes, and if you know the 1st node is known-good and very fast, you could say you improve your connections by 1/3 on average.

    Secondly, you can combine that with the country limiting above. Also I forgot to mention that it may not obey "MiddleNodes" of those (see man torrc), and it may be required to instead use ExcludeNodes to list all the countries you do not want.

    Thanked by 1raindog308
  • BruhGamer12BruhGamer12 Member

    I just logged on to check my Tor relay and someone was uploading something at 80 megabytes per second... so you can def upload very fast if you have the proper circuit. Here is my fingerprint 114D40B14F80EA24CB9FB0DA23AA4A9798F6BF53 hosting from @terrahost :)

  • hyperblasthyperblast Member

    Thank you.
    I'm still not quite sure how to implement this ;)

  • BruhGamer12BruhGamer12 Member

    @hyperblast said:
    Thank you.
    I'm still not quite sure how to implement this ;)

    On linux you can go into the torrc and you should copy what @rm_ did essentially. On windows and mac I do not know. So if you wanted to put mine as an entry node in Norway you would add

    EntryNodes 114D40B14F80EA24CB9FB0DA23AA4A9798F6BF53
    StrictNodes 1

    to the torrc.

  • shruubshruub Member

    @hyperblast said:
    Thank you.
    I'm still not quite sure how to implement this ;)

    It depends on how you use tor. I'm not sure if you can modify the browser's config, but for the tor daemon (just the package name tor), just go to /etc/tor and edit as said previously.

    Also, I suppose you mean a hidden service, since for a clearnet website, you might as well pay like 5 bucks for a good vpn or just use a shitty free one.

  • hyperblasthyperblast Member
    edited February 19

    i use tor browser under windows. torrc i can edit. but with this config no better speed. still around 50 KB/sec.

    # This file was generated by Tor; if you edit it, comments will not be preserved
# The old torrc file was renamed to torrc.orig.1, and Tor will ignore it

ClientOnionAuthDir C:\Users\X\Web\Tor8\Browser\TorBrowser\Data\Tor\onion-auth
DataDirectory C:\Users\X\Web\Tor8\Browser\TorBrowser\Data\Tor
EntryNodes {at},{be},{cz},{de},{nl},{se}
ExitNodes {at},{be},{cz},{de},{nl},{se}
GeoIPFile C:\Users\X\Web\Tor8\Browser\TorBrowser\Data\Tor\geoip
GeoIPv6File C:\Users\X\Web\Tor8\Browser\TorBrowser\Data\Tor\geoip6
MiddleNodes {at},{be},{cz},{de},{nl},{se}

    @shruub yep. downloading the stuff from a hidden service.

    Thanked by 1shruub
  • BruhGamer12BruhGamer12 Member

    @hyperblast said:
    i use tor browser under windows. torrc i can edit. but with this config no better speed. still around 50 KB/sec.

    > # This file was generated by Tor; if you edit it, comments will not be preserved
> # The old torrc file was renamed to torrc.orig.1, and Tor will ignore it
> 
> ClientOnionAuthDir C:\Users\X\Web\Tor8\Browser\TorBrowser\Data\Tor\onion-auth
> DataDirectory C:\Users\X\Web\Tor8\Browser\TorBrowser\Data\Tor
> EntryNodes {at},{be},{cz},{de},{nl},{se}
> ExitNodes {at},{be},{cz},{de},{nl},{se}
> GeoIPFile C:\Users\X\Web\Tor8\Browser\TorBrowser\Data\Tor\geoip
> GeoIPv6File C:\Users\X\Web\Tor8\Browser\TorBrowser\Data\Tor\geoip6
> MiddleNodes {at},{be},{cz},{de},{nl},{se}
>

    @shruub yep. downloading the stuff from a hidden service.

    A hidden service uses 6 relays so maybe not much you can do there - whoever uploaded whatever ur downloading should have uploaded it in smaller chunks. I recommend using a cli on a linux terminal to download it tho instead of the windows tor browser. sometimes it will rando fail.

  • hyperblasthyperblast Member

    @BruhGamer12 said:

    @hyperblast said:
    i use tor browser under windows. torrc i can edit. but with this config no better speed. still around 50 KB/sec.

    > > # This file was generated by Tor; if you edit it, comments will not be preserved
> > # The old torrc file was renamed to torrc.orig.1, and Tor will ignore it
> > 
> > ClientOnionAuthDir C:\Users\X\Web\Tor8\Browser\TorBrowser\Data\Tor\onion-auth
> > DataDirectory C:\Users\X\Web\Tor8\Browser\TorBrowser\Data\Tor
> > EntryNodes {at},{be},{cz},{de},{nl},{se}
> > ExitNodes {at},{be},{cz},{de},{nl},{se}
> > GeoIPFile C:\Users\X\Web\Tor8\Browser\TorBrowser\Data\Tor\geoip
> > GeoIPv6File C:\Users\X\Web\Tor8\Browser\TorBrowser\Data\Tor\geoip6
> > MiddleNodes {at},{be},{cz},{de},{nl},{se}
> >

    @shruub yep. downloading the stuff from a hidden service.

    A hidden service uses 6 relays so maybe not much you can do there - whoever uploaded whatever ur downloading should have uploaded it in smaller chunks. I recommend using a cli on a linux terminal to download it tho instead of the windows tor browser. sometimes it will rando fail.

    @BruhGamer12

    just to be clear, the uploader must also go this route, i.e. upload over 6 relays as soon as the data is in a hidden service?

  • farsighterfarsighter Member
    edited February 19

    @rm_ said:

    @farsighter said: but how much will that help given that subsequent relay nodes and exit node can be from any country?

    It will help about 33%, since each Tor circuit consists of 3 nodes, and if you know the 1st node is known-good and very fast, you could say you improve your connections by 1/3 on average.

    but since final speed is determined by the slowest node, a fast entry node which doesn't stand by itself can't gives any reliable guarantee for anything, let alone promising 33% improvement (although I can agree that knowing that one node is fast is generally better than not knowing anything about any node).

    Thanked by 1tentor
  • BruhGamer12BruhGamer12 Member

    @hyperblast said: just to be clear, the uploader must also go this route, i.e. upload over 6 relays as soon as the data is in a hidden service?

    Well the hidden service can adjust its config where its only 4 relays(facebook, duckduckgo, etc) but this only hides the user and not the server. If the server is trying to hide itself it will be six bidirectionally yes.

    Thanked by 1yoursunny
  • emg88emg88 Member

    @hyperblast Try using a obsf4 bridge

    Thanked by 1hyperblast
  • Carlin0Carlin0 Member

    Tor and quickly don't get along :D

    Thanked by 3yoursunny ehab niranjan
  • hyperblasthyperblast Member

    it seems that the uploader had better speed when uploading. 500MB in 20 seconds. how does that work? or can the server-owner upload to his server without going through tor/hidden-mix?

    .full.part0151.rar                          08-May-2023 23:12           524288000
.full.part0152.rar                          08-May-2023 23:13           524288000
.full.part0153.rar                          08-May-2023 23:13           524288000
.full.part0154.rar                          08-May-2023 23:13           524288000
.full.part0155.rar                          08-May-2023 23:14           524288000
.full.part0156.rar                          08-May-2023 23:14           524288000
.full.part0157.rar                          08-May-2023 23:14           524288000
.full.part0158.rar                          08-May-2023 23:15           524288000
.full.part0159.rar                          08-May-2023 23:16           524288000
.full.part0160.rar                          08-May-2023 23:17           524288000
  • SodaWithoutSparklesSodaWithoutSparkles Member

    @hyperblast said:
    it seems that the uploader had better speed when uploading. 500MB in 20 seconds. how does that work? or can the server-owner upload to his server without going through tor/hidden-mix?

    It could be that he has physical access to the server, or they are separated, tar'ed, then uploaded to create the illustration.

    Thanked by 1hyperblast
  • HotmarerHotmarer Member
    edited February 20

    You can use something like that https://github.com/sn0b4ll/aria2-onion-downloader and try to download every part simultaneously.

    This will be a better solution than using a regular Tor Browser. The tor network or just hidden service sometimes interrupts the connection, so when downloading using curl or wget, you need to add the appropriate parameters.

    Thanked by 2hyperblast 0xC7
  • hyperblasthyperblast Member
    edited February 20

    obsf4 bridge works fine so far. got stable 5mbit. already 30gb leeched.

Sign In or Register to comment.