Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Register

Quick Links


Categories

In this Discussion

› › Residential IP - Cloudflare Protection
New on LowEndTalk? Please Register and read our Community Rules.

Residential IP - Cloudflare Protection

host4cheaphost4cheap Member
in Requests

I'm developing a Telegram bot that executes a Selenium script when I trigger it through Telegram. Unfortunately, it's currently not functioning on my server due to Cloudflare's protection, which blocks traffic originating from the server's IP address.

I've already configured the user-agent header to resemble a valid browser, and the script runs smoothly when executed on a laptop. However, I need to find a hosting solution that offers high availability, allowing me to run the bot without encountering the Cloudflare blockage issue.

Comments

  • emghemgh Member
    edited May 24

    Blocking is normally a scale based on several factors, you might get away without getting a residential IP (expensive)

    First, try using this as the driver: https://github.com/ultrafunkamsterdam/undetected-chromedriver

    Edit/addon: And DONT run headless

    If that’s not enough, keep the driver and combine it with a non-spammy VPN with a free tier, try a few ones (Windscribe for starters)

    If not, detect the Cloudflare block, change IPs, restart driver and try again. This can be done several times if time isn’t a big issue

    Finally, if not, what kind of block?

    If we’re talking captcha, you can get away with an AI-based captcha solver for basically free

    Edit/addon 2: For this, I recommend https://truecaptcha.org/. Very easy to work with their API and $0.33 USD for 1000 captchas is hard to beat..

    Thanked by 1jmaxwell
  • FatGrizzlyFatGrizzly Member, Host Rep

    Try using cloudflare warp+, might help.

    Not sure

    Thanked by 1host4cheap
  • CyberneticTitanCyberneticTitan Member

    Does it need to be Selenium? Cloudscraper works well if you can just work using HTTP requests.

    Thanked by 1emgh
  • emghemgh Member

    @CyberneticTitan said:
    Does it need to be Selenium? Cloudscraper works well if you can just work using HTTP requests.

    Nice find! I didn’t know about this module

    But as I said, it also depends on what he means by blocked too

    If they require a captcha for example, this won’t do him any good

  • jmaxwelljmaxwell Member

    Cloudflare clearance token might work, if the api or whatever you are querying supports it. Or try one of the VPS sold by @Calin as it has the cheapest residential IP and cloudflare friendly. I was playing around you.com api and had the same issue and it worked alright using his VPS.

  • FatGrizzlyFatGrizzly Member, Host Rep

    And chances are, sometimes using privacy pass extension can help

  • host4cheaphost4cheap Member

    @CyberneticTitan said:
    Does it need to be Selenium? Cloudscraper works well if you can just work using HTTP requests.

    Tried that and didnt work. Pasted output here https://pastebin.com/ZAE8Sj9a

  • CyberneticTitanCyberneticTitan Member

    @host4cheap said:

    @CyberneticTitan said:
    Does it need to be Selenium? Cloudscraper works well if you can just work using HTTP requests.

    Tried that and didnt work. Pasted output here https://pastebin.com/ZAE8Sj9a

    Kinda interested in giving this a shot. Can you PM me an address or a similar website that doesn't work?

  • emghemgh Member

    @host4cheap said:

    @CyberneticTitan said:
    Does it need to be Selenium? Cloudscraper works well if you can just work using HTTP requests.

    Tried that and didnt work. Pasted output here https://pastebin.com/ZAE8Sj9a

    Did you try my solutions?

  • CyberneticTitanCyberneticTitan Member
    edited May 24

    After some digging around and testing on several of my machines, residential, VPN, and VPS, the website OP is interested is using firewall rules to block non-residential traffic. Cloudscraper and other Cloudflare anti-bot methods won't work here because it's not blocking bot traffic.

    Aside from running the script on your own home network, you may be interested in buying residential proxies. Cost is usually less than $10/GB.

    Or this scraping platform, which gives you $5 free in credits per month, and also has access to residential proxies: https://apify.com/pricing

    Thanked by 1host4cheap
  • inlandinland Member

    @host4cheap said: I'm developing a Telegram bot that executes a Selenium script when I trigger it through Telegram

    .

    @host4cheap said: I've already configured the user-agent header to resemble a valid browser

    .

    @emgh said: Did you try my solutions?

    He's not even using Selenium in the final script. His pastebin clearly shows python-requests request.

  • ObelousObelous Member

    @emgh said: Edit/addon 2: For this, I recommend https://truecaptcha.org/. Very easy to work with their API and $0.33 USD for 1000 captchas is hard to beat..

    Do they only do image captchas? And I'd probably go with capmonster.cloud or 2captcha instead

  • emghemgh Member

    @Obelous said: 2captcha

    Sure, pay 3x more without any benefit.

  • bgerardbgerard Member

    @emgh said:

    @Obelous said: 2captcha

    Sure, pay 3x more without any benefit.

    It looks like truecaptcha doesn't support recaptcha, hcaptcha, turnstile etc. Is that correct?

  • ObelousObelous Member

    @emgh said: Sure, pay 3x more without any benefit.

    More well-known, and solves more than just text on images (+ capmonster.cloud does these too for the same price as truecaptcha, and faster.)

  • inlandinland Member

    @CyberneticTitan said: After some digging around and testing on several of my machines, residential, VPN, and VPS, the website OP is interested is using firewall rules to block non-residential traffic. Cloudscraper and other Cloudflare anti-bot methods won't work here because it's not blocking bot traffic.

    OP forgot to redact the link from the pastebin. I was able to access the website via Tor after couple of attempts. This is probably just a country-level block.

    Thanked by 1emgh
  • emghemgh Member
    edited May 24

    @bgerard said:

    @emgh said:

    @Obelous said: 2captcha

    Sure, pay 3x more without any benefit.

    It looks like truecaptcha doesn't support recaptcha, hcaptcha, turnstile etc. Is that correct?

    Only image to text, obviously only use it for that.

    @Obelous said: More well-known

    Makes sense if you're choosing a bank, not if you're literally choosing a captcha solver API lol

    If you want to pay 3x for nother because the brand is recognizable, you've came to the wrong forum

    @Obelous said: capmonster.cloud

    Now that's interesting

  • ObelousObelous Member

    @emgh said: not if you're literally choosing a captcha solver API lol

    Except for the fact you recharge into a balance, sure maybe it's not a huge deal when you're depositing a few dollars but I'd still pick something more well-known

  • host4cheaphost4cheap Member

    @FatGrizzly said:
    Try using cloudflare warp+, might help.

    Not sure

    Thank you so much. This works!!!

    Installed warp on the VPS with this and it works
    https://github.com/fscarmen/warp

  • emghemgh Member

    @Obelous said:

    @emgh said: not if you're literally choosing a captcha solver API lol

    Except for the fact you recharge into a balance, sure maybe it's not a huge deal when you're depositing a few dollars but I'd still pick something more well-known

    Yes, paying 3x is better than eventually losing 2 bucks

Sign In or Register to comment.