New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
A simple solution is to devolve control to the user, I assume through cpanel would be easiest. Leave it defaulting to ON but allow it to be turned down/tweaked or OFF. If need be, deal with customers who turn it off but are consuming a lot of resources.
And yes, I'm sure Cloudflare has a similar problem set to you, but the answer is not to create an environment where countless custom written softwares like BitNinja provide arbitrary responses to requests leaving the end users to figure out this extra layer of abstraction a million miles away from any standard, like 200 OK and a captcha instead of the actual resource requested. At least make that a 403 or 503.
I think this is something we can implement. Thank you for the idea! I have created a feature request ticket about it.
It is also easily doable. I'm not sure if it will make any difference but it is doable. What effect do you expect if we return 403 ? But anyway.. we can do it and so at least we return with the right response code. I have created a bug report about it, so it will be fixed within 2 weeks.
Firstly, it indicates to the client something is not quite right. Malicious scripts unless targeting something specific are just going to move along to the next target, but 'legitimate' requests aren't going to mistakenly think the request was served successfully... and that's the point, it wasn't.
That's fair enough. Thank you for the suggestion!
Do you have any other consideration regarding greylisting?
A suggestion which would probably take all the salt and bitterness away.
How about you allow ip block owners and operators (not single ip users) to update you so that you don't generate any notices at all.
It is fine if you want to block them for your clients, but we don't really want to hear about it from you because your legitimacy is 0 from our perspective?
We just received 13 bitninja reports regarding googlebot to ip's we own.
Forwarded them to /dev/null
Stupid shit man, when will you guys learn to stop doing false positives?
And let me guess, they all had 'this is malware or a botnet and this is what you should do' along with some marketing in the email?
Botnet, not sure how googlebot user agent is a botnet when it's coming from their ip's just crawling our client forum.
Marketing was referring to their $30/month to secure our "servers", def not going to pay to remove a ip for a false abuse report.
Probably just user agent spoofing too. A lot of marketers will look at competition for this lazy-cloaking. It's hardly malicious.
Now.. this is rich. Haha
Borderline extortion in my opinion.
Just like those shit mail blacklist sites that require you to pay to be removed from their system.
Spamhaus 2.0
Wait a minute. So we have sent you a report where you see request from google bots? This means someone or some script has made request from your server using your IP to our honeypots pretending as if they were google bots.
What is the evidence of an infection if not this?
Using google bot as an agent string is a regular technique to evade user agent filtering.
You can find more details here about the report: https://doc.bitninja.io/investigations.html
But you totally misunderstood it I think. So the entries are not logs from your server. Of course we can't access them. They are logs from our honeypots. So the question is if you are not the owner of google (and I think this is not the case) Why are your servers sending request to our honeypots using the google bot agent identifier?
I'm not that convinced about it. Many e-mail crawlers, forum spammers and others do this. And believe me for 1 legit crawler there are 1000 malicious on the Internet. But anyway that's not the reason to be put on the greylist, there must be other reasons too.. This is just the top of the iceberg. If @Foul sends me the IP I can tell more about it.
I think at this stage it is safe to say that bitninja does not even know how to manage or control bitninja.
I guess I will leave it at that, all 3 DC's I use auto delete or ignore their reports now, it took all of 10 minutes to convince them as to why.
It is annoying though as they have caused financial damage all over the place before people wised up to the nonsense.
I think your rules are a lot more flaky than you're making them out to be on this thread. e.g. 1) Simply visiting your captcha-200 page several times spawns your marketing email to the source IP abuse contact, 2) Spoofing a user-agent spawns your marketing email.
Why even bother with the poor ruleset you have, just spam all the abuse@ addresses, at least that way we don't have to manually deal with Bitninja either as an IP owner, website visitor or shared hosting user. I'm glad @AnthonySmith's team decided to take the course of action they do, it seems the whole scenario creates more problems than it solves.
You can only visit that Captcha page if previously you did some harm against the server. Like wp brute force, DoS attack, port scan, etc. That Captcha page won't appear by mistake.
The same. A spoofed user agent won't put you IP to greylist. This is just the events we logged after greylisting the IP. Although I still think spoofing the user agent is malicious by itself - and we still don't know what was the request... registration to a forum? user enumeration? @Foul forgot to mention it for some reason. Are you really not curious about who has made that request?
If I was responsible for an IP range, I would like to know about a request with a forged user agent..
We also operate 40+ shared hosting servers and provide hosting for 6500+ users. This system runs on a completely different infrastructure from BitNinja, and at the beginning we received report mails from ourselves too. Traced the malwares, cleaned the infected wp-s, joomlas, etc. and made steps to prevent further infections, and no more reports..
But whatever.. if anyone wants to opt out just drop us a mail, and we stop the reports, just like we ignore-listed Anthony when he asked in June.
Fortunately there are many providers who takes our reports seriously including aws, microsoft, softlayer, digitalocean and many others.
How about visiting a shared hosting package which I use, from a VPS I use, doing a request every couple of minutes, with my VPS provider receiving one of your emails (after my user agent not following any of your links like you'd mentioned before). The description you give of its behaviour seems to be different from real world experience. (btw just to make it abundantly clear, requests to my own website are not malicious, and consider that people might test different user-agents for a huge number of reasons, not just the ones you think of. E.G. serving mobile requests or cloaking content from search engines you don't want them to see)
Bitninja are facing more accusations and with this being LET. Trolling will resume shortly
Wont be for long fortunately
OK, I haven't read all the thread, however, I like to receive reports from most sources. Even as they are they give some useful info, if someone uses a VPN then can still get useful information about his infected home computer, for instance.
That being said, starting with uceprotect et all and ending with bitninja, any list which requests money or trials for "removal" I hold in contempt, maybe some people make more money from this than through a donation mechanism or a legitimate product to be sold, but I do not think that is enough a justification.
Twitter bombing big DC's with #dropbitninja anyone?
After reading 3 pages of posts, I feel that what bitninja does is similiar to what I have been doing since 2009. Collect suspected malware-infected computers' IPs (from my SMTP servers, 1 fake open relay & 1 greylisting server), and send notification mail to the owners of IP blocks daily. Bitninja seems to mainly use HTTP honeypots, but they also try to notify the victims in a timely manner. I think this is good for the internet.
Then why everyone seem to object to what they do? Perhaps it's because many consider their notifications as "false-positive," so think that they are useless and annoying. But is it really so? Take this one for example:
http://xx.169.56.203:80/1phpmyadmin/
I don't think this comes from a web crawler. What web crawlers should do is get the document root, and follow links within, recursively. But I don't believe there will be a link to "/1phpmyadmin/." This looks like a probing, merely testing ways to compromise another system. If you are a provider, you don't want this activity (hacking, trying to gain unauthorized access to networked systems) happen in your network, right? In the case of VPN, it's the same. I am not saying your customers are doing something bad. Perhaps your customers' computers have been compromised, and hackers use victims' VPN to probe the Internet?
It's hard to find systems being compromised oneself. Bitninja's notification is an external hint to help us heed abnormal system behaviors. But with so many objections, maybe they should try to make their message clearer to avoid misunderstanding?
Some people already offering services to suspend any VPS/Dedicated with any provider (of course they will try but no %100 guarantee).
That's the main reason why providers can't rely on "never" known services which they think detects malware/hacks and etc.
@bitninja_george what's name of your hosting company which you're running?
Can I run web-proxy on your hosting and try to browse sites/forums with fake useragent like googlebot?
Can I expect that your dedicated server will be suspend?
@chihcherng you wake up one day and said "Hey I should detect malware-infected computers and send abuse email". Are you hosting provider too ?
Probing attempts to compromise a server are one thing, impoilite crawling/brute-force another, and you blocking the requests is another, but crap about "i think you are user agent spoofing and I think it is is bad, Im going to email your IP space owner and say that you're sending malicious requests and attacking web servers" is something else entirely.
Could you show us an example of your abuse report?
never ever had a complaint from them. just you and maybe 2-3 other "protection services" .
If you do simple requests, no matter what user agent you use, your IP won't be greylisted. If you try something inappropriate requests like an SQL injection, or too many parallel connections or protscan the server etc. then your IP will be greylisted. When your IP is greylisted the bitninja protected server will redirect any inbound traffic from port 80 to the servers local captcha module. Than Captcha module will server the captcha page. So if you do the browsing with an interactive browser, you can see something is not good. After 15 requests your IP land on the servers blacklist. Every request generate an incident that we save in our data center. We only send an email when you reach 80 incidents from 3 different servers.
So to get that report you have to attack 3 different bitninja servers 6 times.
I'm not saying it is always a sign for something phishy, but after a series of real attacks against the honeypots seeing an agent advertising google bot is more than suspicious.
But let's see a real example. Can you send an IP from your range? Well I picked a random IP from our database:
The initial cause of greylisting the IP was this incidnet:
Url: [###lson.biz/wp-includes/js/crop/session.php] File: .../js/crop/session.php Post data: [Array ( [ycsu] => CE1fVggcVB9YSx8NTBEXWl9ZFwZeVksVSEEXMzAeFRsyUTc4Jyw7EzgmChoRW15VJR93X1EYW1FDEAA4BlgSRl4AFVhAFUEQDgMBERccBh4KHG4YBwQEFgAAXQwBQ0BSCRVDQBdLPlsIFw0TQEApA24nP1I4VxEgQDsxXgRgViIBMEoiOSFJXUc0DEcbEFIWGlRRWU9GBxlAGB1ZAhwLBUYAETMbBhsDTxkEXgoYBE5IHFQdWlNYFgEeS1E8NjokAlwgHRImTjcUUj1oG0M5AjBSXUojTkMAIQxRRwhMFRFTSBxNUl8GDAcJEhYEEklbM0cMBgIRRh9LEBwETVVdSltJU1VdGBIHNEoWGk1WMiNcUD5SVR1ZJEZYPlVlEitSNBlCQ0AaPwpaExdKSxpNW1pJTRhPDEcdShoVXx1AEwkBFgQzSAcSRg9cDB9CUV4ASV1ZVAxHBSUPBF81Oy49HzBQNls2O3QaICIBSA00A2JdR1VFOB4pEjc2CVVaBjgTFVwXS0waVkFdVAwBRxFdFh1IG1dTWTURFRBCCxpDTRwDUhQAX0BKU00pdiQsAj0yJAQmPB5rOFQXYQRGezgeL0o/FTxOZy00CVAbORUSRwJRSBpVXB5KShkHEQYsBQ4AAxEZCFoZFEgFQA0AV18WSRtZSQ0wFy06RTRcBhRUCz5WQRAnE0MENQALDxkvGzgJPQF0VCY5WFNXDCVKV0dIQUkcSh1HX1EZBgMPQBQJHgAdBQ0dCy5BDAUTQRJeQwVMWB9PUFJJDTwEbhlKBU1LOTAhPi84PUY6KRUsFTgqPSo4GjAoEhdROlwwUyciLUJWGhI8FhsEUFQDFUFQSVENBUBDGQwUDBsCAxwfWBciDhdOHhwDCgEOTk4XX0ccUFEXLzAbR0gkABISBjxZVAAOAh0FHSROHxgW ASJcZQYpECoNC0pGDTMWG1cUDVgBVUIeSUcRXQsdC0oVAzAFHgMOAQ4AShZHFENVS15TXVZWV1cGHRcoBUkROhcRZSYxI04u(...) ) ]This is an encoded message. If you decode the ycsu it contains 160 byte of spam text and about 20 e-mail addresses. It is clearly nit legit request I think :-)
Sorry but cloudflare doesn't allow me to paste the original content :-) do I altered it here and there so it accepts it now.
Then we greylisted the IP and have captured the following incidents:
2016-11-15 13:34:30 | Url: [###.ni###-y.###.il/wordpress/wp-admin/setup-config.php] 2016-11-15 13:34:30 | Url: [###.ni###-y.###.il/wordpress/wp-admin/setup-config.php] 2016-11-15 13:34:29 | Url: [###.ni###-y.###.il/wordpress/wp-admin/setup-config.php] 2016-11-15 13:34:29 | Url: [###.ni###-y.###.il/wordpress/wp-admin/setup-config.php"] 2016-11-14 23:49:26 | Url: [###stattglatze.com/haartransplantation-vorher-nachher] 2016-11-14 23:06:46 | Url: [###eldivilayout2.biz/wp-includes/js/tinymce/plugins/image/slide-menues.php] 2016-11-10 06:45:41 | Url: [pr###ws.com/wp-content/plugins/carp-wp/inc/ajax86.php] 2016-11-06 06:31:05 | Url: [sh###se.com/wp-admin/edit-form-comment.php] and a lot more.Here is the details of one of them:
Url: [####ivilayout2.biz/wp-includes/js/tinymce/plugins/image/slide-menues.php] Agent: [Mozilla/5.0 (X11; U; Linux i686; en-US) U2/1.0.0 UCBrowser/9.3.1.344] Post data: [Array ( [a] => Php [ajax] => true [p1] => echo serialize(array(..... CF blocked ) ]and this is just a random IP.