New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
Gave up on DO a long time ago myself. Deactivated my account last month, and gave em my credits.
As for Spammyhaus they can kiss it as well.
Good job there. Only 3 small listings there. That was fast.
I love the warmth and fuzziness but what did Digital Ocean do wrong here? An entire /20 is SBL'd and Spamhaus showed that they contacted DO over the course of multiple weeks.
Yes every company has really dedicated people, complete bums, "new" guys and others who are out sick. Every company forgets to monitor something sometimes and tickets get flagged incorrectly and lost, etc. But what did DO specifically mess up and how are they fixing it? A /20 is nearly 5000 customers.
Our Trust & Safety team has a workload in front of them that exceeds their ability to resolve quickly. I'm trying to provide them with extra resources from support, but we have our own workload in front of us and right now I just can't spare anyone in our queues. We're working on building out new processes and delegating some of their responsibilities elsewhere (whether it be functionality that reduces unnecessary workload, or support team handling some tasks that they have been handling). In the meantime, there's just no way to halt the incoming workload so we have to handle it as best we can.
There's light at the end of the tunnel, but one has to continue traveling through the tunnel to get to it. We're basically walking through the tunnel while building a car to drive us to the end. No matter which comes first, the end or the car, we're in a better place when both of those things are resolved. Then, hopefully, we never again meet a tunnel that puts us in the same situation as this one.
While you see 5,000 customers, I see 1 ticket. You've gotta break that into a subset of people who send mail, then another subset of people who send mail to providers that query spamhaus for email rejection. That is a far more significant limiter than you might suspect. Now, one ticket is enough. It means there's a problem. However, zero tickets and 0 customers cancelling within that IP range would mean it is not a problem. You and me see a Spamhaus listing and think "This is terrible" but the reality is that a problem in this area is defined by your customers, not in preconceived opinions.
I am able to know how many tickets are opened by customers using that IP range and I am able to know how many customers are using that IP range and then leaving. It's my job to know things like this. This is precisely why I said in my first reply, if you are experiencing a problem related to this listing then please open a ticket and direct me to it. I want to hear how it is impacting you, not how it makes you feel emotionally. Our emotional response to a listing is based on our preconceived opinions. You and I have the same emotional response to it. However, there are other opinions out there, and other emotional responses. If this listing is impacting you I want to know how it is doing so, not how you feel about the existence of the listing itself. I think that's a reasonable ask.
Now, none of that is to say that the problem will not be dealt with, or that we are accepting this listing and have no intention to reverse it. It is simply to say that if you think this is the #1 priority over everything else, help me argue that with data. Because my data may not be saying that it is the absolute first and only priority. Anything escalated above something else means someone else has to wait longer because of it. Deciding on those things is not as easy as some would make it out to be.
Fair enough.
DO allowed 193 compromised hosts to serve up a botnet over the course of multiple weeks. This led to a blacklisting of a large range. This isn't the first /20 blacklisted. When talking about "impact" - how do I know my IP won't be in the next SBL'd /20 ? Customers that send email need deliverability and this is something that DO can't offer.
I confess that I am kind of trolling here as personally I only send about 5-10 emails - all pointless - per month from a Droplet in my control. But I think it's important to raise this issue for broader consumer awareness.
Front up: I'm not a DO customer. And: I came to know about DO mainly because they offer many good tutorials so I often send newbies over there to read and learn. Compliments!
However: Pardon me? As a customer I have to help you convince someone (top mgmt I guess) that not being listed as spammer or otherwise evil is necessary?
No! The way I see it DO is betraying many of their customers with that attitude; I'm talking about those (majority, I guess, of) customers who expect to be able to use their DO VPS to do all the usual and perfectly legal things such as sending emails.
If you (or any provider, for that matter), can not guarantee your customers that, besides very rare exceptions, their sending emails will work, you should either close down business or change your attitude and quickly and drastically so.
After all, you don't advertise "VPS which might or might not work, largely depending on who else is on your node. Come and play lottery with us!".
(Disclaimer: My basic impression of DO was largely positive and I'm certainly in no way against DO. But what I read here is shocking)
Depends on who the customer base are I guess. Microsoft don't give a fuck, doesn't seem to be hurting Azure that much. Maybe DO's customer-base has transitioned from hobbyists to SME's who use other services for email delivery?
Allowed is not the word I would personally use. If we required you to fill out an application for each droplet pending approval, wait times would be longer and no doubt far more people would be rightfully upset. It would break the entire model of servers on demand. The product that we offer is also highly desired by people who have bad intentions, and every time you find a way to identify them beforehand they will change their methods. It's an uphill battle at all times, and it grows increasingly difficult when it exceeds the scale of reasonable human intervention. Not impossible, just difficult. That's when you know you need to change how you've always done things.
You are absolutely free to raise awareness as you need. Notice I'm making no efforts to delegitimize your complaint or ask you to stop speaking about it. I was tagged in the thread and responded, and at this point I'm only continuing to respond when engaged directly. I value honesty and transparency, I always have. If you ask for that from me, it's what you will get. You deserve that, and I require it of myself.
All I'm saying is don't count us out. I always thought it was BS that problems were more difficult at large scale, while the same problems were easy at small scale. It's not BS, it's totally true. When a problem exceeds your ability to just resolve it by hand, how you deal with it in the short term has to be how you deal with it in the long term too, you can't just brute force your way through it.
I can create a blacklist right now and list every IP on it. Is every provider suddenly negligent for my listing? All I'm asking is "How does this impact you directly." If that's not a reasonable ask to you, then we have nothing to discuss.
a) you (DO) are not one of the major spam lists
b) we indeed have nothing to discuss here. Being able to send normal (not massive or spamming) emails as a normal customer must not be discussed but totally normal.
P.S. I'm not attacking you as a person. And I can understand a "collegial" perspective, understanding each others internal problems. But still, being able to send emails should be a normal thing to do - and so should any hosters efforts to make sure that's possible. (As a customer) I'm not paying DO for having a chance to gain insights in, oh how hard their job is and why so, but simply for a - properly working - service.
Agreed. Please let me know if you're having trouble doing so.
Absolutely, if I happen to be one the thousands of customers in that spam listed IP range.
Anyway only amateurs send emails using their droplet IP address. If you are doing something serious that involves sending emails of any kind then you use the correct services for that. Example mailgun, Amazon SES, etc.
With the way spammers are going lately, I'd only send email from a VPS provider that manually checks each order before approval of opening the account, or ones that block SMTP up front and require strong verification for unblock. Anyone else, you're always highly at the risk of being impacted by the choices of another customer when it comes to something like IP reputation.
I run my own mail servers since many years and I know of many others who do, some of them quite large companies.
I'd agree with you, though, wrt mass mailings.
@jbiloh Just noticed that some of my VPSes on the CC network had their IPs removed from the SBL! (23.95.xx.xx)
Mad love
Sincerely appreciate the efforts CC has gone through to clean up their space. I think they also learned the lesson of how difficult things are to deal with at scale. It got out of hand so quickly that it took them a long time to clean up. It's funny because for all the trash I've talked about Biloh in the past, he's actually a pretty damn cool guy. At one point I thought as mod/admin here I'd be able to stand between CC and the members here to prevent CC from doing problematic things without transparency, not how it turned out at all. Actually grew to love the team.
I guess that the implicit assumption that the OP and others are making is that the listing of an IP on Spamhaus is generally sufficient for negative impact on email deliverability. If this is the case (but perhaps it's not), then it's probably a tiresome exercise to open a ticket about actual negative impact on email deliverability (since it was predictable from the listing of the IP on Spamhaus to begin with).
Yeah, I mean that's my reaction to such a thing personally as well. That said, I can't recall the last time I sent an email through an IP listed on SpamHaus. Does anyone still subscribe to them to block emails? My feeling that they do is mostly irrelevant, for me it would be an assumption based on 2-3 year old data. For all I know, almost no one uses them anymore. If I was going to approach someone who didn't have that past experience with their listings, it would be better for me to point and say "This customer right here cannot send email because of this, and they told me so."
If I'm going to ask someone else to act, I should have solid data and not assumptions.
I think they call that Stockholm syndrome.
Just kidding :P from what I can tell Biloh et al. seem to put up with abuse from many LETers without ever losing their heads or shutting the place down. So props on that front.
"Let it be", with no further acknowledgement doesn't really mean like they're working WITH it, for what it's worth. I mean, look at the shitty unhandled CF stuff and Vanilla 0.01Beta issues we live with daily.
I definitely understand your wanting to have data and not mere assumptions!
I guess that it's a further sign of Spamhaus's hegemony that people experience a panic reaction if they see one of their IPs on a Spamhaus listing, when in reality a mere listing of an IP on Spamhaus may not be the determining factor for negative impact on email deliverability (but it nevertheless seems to be in Spamhaus's interest that people believe that it is the determining factor).
I've used SH for the better part of two decades, and I've had very little trouble with false positives. My customer base is mostly US-based businesses, so YMMV, of course.
My reading is that sending emails on DO is supposed to work but it currently has a partial outage on a certain IP range. Any big internet operation will have 1000s of running services, some of which will be out at any moment. They are always trying to fix stuff, but also have to prioritize effort towards the issues that are affecting customers the most.
I have a running droplet right now but I don't send any email from it. I don't know if my droplet in the affected IP range or not. I can confidently say I'm not personally affected. If I were affected I'd probably open a ticket per the advice above, and hopefully get another address assigned or something like that. That's a slightly inconvenient workaround but if it lets people keep their stuff running, it's a reasonable stopgap.
I don't understand why people are going berserk over this. DO is generally more solid than the average LET host, but stuff breaks just like anywhere else, and there's usually workarounds.
Marteennn @mpkossen how are youuuu?
Who needs sleep. I'll just do this instead. https://img.jarcloud.pw/1iLhalgZ.png
Very commendable. But: Wouldn't it be wiser for DO to add a couple of more people than having you sleep less and work even more?
Thanks @Jarland, we have enjoyed working with you too!
Do apply or send anyone you know our way
https://www.digitalocean.com/company/careers/#trust-and-safety-specialist
While hiring, the work doesn't stop. Takes time to get people into the role.
This would make an excellent LowEndTalk shirt.