New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
This is an out of the blue attack against the network. It's being mitigated, but no, there's no SLA for this kind of thing.
Buyvm @fran has been a member and provider here for many years I assure you this isn't a common occurance.... and they are a good host
Chip
Looks like they're having a DDoS attack. Today i got unstable network, like down every few minutes.
Sure, you can ticket for an SLA and I'll give you a week worth of credit if that helps you out.
full blown extorsion attacks aren't something that happen every day. Our last one was around 1 1/2 years ago when there was someone going around smacking a bunch of LET webhosts. We even gave some hosts free DDOS mitigation on Voxility to carry things.
Path.net has been pretty great getting us onboarded during this. There's been a few bumps as I've been having to make quick router changes to our policy-statements. They contacted their upstreams to push an AS-SET update right away so we could get going.
While we have Cloudflare Magic Transit, they don't do BGP yet (even though I insisted it be a top priority during our initial trial period) which is a big problem in cases like this. I would've easily just moved everything to MT and gone back to doing tickets.
We were already planning network upgrades (bigger ports, etc), we even have a call with Hurricane next week. This will likely speed that along in hopes of giving us room to tank this kinda crap.
I do apologize. Its been a hell of an evening, all for ..$900? PS5 money.
Francisco
or a year worth of acne treatment
Would you say they came in... clutch?
I now believe that the comedy store put a hit on you for shit that fresh.
Francisco
Wooohooo, @Francisco couldn't easily sail over an extortion DDOS ... so his hosting must be bad, cry, cry.
@OP
I've yet to see a provider with 100% uptime and who nevar fails under no circumstances (let alone in a sub $100/month price bracket). Plus, if I were pressed to name 3 providers here who seem to have happy clients only pretty much Francisco would certainly come to mind.
So, just let Francisco work and take care of the situation.
(As for those hackers: may they rather sooner than later get their injection. Caliber 9mm)
Yep, I hear you, @Francisco response to this situation, and kudos from peeps, has given me more confidence, not less.
I think they're still getting hit, uptime robot and hetrixtools just notified me of downtimes.
Aye round 2 it looks like.
Smells like deadpooing.
No round 2, this is Cogent.
Yep, LUX is down, even from within Stallion.
[EDIT] (It was down for 4 minutes and 38 seconds).
Sounds like someone has his botnet on google cloud.
Yeah... Mine's been up and down all day, both IPv4 and IPv6. Hopefully it stabilises soon.
BuyVM is Frantastic, NEVER a bad idea.
Everything is OK now.
I am sure @Francisco will have this sorted out soon. Regardless...If you are currently with Fran's Shared Hosting and have a crucial website, I can temporarily set you up for free (provided your website passes the "dont be a dick" test) on a Direct Admin reseller at El Cheapo until the situation is resolved. It is idling away, anyway.
Just let me know.
BuyVM is so expensive that I would have to sell my kidney if I want a server from BuyVM.
On the other hand, BuyVM has charged enough such that they can afford ten kidneys.
Did you ever wonder what are the slabs made of?
A crucial website ought to have a primary server on NVMe, a secondary standby on SSD, a third replica on spinning rust, a fourth backup on VHS tape, a system to automatically perform failover, a cat to look over this system, a robot to feed the cat, and an engineer to repair the robot.
However, the real bottleneck is how fast your DNS changes can propagate to the readers.
Setting low TTL could improve recovery speed, but then you would worsen performance in normal times due to more frequent DNS lookups.
Sue Fran.
What country are you in? Venezuela?
Anyway, a small update.
We've integrated Path pretty well into the network. We're very likely (almost guaranteed) to keep them as a fallback provider for cases like this (or if someone needs protection that CF doesn't want to provide).
Things are stable and I've fleshed out our internal BGP communities to allow for easier on/offboarding to Path as needed.
Path's been fantastic, even today on a holiday weekend. They've helped get the last few GRE's together and did some debugging for us. They did some code updates on their end to handle an issue some users were reporting in our channel. Top notch.
We don't know what our future with CF entails. As of right now we've been told BGP is a March/April thing, but that's likely to slip with the pandemic, other internal priorities, etc. It seems we're one of the only customers requesting on-the-fly BGP controls outside of their API.
I'll write a mass email to send in the next day or so, my minds fried from all of the LUX upgrades, slabs, this, openvz EOL'ing, & people freaking about cPanel pricing.
I apologize that this happened. I guess someone just really wants to get a PS5.
Francisco
If you think BuyVM is expensive, you have no idea what expensive is.
Relying on DNS changes is silly.
RR DNS is understood by all recent browsers...no change needed if one of your servers goes offline. Will be invisible to end users.
And of course there are many HA mechanisms that don't involve fiddling with DNS.
Caching my friend. DNS caches, all the way to the authoritative server and back, including your browser.
And even if it didn't, "worsen performance" is a bit of a stretch. Visitors are unlikely to notice the milliseconds a DNS query takes.
Unless his kidney is so fucked up that it ain't worth shit.
This is good to know.
My last time testing this was 2007, when I indeed had a crucial website (for freshman to apply for university housing). IE6 stops loading the site once I turn off IIS on the primary.
I never tried again after that.
Cloudflare can do that of course, but only if I pay for Argo…
Yes I know about DNS caching. However, the cache is only valid with TTL. If I set 300 seconds TTL, recovery is within 300 seconds, but client must redo lookup every 300 seconds.
We use Path as well. Totally agree they are top-notch and extremely responsive. They for sure go above and beyond for their customers.
I also want a robot to feed my non-existent cat at this point :O
Agreed.
... but unfortunately necessary (as in "often pretty much the only mechanism available").
I'd have to look at the code but my current understanding is that browsers use the OS gethostbyname (or similar)service.
Yes and no. Looking superficially, yes, but at the end of the day more often than not it's just shifting the problem to other parties like CDNs.
At some point the information "server is offline" and "host now reachable via xyz" must be sent/obtained. Yes, probably a large (e.g. CDN) provider has the necessary infrastructure but still it's not a simple nor a quick operation unless one has the resources of Google, fb, CF - and even they can and do fail occasionally. Also note that one pays not only in terms of money but also in terms of dependency and loss of control.
Not really. DNS lookup still makes up a considerable chunk of click to time of page painted.
And there are other issues to be taken care of too, for example keeping two systems synchronized incl. session data state tables, etc.
But be that as it may, for most users switching host records (preferably with a low TTL) is probably good enough.
I was thinking of other HA mechanisms like floating IPs/heartbeats.
Modern browsers understood RR DNS in the sense that they do this:
So RR DNS is a valid HA strategy. Of course, there is a delay if one IP stops responding and the browser has to switch IPs.
My overall point was that trying to make a single system completely available is folly unless we're talking about specialized hardware (IBM mainframe, HP Nonstop, etc., which we're not), and even there you've still got so many other components from network to storage to other things that it's still folly. It's better to have multiple systems where failure is tolerated.
OP is focusing too much on making one system as HA as possible. Better to assume failures will happen and plan on how to handle them automatically.
I know and you'd be right in another discussion.
Careful there. The basis is some unix OSs returning or filling in a list of "IP addresses" (actually 'hostent' structures) while some return a single one and the more modern getaddrinfo (or similar) always return a list - but that is no statement on how the requester/application/server deals with those lists nor are those lists to be understood as being DNS RR.
Well, that touches RASS (reliability, availability, safety, security) and is way more complicated than what befit LET as a discussion, so I'll keep it short. Being "reachable" from the internet is just one aspect of HA which typically and mostly is concerned about other factors such as node failure. A typical example would be a switch hardware failure and the related problems such as e.g. state tables mirroring. Another nice example (and matching interest on LET better) is a web server being reachable (by whatever means) vs. the web service being redundant (e.g. session DB mirrored).
But you are right and we should largely limit ourselves here to look at the "simple" side of HA like having secondary (or even tertiary) fallbacks etc. In other words, our goal here usually is to keep downtime to less than 5 to 15 minutes rather than to microseconds vs milliseconds.
Yes and one should anyway always assume failures. In fact the internet itself confirms your POV by actually being an implementation of that very assumption.