New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
BuyVM is down ?
jholycloud
Member
in Providers
Wondering if buyvm currently is down. @Francisco
Could not open the website buyvm.net.
And 1 of my service is not available...
Comments
Vegas is down
Likely just a slip of the finger. nothin to worry about..
My Las Vegas VM was down for few hours.
Looks like network speed improved after the upgrade.
Nice, it is online again. Thanks LET
Trust BuyVM will handle it well.
nice upgrade as always.
@Francisco 👌
Had an email update this morning for this issue.
Hello
To start, for the sake of the support staff, if your VPS is now humming along, we ask that you close any open tickets you have about this issue.
Short version:
Networks' been crapping itself for the past day. After 2 - 3 hard failures today we did an emergency JunOS downgrade and are cautiously optimistic that it's resolved.
Longer/technical version:
For the past month or so we've been suffering a weird issue where random IP addresses would stop responding. The issue is that our router would lose/expire out the ARP entry for these IP addresses and wouldn't attempt to learn them again unless forced to.
No amount of traffic getting sent to the IP externally (pings, random connection attempts, etc) would make the router issue an ARP "who-has" message. The only fix was to either ping the IP on the LAN (through our speedtest page for instance), forcing the router to see the ARP message on the network, or to login to the router and ping from there.
No amount of limits, tweaks, process restarts, etc, has helped with this. The only way to date has been the above methods.
This morning we had an issue where all users on 209.141.x.x dropped offline. These subnets had been moved down to our new Cisco core switch to help resolve the issue. Our Juniper router dropped the ARP entry for the Cisco switch, killing BGP. After some time it re-learned then entry reestablished connectivity, bringing those subnets back online.
This happened three times in the past 24 hours, each outage lasting anywhere from 5 minutes to 30 minutes.
Las Vegas is the only location suffering this issue. The key difference between all locations being that Vegas was running JunOS 20.4, where as everywhere else is 18.2~4. In JunOS 20.x they made changes to ARP to allow policing. While we did change this policy, it didn't help us any.
As of an hour ago we performed a downgrade to JunOS 18.4. It obviously went terribly. After some research we found the configuration entry causing issues (our 100G ports kept channelizing) and fixed it, bringing things back online.
As of now, ARP's are resolving near instantaneously, even for new IP addresses. We took this as a good time to change the rest of our ports to 100G, giving us a 400Gbit interlink between our router and core switch.
We're cautiously optimistic that this issue is now resolved.
We thank you for your patience.
BuyVM Team
Wasent a network upgrade.
We had to do an emergency JunOS downgrade on our core router since it was losing ARP entries. I sent a fairly detailed email last night about it.
Anyway, sorry. We’ve been fighting with this arp stuff for about a month now. We brought it up in some provider chat rooms and a few hosts messaged me confirming they saw similar things happen. I only confirmed JunOS versions with one of them. They were also running a 20.x+ release.
The 100gb upgrade on cogent was on Thursday and has smoothed out the last of our congestion issues in LV so that’s nice. That went mostly fine minus the fact cogent yolo’d the 2nd port without telling me. If they had warned me before hand it would’ve been completely transparent.
I don’t want to login to the router for another year. I pray to the network gods before bed every night.
Francisco
@Francisco Just because networking equipment has been brought up on here a lot lately and I have no clue how these things work in terms of what's needed for actual operations - When it comes to hardware costs - About how big of a chunk of the spend goes toward hardware never really "seen" by clients?
As in, switches, hardware solutions to DDoS attacks etc ..
Obviously not asking you to upload your company finances, but like, 5 %? 40 %?
May network gods do your will!
It depends on how you setup your network and what kind of services you offer. Since we are just VPS we can sell a loooot of service in a single rack, meaning we only buy switches once in a while. Vegas used to run an mx480 that I got in 2016. We would still be running it if 100gb ports didn’t cost more than a new router all together.
If you do dedis then you’ll likely be buying way more switches and things like that.
But, that stuffs generally a one time cost. Juniper is trying to turn things into a subscription model though so we will see.
As for ddos mitigation, depends if you want to do it on site or not. Sometimes you can one time pay for hardware, but I think almost everyone has moved to a subscription model now.
Anyway. I’m going to be rolling out 10gbit burstable ports in the next day or so. It’ll be based on how aged your service is (90 days) or if you’re premier. The wait time is to prevent ddos skids from buying things and flooding people with XX gbit.
Francisco
Gotcha, thanks so much!
Another question: Why would selling dedis result in the need for more switches?
A noob like me would think "ah dedis, lesser amount of IP-addresses, lesser packages (I guess not always) etc.
I mean a rack full of 30 servers being hypervisors might be like 500-4 000 instances (or more, or less lol), while a rack full of 30 servers all being sold as dedis would be, well, 30 servers?
Nvm, someone buying a dedi can turn it into as many virtualized environments as he/she pleases, use it strictly for hardware or turning it into a file sharing server, so it might be less plannable?
if you're in the budget dedicated server market then you'll be turning up way more racks, than a rack full of hypervisors where you're hosting 1000's of users.
Francisco
@Francisco
I also detected the mentioned short network outages (Las Vegas location, VPS). Thank You for the detailed and very correct email notification about the problem and the problems background!
Now the question is, if you don't mind me asking, why don't you do dedis
?
Less profit? Maybe it will bring more customers to handle-> so more headache? I am assuming right now you have better density per rack (more clients per 1 U) instead 1 client per 1U and maybe dedis won't be profitable/worth it to you?
I should've sent the email before the maintenance but it wasn't on my mind. It's usually only me on Friday nights.
Basically this.
The dedi market is very cut throughout and i'm not sure what I can bring to that market to make us 'special'. I'd have to compete on price and that's never going to end well.
Francisco
Understandable! Thanks for the answer.
Maybe Not the US. But IMHO some people may have interest in LUX location worth paying the premium price. (not much DCs around Brussels, Lux, Switzerland compared to other places like DE, NL, FR, etc)
@Francisco when upgrading from 40 to 100 or 100 to 400, can you use the same fiber cable and just plug into the new transceiver or you need to run better fiber to support the better speeds?
>
It isn't 400Gbit, it's 4x100Gbit in a bond
But, if you used single mode fiber, then you can just in place swap. Sometimes the fiber quality isn't great which can cause issues at higher speeds, but generally you can just swap and be good.
Some routers, like ours, require that you take the "line card" down, which is the part all the ports plug into. That has to be done to reconfigure things to support the new configuration.
Some routers/switches will allow you to do that on the fly, but I guess Juniper decided to do this on the MX204's just to be a thorn in the ass.
Francisco
Is this taken from idle vps? Or some applications already running?
I would like to know this too. My GB5 score is around 900+.
I have DirectAdmin installed with few low traffic websites running.
And my node is: AMD Ryzen 9 3900X