What happened to Racknerd DC02, Route exploded?

dustinc · November 2023

@sh97 said:
@dustinc on a side note, I've noticed that over the past 6+ months, my VPSs in DC02 have been having a significant drop in download speeds.

I currently have just one service in DC02, but previously when I had multiple services, noticed this issue across different host nodes.

Uploads are easily hitting ~900Mbps, whereas downloads from most networks hover around 200Mbps.
For directly peered networks like Cloudvider LA or Race, I can see ~900Mbps download and upload. (Test files/speedtest/ iperf)

Dunno, seems something is off. I even raised a ticket a few months back, where I was migrated to another node, but it was more or less the same.

When doing some tests from your SJC location, latency/speeds seem much better.

Anyways, just letting you know. Since I don't have any network dependant stuff on this VPS, I'm fine with the current state.

Hi @sh97 -- appreciate this feedback. I will reach out to you shortly via DM to request more info on this. If you’re getting 900Mbps up to a particular remote destination though, then technically there should have no issue reaching 900Mbps down on the same remote destination - so maybe the remote speedtest test server (especially if its a public/free one) has its upload link (and thus down link on our side) congested or busy? Of course, just throwing this out there and this is just an initial thought based on what I am reading of your post so far, which may change once I dive deeper into the details. I’ll definitely investigate thoroughly though once I have more concrete information.

dustinc · November 2023

@HostEONS said:

@dustinc said:
Hi @Doublefire_Chen -- appreciate you being our valued customer

We are not experiencing any widespread issues in Los Angeles DC-02, or any other datacenter location of ours recently. No other customers are reporting similar issues either. I also just checked multiple IPs within LA DC-02 using external tools like: https://ping.pe/ -- These checks have shown no packet loss globally.

Given this, it's possible that the issue you're experiencing might be isolated to your connection or setup. One factor to consider could be any firewall configurations on your server. For instance, if you have ConfigServer Firewall (CSF) installed on your server, it's known to rate limit ICMP requests by default. I would recommend checking your firewall settings, particularly in the csf.conf file (within the "ICMP_IN_RATE" section), to see if this might be a possibility (for example I know there are some free control panels out there that by default bundle in CSF).

I want to clarify that my intention isn't to dismiss the potential of an issue on our end. My mention of firewall configurations, particularly with CSF, is simply to highlight a common oversight we've observed. Often, our clients might be unaware of CSF being installed, as it can come bundled with certain free control panels, frequently with default settings in place. This is a scenario we've encountered more than once, and it's always a good initial check to rule out such possibilities. If there's an issue on our side, we're always the first to take responsibility and make things right. Should it be necessary, we are also prepared to escalate this issue and create a case with our upstream providers. That being said, if after reviewing your firewall settings and any other potential factors within the OS level, you still believe the issue is not originating from your end - please send me an email at [email protected] with your VPS IP, your source IP, and a detailed MTR report. This will help me better understand where and how the packet loss is originating.

I could not stop myself from praising you @dustinc for the kind of professional and well explained response you give all the time .. I know we are competitors but still got to appreciate it

Hi @HostEONS -- very kind of you! We all can learn from each other which makes this industry so fun, and the beauty of this too, is that there’s plenty of customers for all of us. We truly don’t mind sharing knowledge and helping other companies grow, so whenever I can lend a hand, I do. There’s plenty of companies here who have reached out and asked for guidance/help and we’ve done it. If you ever need anything feel free to ask

dustinc · November 2023

@Doublefire_Chen said:
After a communication via email, problem is founded, the issue is in my side. The route provided by my ISP is terrible. Thank you @dustinc very much. The service of Racknerd is awesome!

Hi @Doublefire_Chen -- it's always a pleasure working with you, thank you so much for your transparency and professionalism. I hope to see you around in our Black Friday 2023 event: https://lowendtalk.com/discussion/190011/b-l-a-c-k-f-r-i-d-a-y-by-lets-1-top-provider-racknerd-hundreds-of-giveaways-come-get-some

dustinc · November 2023

@JasonM said:

@HostEONS said: I could not stop myself from praising you @dustinc for the kind of professional and well explained response you give all the time .. I know we are competitors but still got to appreciate it

he indeed is a helpful guy!

Thank You @JasonM for the kind words and for your continued support with RackNerd over the years. We sincerely appreciate your business

dustinc · November 2023

@epaslv said:
I am not sure if this is related, but i have a KVM on DC02 that went offline earlier today. It was timing in and out prior losing pings etc. After a few checks via console it turns out the VPS is stable but but no IP address allocated to the VPS, hence can't connect in or out.

After a few ticket responses, i was fianally told all thay can do is re-install the OS. I was hoping not to have to do this, but with no other options I re-installed the OS.

It didn't resolve the issue and the same problem exists. So problem is outside the VPS it seems and i destryed all data unecessarily.

Now waiting for advice on next steps

Hi @epaslv -- I believe to have found your ticket, and our Customer Success Manager had followed up last night with more details. Your situation (assuming I’m looking at the right ticket) seems to be unrelated to what OP created this thread for, as your particular issue was resolved from within the OS level side. We replied in the ticket with further technical details explaining what and why, so you should be all set. If you need anything else, or if there are still any outstanding issues, please reach out to me at [email protected] and I’ll be happy to personally help.

dustinc · November 2023

@TheXtremeS said:
During night time, on LAX-DC02, I experienced some slow speed to our ISP.

Hi @TheXtremeS -- Depending on where your service is and where you are in the world, as well as the congestion within your region (i.e. China / GFW, etc), latency may vary. If there's an opportunity to evaluate the routes, please share the data directly with me via e-mail at [email protected]. If there's a way that we can adjust the routes, we will definitely do so, though understandably that is not always possible. Sometimes, region side issues can't be solved with certain carriers, though depending on your application, use-case, and budget, we can explore additional direct routes if applicable (such as CN2 GIA dedicated servers for those looking for direct connectivity to and from China).

HostEONS · November 2023

@dustinc said:

@HostEONS said:

@dustinc said:
Hi @Doublefire_Chen -- appreciate you being our valued customer

We are not experiencing any widespread issues in Los Angeles DC-02, or any other datacenter location of ours recently. No other customers are reporting similar issues either. I also just checked multiple IPs within LA DC-02 using external tools like: https://ping.pe/ -- These checks have shown no packet loss globally.

Given this, it's possible that the issue you're experiencing might be isolated to your connection or setup. One factor to consider could be any firewall configurations on your server. For instance, if you have ConfigServer Firewall (CSF) installed on your server, it's known to rate limit ICMP requests by default. I would recommend checking your firewall settings, particularly in the csf.conf file (within the "ICMP_IN_RATE" section), to see if this might be a possibility (for example I know there are some free control panels out there that by default bundle in CSF).

I want to clarify that my intention isn't to dismiss the potential of an issue on our end. My mention of firewall configurations, particularly with CSF, is simply to highlight a common oversight we've observed. Often, our clients might be unaware of CSF being installed, as it can come bundled with certain free control panels, frequently with default settings in place. This is a scenario we've encountered more than once, and it's always a good initial check to rule out such possibilities. If there's an issue on our side, we're always the first to take responsibility and make things right. Should it be necessary, we are also prepared to escalate this issue and create a case with our upstream providers. That being said, if after reviewing your firewall settings and any other potential factors within the OS level, you still believe the issue is not originating from your end - please send me an email at [email protected] with your VPS IP, your source IP, and a detailed MTR report. This will help me better understand where and how the packet loss is originating.

I could not stop myself from praising you @dustinc for the kind of professional and well explained response you give all the time .. I know we are competitors but still got to appreciate it

Hi @HostEONS -- very kind of you! We all can learn from each other which makes this industry so fun, and the beauty of this too, is that there’s plenty of customers for all of us. We truly don’t mind sharing knowledge and helping other companies grow, so whenever I can lend a hand, I do. There’s plenty of companies here who have reached out and asked for guidance/help and we’ve done it. If you ever need anything feel free to ask

Hi @dustinc I very much agree, some people might say hosting industry is saturated, but I think if it's done the right way, there is still a huge gap to fill. It's very kind of you to offer help, I'll definitely contact you if I need anything, and you too can contact me if you ever need a hand with anything.

SplitIce · November 2023

@Doublefire_Chen You really should publish the traceroute (or mtr). Not taking any side here or assigning blame but traffic paths can vary quite a bit by transit load and outages.

Its far more likely that the issue is transit / route related than a specific Datacenter.

For example I've seen alot of US-WEST HE issues this month (we don't use HE directly so only lightly investigated). A good network will route around HE in US-WEST during these incidents (bad will put up with ~20% PL bursts) if that creates a worse route, so be it.

epaslv · November 2023

@dustinc said: I believe to have found your ticket, and our Customer Success Manager had followed up last night with more details. Your situation (assuming I’m looking at the right ticket) seems to be unrelated to what OP created this thread for, as your particular issue was resolved from within the OS level side. We replied in the ticket with further technical details explaining what and why, so you should be all set. If you need anything else, or if there are still any outstanding issues, please reach out to me at [email protected] and I’ll be happy to personally help.

Hi @dustinc,
Thanks for the response. Yes its not related, my apologies. The only commonality was DC02, and when i installed a new OS, but had the same probs it got me concerened.
Thanks again for reaching out.

Howdy, Stranger!

Categories

In this Discussion

What happened to Racknerd DC02, Route exploded?

Comments

Howdy, Stranger!

Quick Links

Categories

In this Discussion

What happened to Racknerd DC02, Route exploded?

Comments