Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Register

Quick Links


Categories

In this Discussion

› › reliablesite.net down ?
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

reliablesite.net down ?

dadihartadadiharta Member
in Outages

My server on reliablesite.net
server location in NewYork.
their website also down.

has anyone experienced the same thing?

Comments

  • totally_not_bannedtotally_not_banned Member

    They are taking a nap.

  • caracalcaracal Member

    Not sure if relevant to the router software update they sent out

    Detail: This notification is to inform you of emergency maintenance that may affect your ReliableSite service. Please review the details of this notification to determine how it will affect your service.

Event Type: Router Software Upgrade
Start: 2024-05-13 02:00:00 Eastern
End: 2024-05-13 14:00:00 Eastern
Location(s): New York City Metro (NY1)
Client Affecting: Yes
Device(s): ##

The JunOS version on the router will be upgraded to fix a software related bug. This bug is known to cause routing related issues and will be installed with high priority to prevent unexpected network downtime. We expect brief unavailability in 15 to 20 minute periods during the upgrade process. The large maintenance period has been scheduled to allow preparation prior to the upgrades and monitoring post upgrades.

If you have any questions regarding this notice, please contact us at https://support.reliablesite.net.
  • dadihartadadiharta Member

    I had missed their notification in email.
    They are upgrading their router software.

  • RoldanRoldan Member
    edited May 13

    My NY server has been down for 3 hours and 38 minutes now.
    I thought they will be offline for 15-20 minutes only as per their email :(

  • sliixsliix Member
    edited May 13

    I can confirm my NY server is also down atm.

    Update: ok nvm received the same email as above.

  • LeviLevi Member

    Reliablesite is not reliable, what a surprise…

    Thanked by 4fluffernutter fadedmaple anda pangkus
  • Maelstrom36Maelstrom36 Member

    They could be experiencing issues they hadn't anticipated. Even so, it's still within the planned outage timeframe.

  • D9DanD9Dan Member
    edited May 13

    @Roldan said:
    My NY server has been down for 3 hours and 38 minutes now.
    I thought they will be offline for 15-20 minutes only as per their email :(

    We have a number of servers in their NYC DC and the original outage was around 50 mins for us then everything came back up until the 2nd outage around 20 mins ago, everything is currently back up.

    From the wording of the maintenance alert, I would imagine there could be multiple small outages during the maintenance window.

  • MrRadicMrRadic Patron Provider, Veteran

    A copy of the scheduled maintenance is available in the dedicated panel under "Maintenance & Status". We had about 7 switches that didn't come back online for whatever reason, 3 are fixed, 4 are being worked on.

    Thanked by 2Maelstrom36 Roldan
  • MrRadicMrRadic Patron Provider, Veteran

    All switches are up, finalizing a few things, and we should be wrapped up.

    Thanked by 2Maelstrom36 sliix
  • JoseDieguezJoseDieguez Member

    they announced a maintenance like 3 hours prior to the maintenance lol.

    and the maintenance was supposed to mean a 15 minutes outage, and it ended to be 75 minutes.

    funny thing is that all our NY servers with them had the same outage, so i guess are all on the same switch. how lucky we are.

    Thanked by 1Gatto
  • RoldanRoldan Member

    Mine has been down for 4 hours and 50 minutes now and is still not back up.

  • JoseDieguezJoseDieguez Member

    @Roldan said:
    Mine has been down for 4 hours and 50 minutes now and is still not back up.

    what does support tells you?

  • RoldanRoldan Member

    @JoseDieguez said:

    @Roldan said:
    Mine has been down for 4 hours and 50 minutes now and is still not back up.

    what does support tells you?

    No response yet.

  • bootboot Member

    @totally_not_banned said:
    They are taking a nap.

    In the city that never sleeps.

    Thanked by 1totally_not_banned
  • RoldanRoldan Member

    I got a response, but just the generic:

    Hello,
    We have received your request.
    We are looking into this and will get back to you shortly.
    Thank you,

  • RoldanRoldan Member

    Does your NY servers still on the same network speed that it's back online now?

    Mine is back after 6 hours of downtime, but the network download speed is very slow.
    I tested it via Iperf3 on the server.

    iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider        | Location (Link)           | Send Speed      | Recv Speed      | Ping           
-----           | -----                     | ----            | ----            | ----           
Clouvider       | London, UK (10G)          | 885 Mbits/sec   | 83.4 Mbits/sec  | --             
Eranium         | Amsterdam, NL (100G)      | 883 Mbits/sec   | 865 Mbits/sec   | --             
Telia           | Helsinki, FI (10G)        | busy            | busy            | --             
Uztelecom       | Tashkent, UZ (10G)        | 771 Mbits/sec   | 59.3 Mbits/sec  | --             
Leaseweb        | Singapore, SG (10G)       | 742 Mbits/sec   | 415 Mbits/sec   | --             
Clouvider       | Los Angeles, CA, US (10G) | 897 Mbits/sec   | 103 Mbits/sec   | --             
Leaseweb        | NYC, NY, US (10G)         | 939 Mbits/sec   | 901 Mbits/sec   | --             
Edgoo           | Sao Paulo, BR (1G)        | 802 Mbits/sec   | 48.7 Mbits/sec  | --             

iperf3 Network Speed Tests (IPv6):
---------------------------------
Provider        | Location (Link)           | Send Speed      | Recv Speed      | Ping           
-----           | -----                     | ----            | ----            | ----           
Clouvider       | London, UK (10G)          | 870 Mbits/sec   | 154 Mbits/sec   | --             
Eranium         | Amsterdam, NL (100G)      | busy            | busy            | --             
Uztelecom       | Tashkent, UZ (10G)        | 757 Mbits/sec   | 55.3 Mbits/sec  | --             
Leaseweb        | Singapore, SG (10G)       | 673 Mbits/sec   | 6.29 Mbits/sec  | --             
Clouvider       | Los Angeles, CA, US (10G) | 886 Mbits/sec   | 114 Mbits/sec   | --             
Leaseweb        | NYC, NY, US (10G)         | 925 Mbits/sec   | 873 Mbits/sec   | --             
Edgoo           | Sao Paulo, BR (1G)        | 803 Mbits/sec   | 30.4 Mbits/sec  | --
  • JoseDieguezJoseDieguez Member

    sadly, haven't made tests to have a point to compare.

  • MrRadicMrRadic Patron Provider, Veteran
    edited May 13

    @JoseDieguez said:
    they announced a maintenance like 3 hours prior to the maintenance lol.

    Here's how the yesterday's timeline went:
    ~9AM Eastern - We started observing abnormal CPU behavior in NY1. It was found to be related to abnormal ARP processing. The source of the issue was found and fixed by 11:30 AM. The router was reporting normal operation.

    ~6:40PM Eastern - We started observing abnormal errors in logs and routing behavior.

    ~8:00PM Eastern - Juniper was engaged and determined the issue was caused by a known software bug in JunOS and recommended an upgrade.

    ~10:30PM Eastern - Planning had been finished and a maintenance period was put together. At this specific point in time, the router was already refusing ARP and adding next-hop to its table. We were limited to the kind of configurations it would accept, the 1-click installer and much of our automation was no longer operational. Considering these issues, the router was still pushing traffic normally, but we didn't know for how long. The time period that we set had to do with traffic levels on our network in NY1, we chose the least busy time possible pushing the maintenance as far back as possible.

    and the maintenance was supposed to mean a 15 minutes outage, and it ended to be 75 minutes.

    "We expect brief unavailability in 15 to 20 minute periods"

    No, it was stated as "15 to 20 minute periods", which is how long the routing engines usually take to reboot. The first one took longer because the bug we were trying to avoid took the backup RE offline ~15 minutes after we gracefully transferred traffic to it and we couldn't bring it back until we finished the upgrade on the primary RE.

    =========

    Trust me when I say that this wasn't what we wanted to to be doing, but it would have led to a much longer and much more painful unplanned outage.

    Thanked by 1Roldan
  • conceptconcept Member
    edited May 13

    @MrRadic Does other ReliableSite locations have routers with the same bug?

  • MrRadicMrRadic Patron Provider, Veteran

    @concept said:
    @MrRadic Does other ReliableSite locations have routers with the same bug?

    I believe they were updated years ago for specific feature requirements, but would need to confirm with my network engineers.

  • caracalcaracal Member

    @MrRadic said: No, it was stated as "15 to 20 minute periods",

    I must confess I did also read it as 15-20 minutes of downtime on my first read through of the email.

  • vipewebvipeweb Member

    I really like reliablesite, in general I have very few problems
    MrRadic

    a comparison, that MrRadic is always here providing information, this is very important.

    I'm waiting for more Miami servers to become available to purchase =)

Sign In or Register to comment.