Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Shells Virtual Desktop
BMail.ag - Secure Email Service
Server.net
CPLicense.net
VPS Server
Buy VPN
Vultr
VMs for AI
HostDare
ReliableSite White-Label Dedicated Hosting for Resellers
InterServer VPS
BMail.ag - Secure Email Service
Best VPN
High-Performance Bare Metal Server Solutions
Karvl.com
Server Mania Cloud Hosting
DataWagon Hosting
AlphaVPS Hosting
Evoxt.com
Clouvider
VPS Hosting with NVMe
Residential IPs in the US & 4G Mobile Proxies in EU & US with Unlimited Bandwidth
ReliableSite White-Label Dedicated Hosting for Resellers
Rabisu - Hosting Solutions
Shells Virtual Desktop
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

AWS US-East-1 disruptions and service degradation

So...

THE mayor zone of AWS is having trouble. Internet globally will be down in 3... 2... 1...

I hate Mondays.

«1

Comments

  • Nobody ever got fired for buying AWS

  • docker hub is down?

  • OhJohnOhJohn Member
    edited October 2025

    @JabJab said: Nobody ever got fired for buying AWS

    I hate MBA idiots.

    Meanwhile:

    Screenshot-2025-10-20-at-10-46-36-Uptime-Monitors-Hetrix-Tools

    All build on lowend providers.

  • sh97sh97 Member, Host Rep

    Can confirm, all of our services are down
    Going to shitpost on the greencloud thread in peace now

    Thanked by 1oloke
  • @JerryHou said:
    docker hub is down?

    Yes and likely related: https://www.dockerstatus.com/

  • r3kr3k Member

    daddy bezos probably tripped over a cable

  • OhJohnOhJohn Member
    edited October 2025

    Impacted AWS services in US-EAST-1 up from 17 to 20 to 32 37 60 (sic). So nice if everything is interrelated.

  • emghemgh Member, Megathread Squad
    edited October 2025

    We couldn't have our meeting because Todoist wasn't working (so we couldn't do CRUD on tasks)

    503 Service Temporarily Unavailable
    

    And our GitHub Actions stuff isn't working because of Docker being down (so we can't build images for our pushes)

    #3 ERROR: failed to authorize: failed to fetch oauth token: unexpected status from POST request to https://auth.docker.io/token: 503 Service Unavailable: <html><body><h1>503 Service Unavailable</h1>

    Nothing prod-related is affected though since it's all OVH

    Thanked by 3sh97 oloke tentor
  • Wonderful. My selfhosted services are UP! Signal is down but my matrix server is UP! This was bound to happen and it will happen again and again.

    Thanked by 3OhJohn sh97 jnd
  • @JabJab said:

    Nobody ever got fired for buying AWS

    HACF fan spotted!

    Thanked by 1OhJohn
  • OhJohnOhJohn Member
    edited October 2025

    @qbit15 said: HACF fan spotted!

    Best show ever. Esp. season 1.

  • Ah, now I know why AWS is down. All those OVH KS API ordering scripts firing up today overloading AWS and the internet.

  • jarjar Patron Provider, Top Host, Veteran

    This is why you host with larger providers, so that you can't experience downtime. The more people on their payroll, the lower the downtime. That is a widely accepted industry fact. Maybe next time go with Amazon.

  • emghemgh Member, Megathread Squad

    @jar said:
    This is why you host with larger providers, so that you can't experience downtime. The more people on their payroll, the lower the downtime. That is a widely accepted industry fact. Maybe next time go with Amazon.

    Tbh it is true to some extent, I feel comfortable hosting with providers that own their own DC, know everything in and out and have their own staff in said DC 24/7.

    Otherwise when things go wrong it's just much more layers in communication.

    Thanked by 4sh97 tentor jar zGato
  • @emgh said: Otherwise when things go wrong it's just much more layers in communication.

    Yeah, but - to some extent - e.g. with AWS you have walls and walls of comm to climb through before you get anywhere. And in 99.99% you probably get nowhere with providers of this size.

    But e.g. providers of the size of IOFLOOD is great: opening a ticket, two minutes later their staff is at the server as the op room/service staff is right next to DC room.

    Thanked by 1emgh
  • sh97sh97 Member, Host Rep

    Also kinda suprised applications like Docker, Perplexity etc don't have proper redundancy setup, and everything is in Virginia.

  • varwwwvarwww Member
    edited October 2025

    aws deadpool. It's always DNS

    https://health.aws.amazon.com/health/status

    Oct 20 2:01 AM PDT We have identified a potential root cause for error rates for the DynamoDB APIs in the US-EAST-1 Region. Based on our investigation, the issue appears to be related to DNS resolution of the DynamoDB API endpoint in US-EAST-1. We are working on multiple parallel paths to accelerate recovery. This issue also affects other AWS Services in the US-EAST-1 Region. Global services or features that rely on US-EAST-1 endpoints such as IAM updates and DynamoDB Global tables may also be experiencing issues. During this time, customers may be unable to create or update Support Cases. We recommend customers continue to retry any failed requests. We will continue to provide updates as we have more information to share, or by 2:45 AM.

  • OhJohnOhJohn Member
    edited October 2025

    @sh97 said: Also kinda suprised applications like Docker, Perplexity etc don't have proper redundancy setup, and everything is in Virginia.

    Ask CF with their SPOF in Portland.

    Thanked by 1quicksilver03
  • jarjar Patron Provider, Top Host, Veteran

    @emgh said:

    @jar said:
    This is why you host with larger providers, so that you can't experience downtime. The more people on their payroll, the lower the downtime. That is a widely accepted industry fact. Maybe next time go with Amazon.

    Tbh it is true to some extent, I feel comfortable hosting with providers that own their own DC, know everything in and out and have their own staff in said DC 24/7.

    Otherwise when things go wrong it's just much more layers in communication.

    Trust =/= uptime

    Everyone goes down. Only a handful end up going down so bad they can't fix it in a reasonable time frame.

    Thanked by 2emgh filtered
  • emghemgh Member, Megathread Squad

    @OhJohn said:

    @emgh said: Otherwise when things go wrong it's just much more layers in communication.

    Yeah, but - to some extent - e.g. with AWS you have walls and walls of comm to climb through before you get anywhere. And in 99.99% you probably get nowhere with providers of this size.

    But e.g. providers of the size of IOFLOOD is great: opening a ticket, two minutes later their staff is at the server as the op room/service staff is right next to DC room.

    Yes, I like medium sized providers like that.

    Thanked by 1OhJohn
  • emghemgh Member, Megathread Squad

    @jar said:

    @emgh said:

    @jar said:
    This is why you host with larger providers, so that you can't experience downtime. The more people on their payroll, the lower the downtime. That is a widely accepted industry fact. Maybe next time go with Amazon.

    Tbh it is true to some extent, I feel comfortable hosting with providers that own their own DC, know everything in and out and have their own staff in said DC 24/7.

    Otherwise when things go wrong it's just much more layers in communication.

    Trust =/= uptime

    Everyone goes down. Only a handful end up going down so bad they can't fix it in a reasonable time frame.

    Trust =/= uptime but we've seen on LET over and over how not having a clue what's actually going on inside the DC and it's network etc lead to things just not getting solved effectively

    Thanked by 1jar
  • emghemgh Member, Megathread Squad

    Also things like no spare parts, no spare server with sufficient resources to migrate to, waiting on remote hands that don't do their job effectively, etc. List goes on.

    Thanked by 1tentor
  • @qbit15 said: Signal is down but my matrix server is UP!

    As an option, Telegram is up as well.
    My TG Desktop is open right now, saw zero issues the last few hours.

  • OhJohnOhJohn Member
    edited October 2025

    As a consequence, @Fiberstate thinks I want my invoice dates in Spanish. Probably some API down hosted in US-EAST-1 that connects countries to languages. Welcome interconnected world.

    Thanked by 1emgh
  • Dockerhub is down for more than 2 hours now.

  • @DataRecovery said:

    @qbit15 said: Signal is down but my matrix server is UP!

    As an option, Telegram is up as well.
    My TG Desktop is open right now, saw zero issues the last few hours.

    Yeah, telegram is full functional. Signal is all in on aws from the beginning. Telegram doesn't rely on cloud providers.

  • Looks like half of internet is down...

  • @JerryHou said: Looks like half of internet is down...

    As was predicted. They'll never learn.

  • @varwww said:
    aws deadpool. It's always DNS

    https://health.aws.amazon.com/health/status

    Oct 20 2:01 AM PDT We have identified a potential root cause for error rates for the DynamoDB APIs in the US-EAST-1 Region. Based on our investigation, the issue appears to be related to DNS resolution of the DynamoDB API endpoint in US-EAST-1. We are working on multiple parallel paths to accelerate recovery. This issue also affects other AWS Services in the US-EAST-1 Region. Global services or features that rely on US-EAST-1 endpoints such as IAM updates and DynamoDB Global tables may also be experiencing issues. During this time, customers may be unable to create or update Support Cases. We recommend customers continue to retry any failed requests. We will continue to provide updates as we have more information to share, or by 2:45 AM.

  • EthernetServersEthernetServers Member, Patron Provider

    10:39
    Breaking
    Amazon Web Services, the cloud company behind the outage, has just released a statement saying: "We are seeing significant signs of recovery.

    "Most requests should now be succeeding.

    "We continue to work through a backlog of queued requests. We will continue to provide additional information," it reports on its website.

    As a reminder, Amazon Web Services (AWS) powers the infrastructure behind millions of websites and apps.

    https://www.bbc.co.uk/news/live/c5y8k7k6v1rt

    Thanked by 1loay
Sign In or Register to comment.