Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Shells Virtual Desktop
BMail.ag - Secure Email Service
Server.net
CPLicense.net
VPS Server
Buy VPN
Vultr
VMs for AI
HostDare
ReliableSite White-Label Dedicated Hosting for Resellers
InterServer VPS
BMail.ag - Secure Email Service
Best VPN
High-Performance Bare Metal Server Solutions
Karvl.com
Server Mania Cloud Hosting
DataWagon Hosting
AlphaVPS Hosting
Evoxt.com
Clouvider
VPS Hosting with NVMe
Residential IPs in the US & 4G Mobile Proxies in EU & US with Unlimited Bandwidth
ReliableSite White-Label Dedicated Hosting for Resellers
Rabisu - Hosting Solutions
Shells Virtual Desktop
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Microtronix Issues Overnight

jfreak53jfreak53 Member, Patron Provider

Hey all! We had a pretty significant outage overnight, finally seeing the light of day here so figured I'd make a post myself before someone else did. We are sending out an email now to all customers explaining this as well, also we've been updating our telegram channel as we went and had the time, but I'll post it here as well :smile:

Between roughly 1:00 AM and 7:30 AM EST, we dealt with a network issue caused by an unusually high volume of internal Layer-2 traffic within a specific customer segment. That traffic pattern put an unexpected load on part of the network and led to intermittent connectivity for some users, high level part of it. We did not get our normal alerts for DC route downs, since none of our routes nor edge equipment was actually down during this time, only loads and specific components went down intermittently, so effectively the DC wasn't down and no alerts went out as critical normally would of. So our overnight team didn't truly know it was critical infrastructure until early morning hours.

Once identified, we isolated the affected segment of the network, stabilized it, and put additional protections in place to prevent this type of condition from impacting service going forward. The network has been stable since then.

There was no data exposure or security impact as a result of this event.

We know this was a longer outage than anyone wants to see. Apologies, we do appreciate everyone’s patience while we worked through it. We’re also reviewing monitoring and alerting so we can catch and react to similar situations better in the future.

If you’re still seeing anything unusual, please open a ticket or reach out to support. Please open a ticket, no messages here or posting for support, its too much to deal with on top of the standard ticket response. If there are issues, please open a ticket :smile:

Thanks all!

Comments

  • maladiecmaladiec Member
    edited January 29

    Thats -- what i call communication, and care! I am not a customer - but keep it up!

    Things happen, life happens!

    Thanked by 1jfreak53
  • @jfreak53 said: Once identified, we isolated the affected segment of the network, stabilized it

    Is this fancy talk for blackholed?

  • Ex cust got the email too :p

Sign In or Register to comment.