New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
Sorry my ignorance sparked this. Hostigation is the next best thing to having sex! Now i have saved this: https://twitter.com/hostigation
Gosh darn it. My nagios node is in LAX. I'll have to set up a redundant Nagios config. Good thing my secondary DNS is with another provider.
I have people I trust at Quadranet on the same page now, so I'll go with you on the 2 hour estimate, but this second, I really don't know.
losing customers
very bad luck
Oh, god, who is next on Murphy's hit list ?
January is almost over, place your last bets...
Hope February wont start over...
Notice to self: never do scheduled maintenance/moving in January... Downtime will happen anyway, so why not take advantage of that...
@Jack I know most of what is going on, and shit hit the fan when I rebooted the router after getting a IP/KVM on it and the busybox promt came up, and the spider kvm resets anytime I reboot. Passed onto someone at Quadranet that know's what he is doing what I need since the IP/KVM is not cooperating, and told him not to worry about the cost, just sell me another box and where the vyatta config is on the original box, and he is on it.
You tempt fate way too much, I see all these earlier problems and I kept my mouth shut not to draw attention to myself, and still my plan failed
Failed HD then?
Refugee discount code: GTFO
Hope everything clears up brother.
I don't think so, Vyatta reads the ISO off the drive while booting, and I am able to mount the drive within busybox and see the config, but unable to save it off the system from the spider kvm, so quadranet is helping with that and moving to a new box.
LA has always been an Opteron 1200 series
Have you considered adding a second router for redundancy? Or is it too much trouble?
server down for 3hrs and counting. Is there any estimate time for server/router/network up?
Have config off old router, new router is being racked now, soon as they give IPMI info I will get it loaded up. Time in minutes/hours? Soon™
Tim is updating his Twitter account at https://twitter.com/hostigation
@miTgiB How is the OS load going?
yep my node is up. I'm not leaving Hostigation, shit happens.
These outages aren't Murphy. They are in the course of doing routine work, planned moves, etc.
Stuff happens, even to the most seasoned and proven folks. Always points to live spare necessity and system to failover and back from such.
I often wonder how much these outages are increased by providers being far from their own gear (i.e. can't drive down the road 20 minutes and fix it yourself).
Lets hope that the providers snagged offline lately all take steps to improve redundancy.
@MiguelQ: From Twitter:
Hostigation @hostigation 7m
All is fixed, will reboot the new config once to make sure all is well
Doesn't help with remote hands that are not familiar with things.
Report from my monitoring server [GMT+8]
Downtime : 7 hours
Shit happen, however Hostigation have almost flawless uptime record every other month
p/s : how do I embed image in a post? the built-in function didn't work?
updated
HTML < img > tags
You can have redundancy for your redundancy but you'll never outrun the simple fact that whatever can go wrong will go wrong, and if you reduce it to hurricane, better find shelter.
Redundancy is N+1, that equals 3.
Most folks seem to be running one of anything typically which is part of the problem.
Providers smacked by brownouts, just A leg power. No B.
Providers smacked by router on the fritz, just one router.
Redundancy comes at a cost and I don't expect brand new providers to have such, especially on the lowend of the spectrum.
What program is that! Looks sexy!
Agreed
Mine is up! Thanks Tim for fixing, will point back my site to hostigation vps.
that is the key, this is lowend, and while a few of us can afford some more redundancy, it isn't offered at these rates. @KuJoe is the only person I know that takes redundancy to the next level.
@KuJoe has been getting slapped by the N+1 lacking lately it seems. I think he needs to spend more quality time with the other two parts of the redundancy (i.e. router snafu recently). Think he has the N part but not the spare +1.
But agreed about the market and what it affords.
Hoping to see more providers offering services where they are physically with access. Hands on facility folks are spread like jam in these datacenters and often random in competency and familiarity. I hate being the remote seeing eye dog for them and I detest schooling them during a three alarm fire.
We have a fully redundant network but it means nothing if BGP isn't working (and BGP is supposed to be so simple, it angers me that we've had so many problems with it). We have 2 routers that connect to 2 different networks with 2 different uplinks but for some reason Vyatta would terminate the BGP sessions out of nowhere and no matter what we did the DC's Cisco routers would not let us re-establish them (our routers would sit in an Active state but the Ciscos would show us terminating the session).
We've fixed this by removing BGP from the equation and ordered some non-Vyatta routers to install along side our current routers. We also have a Cisco router that replaced our 2nd Vyatta router so if we have any issues between 2 Cisco devices (configured by our DC tech) we know it's not an issue on our side. Additionally we have a hard drive sitting by itself in the cabinet with a known working config of Vyatta in the event something happens we can throw it in one of our backup servers and move some cables and get back online in the 15-20 minutes it takes one of us to drive to our DC (or just ask the on-site tech to do it but we prefer to do it ourselves).
This is the primary reason we moved from Jacksonville to Tampa, the 15 minute drive is awesome. It's also the reason why we picked Denver as our next location (I will be moving to the area in April and will be working about 20 minutes from the DC we're in).
Really liking your approach to things @KuJoe! My kind of plan/person.
Yeah that's pretty good redundancy.