New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Murphyology applied
Well, it is simply too much:
BuyVM saga continues
SecureDragon seems is in big trouble too
Inception hosting has problems
Prometeus had a failed IPv6 router 3 days ago
There are other usuals here experiencing difficulties, either related or unrelated.
As I recall, there was never such a spread out problem in the history of LEB/LET and while the causes seem to be unrelated, the sheer scale of it gives me the creeps.
What's next ?
I mean, what are the chances that you switch data centers to a stable facility, you manage to put almost everything in place and the power fails a few times on you ?
Sometimes Murphy does work overtime, but it is getting scary.
Comments
It's a tough week for everyone
Making an early declaration that I'm going to have an AWESOME week.
So you're welcome. Murphy is going to single me out now.
Maybe the mayans were wrong with a few months? ;-)
To quote a song
"Never had to, I better knock on wood...
Cause I'm sure it isn't good
Am I'm glad I havn't yet...
" -MMB
This is why you need over 30 VPS servers and a dedicated or 3. And very very low TTL. Would have run out of Xanax due to the Buffalo BuyVM issue otherwise.
you can buy off Murphy with your 1st born
as always, things can go wrong,
start with: Top provider announcement, new year offers, buyVM relocation problem, some top provider problems, next?
I'm keen to see the 2013 Q1 top provider list
@graca: I wish a provider that had just a few customers, would win, that would be interesting.
I hope not. First week for me!
I'd like to introduce you to someone, he'll be sleeping on your couch from now on, his name is Murphy.
http://nooooooooooooooo.com/
This stuff happens folks. Seems like a busy time of year --- stuck in colder darker inside realm where people plan and plot moving gear and muck with things.
Unsure how much SecureDragon's new Colorado location may have contributed to their recent issues.
Prometeus, what was the cause of their outage? Failed router and no on site backup spare?
The cause of outage was that the router locked at 5 AM just when i went to bed and since it was already an hour or so till power cycle woke it, the time was used to move some functions on the main router for a bit of insurance.
It didnt fail completely, just locked up a bit, enough for ipv6 connectivity to be lost.
Now there is some redundancy, at the beginning IPv6 was considered more like an add-on, something you can do without, but the things evolved and there is more redundancy and monitoring in place after this.
Good info and ideas @Maounique.
We don't use IPv6 yet, so never noticed.
I read this as murphy apologised. I nearly had a heart attack.
I'm sure compared to Prometeus and BuyVM, RamNode qualifies as having "just a few customers", so maybe we'll win next time and fulfill your wish.
What's next ?
Welcome to statistical probability. The next time a month goes by without any major issues here be sure to ask, "OMG what's happening?"
"relocation problem" ?
Think about how many nodes we had in SJ, then think about how we had only 3 people doing the move
Of those 3, only I had experience setting up and configuring a rack, as well as configuring switches/routers/etc.
Aldryic learned a lot very quickly and handled most of the rebuilds we did while in Vegas. After the breaker blew we took it as a chance to replace the motherboards in all of the KVM nodes so we have proper IPMI's in all of them.
Really, if I had 8 people on hand, everyone was well versed in rack deployments and the likes and we tripped over some of the things we did on this pull? Then shame us to the world but really, you had 1 manager and 2 "first day on site" employees. While I would have liked to complete things sooner as well as handled @sonicvps quicker, there's only so much any of our bodies could keep up with.
Francisco
@Francisco why such a small/rookie team for this big move?
You can train basic tech skills in a pinch, You can't teach loyalty and someone to be respectful of your customers & equipment. The only way to get the last two is to get someone heavily involved in the company.
Aldryic is a trained long distance driver and had the equiv of a class A license while in the military. Karen, while small framed, still pulled her share of the load by being efficient, constantly breaking down boxes to clean up the area as well as installed a good half of all rails. Today she helped me wire all of the IPMI's.
Francisco
/me waits for the crying over all the power they draw
We're actually at a net positive on power, even with the ipmi's
The new cage we got has access to a lot of power. Cheap power.
Francisco
@Nick_A: I was actually at your website earlier considering a KVM VPS. lol
@Francisco: it's still amazing how well this whole thing went.
I'm just thinking back to you stating IPMI draws 2A or something outrageous a year or so ago, which made me witch hunt the specs, and could only find 100mA in the manual
Yer, 2A total for the whole rack
Back in FMT it was like 0.3a or so per IPMI.
It has greatly improved and we've seen no real difference in usage.
Francisco
IPMI in the 100ma area sounds more like it.l
I am still impressed with the move @Francisco. Crack sniper team. You folks got a freaking back and ab workout racking all that gear.
Haters will hate and let em.
Love this
I honestly feel partly responsible. I wanted to fly over for this move, but the dates were too close to the New Year, had it been in March or later, I would have been there for sure.
It's all worked out though and I haven't actually had any complaints - quite the opposite.
Some other challenge in my works too..
"
But I just take it as "It would make me stronger and better than before, because I'm raising from my current position
+1