New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
Although 2 nodes would work fine with our setup without issues, this is indeed a 4-node chassis. We aim for 4-12 per location.
The 3 nodes online were just because I had just cabled everything, all 4 nodes will be operational at turn-up completion.
In regards to the nodes themselves, I forget the exact spec, but I'd say you are decently close on the filtering capacity, it varies, of course, based on the attack type. Phoenix is a very insulated site with Salt Lake City, Los Angeles, and Dallas surrounding it, so it doesn't require as much scrubbing capacity as hotter sites.
That makes sense. Do you use something beefier than the SN2700's for your hotter sites? I have one coming for my home lab tomorrow, probably get the second one next week.
We've started a transition to MX204s for smaller sites and 304s for the larger ones, but the SN2700s have done relatively well.
My server has been unreachable for 4d 5h 47m 13s
NY. @KyanG7
I hope you have backups. New York is being deadpooled like Phoenix.
Let’s count the lies from Kyan (Director of Ops @ Tempest) since Phoenix went down:
“We think it’s a GTT issue.”
“Phoenix will be back online in 24 hours!”
“It’s the beta equipment acting up, guys!”
“Our specialists are working on it right now! Just a few hours guys...”
“It’ll be back up shortly!”
”Oh no its not beta equipment, Its all PhoenixNAP's their fault that we couldn't pay them!” Just like Equinix and Digital reality?
“They’re thinking about moving everything to a new DC. However, PhoenixNAP wants to negotiate, so we might stay there.”
3 months later Phoenix is still down.
Blatantly false statement. End of story.
First step of troubleshooting was checking GTT as we received an email
You seem to misunderstand the word "estimate" that was given when we were still investigating
Was an issue with our new network equipment which we resolved but then another issue arose at the same time.
We are still in Equinix and Digital Realty facilities, not really sure what else to say to that. PhoenixNAP has personally gone through huge hoops to screw us around.
Negotations with companies happen all the time, its the reason we executed the Equinix move in the first place, if you put your mind to it on when we signed contracts for locations, you'll find it makes sense why we were negotiating with all datacenters at the same time.
So you still haven't answered any of the questions. When is New York or Phoenix going to come back online? Could you tell me what the issue is? Besides, your company is not paying its bills! But if you're the director of operations, you should have a clue about the issue, then right? Oh gosh, the vagueness in your response is just insulting to your customers past and present.
Consider the rabbit pulled. Singapore has returned, Phoenix is awaiting one shipment estimated arrival Monday, Tokyo is being configured.> @PickleFarts said:
New York is just waiting for the temporary transit to be run; we paid a very large fee for it to be expedited. The only vagueness is that we cannot control the speed at which a service is provisioned.
My own comments were screenshotted and posted here that Path Phoenix will return shortly. My current estimate is Wednesday, as I needed to order a few last parts. We could have done it the day I racked everything, but I wanted to make sure it was done fully and correctly the first time.
Path has supposedly "not been paying bills" for 3 years, yet here we are.
I’m still hearing a lot of cap. People should 100% take everything that @KyanG7 says with a grain of salt.
You only moved from Equinix and Digital Realty after power got cut in multiple locations, including Phoenix. But now it’s being framed like that was the plan all along?
And blaming the datacenter providers? Bit odd that several of them dropped you in the same week. Even if everything you’re saying is true, it still screams mismanagement.
Letting contracts expire or not paying them while you’re “in negotiations,” hoping they don’t cut you off, that’s not strategy, that’s just gambling with your customers’ data and uptime.
And let’s be real, Path or Tempest didn’t even try to be transparent. Five outages before a proper statement, and before that, straight up lying to customers. Now with New York, same story again. A fiber issue taking five days and you’re being super vague about it? Doesn’t add up.
Also, saying it was just an “estimate” while you were still figuring things out doesn’t make it better. That just proves the communication was a mess. We’re not asking for perfection, just honesty.
And about the negotiations, if all those talks came to a head at the same time, then something went seriously wrong. Either it wasn’t planned properly, or it completely backfired. Either way, customers got burned due to your mistakes.
Singapore has not returned. All traffic from APAC still routes to either USA or Europe.
https://atlas.ripe.net/measurements/86481969
All comments I've made were accurate to my thoughts at the time. There is no winning with estimates, which is also in that chat but has been conveniently left out of the screenshots. If I give estimates and they aren't fulfilled, we are the bad guys. If we don't give estimates, we are the bad guys; there is no winning, unfortunately. We do what we can.
We cannot be transparent while we are in active court cases with another provider. I think if you do some quick research and see the victories we've had so far, it should prove my point. As things come to a close, we've become far more public with the information we can share.
Either way, we will continue to do our thing. I believe in our product.
Oops, that is a mistake on my part, Singapore WILL return when it is safe to without causing major disconnects. I apologize.
Yes no problem we are used to it. You guys make a lot of mistakes.
Growing companies will come with related pains, we will continue to do what we can to resolve any issues! Feel free to reach out if you are directly impacted and I'll see what we can do for you.
What victories?
The one where you settled with your landlord over the $250,000 bill?
Don’t forget to pay your settlement before July 25!
Or the "victory" where 90% of Tempest customers are now hosting with GameServerKings/Cosmic?
While you’re struggling to keep your locations online and pay invoices, René got an extra-large paycheck these months.
Anyway, good try hiding behind active court cases. Yet another excuse to avoid providing any proof for your claims.
Regardless of whether what you say is true or not, I can’t justify paying for a server that can go kaput quite literally at any second and be on the whims of random nonsensical discord and forum statements of whether it’d come back or not. All this chaos and mismanagement is comparable to what’s coming out of Washington DC.
To be clear, we’re not talking about a $15/yr VPS provider having an outage, which is a clear case of “you get what you pay for.” We’re talking about dedicated servers that are invoiced at or close to triple digits monthly, far more than OVH GAME servers. This amount of downtime, unreliability, and poor PR is unacceptable for anyone who’s running anything more than an [expensive] disposable hobby project. People paying premium dollar expect premium quality service, not the service of a basement provider.
That you fail to understand this and make a laughable statement such as “I believe in our product” shows how out-of-touch and tone-deaf you are to those of us being burned and infuriated by these long prolonged sudden outages. But I suppose as long as your paycheck hits your bank account, fuck the rest of us that funded it with our invoice payments, right?
This will be my last post on this thread for the foreseeable future as I have nothing more to add that others (like @SoepBlik ) haven’t already added.
"What victories", My statement was completely unrelated to the Esplanade, we can't share details based on the actually important lawsuits, which you decided to skip over that was related to the conversation (extortion of a provider). But just because you decided to, Esplanade is supposed to be a high-quality, and I'd say higher-end office building. Instead, they couldn't supply (a tech company) with Wi-Fi, fix the shattered glass, or have a functional lobby for months. I also don't get it, do we not have any money, or do we have $250,000 to pay out?
90% of Tempest customers is bold, I'd like to see the numbers on that one. Cosmic hasn't had a new client in their AS-SET in 2 months as far as I'm aware. I think you should of gone for Neo-Protect which a majority of the cheap clients go to which is pretty standard in business or else why would people buy Gucci?
I'll continue to "hide" behind what is best for the company, I won't sabotage a lawsuit because of someone on LET that has so far only posted about Path.
Honestly yeah, who will now host with such a shady company lol. This is more scarier than ihostart.
OVH game servers (not rise-game) are now triple digits though to be fair. Though 7950x for $500/month is indeed still way higher and highway robbery.
But honestly if you are still with Tempest/Path right now, that is your own fault. Signs of insolvency like this appeared last year. If you put your head down and ignored everything that was happening, the problem didn't go away.
I agree with you, the discord can be nonsensical at times, we do not allow tech support via discord for this exact reason, all customers that reach out through the correct support lines get responses (tickets typically). In my personal opinion, the discord just shouldn't be used as it causes further issues with communication and mismatched information but then that also causes issues as people will think we are "hiding" things. Open to suggestions for this one.
I completely understand that there is no doubt some clients will be lost, but most will stay; we wouldn't have survived otherwise if we didn't have a purpose. I've only had a few months to take over at Tempest and do what we can to increase reliability, starting with redundant providers, which have already been in the pipeline and is already deployed at a few sites of course, not that it would help anyone's argument to bring that up.
I actually was a customer of Tempest and GSK before, ran my own hosting company, and own game servers. I am fully aware of the strain it brings on our customers, but without believing in the product, how can you expect it to improve? My paycheck relies on the fact that sites are online and invoices are getting paid, as you said, so why would I not do everything in my power to get things online, get customers paying invoices, and not have to hand out SLA?
We've been told by people outside the company that we've been "insolvent" for 3 years. (People that have actual no idea of the innerworkings of the company) We continue to stand here alive and relatively well. We actually have brand new sites coming online here shortly!
The 7950X price in London is truthfully outdated and is a placeholder as we decide on the best course of action for new hardware, of course, privately to those who request it, we have deployed newer/different hardware on a similar caliber for less.
Path seems to be a world record-breaker where other companies apparently screw Path over.
Office buildings, 3 different data center companies, transit providers, etc., etc. Too much coincidence. Where there's smoke, there's fire.
Doesn’t matter if it’s Neoprotect or Cosmic, the reality is Tempest lost a ton of customers. The AS-SET doesn't really mean much either, since most dedicated server clients don’t have their own ASN anyway.
Here’s what the Tempest Rust customer base looked like after those major outages across several locations:
Also, Path dropping from 110 downstreams (and it was even 120 at one point) to 67? That’s wild.
12 Jul 2024:
Now:
Good thing Rust isn't the only game in the world then, in comparison to the start of Tempest we haven't had a heavy "reliance" on Rust for quite some time. If you could please edit the Rust graph to show Path Network as a whole instead of just Tempest would probably show 90% of Rust alone doesn't even make sense.
Downstreams come and go. I can clearly state we obviously didn't lose 60 customers during the migrations, not even close to that.
Also from 10 upstreams down to 3, 2 of which are t2s.
Have you ever paid out SLA credit? I can't remember any cases in Germany.