New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
★ VirMach ★ RYZEN ★ NVMe ★★ $8.88/YR- 384MB ★★ $21.85/YR- 2.5GB ★ Instant ★ Japan Pre-order ★ & More
This discussion has been closed.
Comments
@VirMach - What ever you did on Node 39 worked. Network has been stable for ~14 hours. The steal rate has gone from rather high to virtually zero. Load average has dropped to be consistent with expectations based on what I am currently running. Looking good. Thank you
Larger image
You're learning new words! Good on you!
@VirMach Seeing that TYOC034 has been configured with IPV6, will other nodes be configured successively?
Brother, I understand your hard work very much. I won't urge you when my order will be activated. However, I really find that the number of 384m activated is more than 768m. I hope it is activated according to the principle of first come, first served .
Sadly,my 768m order number is earlier than my friends' activated 384m,but it is still pending.
I apologize if I'm said wrong! Do you guarantee you are not discriminating against Chinese customers?
can u tell about the reason why u think that VIrmach are discriminating against Chinese customers?
Please stop participation in spreading rumors.We have never discriminated against MJJ or Chinese customers.
There is no point in talking to you, as long as the boss's guarantee
You misunderstood me,
Your question is the same as asking nothing.
Regardless of whether VirMach discriminates against Chinese customers, he will say "I do not discriminate against customers from China."
Therefore, he will most likely choose not to say anything.
Tokyo Update For Existing Services - Thorough run-down of all nodes.
TYOC033
32 virtual servers are offline. There is a disk error similar to previous errors that was fixed on all nodes. I need to check further and resolve this.
TYOC034
Only 1 virtual server is offline and needs to be recreated. Three services appear to be maxing out CPU for extended periods of time and may be in kernel panic.
TYOC035
30 virtual servers are offline. There is a disk error similar to previous errors that was fixed on all nodes. I need to check further and resolve this.
TYOC037
5 virtual servers are offline. But they do not need to be recreated, it's just most likely not installed OS or powered down by the user. This has three virtual servers maxing out CPU that may be in kernel panic.
TYOC039
26 virtual servers are offline. Only 2 appear to require recreation. The rest are most likely mostly power-offs from kernel panic issue and just need to re-install OS and/or power them on. One services appear to be maxing out CPU for extended periods of time and may be in kernel panic.
TYOC040
61 virtual servers are offline. Zero need to be recreated. I assume most are power-offs from kernel panic issue, but this is an alarmingly high number powered off. In addition, one server seems to be maxing out CPU and this server feels laggy enough though the load is low, so we may need to look into it further at this point if powering down the single person maxing out CPU does not alleviate the problem.
Here's a list of what's done or being done based on the above information.
Maintenance:
TYOC033 and TYOC035 will require a reboot. Everyone will be emailed for emergency maintenance after I confirm the problem. It's most likely settings that did not properly write to the kernel parameters, or BIOS options that need to be changed. For these two, we'll skip any powerdowns for OS kernel issues for now.
The disks that ran into issue were XPG Gen4 on both TYOC033 and TYOC035. We may just need to apply a more aggressive patch. For future reference no further nodes being sent to Tokyo or anywhere will be using XPG Gen4 for the time being.
Anti-abuse cleanup:
Three virtual servers powered down on TYOC037. Four Three virtual servers powered down on TYOC034. One virtual server powered down on TYOC039. One virtual server powered down on TYOC040. This was monitored over a period of 15+ minutes to avoid false positive as much as possible but we understand the powerdowns are abrupt, so all these people have been messaged via ticket.
After these poweroffs, the nodes calmed down by approximately:
30% load - TYOC039.
60% load - TYOC034.
10% load - TYOC040.
15% load - TYOC037.
TYOC040 still feels laggy, I need to look into this further. May be related to other operating systems online that are incompatible but not necessarily overloading CPU by maxing out. Next step is to power down anyone still on the default CentOS 5.8 selection.
Recreations:
TYOC033 and TYOC035 need to be rebooted first before recreations. TYOC034 has had the single virtual server that needed to be recreated, but I actually don't see why this VM exists, it's possible it was generated as a duplicate on accident by the system. TYOC039 only needed only recreated for now, and I think this was belonging to @FrankZ so it's probably from the beta, and I'm deleting it. Let me know if you need it created but it looks like it's missing from WHMCS and just a duplicate. One other seems like something else wrong, keeping it open for now to decide what to do.
All operating systems are being tested.
I'm going through and making sure all plans on all servers have the correct OS groups available. All of these will be tested, on both 384MB plan and 2560MB plan. This will take some time.
Suspensions.
The next time there are powerdowns, we will be suspending the service instead. But don't worry, this is just a suspension to get your attention to re-install/fix your OS. These will happen when TYOC033 and TYOC035 are rebooted as I suspect a lot of VMs that were going into kernel panic will come back online and still have the problem. We'll work with you in resolving the OS issue. You can create a priority ticket if you are suspended, without any additional charge, and there won't be any penalties on your account.
Any further updates on this thread will be provided in a few hours at the very soonest. A lot has to get done. Please do not create any tickets about it at this time, it will not be helpful if we're already aware of the issue.
We have no reason to discriminate against any customer. I'm a bilingual immigrant and we have someone hired currently of Chinese origin. We have a lot of customers from China, and a lot of Chinese customers who are from the US, including some of our largest customers. The two nodes you mentioned are not even originally used for placing any of the impatient customers we originally described, and as you mentioned yourself, "MJJ represents the majority of consumers who are looking for value for money no matter what country!" We just joked about one node being the "MJJ" node, because we placed all the people that requested to be activated early and were impatient and created tickets on one, including those that were rude. A lot of these people were self-proclaimed "MJJ." That is the only category we used for that, not race or country. This included everyone from multiple countries who created a ticket or were impatient, and the idea was scrapped. This node had no limitations. This node was TYOC034, which is ironically the only node people did not complain about since I never filled it correctly and got rid of that idea to place people who are impatient on the same node. I just thought it was a funny joke to place everyone who was impatient on one node because they would all be neighbors and complain about eachother. This never actually happened because the node was not filled.
It seems like these people still got upset though and spread rumors about every node being the "MJJ" node and throttled, which is not true. Then others further spread the rumor and said we are specifically discriminating against Chinese customers, which is also untrue.
Please have a look of Ticket #772256, The VM offline too.
Regardless of whether VirMach discriminates against Chinese customers, he will say "I do not discriminate against customers from China."
Therefore, he will most likely choose not to say anything.
I don't like remaining quiet, even if speaking further may cause further confusion if people misunderstand me.
And to be clear, if we did discriminate against customers based on their country, then we would obviously take actions to make that known as in that case we would obviously not want them as customers? We've had plenty of "actions" done that were quite the opposite, and only made it easier for customers of that region and all regions to purchase and use our services. We're a global company. We're an internet company. We do not believe we can be a cloud hosting company and at the same time cut off an entire portion of the internet from using our services, it just doesn't make sense, nor would we want to do that in any case.
We will, however, always poke fun at any customer who breaks our rules, spams, tries to cut in line, etc.
Please have a look at the long message I just posted which already addresses your issue.
OK. Thank you for your reply, I apologize for my over the top comments, I just want every customer to be treated equally, no matter what country he is from! I will shut down my VPS and stop pushing you until the VPS is working!
That's good to hear. That means since this is one of the older nodes with mostly older customers, it's stabilized and others will soon follow. TYOC033 and TYOC035 have similar disk issues and should be stable afterward as well.
We did also change the uplink to 40Gbps which lowered congestion, and this node is not having strange high interrupt level like TYOC040.
Right now on my end, I see every node is able to use the network properly, with ALL nodes using networking up to the FULL port speed of the server in the last few hours, indicating that there aren't any network issues. This even includes TYOC040, but I do see that TYOC040 uses less networking than the others which coincides with the weird issue it's facing. As in, TYOC040 did use full network speed around the time @bluecat mentioned it, and then around the time he said it got worse, it did lower. I've known about this being a potential problem for a day or two now since @bluecat started reminding me several times, and last time it improved, someone was powered down. However, this time it seems like the powerdown did not help as much. I'll figure it out. This node may just require powerdowns of CentOS 5.8 and/or a reboot.
Hi @VirMach
Can you answer when will my order be approved
I placed it on 7th April
I have already asked you 2 times before but haven't received your answer yet
My Invoice ID is #1414080
Thank you in advance !
I closed the ticket and waiting emergency maintenance finished.
My VM at TYOC033 is offline after reinstall OS yesterday, and I am not receive the active email untile now.
@kaokao2222 - If that is the 4GB VPS from the current sale your speaking of, you need to post your order number on the other forum to be activated.
EDIT: If you are unaware of how to do that let me know.
Incase someone who using Translate that might not fully translate correctly.
Here is the optimized translation from Virmach's announcement about MJJ Node.
我们没有理由歧视任何客户。我是一个双语移民，我们目前有雇用一位华人员工。我们有很多来自中国的客户，也有很多来自美国的中国客户，包括我们一些最大的客户群。你提到的这两个节点，根本不是用来安置我们原来所说的任何一个没有耐心的客户，正如你自己提到的，"MJJ代表了大多数消费者，他们无论在哪个国家都在寻找性价比！" 我们只是开玩笑说有一个节点是 "MJJ "节点，因为我们把所有要求提前激活VPS和不耐烦并提交了工单的人都放在一个节点上，包括那些粗鲁/不礼貌的人。这些人中有很多是自称 "MJJ "的。这是我们使用的唯一标记类别，而不是种族或国家。这包括来自多个国家的每个人，他们提交了工单或不耐烦的客户，但这个想法已经被取消了。这个节点没有任何限制。这个所谓的“MJJ”节点是TYOC034，但很讽刺的是，这节点是唯一没有人抱怨的节点，因为我从来没有正确地填满服务器的使用量，并摆脱了这个“MJJ专属节点”的想法，把那些没有耐心的人放在同一个节点上。我只是觉得把所有不耐烦的人放在一个节点上是个有趣的笑话，认为他们都会成为邻居并互相抱怨。但这实际上从未发生，因为该节点没有被填满MJJ用户。
By Virmach's
Yes, bro. That's a Special 4096 Order. Do you mean that I have to post it to LowendSpirit, right
Btw, how can I find the order number of my order, bro
I can only see the Invoice ID
The order number is only displayed on the order confirmation page when you did the order. It's ok, Virmach will most likely activate your order with the Invoice ID if you post it in the VirMach sale thread on LowendSpirit.
Thank you so much, bro !
I will do it right now
Hey friend, after you confirmed that you didn't reply to my ticket for 24 days? Is there anything you want to tell me in the future?
TYOC035 and TYOC033 update:
These are being rebooted now in 10~ minutes, and will have two disks taken out of the disk pool and VMs rebooted. The XPG disks are receiving another error. They will be replaced by Samsung Pro or WD or equivalent. We don't suspect anyone to have data loss as it appears the disks failed relatively quickly. Once again, we're throwing all these disks out of the servers we're sending to the datacenters so hopefully we won't have further issues with them.
So to confirm, this is not the same issue as before, it appears it is a new issue related to the disks failing. If anyone does have important data on it that they need to retrieve, the disk will be kept and sent back to us so we can likely still recover data, just open a ticket.
This means that we just lost some space in Tokyo. We're sending more servers this week, and we should still get close to all deployments as I'm setting up another server now and waiting for IPMI reset by the DC on another. I'm checking with the DC if they have the other disks we sent on-hand so we can put the new disks in these new nodes and avoid further problems but they may be at an office for storage away from the datacenter and it's the weekend. We'll see. (Later on we'll streamline this process.)
Everyone "offline" on these will be recreated on another node.
It looks like both of these disks filled 300GB exactly before dying, fun. But that means not that many people affected either.
Is it just me or Alma's NetworkManager really wasting unreasonably high cpu just for sitting there?
I have to uninstall it and use the good old network-scripts to let my load to idle at the expected level.
What's your question? It was already answered in the ticket, I'm wondering what you'd like clarified. You made a ticket asking for a refund to change your service to another, so someone (not me personally) replied and processed it as a refund to store credit so you could change plans. Then later you changed your mind and wanted a refund to your original payment method which was not offered.
Next time if you want a refund to original payment method there's no need to mention purchasing other service as that makes your intentions unclear and could result in it being processed in the way you said and not the way you felt.