New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
★ VirMach ★ RYZEN ★ NVMe ★★ $8.88/YR- 384MB ★★ $21.85/YR- 2.5GB ★ Instant ★ Japan Pre-order ★ & More
This discussion has been closed.
Comments
You're the most patient person I've ever met. You must be a really nice person in real life!
Is anyone else seeing issues on SEAZ001 ?
My VPS shot up to a 1min load average of 4 then died. On reboot I get the following errors.
Error: attempt to read or write outside of disk hd0
Failure reading sector 0x7807b2 from hd0
Error: attempt to read or write outside of disk hd0
Error: attempt to read or write outside of disk hd0
Error: file /boot/grub2/i386-pc/normal.mod not found
Entering rescue mode ..
I am set to manually update, and no updates have been done since last reboot.
Mine's pending migration (at some point). Powered it up - no bootable device (seen in HTML5 console). Reinstalling Almalinux, as I type..
Not helpful. Nothing has been updated for over a month with the LA storage plan. It has been "Shipping soon" on the provided web page for too long.
Here was the update in this thread from March 23rd...
Los Angeles - 3 servers sent, 1 server racked. PDU and switch but switch requires configuration and IPv4 LOA still pending to some degree. Storage node is essentially ready and waiting to be sent with bulk servers.
.. Aye, something funky going on -won't boot up, after leaving the reinstall for a bit. Given I've asked to be moved to NYC, I'm not bothered though.
Trying to get to this as soon as possible.
Correct. As for network issue, it's not that it's throttled in any way like that directly. It's just that how a NIC functions isn't just straight 1Gbps, it depends on what's being transferred and how. It's very complicated and the simplest way to describe it is that when a high quantity of VMs are involved, all with their own virtual interface, it has to schedule everything properly and do all these tasks at once. We just have the provided NIC on the motherboard and that doesn't always do a perfect job so we had to configure it in a certain way and if it's being used by many people at once then it's possible that you cannot burst to the full maximum speed.
I've been reviewing all this for a couple weeks now and moving forward we can consider purchasing and installing a more "dedicated" NIC but that requires a lot of planning and it may not necessarily fit.
If we could we would have gone with better boards, but they literally do not exist for Ryzen. There are some that do have 10G NICs onboard, but those motherboards are confirmed to be on backorder by the manufacturer. As in, specifically those NICs were not available for purchase by the motherboard manufacturer so the only ones available while we were building these were price-gouged to be multiple times above original MSRP.
I'm doing speed test at like 3 or 4 AM JST that's like super off-peak time for Japan as well as for MJJs (only one hour time gap). Whenever speed is about to go beyond 500Mbps, it falls down to 500Mbps so I assume it's been throttled rather than simply a congestion issue.
Do the new nodes have the same issue? If not perhaps I should hop on to them as well? I saw no with-data migration option and TYOC038 seems stable enough so I'm trying to avoid migration as possible. If burst speed will be forever capped at 500Mbps I might migrate too. How long will ryzen migration button be there?
I wouldn't recommend using cryptocurrency for payments. Because there's a time limit (8 hours) which I wasn't aware of, the transaction was not confirmed before the payment timed out, so I lost my money. I had opened a ticket a month ago with billing department about this problem, but it was closed a couple days back with no reply from them.
Yes, i think drive kaput... I/O error and i see load in CPU ( i cant connect by ssh/vnc )
I am sure that VirMach did not forget about your order, VirMach mentioned that there are still orders pending particularly ones that were held for manual review. I am sure you'll get activated in the next few days aswell.
I was busy dealing with other things in my life and didn't even realize Virmach activated my JP order...
@VirMach If it's possible can you also run the script to activate pending pre-orders in Atlanta and Dallas?
I want return to Tokyo in advance，the ticket #266828 which named switch from tokyo to sanjose has been closed and can't be replied,
Those that were created on TYOC038 on the broken disk have been re-created if they were ever created. There are some that never created properly from the beginning that will be recreated next. Then I'm moving onto SJCZ003 and SJCZ007.
Oh fun! I'll add it to the list.
Looks like SEAZ001 had an XPG disk... There's about 100 of these that I have shelved right now. These are the 4TB Gen3 disks, that don't immediately break like the 2TB Gen4 disks. But it looks like they last a couple months!
Still trying to even contact XPG, their form is broken and I have no contacts, only redistributor contacts who don't care.
(edit) Good news is not many nodes have these AND that we have excess disks in general so swapping them out won't be a problem, I'll have to do a roundup and just pre-emptively pull the remaining XPG disks.
(edit2) if I had to guess I'd say maybe 5-7 total nodes with these disks.
(edit3) the XPG disk is still there on SEAZ001 so either it's only beginning to fail or it just overheated and dropped off Linux. Or something to that effect.
(edit4) should be coming back up soon, if it goes down again we know something's immediately wrong but I'll still take a further look and check it on SMART, etc.
Sorry, I was supposed to look into this sooner but then everything started malfunctioning. I'll look into this now.
We can't do these right now, San Jose is having problems.
Tokyo should be at 100% right now but I found another problem earlier today or last night, a lot of what we call "zombies" created. This is a bug with WHMCS provisioning module and/or SolusVM that dates back to several years ago and still going strong (not fixed by anyone.) It causes VMs to create duplicates of themselves sometimes. So there's a good amount of these that threw off our estimates by around 200 VMs.
This means we're short by around 200 VMs and it doesn't help that TYOC038 has a missing disk as well as one or two others. We need to schedule maintenance to add new disks to these to finish off the rest, but I'm still trying to fit as many of these as we can here and there.
We need to do a cleanup of these "zombies" or duplicates to make more room but that will take a few days. We have to do them in a way where we don't just immediately terminate them just in case someone is for some reason using the duplicate service instead of the primary one.
The perks of using 10 year old PHP software!
SEAZ001 looks like it's running into issues again as soon as the disk is thrashed trying to boot everyone up.
Good news for Seattle is that we do have more servers here that are not utilized at all that we can migrate people to if we can get it stable for at least reading off the disk. But that's going to be a daunting task.
They might have to be added to the re-create queue and people can request their data and I'll do the same plan as the others, shipping disks out and extracting them here at the office. I don't know, I'll try to provide an update today and I apologize in advance for however long this may take...
@VirMach Ryzen Special 2560 Not open, what time?
@VirMach The last Ticket
#794490
I think I'll avoid XPG disks next time building my rig.
Chia ploters where also unhappy with XPG....
btw samsung 500GB at hetzner dedi is atm at over 2500TB write no error no media wear, nothing just over 20x projected life going strong
just do not tell them or they kill me
There was a single service stuck uncreated for Denver, this has been corrected/activated.
About 15 in Dallas, corrected/activated.
About 11 in Atlanta, corrected/activated.
It's so funny, I remember avoiding XPG (then just called ADATA) something like 15 years back... and I researched them to an extreme level and I specifically bought the disks that were using good controllers and had good reviews. Still looks like they're awful, I should have just stuck to my gut on these initially. I did do an extreme amount of testing on them as well and they worked fine in all my testing... I guess it's just something about them paired up with a hundred LVMs for virtual machines doing different disk operations all at once that gets them riled up and broken.
Maybe their company philosophy is to use garbage past the first NAND and hope their customers do not use them enough to hit the warranty swap period or maybe they just decline all RMA requests and strategically break their contact forms.
arent they still called Adata? and the model is XPG8200 PRO or something
and almost bought them but they got bad issues changing controller and flash
Biennial pre-order becoming one of the last 200 to be activated is honestly a bit disappointing.
I think everyone avoided ADATA so they kept their company name ADATA and they specifically launched the "XPG" branding for their recent NVMe drives.
But the model is SX8200 Pro, and their brand name is definitely "XPG" now. Some places do advertise it as ADATA XPG and others just go by XPG SX8200 Pro for that specific model. It may still mention that the "seller" is ADATA Technology Co Ltd though.
I've already explained the misunderstanding here, somewhere in between page 100 and 160 if you want to go take a look.
I want to add onto this and say that we did still technically avoid them, their pricing and availability was just timed perfectly to sway us.
When we purchased the 4TB models, all other 4TB models were out of stock due to COVID and the ones that were in stock were being price-gouged. So we bit the bullet. Then with the 2TB Gen4 models, the Western Digital were +20% more cost and also something like 2 week backorder. So we did it again, and with our luck the Western Digital became in stock and reduced 20% in price as soon as we shipped out the XPG.