New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
I tried to do a
lsin a folder on /storage with 3 or 4 files in and it took around 45 seconds.same here.
/storage partition is barely usable.
Just writing a simple text file on it, took almost 1 minute.
Same here.. benchmark tests won't even start & node was rebooted automatically
Hello , today we fixed problem about I/O , lot of people make transfer files and ARC it's on full load , for fix problem only need make a ram memory upgrade
Regards
Shouldn't you install enough RAM before commissioning?
Or is the problem only that too little ARC was assigned?
I am partly not sure how exactly your statements are to be understood.
>
I had installed default 256 GB ram, but after I enabled dedup in ZFS Config,ram consumption almost doubled
Regards
fio Disk Speed Tests (Mixed R/W 50/50):
Geekbench 6 test failed. Run manually to determine cause.
YABS completed in 59 min 55 sec
I would advise you not to test such customizations on systems with customers, even if this causes some costs.
Even at your prices, I can't understand how anyone should really use your service if there are always such extreme problems (and this has been going on for a long time now).
>
I tested dedup on a test node, everything works perfectly and HDDs no longer disconnect from raid, so I moved this configuration to production nodes
The only thing that still needs to be done is upgrade to RAM
Regards
Whats the ETA?
>
We try in next 2 hours finish this process
Regards
I just copied a 13 MB file to the "big disk". It took almost two and a half minutes that is, the speed was solidly below 100 KB/s.
@Calin I didn't buy a ticket for an adventure trip but for a storage VPS!
I'm increasingly under the impression that you got it the wrong way around. First a reasonable provider gets their act together and their node working properly and tested. Only then they start sales!
What's the ETA on the RAM upgrades?
Exactly my point, we where expecting a solid system to store backups etc not a server that is slow and offline most of the time.
Why use ZFS if you have little experience with it...
my whole vps is barely usable right now.
Was expecting some issues, but not that much troubles, honeslty
Certainly not to gloat, but weren't the price and... provider's background indicative of at least some adventure ahead?
>
We know this and from next week we will have a new employee who will take care of the ZFS side , Because remote work is not the best solution, there will be someone who will work physically , for test HDDs , ZFS configurations , recommendations etc...Because we plan to rely heavily on these storage offerings and build more PBs in future , At least until we put the servers on colocation it will be a full-time physical employee
Regards,
Calin
Agreed, but (a) the pricing made it seem highly likely that many LETters would buy anyway, and (b) I, possibly, even likely wrongly assumed that @Calin had learned his lesson after his past, uhm, not exactly smooth sailing.
I agree, I purchased the VPS with reasonable expectations with possibly:
I did not expect anything more and set my expectations reasonably and jumped into an exciting adventure!
@jsg
A bumpy ride is more exciting than a smooth sailing
Guys order the cheapest storage on the internet and complain about quality
Hang in there @Calin but keep us posted..
521,699,328 5% 30.47kB/s 80:55:43
avg-cpu: %user %nice %system %iowait %steal %idle
0.00 0.00 0.13 74.84 0.13 24.91
Hello @nullptr it's possible give us more screenshots about ? That it s very wired , the main node CPU it s maximum 20% usage and I/O it's 16%
https://uptime.ihostart.com/report/uptime/e91598904c79c0f4ac6b5c2a2851e715/
Any other people have this problems?
@nullptr possible share htop and iotop screenshots
I can't even run htop in tmux. Otherwise, it will freeze. And my VPS got reboot automatically about one hour ago, don't know why.
When I found things are going wrong and htop will freeze. I ran iostat, and then reboot. Now I completely lost access after rebooting it 10 minutes ago.
Update:
20 minutes after rebooting, SSH port is still closed. Can't access.
@Calin any update on the ram upgrade? i have disk io of 100kb/s (without network)
I've rebooted mine too and the VPS doesn't respond to ping.
(fstab OK, VPS has already been rebooted)
Just before there are IO waits
To properly deduplicate 0.5 PB of data, you'd need many TB of RAM, something like 10 TB, or even more to account for various other ZFS caching, which is quite required for good performance. With way less RAM installed than what is required, you're now probably deduplicating through disk (instead in RAM) which is what brought disk performance to a crawl. It's a miracle it works at all!
To make matters worse, once you turn ZFS deduplication on, I think it's not possible to turn it (completely) off anymore, because on-disk format has changed, and deduplication tables are required to access data.
Which is basically a reminder to everybody that this experiment on production service will probably end very badly (unless some TB of RAM can be obtained and installed quickly?). Have your backups ready.
Unfortunately I did a reset, now it is not booting up anymore, console will not run. I was even not able to mount the /storage partition before my reset.