New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
this.
I really like @shovenose for starting this thread, and would also like to thank the people here who replied back without being sarcastic or telling him that "if you don't know ...." you shouldn't be a host or the silly kids stuff.
No one is perfect, but at least he came out and asked for help the right way.
Good luck @shovenose
Regards
@rsk thank you, I think you just made me smile
I learned a lot this thread, I will never ask these questions again, and if I can help somebody if they have the same question, I would most certainly do my best to do so.
Yep. Props to no one for being sarcastic or an ass. I picked up some stuff too
A difference here is that @shovenose actually listens. A lot of the people in the past who have asked will ask for advice/information, then retort with "well you're WRONG and i'm going to do it this way anyway", even though we've been there/done that.
True, true. But I admired that he didn't "hide". Kudos to that
Well this isn't resolved after all.
Noticed high load and decided to check it out. Damn thing's RAID is messed up again.
at this point I'm fairly certain one of the drives is bad, even though SMART is fine. Agreed?
Personalities : [raid1]
md2 : active raid1 sdb2[1] sda2[0]
1932012480 blocks [2/2] [UU]
[=>...................] check = 7.2% (139808000/1932012480) finish=471632.7min speed=63K/sec
20971456 blocks [2/2] [UU]
resync=DELAYED
[root@openvz1 ~]# smartctl -H /dev/sda
smartctl 5.42 2011-10-20 r3458 [x86_64-linux-2.6.32-042stab068.8] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net
SMART overall-health self-assessment test result: PASSED
smartctl 5.42 2011-10-20 r3458 [x86_64-linux-2.6.32-042stab068.8] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
16:28:17 up 4 days, 19:16, 1 user, load average: 7.33, 7.38, 7.03
[root@openvz1 ~]#
No. Software raid1 is that terrible for production; we used it for a short while on shared web hosting, but it was just not a viable solution.
What's the output of mdadm --detail ?
@Damian
mdadm: No devices given.
at this point I'm fairly certain one of the drives is bad, even though SMART is fine. Agreed?
Nope, Linux RAID1 does regular checks on the consistency of the two disks, basically verify the mirrored data of both disks to ensure that it's the same. Notice the "check" instead of "resync".
It's pretty normal for software raid 1 to basically die like this during a check, your two disk setup is simple insufficient on the IO level to handle full load of regular disk processing + a rebuild/verify. It's why I basically stopped using software raid for all sort of VM use.
Google "speed up mdadm resync", I'm on my phone atm but it allows you to set the minimum / maximum sync speeds. If RHEL 6 turn bitmaps on, not sure if it will let you turn this on during a sync.
@Kenshin
OK, well shit that sucks and I can't do anything about it. Can I cancel this check somehow?
Increasing sync speed probably won't help much since the CPU is already through the roof. Enabling bitmapping should help a little, but again the full CPU is going to make things hard.
Can't remember how to cancel, not even sure if it's possible. I quickly moved away from software raid when I ran into this so the memory is vague.
I looked it up, it's possible, but I would rather not screw it up and make it start over.
Besides, it's going a bit faster now.
Personalities : [raid1] md2 : active raid1 sdb2[1] sda2[0] 1932012480 blocks [2/2] [UU] [===>.................] check = 15.8% (306156416/1932012480) finish=292.6min speed=92608K/secIndeed, you have to specify a device.
mdadm --detail /dev/md2
Wow I didn't even know this about mdadm.
@shovenose If you're looking for a replacement server then just drop me a PM. I can have something racked and ready by morning/mid afternoon.
@Damian:
/dev/md2:
Version : 0.90
Creation Time : Sun Dec 23 12:33:14 2012
Raid Level : raid1
Array Size : 1932012480 (1842.51 GiB 1978.38 GB)
Used Dev Size : 1932012480 (1842.51 GiB 1978.38 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 2
Persistence : Superblock is persistent
Update Time : Sun Feb 3 17:27:48 2013
State : clean, checking
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Check Status : 19% complete
UUID : 641aaab6:e4182967:a4d2adc2:26fd5302
Events : 0.111
Number Major Minor RaidDevice State
0 8 2 0 active sync /dev/sda2
1 8 18 1 active sync /dev/sdb2
[root@openvz1 ~]#
about 16
less than 50GB
@Jack correct
http://www.ovh.com/us/dedicated-servers/sp_16g.xml
It's that.
@Jack even DataShack is too expensive at the prices I sell VPS. Budget isn't there to get something better, until I get orders enough to fill up a couple nodes.
Once all 32 IPs on that server are used up, I'll get my next node at WholeSaleInternet with Software RAID10 that should be good right?
Well if RAID1 is fine then what is the problem with my server?
Not at all trying to be rude;I am simply confused.
On a different note, finally this is going better!
17:54:14 up 4 days, 20:41, 1 user, load average: 1.00, 1.04, 1.16
@Jack 4 x 1TB R10.
It's a E3, pretty sure they all are.
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Check Status : 19% complete
This block here gives you all the info you need:
It has two devices
Both of them are working
The 'normal' integrity check is in progress
It's 19% done.
The problem is simple, when Linux does a verify, it uses up both Disk IO and CPU. With 16 VMs running, you quickly run into IO and CPU issues because the VMs are trying to fight for IO/CPU with the checking.
Software RAID is good when you're not stretching the disk IO like for webhosting use, otherwise always hardware RAID. Hardware RAID does the verify on the hardware level, and uses it's own on-chip CPU for comparing, not to mention also provides cache for the RAID array.