All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
kvm node running very odd with raid5
I just installed a second kvm node with 3x2TB in a raid5 config. I'm using solusvm and I have one vps installed on the node and my loads are: 129.26 122.12 91.94. I know the vps isn't doing anything, it's the node. When I remove all vps from the node the cpu loads are normal around 1.50.
When I run top -c I get:
24337 root 20 0 1450m 571m 4584 S 17.0 0.4 5:15.75 /usr/libexec/qemu-kvm -global virtio-blk-pci.scsi=off -nodefconfig -nodefaults -nographic -machine accel=kvm:tcg -cpu host,+kvmclock -m 500 -no-reboot -kernel /var/tmp/.guestfs-0/kernel.24325 -initrd /var/tmp/.guestfs-0/initrd.24325 -device virtio-scsi-pc 22022 root 20 0 1449m 526m 4584 S 16.6 0.4 9:17.96 /usr/libexec/qemu-kvm -global virtio-blk-pci.scsi=off -nodefconfig -nodefaults -nographic -machine accel=kvm:tcg -cpu host,+kvmclock -m 500 -no-reboot -kernel /var/tmp/.guestfs-0/kernel.22010 -initrd /var/tmp/.guestfs-0/initrd.22010 -device virtio-scsi-pc
These are the two processes that are killing the CPU.
When I create a vps it starts with an iso mounted but when i try to install a template it doesn't start so I manually start it using:
virsh create /home/kvm/kvmID/kvmID.xml
The vps starts but only shows the iso mounted and not the installed template. I synced my templates with the new kvm node and transferred all my templates and isos to it. I can't get the new node to work, the old one works totally fine though. I installed them the same way, so my conclusion is that it's the raid5 config. Any thoughts?
Comments
Is this node at OVH?
swraid or hw raid ?
swraid and yes OVH, that is scary you asked that. Is this common with OVH?
I have to ask you one thing, do you know what are the steps to check and diagnose the current state of operation of Linux software RAID?
If yes, why did you not try them and attach their results into the first post?
If no, I hope you can reassure us you aren't actually trying to use this in production, and have no plans whatsoever of selling anyone VPSes situated on those nodes. Just to reinforce the faith in humanity in some of us. Thanks.
Perhaps you are using their custom kernel, on SYS we can choose the default kernel and I am sure on OVH will be the same.
I should've added the diagnoses for the raid config checks out 100%, so posting anything would be useless. Even soluslabs is stumped on this one as everything looks fine inside solusvm as well.
One very interesting thing I found with OVH servers and this probably won't come as a surprise but I faulted the raid config on my first attempt and when I repartitioned the drives wiping them clean with a new os install I found that remnants of the last install stayed on the newly partitioned drives.
The volume group I created on the drives during the first attempt was there AFTER I wiped and repartitioned. I told OVH about this and they said it wasn't possible.
I apologize rm_ your faith in humanity will have to be reinforced in other ways
Hmmm... interesting I will check this out. I am using OVH's centos 6.5 install so I suspect yes it is their custom kernal. I found this guide maybe I'll give it a shot, got nothin' to loose my server is unusable as it is right now.
On SYS there is a checkbox to choose the default kernel after we partitioning the drives, this options only available if we choose custom installations.
I don't see anywhere to do that in the OVH manager even when I choose a custom install, so I'm guessing it's not available.
OVH kernels should never be used if you want to host virtual servers using any platform (KVM, XEN, OpenVZ etc). And I had guessed as soon as I read your original post that the issue is most probably because of OVH kernel but I asked if it is an OVH kernel just to confirm Based on my 3+ years of experience with OVH servers I would recommend that you reinstall with your OS's native kernel instead of just trying to change the kernel as sometimes doing this could cause kernel panic & OVH would run an intervention rebooting your server with an OVH kernel again
And SYS installer does have this bug that when you reinstall/repartition it only formats the
/ partition.
Also, just to clear the doubts please post the output of
# cat /proc/mdstat
So we must destroy another partition first before reinstall it? or?
If you have a server from SYS you should do always do that until the fix their installer (remember their is no problem with the installer they use at OVH but the one at SYS only has that stupid bug)
If I do end up reinstalling I will also be running a shred to make sure everything is clear. How can I reinstall a native kernal with ovh when they only have their recipes of linux to choose from? Did I miss something?
Here is the readout, possible I missed something but everything looks okay to me.
Everything is fine with RAID so the issue is with Kernel. And you just have to check a box that you will see during the reinstallation of OS from OVH panel
What I was planning on doing is a netboot then shred, and then install centos again and before I do any work on the new installation change the kernel so that way if I mess up something I can just wipe it again.
Here are the steps in the OVH panel:
After the last step it installs, and that's it.
What happens when you choose "Install one of your templates" in the first step?
It is only a saved partition setup and nothing more. No check box.
As you have the server from OVH, you can always use iKVM to install your desired OS. Download the CentOS NetInstall & then install via iKVM using OVH mirror.
http://mirror.ovh.net/ftp.centos.org/6.5/isos/x86_64/
No need, there it is!:
It was a step I forgot about. Thanks a lot for your help K2, and everyone else who contributed. I appreciate all of your help. I will post back after I try this again, if it works OVH should stop putting their weird stuff into those kernels.
Unfortunately the original kernel did nothing to solve the issue, fun times at OVH for me!
ok, raid is rebuilding "growing". Optimize your softraid settings:
http://www.cyberciti.biz/tips/linux-raid-increase-resync-rebuild-speed.html
After it is done, load will be 0.
My raid1 servers work fine, but my server with raid5 is the problem one. What settings do you recommend for a raid5? Also, my vps are not even starting on the raid5 server. Thanks for the info.
Did you creating the LVM manually?
Yes I did, but I did nothing different from my kvm server that is running perfectly right now.
pm me with your skype i will help you with raid optimization
I had to go through paypal to get a refund for the server from ovh but I got it. I got a new one with only two drives and it's working, the only thing is the cpu loads still go a little high when performing an install.
What is a good raid.speed_limit_min?
I've just increased my raid.speed_limit_max to 500000KB and it seemed to help the loads a little, should I go higher?