Many random process [kworker/x] in top

jazz1611

hello,

i'm getting many random process [kworker/x] in "top -c" with CentOS 7. Will be my server is hacked or crashed?

Thank you.

rcxb

kworker processes are normal on any Linux system. That they are using up so much CPU is strange. You should show the whole TOP screen. Is there a lot of I/O wait? Most likely they are caused by disk or network I/O or interrupts because of other services, such as that mysql process.

SplitIce

kworker's are kernel [worker] threads FYI

jazz1611

@rcxb

i see everything is normal. nothing is overload.

jazz1611

Hello,

I reset the server and now it can't boot to OS. the IPMI not work too. but in email from support they show me screenshot of IPMI like this

root@rescue:~# parted -l
Model: Unknown (unknown)
Disk /dev/nvme0n1: 512GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system     Name     Flags
 1      1049kB  537MB   536MB   fat32           primary  boot, esp
 2      537MB   1073MB  536MB   xfs             primary  raid
 3      1073MB  9662MB  8589MB  linux-swap(v1)  primary
 4      9662MB  512GB   502GB   xfs             primary  raid


Model: Linux Software RAID Array (md)
Disk /dev/md4: 502GB
Sector size (logical/physical): 512B/512B
Partition Table: loop
Disk Flags:

Number  Start  End    Size   File system  Flags
 1      0.00B  502GB  502GB  xfs


Model: Linux Software RAID Array (md)
Disk /dev/md2: 536MB
Sector size (logical/physical): 512B/512B
Partition Table: loop
Disk Flags:

Number  Start  End    Size   File system  Flags
 1      0.00B  536MB  536MB  xfs


Model: Unknown (unknown)
Disk /dev/nvme1n1: 512GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system     Name     Flags
 1      1049kB  537MB   536MB   fat32           primary  boot, esp
 2      537MB   1073MB  536MB   xfs             primary  raid
 3      1073MB  9662MB  8589MB  linux-swap(v1)  primary
 4      9662MB  512GB   502GB   xfs             primary  raid

root@rescue:~# ls /mnt/boot/
config-3.10.0-1062.12.1.el7.x86_64  initramfs-0-rescue-4a08265c0d75b08098797bef5e08de11.img  initramfs-.img                          System.map-3.10.0-1127.10.1.el7.x86_64
config-3.10.0-1062.9.1.el7.x86_64   initramfs-3.10.0-1062.12.1.el7.x86_64.img                symvers-3.10.0-1062.12.1.el7.x86_64.gz  vmlinuz-0-rescue-4a08265c0d75b08098797bef5e08de11
config-3.10.0-1127.10.1.el7.x86_64  initramfs-3.10.0-1062.12.1.el7.x86_64kdump.img           symvers-3.10.0-1062.9.1.el7.x86_64.gz   vmlinuz-3.10.0-1062.12.1.el7.x86_64
efi                                 initramfs-3.10.0-1062.9.1.el7.x86_64.img                 symvers-3.10.0-1127.10.1.el7.x86_64.gz  vmlinuz-3.10.0-1062.9.1.el7.x86_64
grub                                initramfs-3.10.0-1062.9.1.el7.x86_64kdump.img            System.map-3.10.0-1062.12.1.el7.x86_64  vmlinuz-3.10.0-1127.10.1.el7.x86_64
grub2                               initramfs-3.10.0-1127.10.1.el7.x86_64.img                System.map-3.10.0-1062.9.1.el7.x86_64
root@rescue:~# mount /dev/md4 /mnt/
root@rescue:~# ls /mnt
bin  boot  dev  etc  home  letsencrypt  lib  lib64  media  mnt  opt  patch  proc  root  run  sbin  srv  sys  tmp  usr  var  vddos  vMonitor.DB  www
root@rescue:~# chroot /mnt/
[root@rescue /]# grub2-install /dev/md2
Installing for i386-pc platform.
grub2-install: error: cannot find a device for /boot/grub2 (is /dev mounted?).
[root@rescue /]# cat /etc/fstab
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
/dev/md4        /       xfs     defaults        1       2
/dev/md2        /boot   xfs     defaults        1       2
/dev/nvme0n1p3  swap    swap    defaults        0       0
/dev/nvme1n1p3  swap    swap    defaults        0       0
proc            /proc   proc    defaults                0       0
sysfs           /sys    sysfs   defaults                0       0
UUID=DB6A-640A  /boot/efi       vfat    defaults        0       0
tmpfs           /dev/shm        tmpfs   defaults        0       0
devpts          /dev/pts        devpts  defaults        0       0
[root@rescue /]#

I can boot to rescue mode but can't fix grub. Someone can help me to fix it? Thank you.

Regards,

Falzo

seeing that you use xfs, that could have been the cause for the load. scrubbing or whatever. rebooting most likely didn't help if something was trying to recover or realign.

for grub, you are simply doing it wrong. if you already mounted your boot partition to /mnt/boot , you shouldn't mount something else to /mnt then. needs to be done the other way around.
you also need to bind mount proc sys dev before you chroot into that, that's also why grub2-install is complaining and the error message is quite clear about it ;-)

hope you have a good learning curve - good luck.

jazz1611

@Falzo so in this case, what should i do get the server to normal?

jazz1611

the OVH support was fixed for my server. thank all guys.

Here are the details of this operation:
Software diagnosis
Date 2020-09-22 08:00:40 EDT (UTC -04:00), david L made Software diagnosis:
The server gets stuck during the boot phase with the message : Stuck on a GRUB issue.

The same behavior persists during subsequent boots.

A reboot on a standard OVH kernel (bzImage) corrects the situation.

The server is booted on OVH kernel and is on the login screen. Ping OK and services are up.

Falzo

for trying to fix grub, just google it. as I wrote before, you most likely mounted your root partition and boot in the wrong order and missed to also bind mount dev/sys/proc before chrooting into the system.

for the kworker load you probably have to check what's the xfs and raid status looks like, while it occurs...