New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
After Ubuntu kernel update:
iperf -c iperf.online.net
Client connecting to iperf.online.net, TCP port 5001
TCP window size: 43.8 KByte (default)
[ 3] local 213.32.0.80 port 58762 connected with 62.210.18.40 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 2.01 GBytes 1.72 Gbits/sec
That's very impressive. Thank you guys!
Thanks. I did what you guys suggested and it worked for me.
so, how is the performance with linux iso distribution? can the little arm cpu handle all those peers?
check /lib/modules for the name of the folder inside, then do ln -s /lib/modules/foldername /lib/modules/4.5.2-armada375
not an elegant solution, but did work so far for my use case...
you are totally right, the workaround is quite dirty and not really how it is supposed to be done. yet it was the fastest way to check if somehow the kernel is the root cause here.
afaik one could try to start a vnc server from rescue mode to connect to via ssh tunnel and run a netinst through it? but from what I saw the diskspace in rescue is very limited and 2GB RAM are not much either to create additional ramdisk...
or maybe try to narrow down, which kernel module is at fault and if it can be replaced manually somehow.
for why it is not working: maybe it makes a difference, if you just remove the armada kernel package or even purge everything related to it, including the headers/modules stuff. after a reboot I did find a broken link to the uImage in /boot, but I have no experience with uImage/ELF and stuff. also too lazy to just dig into it yet ;-)
as written above, the package swap most likely isn't a clean solution, yet I intend to use the box only as storage, so won't really run anything besides ssh anyways.
good enough for me, sorry for everyone if it isn't working for you and please be aware that the whole replacement might come with yet unknown side effects.
TL;DR; as you all are buying unmanaged dedicated servers, I am sure you know what you are doing here :-D :-D :-D
Yeah I'm going to use mine just for storage (I'm going to be rsyncing backups from my servers to the SYS ARM) and that's it so I won't be running anything else. Hopefully no future updates or anything mess it up.
Yes. I did. I had the same result - first I installed with the Debian 9 template and distribution kernel selected. What got installed was the ovh kernel 4.9.58-armada375 through:
$ cat /etc/apt/sources.list.d/ovh.list
deb http://last.public.ovh.hdaas.snap.mirrors.ovh.net/debian/ stretch main
when you uninstall it with
$ apt purge armada
install the Debian kernel instead with
$ apt install linux-image-4.9.0-6-armmp linux-headers-4.9.0-6-armmp
and reboot the machine, what happens is that an earlier ovh kernel (4.5.2-armada375) is booted. Where does that kernel come from? Good question.
Anyone?
On the other hand the Debian kernel doesn't appear to be used at all - neither when you do a fresh setup (with distribution kernel selected) nor when you install it manually afterwards.
My test after removing the kernel 4.9.58-armada375 kernel, installing the debian ones instead, and rebooting:
$ uname -r -v
4.5.2-armada375 #1 SMP Tue Oct 25 11:52:56 CEST 2016
$ iperf -c iperf.online.net
Now isn't that funny. Now the throughput is fine. I swear it was stuck at 5 Mbits/sec when I ran the same speedtest an hour ago.
Whatever. Perhaps this earlier 4.5.2-armada375 kernel doesn't have speed throttling. Unfortunately, it is really old (2016), and I would rather be able to use a patched kernel and to have an unthrottled network.
PS: I tried using the Debian stock kernel via KEXEC, but ovh's kernel doesn't support KEXEC.
Btw, you can see an OVH engineer talking about their patched kernel for our boxes here:
http://lkml.iu.edu/hypermail/linux/kernel/1605.0/00407.html
And there's also included the kernel patches for it:
http://lkml.iu.edu/hypermail/linux/kernel/1605.0/00407/patches.tgz
Looks like they are patching tcp.c and tcp_output.c ...
I tried but it still error'd with
But thanks for the help, im going to give debian a try and ditch ubuntu.
Soooo I see mainly gso related things - have people tried disabling gso? ^_^
After uninstalling the updated ovh kernel, what you get is an older one that's probably booted through netboot I am not sure. What's missing are the right modules now. You could install it via
$ wget http://last.public.ovh.hdaas.snap.mirrors.ovh.net/ubuntu/pool/main/l/linux-modules-armada375/linux-modules-armada375_4.5.2-4_armhf.deb
$ dpkg -i linux-modules-armada375_4.5.2-4_armhf.deb
No difference with the latest ovh kernel with gso disabled. The patches posted in the mailing list were for 4.5.2 (which is installed when we remove 4.9.58 and which doesn't have the speed throttle). I presume there are some additional patches in 4.9.58. The question is - with the patchset for 4.5.2, couldn't we recompile our own up-to-date kernel that would work here?
That thread is from May 2016, are you sure it's related?
That thread is about the 4.5.2 kernel that we get through u-boot when we uninstall the newer 4.9.58 kernel (which is automatically installed in the setup). 4.5.2 = no speed throttle, but compatible with our arm cpu. 4.9.58 = throttle. The 4.5.2 patchset could perhaps be used to make our own more current kernel that would also work with this server, since our other only option is to use 4.9.58 which has the trottle issue (and for which we don't have the patchset).
I looked into compiling an own kernel, but it does seem not so easy, as you most likely need config files for the board vendor, which obivously is OVH with some kind of development board for that marvell armada 375...
I have a ratio of 350 on the 1.8GB HBO World S02E09 720p DEFLATE torrent (added exactly 37 hours ago).
proof?
Apparently someone is already hard limited to 250Mbit, I guess that has to do that we found a wait down the rabbit hole.
Let see if more people get limited to 250Mbit.
Awesome, fixed it, thanks for the help.
Honestly. with you guys being such knobs to me earlier I almost didn't write this reply.. but feeling nice.. so here you go....
First, when you remove the kernel in use on the server after installing Stretch as you have been doing, what actually occurs (my opinion) is that their boot loader looks for their specific kernel versions in /boot and if they are not found it Netboots the 4.5.2 kernel you end up with. I have verified it is indeed the 4.5.2 kernel by trying to load modules made for 4.9.0 under it and failing.
Now, that said, here are a few things to help you out:
1. http://last.public.ovh.hdaas.snap.mirrors.ovh.net/debian/pool/main/l/
After visiting here, if you check linux-modules-armada375/ you will find most of the basic modules for the 4.5.2 kernel in a .deb, additionally, at that same link you can find the kernel headers for 4.5.2 as well. You will want to install both of these packages once changing to use the 4.5.2 kernel.
2. You can download the Linux Kernel from kernel.org:
https://mirrors.edge.kernel.org/pub/linux/kernel/v4.x/linux-4.5.2.tar.gz
Now, if you need your own kernel modules that are not already supplied with the above package, what you can do is:
Then, once saved, you can build the modules 'make -j2 modules' and they will be loadable by your kernel.
You can likely do a 'make modules_install' but my bet is it will make a new modules folder that doesn't exactly match your current kernel, in my case I simply copied the built modules into the existing '/lib/modules/4.5.2-armada375' folder and then did a 'depmod -a' which then allows you to load them with modprobe.
With this done you can create the needed xts, gf128mul modules needed for LUKS if you want and you also have the benefit of your upload working at full speed.
my 2 cents.
Cheers!
Awesome thread! Nice to see people helping each other.
out of curiosity I just reinstalled again, debian 9 and not ticking the distro kernel checkbox... guess what, box came up directly with the 4.5.2 kernel and no speed limit whatsoever.
the uImage file for 4.9.58 is available on the template and linked in /boot though. I guess that checkbox then simply decides whether the system looks for that uImage during boot or directly uses the default kernel via netboot.
that said the given hints from @TheLinuxBug esp. about obtaining the .config from the previous modules folder might come in handy on trying to compile an own kernel from scratch and replace the uImage with it...
Yes we tried with @Neoon on IRC, it only makes things worse.
As for what actually happens, as installing the Debian kernel doesn't actually boot it, it's very puzzling that doing so changed anything. However you did one more thing, which is removing the armada modules. And one guess that I have is that there may have been cpufreq modules for it, which caused the CPU to slow down too much under no load (and iperf is almost no load, right?)
Check by installing
cpufrequtils, then comparingcpufreq-infoon a "bad" server, and after the "fix". If it's indeed the issue, then simply settingcpufreq-set -g performanceshould also fix it.It would be nice if we could compile our own kernel. The 4.5.2 kernel is quite old and I wouldn't trust it to work perfectly with, for example, packages in Debian Stretch. Tried to install xtables-addons-dkms, but it failed to build modules with 4.5.2.
just running a test compilation for a 4.9.109 kernel now, as this is the newest 4.9 afaik and should still be close to the config-options of that 4.9.58 in the template.
will run through the whole process and see if any errors are popping up and what files I get from it (uImage etc.) - if that works I probably need to do another reinstall again to enable the distro-kernel again , do the whole compile stuff again (not sooo fast on ARM) and try to replace the image then.
will give an update on the results ofc ;-)
did I say I'll let it go?
cpufreq-info for default stretch template with default kernel 4.5.2, no changes to kernel or modules so far, no limiting issue though:
Both a good and bad server report the same
Whatever the actual fix was, I can get 200mbps downloading a real file from this BHS machine to Online Paris now, versus 50mbps before removing the armada modules.
aria2:
[#8a1874 3.1GiB/3.2GiB(98%) CN:4 DL:26MiB ETA:1s]You did not "remove" the armada modules simply because there aren't any. The network support is built in:
What you did is you booted an earlier kernel release from the standard one:
Uh, France is getting low on stock, so if you need one, get it now.
4 and 6TB is already gone, the 4TB is not restocked since about 1 week sadly.
Wonder if these servers can DHCP to the assigned IP... That way you could perhaps dd an ARMv7 liveCD hybrid ISO onto sda and try to boot from that.. not sure if would work though with all the bootloader limitations/oddities on these things.
Also for those asking about a light panel for these things, Froxlor or Ajenti might perform OK'ish, but not sure if compatible with ARM or not.
A couple others that look interesting:
https://github.com/netserva/hcp/blob/master/README.md
https://dietpi.com
Again, not sure if any of these would work on these SYS ARMv7 servers or not.