New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
XEN Full virtualization -- unstable network: overload?
Hi
I have got new VPS XEN Full virtualization server / 100Mb port
What I am seeing for the third day is low bandwidth rates (lower 5MB/s) and 15ms latency to DE-CIX. Also see large StDevs on every hop -- the network is very unstable, (it is Accelerated based, so I guess uplink is not a problem). Do you think the problem is with the host machine?
============================================================================================================== Sat Aug 8 01:59:01 GMT-3 2015 + mtr -rwc10 decix.de Start: Sat Aug 8 01:59:01 2015 HOST: XXXXXXXXXXXXXXX.de Loss% Snt Last Avg Best Wrst StDev 1.|-- XX.XX.XX.XX 0.0% 10 5.8 3.4 0.2 6.7 2.7 2.|-- 82.211.0.1 0.0% 10 12.1 10.5 0.3 48.8 14.5 3.|-- fra3.xe-0-0-0.accelerated.de 0.0% 10 11.7 7.9 0.4 16.9 7.0 4.|-- sgw2-te-0-0-2-3-ixp.fra.de-cix.net 0.0% 10 13.8 8.4 1.3 21.5 6.8 5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 + mtr -rwc10 4.2.2.2 Start: Sat Aug 8 01:59:12 2015 HOST: XXXXXXXXXXXXXXX.de Loss% Snt Last Avg Best Wrst StDev 1.|-- XX.XX.XX.XX 0.0% 10 6.7 5.8 2.7 8.4 2.1 2.|-- 82.211.0.1 0.0% 10 8.0 17.2 8.0 58.1 14.6 3.|-- xe-4-2-1-311.edge4.Frankfurt1.Level3.net 0.0% 10 13.4 12.4 8.3 16.1 2.5 4.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 5.|-- b.resolvers.Level3.net 0.0% 10 13.8 14.0 9.9 21.3 3.3 + grep SUM + iperf -c ping.online.net -i 1 -P 10 -f M -t 5 [SUM] 0.0- 1.0 sec 7.50 MBytes 7.50 MBytes/sec [SUM] 1.0- 2.0 sec 5.00 MBytes 5.00 MBytes/sec [SUM] 2.0- 3.0 sec 4.50 MBytes 4.50 MBytes/sec [SUM] 3.0- 4.0 sec 4.62 MBytes 4.62 MBytes/sec [SUM] 4.0- 5.0 sec 4.62 MBytes 4.62 MBytes/sec [SUM] 0.0- 5.6 sec 27.5 MBytes 4.88 MBytes/sec + dd if=/dev/zero of=/root/test bs=64k count=16k conv=fdatasync 16384+0 records in 16384+0 records out 1073741824 bytes (1.1 GB) copied, 10.5804 s, 101 MB/s ============================================================================================================== Sat Aug 8 02:59:01 GMT-3 2015 + mtr -rwc10 decix.de Start: Sat Aug 8 02:59:01 2015 HOST: XXXXXXXXXXXXXXX.de Loss% Snt Last Avg Best Wrst StDev 1.|-- XX.XX.XX.XX 0.0% 10 0.2 4.6 0.2 7.6 2.3 2.|-- 82.211.0.1 0.0% 10 0.8 11.2 0.8 21.8 5.1 3.|-- fra3.xe-0-0-0.accelerated.de 0.0% 10 0.4 12.6 0.4 17.4 5.1 4.|-- sgw2-te-0-0-2-3-ixp.fra.de-cix.net 0.0% 10 1.3 13.1 1.3 16.8 4.4 5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 + mtr -rwc10 4.2.2.2 Start: Sat Aug 8 02:59:11 2015 HOST: XXXXXXXXXXXXXXX.de Loss% Snt Last Avg Best Wrst StDev 1.|-- XX.XX.XX.XX 0.0% 10 7.1 3.9 0.1 9.0 3.9 2.|-- 82.211.0.1 0.0% 10 16.3 8.7 0.3 16.3 5.7 3.|-- xe-4-2-1-311.edge4.Frankfurt1.Level3.net 0.0% 10 10.6 7.6 0.6 15.5 5.6 4.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 5.|-- b.resolvers.Level3.net 0.0% 10 12.7 8.2 0.7 18.4 6.9 + grep SUM + iperf -c ping.online.net -i 1 -P 10 -f M -t 5 [SUM] 0.0- 1.0 sec 7.50 MBytes 7.50 MBytes/sec [SUM] 1.0- 2.0 sec 5.00 MBytes 5.00 MBytes/sec [SUM] 2.0- 3.0 sec 4.25 MBytes 4.25 MBytes/sec [SUM] 3.0- 4.0 sec 4.62 MBytes 4.62 MBytes/sec [SUM] 4.0- 5.0 sec 4.62 MBytes 4.62 MBytes/sec [SUM] 0.0- 5.5 sec 27.2 MBytes 4.91 MBytes/sec + dd if=/dev/zero of=/root/test bs=64k count=16k conv=fdatasync 16384+0 records in 16384+0 records out 1073741824 bytes (1.1 GB) copied, 22.7015 s, 47.3 MB/s ============================================================================================================== Sat Aug 8 03:59:01 GMT-3 2015 + mtr -rwc10 decix.de Start: Sat Aug 8 03:59:01 2015 HOST: XXXXXXXXXXXXXXX.de Loss% Snt Last Avg Best Wrst StDev 1.|-- XX.XX.XX.XX 0.0% 10 8.1 6.3 2.0 9.1 2.0 2.|-- 82.211.0.1 0.0% 10 12.4 13.1 8.7 21.8 3.5 3.|-- fra3.xe-0-0-0.accelerated.de 0.0% 10 14.3 13.8 9.1 17.8 3.0 4.|-- sgw2-te-0-0-2-3-ixp.fra.de-cix.net 0.0% 10 9.6 11.8 8.9 17.8 2.9 5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 + mtr -rwc10 4.2.2.2 Start: Sat Aug 8 03:59:12 2015 HOST: XXXXXXXXXXXXXXX.de Loss% Snt Last Avg Best Wrst StDev 1.|-- XX.XX.XX.XX 0.0% 10 0.1 5.5 0.1 10.2 3.5 2.|-- 82.211.0.1 0.0% 10 0.3 10.2 0.2 15.3 5.4 3.|-- xe-4-2-1-311.edge4.Frankfurt1.Level3.net 0.0% 10 0.6 13.8 0.6 36.0 11.3 4.|-- ae-1-60.edge5.Frankfurt1.Level3.net 10.0% 10 0.7 11.3 0.7 19.0 6.7 5.|-- b.resolvers.Level3.net 0.0% 10 0.7 10.2 0.6 16.0 5.6 + grep SUM + iperf -c ping.online.net -i 1 -P 10 -f M -t 5 [SUM] 0.0- 1.0 sec 14.6 MBytes 14.6 MBytes/sec [SUM] 1.0- 2.0 sec 11.1 MBytes 11.1 MBytes/sec [SUM] 2.0- 3.0 sec 11.1 MBytes 11.1 MBytes/sec [SUM] 3.0- 4.0 sec 5.75 MBytes 5.75 MBytes/sec [SUM] 4.0- 5.0 sec 4.88 MBytes 4.88 MBytes/sec [SUM] 0.0- 5.6 sec 48.8 MBytes 8.74 MBytes/sec + dd if=/dev/zero of=/root/test bs=64k count=16k conv=fdatasync 16384+0 records in 16384+0 records out 1073741824 bytes (1.1 GB) copied, 27.411 s, 39.2 MB/s
Comments
How the fuck should we know? Contact the support of the provider!?
Wait what? This isn't support? They told me to contact LowEndHelpDesk first hand before anything. Did no rite place? /troll
Did you only see this via DE-CIX?
We had a customer report some packet loss via DE-CIX last night (but we couldnt replicate it ourselves via any of our monitoring servers, and our upstream couldnt relate it to anything on their network).
But @nexusrain is right, you should be contacting your host with these MTRs.
It is almost constant. Here are the latest results:
That was the first thing I did, asking support. I started tracert-ing de-cix because asked by the support. Before i was testing other destinations with the same results.
Chatting with support the third day about the problem, provided all logs, started tracerouting de-cix as asked, at the end was told it is OK, getting much more stable results on vpns from 23Media and filoo: 1-2ms to de-cix and 10ms less behind it to the same destinations and 10MB/s throughput. ip-projects vpn seems hugely overloaded (or misconfigured), but their support says results are normal, I am not sure
You can't compare ip-projects or one of their resellers to accelerated directly, I had similar issues a while ago (2 years) and ended up speaking to accelerated directly they were actually pretty annoyed that the reseller of a customer was suggesting there network setup was the same as using accelerated directly, they gave me a free test server for a week to prove the point and the difference was insane.
That said.. What nic type are you using, have you installed PV on HVM drivers and has your host enabled that on your HVM vps and do you see the same latency results over tcp, what is the host node connected at?
ip-project uses XEN full virtualizatoon
As such, I thought there were no need to install drivers.
Since I get sometimes normal results like these
there should be no problem with setup, just a banal overload of the host machine most of the time.
Hello,
How is host load average ? Overloaded cpu might impact everything including disks and network performance.
Edit : If the host load average is greater than 1 that will not change HW performance but that can impact vm performance so checking with #top command on the host is a good start i think.
Well sorry but you are wrong.
I have no access to host console, I am vps customer. I have been testing three different vps providers last week, each has its ups and downs, will post results soon.
That does not prevent you from checking your nic type and block drivers etc, in solusvm and other panels I have used its fairly standard to be able to change or update this as a customer.
It does not make sense either.
Anyway I am done with testing. Here are my observation about three vps providers tested:
Rootwelt (23Media)
Pros: Good uplink. The network showed best results for my location and was stable. They say they put no limit on traffic so I was afraid that users with large usage would be attracted, but in reality I did not noticed any problems with the network.
Cons: One and huge: distros. OS images they provide are modified. In debian there are 100kb of junk in ENV which you see as output of “set” command. They assure you that this is “clean minimal installation”, but in fact tampered even with aptitude
and do not allow you to make clean install from official distribution, when asked. not good
Ip-projects (Accelerated)
Pros: Good uplink. Clean minimal installation. XEN full virtualizaton. Seem to follow the standards.
Cons: Host machine is largely oversold/overloaded and they won'tt admit it. Do not expect stable network performance from them, could see hdd rates go down to 30 MB/s more than once.
Filoo (Filoo)
Pros: Good uplink. Have seen no problem with the load of the server at all. Was able to attach installation image myself and install distro from scratch myself (image they provide was also modified). Console works with internet browser without java, which is a very very nice thing, provided that in emergency situation when you will need console, there might be no ssh client at hand.
Cons: More expensive. Less traffic included in the price. Also I noticed with tcpdump the packets from other subnets coming to my interface, which does not hurt much of course but was kinda weird.
Downgrade to lower version of Xen and then recheck.
Make sure setup and configuration of Xen server is correct.
I am using Xen and never face any issue but you can try to get a help from Xen community forum. They can give you suggestion.
Most of LET users use KVM.
Well as your refusing to look into your own servers setup for some reason and keep saying full virtual like that's a catch all I can only take your results with a pinch of salt.
The reasons were all given above, including the fact that connectivity problems are not constant, although periodic. When they coincides with other indicators in vps which suggest that the problem lie beyond only network drivers (such as low hdd rates) I can see no point in looking into them, but I am thankful for your suggestion.
One comment: the issue with network misconfiguration @Filoo seems to be a bigger problem. I can tcpdump udp and tcp packets addressed to other machines, even voip protocol included with passwords etc. Thats SERIOUS. And they have not been able to fix IT in more than two weeks. Very VERY bad
Who is curious: these are packets to/from addresses neither of which is mine.
Well, is it any good???
^C