High latency on RareCloud.io BF 11 EUR server

Comments

• ___XbNz___ Member

  December 2024

  Another update on this for those who are not part of the reflexive hoster defense brigade.

  The source of the following ICMP results is the server itself. The destinations are IP addresses on the same(!) /24 CIDR. We can now rule out the theory of the upstream provider as the cause, pointed out by @d2411 as a potential explanation:

  @d2411 said: Wiit looks like my loc

  2.57.123.179 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 3.80/75.8/257
  2.57.123.180 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 3.56/76.3/257
  2.57.123.181 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 3.49/78.7/273
  2.57.123.182 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 3.33/79.3/286
  2.57.123.183 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 3.19/79.7/286
  2.57.123.184 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 3.11/80.7/286
  2.57.123.185 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 2.97/80.7/286
  2.57.123.186 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 2.82/80.6/285
  2.57.123.187 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 3.53/80.5/285
  2.57.123.188 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 2.52/82.5/288
  2.57.123.189 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 2.37/82.2/285
  2.57.123.190 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 2.25/82.3/285
  2.57.123.191 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 2.20/82.4/285
  2.57.123.192 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 1.95/82.3/287
  2.57.123.194 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 1.64/82.2/287
  2.57.123.195 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 1.53/82.1/287
  2.57.123.196 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 1.38/82.3/294
  2.57.123.197 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 1.25/82.7/294
  2.57.123.198 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 1.14/82.6/294
  2.57.123.199 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 1.02/82.4/294
  2.57.123.200 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.951/82.3/294
  2.57.123.201 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.876/82.1/294
  2.57.123.202 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.817/82.0/294
  2.57.123.203 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.764/81.9/293
  2.57.123.204 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.652/81.8/293
  2.57.123.205 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.540/81.7/293
  2.57.123.206 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.509/81.7/293
  2.57.123.207 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 0.456/83.1/293
  2.57.123.208 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.446/81.6/293
  2.57.123.209 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.407/81.5/293
  2.57.123.210 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.313/81.6/292
  2.57.123.211 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.225/81.1/292
  2.57.123.212 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.262/81.3/292
  2.57.123.213 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.306/81.2/292
  2.57.123.214 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.243/83.2/292
  2.57.123.215 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 0.263/82.9/292
  2.57.123.216 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.238/83.1/292
  2.57.123.217 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.323/85.9/292
  2.57.123.218 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 0.311/85.9/292
  2.57.123.219 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 0.302/85.8/291
  2.57.123.220 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 0.306/85.7/291
  2.57.123.221 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 0.287/81.0/291
  2.57.123.222 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 0.223/85.4/291
  2.57.123.223 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 0.300/85.3/291
  2.57.123.224 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 0.215/81.5/291
  2.57.123.225 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 0.691/86.0/291
  2.57.123.226 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 0.629/86.1/291
  2.57.123.227 : xmt/rcv/%loss = 20/18/10%, min/avg/max = 0.590/83.2/296
  2.57.123.228 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 0.482/88.1/295
  2.57.123.229 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.497/87.8/295
  2.57.123.230 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.346/87.9/295
  2.57.123.231 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.340/87.9/295
  2.57.123.232 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.320/87.8/295
  2.57.123.233 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 2.28/89.9/295
  2.57.123.234 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 2.12/86.9/295
  2.57.123.235 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 2.00/86.9/295
  2.57.123.238 : xmt/rcv/%loss = 20/18/10%, min/avg/max = 1.53/81.5/294
  2.57.123.240 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 1.25/86.4/294
  2.57.123.242 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 0.958/86.2/293
  2.57.123.243 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.903/89.1/293
  2.57.123.244 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.783/89.0/293
  2.57.123.245 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.712/88.8/293
  2.57.123.246 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.644/88.7/293
  2.57.123.247 : xmt/rcv/%loss = 20/18/10%, min/avg/max = 0.549/72.9/179
  2.57.123.248 : xmt/rcv/%loss = 20/20/0%, min/avg/max = 0.484/88.4/293
  2.57.123.249 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 0.493/77.6/179
  2.57.123.250 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 0.383/77.4/179
  2.57.123.252 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 0.318/77.1/178
  2.57.123.253 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 0.229/78.3/203
  2.57.123.254 : xmt/rcv/%loss = 20/19/5%, min/avg/max = 0.249/78.2/203
  

  I don't think this requires much analysis, but if someone wants to come here and ask why this even matters:

  • A load balanced setup with 2 or more servers in the same data center would face a 10% packet loss and up to 200 ms response delay
  • Multi node setups with a database on a separate server would experience read or write issues

  I've also eliminated the possibility that a misconfigured firewall is deprioritizing ICMP while handling TCP as expected. TCP connections also face the same packet loss.

  👇🏼 The goalposts will be moved below 👇🏼