How to setup two GRE tunnels with different IP addresses pointing to the same destination server?

oddmario

Hello,

I have a BuyVM KVM slice with two DDoS protected IP addresses for the purpose of using the BuyVM DDoS protected IP address instead of the IP address of my main (let's call it "backend") server. I am doing this through a GRE tunnel.

Using https://community.hetzner.com/tutorials/linux-setup-gre-tunnel & https://wiki.buyvm.net/doku.php/gre_tunnel - I was able to come up with the below scripts to set my tunnel up:

On the BuyVM VPS:

#!/bin/bash

# This script is placed on the GRE VPS (BuyVM)

#
# Variables
#

GRE_VPS_IP="[buyvm ddos protected ip here]"
BACKEND_IP="[backend server ip]"
GRE_VPS_IP_MAIN_INTERFACE="eth0"

GRE_TUNNEL_INTERFACE_NAME="gre1"
GRE_TUNNEL_GATEWAY_IP="192.168.168.0"
GRE_TUNNEL_GREVPS_IP="192.168.168.1"
GRE_TUNNEL_BACKEND_IP="192.168.168.2"

# ----------------------------------

sysctl -p
systemctl stop firewalld
systemctl disable firewalld
modprobe tcp_bbr
tc qdisc replace dev $GRE_VPS_IP_MAIN_INTERFACE root fq
ip link set $GRE_VPS_IP_MAIN_INTERFACE txqueuelen 15000
modprobe ip_gre
iptables -F

ip tunnel add $GRE_TUNNEL_INTERFACE_NAME mode gre local $GRE_VPS_IP remote $BACKEND_IP ttl 255
ip addr add $GRE_TUNNEL_GREVPS_IP/30 dev $GRE_TUNNEL_INTERFACE_NAME
ip link set $GRE_TUNNEL_INTERFACE_NAME up

iptables -t nat -A POSTROUTING -s $GRE_TUNNEL_GATEWAY_IP/30 ! -o gre+ -j SNAT --to-source $GRE_VPS_IP
iptables -t nat -A PREROUTING -d $GRE_VPS_IP -j DNAT --to-destination $GRE_TUNNEL_BACKEND_IP
iptables -A FORWARD -d $GRE_TUNNEL_BACKEND_IP -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -s $GRE_TUNNEL_BACKEND_IP -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT

tc qdisc replace dev $GRE_TUNNEL_INTERFACE_NAME root fq
ip link set $GRE_TUNNEL_INTERFACE_NAME txqueuelen 15000
ethtool -K $GRE_TUNNEL_INTERFACE_NAME gro off gso off tso off
ethtool -K $GRE_VPS_IP_MAIN_INTERFACE gro off gso off tso off

On the backend server:

#!/bin/bash

# This script is placed on the backend server

#
# Variables
#

GRE_VPS_IP="[buyvm ddos protected ip here]"
BACKEND_IP="[backend server ip]"

GRE_TUNNEL_INTERFACE_NAME="gre1"
GRE_TUNNEL_GATEWAY_IP="192.168.168.0"
GRE_TUNNEL_GREVPS_IP="192.168.168.1"
GRE_TUNNEL_RTTABLES_ID="100"
GRE_TUNNEL_RTTABLES_NAME="GRE"

# ----------------------------------

modprobe ip_gre

if ! grep -Fxq "$GRE_TUNNEL_RTTABLES_ID $GRE_TUNNEL_RTTABLES_NAME" /etc/iproute2/rt_tables
then
     echo "$GRE_TUNNEL_RTTABLES_ID $GRE_TUNNEL_RTTABLES_NAME" >> /etc/iproute2/rt_tables
fi

ip tunnel add $GRE_TUNNEL_INTERFACE_NAME mode gre local $BACKEND_IP remote $GRE_VPS_IP ttl 255
ip addr add 10.0.0.2/30 dev $GRE_TUNNEL_INTERFACE_NAME
ip link set $GRE_TUNNEL_INTERFACE_NAME up

ip rule add from $GRE_TUNNEL_GATEWAY_IP/30 table $GRE_TUNNEL_RTTABLES_NAME
ip route add default via $GRE_TUNNEL_GREVPS_IP table $GRE_TUNNEL_RTTABLES_NAME

tc qdisc replace dev $GRE_TUNNEL_INTERFACE_NAME root fq
ip link set $GRE_TUNNEL_INTERFACE_NAME txqueuelen 15000
ethtool -K $GRE_TUNNEL_INTERFACE_NAME gro off gso off tso off

This worked fantastically when I put GRE_VPS_IP as the BuyVM DDoS protected IP, and when I put BACKEND_IP as the original backend server IP address.

---

So the problem is, I ordered another DDoS protected IP address on the same BuyVM KVM slice and now I want both IP addresses of the BuyVM VPS to point to my backend server.

I linked the newly ordered IP address to the BuyVM VPS operating system as eth0:1 and I tested it using curl too.

Then to setup a second GRE tunnel, I tried copying both of the scripts that I wrote above and I changed the variables to this instead:

GRE_TUNNEL_INTERFACE_NAME="gre2"
GRE_TUNNEL_GATEWAY_IP="192.168.169.0" # uses 169 instead of 168
GRE_TUNNEL_GREVPS_IP="192.168.169.1" # uses 169 instead of 168
GRE_TUNNEL_RTTABLES_ID="200"
GRE_TUNNEL_RTTABLES_NAME="GRE2"

But upon running the newly copied and modified scripts, the GRE tunnel entirely breaks and neither of the DDoS protected IPs work.

I believe the issue originates from the ip route add default via $GRE_TUNNEL_GREVPS_IP table $GRE_TUNNEL_RTTABLES_NAME line on the backend GRE setup script (the issue doesn't happen when this line is commented, however only the first DDoS protected IP works)

But I can't quite understand how the route is supposed to be. I made sure that the route of the second GRE tunnel is using the GRE2 rt table instead of the GRE one.

I usually try not to mess with networking but I really need this 🫠

Any help would be much appreciated!

Thanks in advance

superuser

I don't know what to do, so I'll just write a comment to promote the topic

yoursunny

Once upon a time, we moved an IPv4 over a tunnel, without these crazy iptables rules.
https://yoursunny.com/t/2023/borrow-ipv4/

kevinds

Yeah.. That isn't going to work.

Just route the traffic over the existing tunnel.

SplitIce

Conntrack your tunnels and make them sticky with a connmark that you restore to the packet mark for routing purposes.

vsys_host

@oddmario
Why do you need a second tunnel? Route 192.168.168.3 to the same tunnel and DNAT second protected IP there.

oddmario

@vsys_host said:
@oddmario
Why do you need a second tunnel? Route 192.168.168.3 to the same tunnel and DNAT second protected IP there.

Oh dear I really was overcomplicating things, I knew it 😭

Running this on the BuyVM VPS after setting the GRE tunnel up did it:

iptables -t nat -A POSTROUTING -s $GRE_TUNNEL_GATEWAY_IP/30 ! -o gre+ -j SNAT --to-source [second protected IP]
iptables -t nat -A PREROUTING -d [second protected IP] -j DNAT --to-destination $GRE_TUNNEL_BACKEND_IP

where $GRE_TUNNEL_BACKEND_IP is just 192.168.168.2 - no additional IP addresses were needed to make it work.

it really was just two commands. thank you so much for the idea @vsys_host