New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Proxmox NAT help setup
My current proxmox network setup:
source /etc/network/interfaces.d/*
auto lo
iface lo inet loopback
auto primary
iface primary inet static
auto vmbr2
iface vmbr2 inet static
address 10.0.0.254/24
bridge-ports none
bridge-stp off
bridge-fd 0
post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o vmbr0 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o vmbr0 -j MASQUERADE
#Windows 2012 - 101
post-up iptables -t nat -A PREROUTING -p tcp --dport 13391 -j DNAT --to 10.0.0.101:3389
post-down iptables -t nat -D PREROUTING -p tcp --dport 13391 -j DNAT --to 10.0.0.101:3389
#Debian Server + Plex + Jellyfin + ruTorrent - 100
post-up iptables -t nat -A PREROUTING -p tcp --dport 32400 -j DNAT --to 10.0.0.102:32400
post-down iptables -t nat -D PREROUTING -p tcp --dport 32400 -j DNAT --to 10.0.0.102:32400
post-up iptables -t nat -A PREROUTING -p tcp --dport 12222 -j DNAT --to 10.0.0.102:22
post-down iptables -t nat -D PREROUTING -p tcp --dport 12222 -j DNAT --to 10.0.0.102:22
post-up iptables -t nat -A PREROUTING -p tcp --dport 19999 -j DNAT --to 10.0.0.102:10000
post-down iptables -t nat -D PREROUTING -p tcp --dport 19999 -j DNAT --to 10.0.0.102:10000
post-up iptables -t nat -A PREROUTING -p tcp --dport 80 -j DNAT --to 10.0.0.102:80
post-down iptables -t nat -D PREROUTING -p tcp --dport 80 -j DNAT --to 10.10.10.102:80
post-up iptables -t nat -A PREROUTING -p tcp --dport 443 -j DNAT --to 10.0.0.102:443
post-down iptables -t nat -D PREROUTING -p tcp --dport 443 -j DNAT --to 10.10.10.102:443
post-up iptables -t nat -A PREROUTING -p tcp --dport 12224 -j DNAT --to 10.0.0.102:22
post-down iptables -t nat -D PREROUTING -p tcp --dport 12224 -j DNAT --to 10.0.0.102:22
post-up iptables -t nat -A PREROUTING -p tcp --dport 19996 -j DNAT --to 10.0.0.102:10000
post-down iptables -t nat -D PREROUTING -p tcp --dport 19996 -j DNAT --to 10.0.0.102:10000
post-up iptables -t nat -A PREROUTING -p tcp --dport 7031:7500 -j DNAT --to 10.0.0.102
post-down iptables -t nat -D PREROUTING -p tcp --dport 7031:7500 -j DNAT --to 10.0.0.102
post-up iptables -t nat -A PREROUTING -p udp --dport 7031:7500 -j DNAT --to 10.0.0.102
post-down iptables -t nat -D PREROUTING -p udp --dport 7031:7500 -j DNAT --to 10.0.0.102
post-down iptables -t nat -D PREROUTING -p tcp --dport 8096 -j DNAT --to 10.0.0.102:8096
post-up iptables -t nat -A PREROUTING -p tcp --dport 8096 -j DNAT --to 10.0.0.102:8096
post-down iptables -t nat -D PREROUTING -p tcp --dport 8920 -j DNAT --to 10.0.0.102:8920
post-up iptables -t nat -A PREROUTING -p tcp --dport 8920 -j DNAT --to 10.0.0.102:8920
#Nao sei qual VM
post-up iptables -t nat -A PREROUTING -p tcp --dport 8000 -j DNAT --to 10.0.0.103:80
post-down iptables -t nat -D PREROUTING -p tcp --dport 8000 -j DNAT --to 10.0.0.103:80
#VM Windows 2022 - 105
post-up iptables -t nat -A PREROUTING -p tcp --dport 13390 -j DNAT --to 10.0.0.105:3389
post-down iptables -t nat -D PREROUTING -p tcp --dport 13390 -j DNAT --to 10.0.0.105:3389
post-up iptables -t nat -A PREROUTING -p tcp --dport 4000 -j DNAT --to 10.0.0.105:4000
post-down iptables -t nat -D PREROUTING -p tcp --dport 4000 -j DNAT --to 10.0.0.105:4000
post-up iptables -t nat -A PREROUTING -p udp --dport 4000 -j DNAT --to 10.0.0.105:4000
post-down iptables -t nat -D PREROUTING -p udp --dport 4000 -j DNAT --to 10.0.0.105:4000
#VM - Jean - 104
post-down iptables -t nat -D PREROUTING -p tcp --dport 1431 -j DNAT --to 10.0.0.106:1431
post-up iptables -t nat -A PREROUTING -p tcp --dport 1431 -j DNAT --to 10.0.0.106:1431
post-down iptables -t nat -D PREROUTING -p tcp --dport 1433 -j DNAT --to 10.0.0.106:1433
post-up iptables -t nat -A PREROUTING -p tcp --dport 1433 -j DNAT --to 10.0.0.106:1433
post-down iptables -t nat -D PREROUTING -p tcp --dport 1434 -j DNAT --to 10.0.0.106:1434
post-up iptables -t nat -A PREROUTING -p tcp --dport 1434 -j DNAT --to 10.0.0.106:1434
post-down iptables -t nat -D PREROUTING -p tcp --dport 1435 -j DNAT --to 10.0.0.106:22
post-up iptables -t nat -A PREROUTING -p tcp --dport 1435 -j DNAT --to 10.0.0.106:22
#VM - PBS - 106
post-down iptables -t nat -D PREROUTING -p tcp --dport 8007 -j DNAT --to 10.0.0.107:8007
post-up iptables -t nat -A PREROUTING -p tcp --dport 1436 -j DNAT --to 10.0.0.107:22
#iface vmbr6 inet6 static
# address 2a10:e780:10:16::4/64
iface enp1s0f0 inet manual
auto vmbr0
iface vmbr0 inet static
address XXX.XXX.XXX.XXX/32
gateway XXX.XXX.XX1
pointopoint XXX.XXX.XX1
bridge-ports primary
bridge-stp off
bridge-fd 0
#up ip route add 172.0.0.1/32 dev vmbr0
#post-up echo 1 > /proc/sys/net/ipv4/ip_forward
#post-up echo 1 > /proc/sys/net/ipv4/conf/vmbr0/proxy_arp
# dns-* options are implemented by the resolvconf package, if installed Added during provisioning
#auto vmbr1
#iface vmbr1 inet static
# address 10.0.0.1/16
# bridge-ports none
# bridge-stp off
# bridge-fd 0
One of my containers networks setup:
root@CT100:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback
#iface lo inet6 loopback
auto eth0
iface eth0 inet static
address 10.0.0.102/24
gateway 10.0.0.254
root@CT100:~# ping -4 -c5 10.0.0.254 PING 10.0.0.254 (10.0.0.254) 56(84) bytes of data. 64 bytes from 10.0.0.254: icmp_seq=1 ttl=64 time=0.051 ms 64 bytes from 10.0.0.254: icmp_seq=2 ttl=64 time=0.033 ms 64 bytes from 10.0.0.254: icmp_seq=3 ttl=64 time=0.045 ms 64 bytes from 10.0.0.254: icmp_seq=4 ttl=64 time=0.038 ms 64 bytes from 10.0.0.254: icmp_seq=5 ttl=64 time=0.060 ms --- 10.0.0.254 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4091ms rtt min/avg/max/mdev = 0.033/0.045/0.060/0.009 ms
root@CT100:~# ping -4 -c5 google.com PING google.com (142.250.200.14) 56(84) bytes of data. 64 bytes from lhr48s29-in-f14.1e100.net (142.250.200.14): icmp_seq=1 ttl=115 time=5.51 ms 64 bytes from lhr48s29-in-f14.1e100.net (142.250.200.14): icmp_seq=2 ttl=115 time=5.47 ms 64 bytes from lhr48s29-in-f14.1e100.net (142.250.200.14): icmp_seq=3 ttl=115 time=5.61 ms 64 bytes from lhr48s29-in-f14.1e100.net (142.250.200.14): icmp_seq=4 ttl=115 time=5.59 ms 64 bytes from lhr48s29-in-f14.1e100.net (142.250.200.14): icmp_seq=5 ttl=115 time=5.61 ms --- google.com ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4008ms rtt min/avg/max/mdev = 5.468/5.557/5.613/0.058 ms
root@CT100:~# apt update -y Err:1 http://ftp.debian.org/debian bullseye InRelease Could not connect to debian.map.fastlydns.net:80 (146.75.122.132), connection timed out Unable to connect to ftp.debian.org:http: Err:2 http://ftp.debian.org/debian bullseye-updates InRelease Unable to connect to ftp.debian.org:http: Err:3 http://security.debian.org bullseye-security InRelease Could not connect to debian.map.fastlydns.net:80 (146.75.118.132), connection timed out Could not connect to security.debian.org:80 (151.101.2.132), connection timed out Could not connect to security.debian.org:80 (151.101.194.132), connection timed out Could not connect to security.debian.org:80 (151.101.130.132), connection timed out Could not connect to security.debian.org:80 (151.101.66.132), connection timed out Reading package lists... Done Building dependency tree... Done Reading state information... Done All packages are up to date. W: Failed to fetch http://ftp.debian.org/debian/dists/bullseye/InRelease Could not connect to debian.map.fastlydns.net:80 (146.75.122.132), connection timed out Unable to connect to ftp.debian.org:http: W: Failed to fetch http://ftp.debian.org/debian/dists/bullseye-updates/InRelease Unable to connect to ftp.debian.org:http: W: Failed to fetch http://security.debian.org/dists/bullseye-security/InRelease Could not connect to debian.map.fastlydns.net:80 (146.75.118.132), connection timed out Could not connect to security.debian.org:80 (151.101.2.132), connection timed out Could not connect to security.debian.org:80 (151.101.194.132), connection timed out Could not connect to security.debian.org:80 (151.101.130.132), connection timed out Could not connect to security.debian.org:80 (151.101.66.132), connection timed out W: Some index files failed to download. They have been ignored, or old ones used instead.
So basically I'm not getting internet connectivity outside the container.
Comments
This is a known issue some people reported. Try to dismount the container then start again.
Already have done this. Besides have rebooted the node. No lucky.
If you remove forwarding port 80 and 443, does it work?
I am asking because I have received feedback about this but was not able to reproduce. Maybe I could finally understand what is happening.
I'm going to try this and let you know.
I also have been having a similar issue which may or may help. The setup is as follows:
KS-LE (Proxmox 7) -> VM (Debian 11) -> Docker Containers
The docker containers are mailcow, ngnix proxy mgr., nextcloud.
The VM is setup using the NAT guide @Maounique kindly shared for IPV4. I setup the ports on the Proxmox host setting IPV4 port forwarding rules to the VM. Running nmap and telnet (external IPV4 address) the ports are exposed and services are communicating. Great!
The issues start however when introducing the docker containers on the VM. The containers will ping and connect to external IPV4 addresses. These also appear reachable when running nmap and telnet (services are reachable).
However for whatever reason and only my observations, DNS resolution from the container (those I've mentioned above) are unable to resolve on IPV4. I've tried enabling IPV4/6 forwarding on the VM, remove ufw completely, etc.
These same containers I've used on SYS/OVH in the same setup except with failover IP and MAC at the VM level (no IPV4 NAT) - no issues.
Hope that maybe helps?
This problem is specific to HTTP/HTTPS connections. Also, from the OP I can see ping works to resolve the domain, the same thing other people reported.
I have been unable to reproduce the issue, a similar setup for a desktop debian 11 works perfectly to browse even with ports 80 and 443 forwarded. Other people reported the problem went away by itself which is even more puzzling.
I am curious :P
First of all stop copy and pasting from internet without checking in detail.
Second, clean up that mess into a readable format. Multiple ports can go into a single iptables rule.
Third, the ports 80 and 443 are being forwarded to 10.10.10.102 - which most likely is a typo, but potentially breaks things as @Maounique wonders about.
PS: you also want to specify the interface or your public IP as destination, so iptables does not forward just everything but rather incoming traffic.
this one:
should replace the rules, where the destination port matches the sources.
My guide is specifically one rule per line because it is for newbies, if something is wrong wont break everything. Of course, for tons of similar rules, it is much better.
Forwarding to the wrong IP should not break outgoing traffic. Many people had the same issue, sometimes solving by itself without doing anything. That is puzzling.
Indeed.
PS I didn't notice the typo :P
didn't know this is from a guide you wrote... and while I understand the idea with single line it greatly increases the risk to add mistypings or copy something wrong etc.
exactly ;-) ;-)
I'd suggest to at least do seperate post-up and post-down blocks and combine related ports per service into single rules whenever possible. it's much easier to debug.
I would not really believe that. people tend to start fiddling around and eventually might fix things without even noticing, that they did something
I will post another revision possibly with both approaches.
Yep, after removing 80/443 forwarding rules the issue is gone. Any fix for this?
Yes, first, fix the address, it was a typo, 10.10.10 instead of 10.0.0 in post-down and then modify:
into this:
@Falzo already found the issue and seeing your complex setup I understood that his approach is better than my one-line per rule to simplify things and reduce the risk of everything failing in case of a typo but running the risk of becoming overly complicated in case of complex setups.
When people have so many interfaces (I saw at least 4 there), we need to specify which one otherwise packets would fly more or less at random. In a simple setup it works ALSO it can work even if you have multiple interfaces in some cases, depending on the race condition results, therefore fixing by itself is not impossible but it is likely the fix wont be permanent.
I will amend my tutorial to highlight this issue and offer alternatives for more complicated cases which I have not foreseen newbies would need.