How to assign a dedicated IP to a container inside of Proxmox?

AleksaDjordjic · December 2021

Hi,

I've been playing around and having fun with Proxmox recently and bought a server with multiple IPs to experiment with networking.

So far I've been quite unsuccessful as to making anything else other than a NAT setup, where each CT has its port range on a shared IP of the host.

How would assigning a dedicated IP to CT be done?

Here's my example setup, and what I've tried so far:

IPs (Examples):
172.0.0.1 - Gateway
172.0.0.100 - Main IP (Proxmox host)
172.0.0.200 - Second IP (Want this assigned to a CT)

I've installed Proxmox on top of Debian, where I got this as the default network configuration:

auto eth0
iface eth0 inet static
   address 172.0.0.100
   gateway 172.0.0.1
   netmask 255.255.255.0
   dns-nameservers 1.1.1.1 1.0.0.1

auto eth0:0
iface eth0:0 inet static
   address 172.0.0.200
   gateway 172.0.0.1
   netmask 255.255.255.0
   dns-nameservers 1.1.1.1 1.0.0.1

Now, I've been searching around to see how people usually configure this, and it seemed like they had two bridges, one for local and other for public access so I took a similar approach, having something like this in the end:

auto eth0
iface eth0 inet static
   address 172.0.0.100
   gateway 172.0.0.1
   pointopoint 172.0.0.1
   netmask 255.255.255.0

auto vmbr0
iface vmbr0 inet static
    address 10.0.1.0/24
    bridge_ports none
    bridge_stp off
    bridge_fd 0
    post-up iptables -t nat -A POSTROUTING -s '10.0.1.0/24' -o eth0 -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s '10.0.1.0/24' -o eth0 -j MASQUERADE

iface vmbr1 inet static
    address 172.0.0.100
    netmask 255.255.255.0
    bridge_ports none
    bridge_stp off
    bridge_fd 0
    up ip route add 172.0.0.200/32 dev vmbr0

When creating the container, I gave it two network interfaces, one on the vmbr0 bridge, and other on the vmbr1 bridge.

On the interface that uses vmbr0, I set its gateway, IP and subnet and it seemed to work fine.
Though on the second interface that is using vmbr1, I set the IP to 172.0.0.200 and gateway as 172.0.0.1... which doesn't seem to work...

I've been messing with this for a couple of days now, brute-force trying every possible configuration that I could come up with, though none seem to work as they should.

What am I doing wrong, and how should I configure something like this?
Thank you in advance!

I'm not even sure if the second IP is even properly assigned to my server anymore, should probably check that somehow...

Samidare · December 2021

Isn't this setup for routed subnet?
If your additional IP is in same subnet as main IP,
just using a bridge with "bridge_ports eth0" should work.

mxmla · December 2021

Hi,

is it a Server from Hetzner?

Are you having additional single IPs or a subnet?

AleksaDjordjic · December 2021

@Samidare said:
Isn't this setup for routed subnet?
If your additional IP is in same subnet as main IP,
just using a bridge with "bridge_ports eth0" should work.

I'm not sure, I guess so, I was taking a look at some auto generated script for additional subnets and IPs so though it might work.

So something like this should be fine? (Lets ignore the local/NAT setup for now, only one interface with public internet access and dedicated IP)

auto vmbr0
iface vmbr0 inet static
    address 172.0.0.100
    netmask 255.255.255.0
    bridge_ports eth0
    bridge_stp off
    bridge_fd 0

And then in the CT, give it that bridge and IP of 172.0.0.200?

@mxmla said:
Hi,

is it a Server from Hetzner?

Are you having additional single IPs or a subnet?

Its not from Hetzner and its a single additional IP, though on the same subnet as the main IP.

Are there any differences if its on the same subnet or not? I'm guessing a different netmask and/or port would be required for that.

Samidare · December 2021

Also remove config from eth0
so, something like this

auto eth0
iface eth0 inet manual

auto vmbr0
iface vmbr0 inet static
    address 172.0.0.100
    gateway 172.0.0.1
    netmask 255.255.255.0
    bridge_ports eth0
    bridge_stp off
    bridge_fd 0

@AleksaDjordjic said: And then in the CT, give it that bridge and IP of 172.0.0.200?

and set gateway to the same one as the main IP (172.0.0.1).

AleksaDjordjic · December 2021

Gotcha, will try that in a bit and post my results.
Thanks!

AleksaDjordjic · December 2021

@Samidare said:
Also remove config from eth0
so, something like this
auto eth0
iface eth0 inet manual

auto vmbr0
iface vmbr0 inet static
    address 172.0.0.100
    netmask 255.255.255.0
    bridge_ports eth0
    bridge_stp off
    bridge_fd 0
@AleksaDjordjic said: And then in the CT, give it that bridge and IP of 172.0.0.200?

and set gateway to the same one as the main IP (172.0.0.1).

I've now got behavior like this: https://imgur.com/ZhPcN2X
Red is the IP of the VM, from the example equivalent to 172.0.0.200, blue is some IP not similar in any way - Did a whois on it and it is from the same provider though.

Should... that be my gateway...?
Doesn't make sense to me though...

mezoology · December 2021

on your example of the bridge conf, you didn't specify your gateway.

AleksaDjordjic · December 2021

@mezoology said:
on your example of the bridge conf, you didn't specify your gateway.

My bad on that, though did actually put it in the config I was testing with.

AleksaDjordjic · December 2021

Yellow: Main IP (172.0.0.100 from the example)
Green: Gateway IP (172.0.0.1 from the example)
Red: Second IP (172.0.0.200 from the example)

/etc/network/interfaces
https://imgur.com/a/JVDq2ih

Proxmox CT network device:
https://imgur.com/a/Fo6dMQK

Samidare · December 2021

I'm not sure.
Maybe the provider is whitelisting with MAC address?

Try setting to the same one as eth0.
Check MAC address with
cat /sys/class/net/eth0/address

auto vmbr0
iface vmbr0 inet static
    address 172.0.0.100
    gateway 172.0.0.1
    netmask 255.255.255.0
    hwaddress ether aa:bb:cc:dd:ee
    bridge_ports eth0
    bridge_stp off
    bridge_fd 0

AleksaDjordjic · December 2021

@Samidare said:
I'm not sure.
Maybe the provider is whitelisting with MAC address?

Try setting to the same one as eth0.
Check MAC address with
cat /sys/class/net/eth0/address
auto vmbr0
iface vmbr0 inet static
    address 172.0.0.100
    gateway 172.0.0.1
    netmask 255.255.255.0
    hwaddress ether aa:bb:cc:dd:ee
    bridge_ports eth0
    bridge_stp off
    bridge_fd 0

Nope, same, though just realized my mistake in the last reply:

@AleksaDjordjic said: I've now got behavior like this: https://imgur.com/ZhPcN2X
Red is the IP of the VM, from the example equivalent to 172.0.0.200,

Incorrect, the red (just on that picture) is the gateway IP, not the one assigned to the container.
So its From {gateway_ip}: icmp....
Isn't there a way that I can ignore ICMP redirects? Aren't they just used for picking an optimal route, not fully blocking access or am I getting something wrong here?

Still quite new to networking.

AleksaDjordjic · December 2021

Setting the container MAC address to the MAC returned by cat /sys/class/net/eth0/address, and then doing ping does show that it's coming from the container IP, so example, From 172.0.0.200: icmp... though its Destination host unreachable

Samidare · December 2021

Umm... I'm not an expert so I can't give advice anymore.
I have experience only with routed subnets.

@AleksaDjordjic said: I'm not even sure if the second IP is even properly assigned to my server anymore, should probably check that somehow...

Just to confirm, was the additional IP working with this default config?

@AleksaDjordjic said: I've installed Proxmox on top of Debian, where I got this as the default network configuration:

AleksaDjordjic · December 2021

@Samidare said: Just to confirm, was the additional IP working with this default config?

Just tested and uh... Nope, it doesn't work.
Got in contact with the provider, confirmed it doesn't work - Will have it fixed soon - Oh well, gonna wait for a bit and re-try later.

mezoology · December 2021

i'd put the second IP as the main node IP and see if it works.

AleksaDjordjic · December 2021

@mezoology said:
i'd put the second IP as the main node IP and see if it works.

Exactly what I did, reverted the old config, switched the IPs, installed NGINX and made it listen on any IP.
The old main IP responded, the other one didn't.

AlwaysSkint · December 2021

@AleksaDjordjic said: The old main IP responded, the other one didn't.

What's the output from ip a ? (Mask the 3rd octet.)

AleksaDjordjic · December 2021

@AlwaysSkint said:

@AleksaDjordjic said: The old main IP responded, the other one didn't.

What's the output from ip a ? (Mask the 3rd octet.)

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether {mac_address} brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    altname ens3
    inet {second_ip}/24 scope global eth0
       valid_lft forever preferred_lft forever
    inet {main_ip}/24 scope global secondary eth0
       valid_lft forever preferred_lft forever

Edit:

@AleksaDjordjic said: The old main IP responded, the other one didn't.

This was confirmed by the provider, they didn't properly setup the 2nd IP to my server and will fix it soon

AlwaysSkint · December 2021

I'd be tempted to stick with your 1st configuration, rather than the secondary eth0 method i.e.

auto eth0
iface eth0 inet static
   address 172.0.0.100
   gateway 172.0.0.1
   netmask 255.255.255.0
   dns-nameservers 1.1.1.1 1.0.0.1

auto eth0:0
iface eth0:0 inet static
   address 172.0.0.200
   gateway 172.0.0.1
   netmask 255.255.255.0
   dns-nameservers 1.1.1.1 1.0.0.1

AleksaDjordjic · December 2021

@Samidare said:
I'm not sure.
Maybe the provider is whitelisting with MAC address?

Try setting to the same one as eth0.
Check MAC address with
cat /sys/class/net/eth0/address
auto vmbr0
iface vmbr0 inet static
    address 172.0.0.100
    gateway 172.0.0.1
    netmask 255.255.255.0
    hwaddress ether aa:bb:cc:dd:ee
    bridge_ports eth0
    bridge_stp off
    bridge_fd 0

Well the secondary IP not working was fixed by the provider, though now every time I try to create a bridge, it bricks itself -_-

Used the config that worked earlier, when I technically had only one IP and it worked, now using it again, nope, nothing.

Tried as before, without and with giving the MAC address to the vmbr0... Think I'll just give up for now and try at some later date, I've got no clue what could be the reason of this.

AleksaDjordjic · December 2021

Reinstalled proxmox, used that config again, works like a charm - Able to assign a dedicated IP to a container, perfect!

Thanks everyone for your help in this.

Howdy, Stranger!

Categories

In this Discussion

How to assign a dedicated IP to a container inside of Proxmox?

Comments

Howdy, Stranger!

Quick Links

Categories

In this Discussion

How to assign a dedicated IP to a container inside of Proxmox?

Comments