Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Mac address abuse message from Hetzner
New on LowEndTalk? Please Register and read our Community Rules.

Mac address abuse message from Hetzner

comXyzcomXyz Member

Hi everyone,

A week ago, I decided to use IPv6 for my personal VMs setup with Proxmox on a Hetzner dedicated server, because of the recent IPv4 price increase.

I just received Abuse Message MAC-Errors: MAC-Report from their Network-Abuse department.

They said I need to "route the requests, so the MAC used is the Hardware MAC."

I did some research, and found this https://community.hetzner.com/tutorials/install-and-configure-proxmox_ve

But if I follow the guidance above, both the host and the VMs don't have internet access.

So I did it my way, and both the host and the VMs have internet working on IPv6

The host:

/etc/network/interfaces

auto lo
iface lo inet loopback

iface enp35s0 inet manual

auto vmbr0
iface vmbr0 inet6 static
address 2a01:masked::/64
gateway fe80::1

/etc/sysctl.d/ipv6.conf

net.ipv6.conf.all.forwarding=1

The VM:

And now Hetzner support told me
"Please make sure that your VMs are in a different Layer-2 domain than your physical NIC."

Does anyone has an idea what should I do now?

Any help would be appreciated.
Thanks

Comments

  • Happened for me with ESXi, you basically need a VM to act as a router for it to work on Hetzner. You can use either pfSense to do this or some Linux flavor with manual network setup.

    I only work with ESXi so I followed this guide to pull it through: https://community.hetzner.com/tutorials/install-and-configure-routervm-for-vmware-vsphere

    Thanked by 1comXyz
  • @comXyz said: address 2a01:masked::/64

    Hi @comXyz !

    Maybe try adding a "2" to the vmbr0 address in the server's /etc/network/interfaces:

    address 2a01:masked::2/64
    

    The node gateway remains as fe80::1. However, the node's vmbr0 address 2a01:masked::2 replaces fe80::1 as the gateway inside the VM.

    This seems to work for me without any complaints from Hetzner. :) I hope it works for you! :)

    Best wishes!

    Thanked by 2comXyz coolice

    Tom, not Oles. Happy New York City guy visiting in Mexico!

  • I use OPNSense for that, following this guide: https://dominicpratt.de/hetzner-und-proxmox-ipv6-mit-router-vm-nutzen/

    Thanked by 1comXyz
  • comXyzcomXyz Member
    edited September 4

    @Not_Oles I did it
    And they said they still see traffic with wrong MACs

    tracert google.com

    Tracing route to google.com [2a00:1450:4001:82b::200e]
    over a maximum of 30 hops:

    1 * <1 ms <1 ms 2a01:masked::
    2 <1 ms <1 ms <1 ms 2a01:4f9::a:4:b
    3 34 ms 4 ms <1 ms core31.hel1.hetzner.com [2a01:4f8:0:3::3dd]
    4 20 ms 20 ms 20 ms core9.fra.hetzner.com [2a01:4f8:0:3::2f6]
    5 22 ms 20 ms 27 ms 2a01:4f8:0:3::2fd
    6 * 29 ms * 2001:4860:1:1::624
    7 29 ms 29 ms 29 ms 2a00:1450:8153::1
    8 * * 29 ms 2001:4860:0:1::500a
    9 25 ms 25 ms 25 ms 2001:4860:0:1::5013
    10 25 ms 25 ms 25 ms fra24s08-in-x0e.1e100.net [2a00:1450:4001:82b::200e]

    Thanked by 1Not_Oles
  • You are sure you setup bridge that way

    Guide bellow
    host still have :2 ip

    iface enp1s0 inet6 static
      address <address from the IPv6 subnet> #e.g. 2001:db8::2
      netmask 128
      gateway fe80::1
      up sysctl -p
    

    bridge starts with 3

    iface vmbr0 inet6 static
      address <address from the IPv6 subnet>
      #e.g. 2001:db8::3, should not be the same as on eth0
      netmask 64
      up ip -6 route add 2001:db8::/64 dev vmbr0
    

    VMs with next one and uses 3 as gateway ?

    If yes and it is not working you can try adding nat and masquerade it should hide anything from VM inder the 3 but I have it on IP4 with single failover IPs will try to addapt it for you for 6

    Thanked by 2comXyz Not_Oles

    Future-Proof Yourself! Buy! Buy!

  • Hetzner now charges for KVM access after first 3 hours.

    "Only the first 3h per 24h cycle are free. After that there is a fee of 8.40€ + VAT per 3h extension. Do you accept this charge?"

    I guess I will just cancel this server at Hetzner and get a new one from another provider then.

    Thanked by 1Not_Oles
  • Not_OlesNot_Oles Member
    edited September 4
    Thanked by 1comXyz

    Tom, not Oles. Happy New York City guy visiting in Mexico!

  • Mr_TomMr_Tom Member, Provider
    edited September 4

    Make sure you add:

    bridge_ports none
    bridge_stp off
    bridge_fd 0
    

    To your vmbr0 interface. On mine this is on the inet block but if you're just using inet6 add it there.

    I also have

    post-up echo 1 > /proc/sys/net/ipv6/conf/all/proxy_ndp
    post-up echo 1 > /proc/sys/net/ipv6/conf/all/forwarding
    post-up echo 1 > /proc/sys/net/ipv6/conf/default/forwarding
    

    On the inet6 block too.

    The above works for me in Helsinki. Using ::2 on enp4s0 and ::3 on vmbr0

    Edit: You shouldn't need KVM access to fix this. Worst case boot into rescue mode and chroot into your current install and make the changes.

    Edit 2: reading earlier comments about gateway - I have the gateway set on enp4s0 and no gateway set on vmbr0. VMs use the vmbr0 address as a gateway.

    Edit 3: As per hetzner guide, you'll also need to add a route for the IPv6 addresses (their guide shows up ip -6 route add 2001:db8::/64 dev vmbr0 on the vmbr0 inet6 block).

    Thanked by 2Not_Oles comXyz

    VM Specialist - Custom, managed and storage VM solutions | Latest Offers

  • @comXyz said: KVM access

    Probably you already know it is possible to work just with the Rescue System. If you change /etc/network/interfaces and lose connectivity, then you can activate the Rescue System, mount the boot drive, and then make further changes to the /etc/network/interfaces file.

    Thanked by 1comXyz

    Tom, not Oles. Happy New York City guy visiting in Mexico!

  • @comXyz said:
    Hetzner now charges for KVM access after first 3 hours.

    "Only the first 3h per 24h cycle are free. After that there is a fee of 8.40€ + VAT per 3h extension. Do you accept this charge?"

    I guess I will just cancel this server at Hetzner and get a new one from another provider then.

    It's always been like this why would you need permanent console more than 24hours?

  • @plumberg said:

    @comXyz said:
    Hetzner now charges for KVM access after first 3 hours.

    "Only the first 3h per 24h cycle are free. After that there is a fee of 8.40€ + VAT per 3h extension. Do you accept this charge?"

    I guess I will just cancel this server at Hetzner and get a new one from another provider then.

    It's always been like this why would you need permanent console more than 24hours?

    No, In the past, it took me few "3 hours" times to get Windows installation done via their KVM.

    So basically I could request 3 hours, if the job hasn't done, I can request another 3 hours, and just wait for their KVM info. Now it doesn't work that way, you can only request total 3 hours per 24 hours.

  • Mr_TomMr_Tom Member, Provider

    If you're using proxmox just install Windows via noVNC?

    Thanked by 1comXyz

    VM Specialist - Custom, managed and storage VM solutions | Latest Offers

  • So I have this config

    /etc/network/interfaces

    auto lo
    iface lo inet6 loopback
    iface enp35s0 inet6 static
    address 2a01:masked::2
    netmask 128
    gateway fe80::1

    auto vmbr0
    iface vmbr0 inet6 static
    address 2a01:masked::1
    netmask 64
    bridge_ports none
    bridge_stp off
    bridge_fd 0
    up ip -6 route add 2a01:masked::/64 dev vmbr0
    post-up echo 1 > /proc/sys/net/ipv6/conf/all/proxy_ndp
    post-up echo 1 > /proc/sys/net/ipv6/conf/all/forwarding
    post-up echo 1 > /proc/sys/net/ipv6/conf/default/forwarding

    The host doesn't have internet connection
    From the host

    ping6 google.com
    connect: Network is unreachable

    The host can't ping 2a01:masked::2 (physical NIC address)
    But it can ping 2a01:masked::1 (vmbr0 address)

    From inside the VM, it can ping 2a01:masked::1, but not 2a01:masked::2

    When I run this from the host

    ifup enp35s0
    error: enp35s0: cmd '/bin/ip route add default via fe80::1 proto kernel dev enp35s0 onlink' failed: returned 2 (Error: IPv6 is disabled on nexthop device.
    )

    Any ideas?

    Thanked by 1Not_Oles
  • Mr_TomMr_Tom Member, Provider
    edited September 4

    Here's a redacted version of my /etc/network/interfaces (mostly removing additional routing post-up lines and hiding IPs:

    ### Hetzner Online GmbH installimage
    
    source /etc/network/interfaces.d/*
    
    auto lo
    iface lo inet loopback
    iface lo inet6 loopback
    
    auto enp4s0
    iface enp4s0 inet static
      address 95.217.xxx.yyy
      netmask 255.255.255.192
      gateway 95.217.xxx.zzz
      up route add -net 95.217.xxx.aaa netmask 255.255.255.192 gw 95.217.xxx.zzz dev enp4s0
      post-up echo 1 > /proc/sys/net/ipv4/ip_forward
      post-up echo 1 > /proc/sys/net/ipv4/conf/enp4s0/proxy_arp
    
    iface enp4s0 inet6 static
      address 2a01:4f9:ff:abcd::2
      netmask 64
      gateway fe80::1
    
    auto viifbr0
    iface viifbr0 inet static
      address 95.217.xxx.yyy
      netmask 255.255.255.192
      bridge_ports none
      bridge_stp off
      bridge_fd 0
    
    iface viifbr0 inet6 static
      pre-up modprobe ipv6
      address 2a01:4f9:ff:abcd::3
      netmask 64
      post-up echo 1 > /proc/sys/net/ipv6/conf/all/proxy_ndp
      post-up echo 1 > /proc/sys/net/ipv6/conf/all/forwarding
      post-up echo 1 > /proc/sys/net/ipv6/conf/default/forwarding
      up ip -6 route add 2a01:4f9:ff:abcd::/64 dev viifbr0
    

    Although, I actually split the /64 from Hetzner into a smaller /96 and route that (instead of the final line above) and give each VM a single address from that /96 but the above should work. I only do that so each VM has an address from the /96 and then a seperate /64 is routed to them but you need a /56 from hetzner to do this.

    Thanked by 2comXyz Falzo

    VM Specialist - Custom, managed and storage VM solutions | Latest Offers

  • I have predefined IPs not subnet and has

    post-up iptables -t nat -A POSTROUTING -s 'IP4/32' -o vmbr0 -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s 'IP4/32' -o vmbr0 -j MASQUERADE

    under the UP IP that should prevent any exposure of the VM under the bridge and trigger hetzner abuse for auto generated VM mac

    Do not use KVM use rescue ...

    Thanked by 1comXyz

    Future-Proof Yourself! Buy! Buy!

  • I made all the changes based on @Mr_Tom configuration.

    At this moment both the host and VMs have internet connection.
    When I do tracert from within the VMs, I see it's going through the vmbr0 address.

    How can I check if the traffic inside VMs using the routed configuration?
    Hetzner support simply says "at this moment the problem is still active."

  • did you change the gateway in your VMs ?

    UltraVPS.eu KVM in US/UK/NL/DE: 15% off first 6 month | Netcup VPS/rootDS - 5€ off: 36nc15279180197 (ref)

  • @Falzo said: did you change the gateway in your VMs ?

    Yes I did

  • hmm weird. I have a very similar config to what @Mr_Tom posted. also splitting the /64 into smaller pieces though and having individual bridges. however, more or less same syntax in the end and never had problems with that.

    maybe share your actual settings for host and client again, so we can llok closer, where you are at right now.

    do you involve any kind of IPv4 as well? maybe it's not even IPv6 related...

    UltraVPS.eu KVM in US/UK/NL/DE: 15% off first 6 month | Netcup VPS/rootDS - 5€ off: 36nc15279180197 (ref)

  • I asked Hetzner for the list of MAC addresses that having problems.

    And they said
    "Hello,
    problem looks fixed since 04 Sep 19:16."

    So I guess the config has been correct, and I don't have to change anything .

    Thanks @Falzo

    Thanked by 1Falzo
  • @comXyz said:
    I asked Hetzner for the list of MAC addresses that having problems.

    And they said
    "Hello,
    problem looks fixed since 04 Sep 19:16."

    So I guess the config has been correct, and I don't have to change anything .

    Thanks @Falzo

    ah, good then! essentially it comes down to always use the host itself as gateway within the VMs and not the main one, to make sure nothing goes out with a virtual mac.

    the only case where this is different are single addon IPv4, where you'd request a specific virtual mac from Hetzner and apply that to your VMs network interface.

    anyway, glad you got it working finally...

    Thanked by 1comXyz

    UltraVPS.eu KVM in US/UK/NL/DE: 15% off first 6 month | Netcup VPS/rootDS - 5€ off: 36nc15279180197 (ref)

  • Hi all,

    looking for some help here. I've got a similar message from Hetzne but don't have a clue how to fix it. Could someone help with step by step instructions on the right commands for Ubuntu 18.04?

  • @eb98jdb said:
    Hi all,

    looking for some help here. I've got a similar message from Hetzne but don't have a clue how to fix it. Could someone help with step by step instructions on the right commands for Ubuntu 18.04?

    Do you run Proxmox?

  • OseriOseri Member

    Yesterday I got a similar complaint about my dedicated server (below, redacted):

    "We have detected that your server is using a differente MAC address from those allowed by your Robot Account.
    Allowed MACs:
    90:1b:xx:xx:xx:xx
    Unallowed MACs:
    52:54:00:b3:8f:98
    52:54:00:bd:84:58"

    I had installed QEMU/KVM on my Ubuntu 20.04 server and seems the offending MACs are the ones in the bridged virtual NAT virbr0. What I want to do is create a VM which can get reached at some port 3389 from the outside internet but still have access to the internet in the server. Still trying to figure out how to do this with the existing MAC (read: avoid handing out €19 to setup a new IP/MAC on my server)

    BTW, I found this is pretty easy with VirtualBox (in the VM configuration option Network | Attached to: NAT | Advanced: Port forwarding) but I'm not sure if this will also trigger a MAC abuse warning.

Sign In or Register to comment.