Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Register

Quick Links


Categories

In this Discussion

› › $100 if you fix this Proxmox VM issue - Page 3
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

$100 if you fix this Proxmox VM issue

135

Comments

  • mwmw Member

    @Andreix said:

    @mw said:

    @Andreix said:

    @mw said:

    @Andreix said:
    May be a long shot... but is the subnet properly authenticated and visible in DFZ?
    Are you sure there's no one else (not even RADB or ALTDB) that validates another origin?

    i dont think its to do with BGP/routing, the prefix works fine on all the others boxes in the rack we have VMs running on, and i tested with the vendors own prefix to rule out issues with mine

    and it worked with provider's prefix?

    PS: Just as a suggestion, a service provider is usually a referenced to as "provider"... "vendor" is used more to refer to a product developer or manufacturer (eg. HP, Dell, WHMCS ... etc.)

    it didnt work with their prefix, no

    DM me with your t.me or discord ID and I'll reach you tomorrow morning.

    If it's not fixed I'll reach out

  • nszervernszerver Member

    @mw said:

    @nszerver said:

    @mw said:
    $100 to the first person with a working fix

    I have a server running Proxmox that refuses to give me networking on a VM. The vendor confirmed the VM's IP shows up in the ARP table but it cannot ping anything, not even the gateway

    0-access-r02b08-drt-ams5.royalehosting.net(config)#show arp vrf internet <IP removed>

Address         Age (sec)  Hardware Addr   Interface
<IP removed>      0:00:51  bc24.116f.970e  Vlan100, Ethernet1/1

    It's a brand new Proxmox install, all I did was create the VM and configure it the way all our other VMs are configured

    Host:

    root@host:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback

auto vmbr0
iface vmbr0 inet static
        address 45.94.x.2/24
        gateway 45.94.x.1
        bridge-ports enp1s0f0np0
        bridge-stp off
        bridge-fd 0

iface enp1s0f0np0 inet manual

iface enp1s0f1np1 inet manual


source /etc/network/interfaces.d/*

    VM:

    • ens18 is bridged to vmbr0

      root@vm:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback

auto ens18
iface ens18 inet static
        address 45.94.x.3/24
        gateway 45.94.x.1

    It's a 9950X on a Supermicro board with 2x 40G NICs, nothing exotic.

    I've tried:

    • Using the vendors provided prefix
    • Using our prefix
    • With and without LACP on the host
    • Reinstalling the host and VM
    • Swapping IPs between the host and VM
    • Bridging the 2nd 40G interface to vmbr1 and giving that to the VM
    • Praying
    • A CT container
    • virtio and Intel E1000 NICs
    • Changing MTU
    • Spoofing the host MAC's unused interface for the VMs NIC

    Other boxes in this rack have VMs running fine. What could I be missing?

    sysctl net.ipv4.ip_forward=1

    Already tried this, it didn't work and isn't needed as I'm not forwarding

    @Falzo said:

    @mw said:

    @Falzo said:

    routed setup

    This works, the VM now has networking.

    guess you owe me $100 then.

    Wish it was a valid solution

    @roblowend said:
    I had the EXACT same issue. Are you using TempleOS on the VM per chance?

    i can't tell if you're being serious or not

    @nszerver said:
    If you give VPS to Gateway server IP address?
    so in proxmox:

    VM IP: 45.94.x.5/32 Gateway: 45.94.x.2 Nameserver 1.1.1.1 So did you try it?

    This sounds like the routed configuration Falzo mentioned, if so, it gives the VM networking but won't work in our use case

    The provider is busy looking into it so I'm hoping they come up with a fix

    In Routed a try is worth

  • mwmw Member

    @nszerver said:

    @mw said:

    @nszerver said:

    @mw said:
    $100 to the first person with a working fix

    I have a server running Proxmox that refuses to give me networking on a VM. The vendor confirmed the VM's IP shows up in the ARP table but it cannot ping anything, not even the gateway

    0-access-r02b08-drt-ams5.royalehosting.net(config)#show arp vrf internet <IP removed>

Address         Age (sec)  Hardware Addr   Interface
<IP removed>      0:00:51  bc24.116f.970e  Vlan100, Ethernet1/1

    It's a brand new Proxmox install, all I did was create the VM and configure it the way all our other VMs are configured

    Host:

    root@host:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback

auto vmbr0
iface vmbr0 inet static
        address 45.94.x.2/24
        gateway 45.94.x.1
        bridge-ports enp1s0f0np0
        bridge-stp off
        bridge-fd 0

iface enp1s0f0np0 inet manual

iface enp1s0f1np1 inet manual


source /etc/network/interfaces.d/*

    VM:

    • ens18 is bridged to vmbr0

      root@vm:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback

auto ens18
iface ens18 inet static
        address 45.94.x.3/24
        gateway 45.94.x.1

    It's a 9950X on a Supermicro board with 2x 40G NICs, nothing exotic.

    I've tried:

    • Using the vendors provided prefix
    • Using our prefix
    • With and without LACP on the host
    • Reinstalling the host and VM
    • Swapping IPs between the host and VM
    • Bridging the 2nd 40G interface to vmbr1 and giving that to the VM
    • Praying
    • A CT container
    • virtio and Intel E1000 NICs
    • Changing MTU
    • Spoofing the host MAC's unused interface for the VMs NIC

    Other boxes in this rack have VMs running fine. What could I be missing?

    sysctl net.ipv4.ip_forward=1

    Already tried this, it didn't work and isn't needed as I'm not forwarding

    @Falzo said:

    @mw said:

    @Falzo said:

    routed setup

    This works, the VM now has networking.

    guess you owe me $100 then.

    Wish it was a valid solution

    @roblowend said:
    I had the EXACT same issue. Are you using TempleOS on the VM per chance?

    i can't tell if you're being serious or not

    @nszerver said:
    If you give VPS to Gateway server IP address?
    so in proxmox:

    VM IP: 45.94.x.5/32 Gateway: 45.94.x.2 Nameserver 1.1.1.1 So did you try it?

    This sounds like the routed configuration Falzo mentioned, if so, it gives the VM networking but won't work in our use case

    The provider is busy looking into it so I'm hoping they come up with a fix

    In Routed a try is worth

    Routed works, but isn't something I can use. I need to understand what I'm doing wrong that makes on-link not work

  • nszervernszerver Member

    @mw said:

    @nszerver said:

    @mw said:

    @nszerver said:

    @mw said:
    $100 to the first person with a working fix

    I have a server running Proxmox that refuses to give me networking on a VM. The vendor confirmed the VM's IP shows up in the ARP table but it cannot ping anything, not even the gateway

    0-access-r02b08-drt-ams5.royalehosting.net(config)#show arp vrf internet <IP removed>

Address         Age (sec)  Hardware Addr   Interface
<IP removed>      0:00:51  bc24.116f.970e  Vlan100, Ethernet1/1

    It's a brand new Proxmox install, all I did was create the VM and configure it the way all our other VMs are configured

    Host:

    root@host:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback

auto vmbr0
iface vmbr0 inet static
        address 45.94.x.2/24
        gateway 45.94.x.1
        bridge-ports enp1s0f0np0
        bridge-stp off
        bridge-fd 0

iface enp1s0f0np0 inet manual

iface enp1s0f1np1 inet manual


source /etc/network/interfaces.d/*

    VM:

    • ens18 is bridged to vmbr0

      root@vm:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback

auto ens18
iface ens18 inet static
        address 45.94.x.3/24
        gateway 45.94.x.1

    It's a 9950X on a Supermicro board with 2x 40G NICs, nothing exotic.

    I've tried:

    • Using the vendors provided prefix
    • Using our prefix
    • With and without LACP on the host
    • Reinstalling the host and VM
    • Swapping IPs between the host and VM
    • Bridging the 2nd 40G interface to vmbr1 and giving that to the VM
    • Praying
    • A CT container
    • virtio and Intel E1000 NICs
    • Changing MTU
    • Spoofing the host MAC's unused interface for the VMs NIC

    Other boxes in this rack have VMs running fine. What could I be missing?

    sysctl net.ipv4.ip_forward=1

    Already tried this, it didn't work and isn't needed as I'm not forwarding

    @Falzo said:

    @mw said:

    @Falzo said:

    routed setup

    This works, the VM now has networking.

    guess you owe me $100 then.

    Wish it was a valid solution

    @roblowend said:
    I had the EXACT same issue. Are you using TempleOS on the VM per chance?

    i can't tell if you're being serious or not

    @nszerver said:
    If you give VPS to Gateway server IP address?
    so in proxmox:

    VM IP: 45.94.x.5/32 Gateway: 45.94.x.2 Nameserver 1.1.1.1 So did you try it?

    This sounds like the routed configuration Falzo mentioned, if so, it gives the VM networking but won't work in our use case

    The provider is busy looking into it so I'm hoping they come up with a fix

    In Routed a try is worth

    Routed works, but isn't something I can use. I need to understand what I'm doing wrong that makes on-link not work

    /24 change to /32?

    VM IP: 45.94.x.3/32 Gateway: 45.94.x.1 Nameserver 1.1.1.1

  • mwmw Member

    @nszerver said:

    @mw said:

    @nszerver said:

    @mw said:

    @nszerver said:

    @mw said:
    $100 to the first person with a working fix

    I have a server running Proxmox that refuses to give me networking on a VM. The vendor confirmed the VM's IP shows up in the ARP table but it cannot ping anything, not even the gateway

    0-access-r02b08-drt-ams5.royalehosting.net(config)#show arp vrf internet <IP removed>

Address         Age (sec)  Hardware Addr   Interface
<IP removed>      0:00:51  bc24.116f.970e  Vlan100, Ethernet1/1

    It's a brand new Proxmox install, all I did was create the VM and configure it the way all our other VMs are configured

    Host:

    root@host:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback

auto vmbr0
iface vmbr0 inet static
        address 45.94.x.2/24
        gateway 45.94.x.1
        bridge-ports enp1s0f0np0
        bridge-stp off
        bridge-fd 0

iface enp1s0f0np0 inet manual

iface enp1s0f1np1 inet manual


source /etc/network/interfaces.d/*

    VM:

    • ens18 is bridged to vmbr0

      root@vm:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback

auto ens18
iface ens18 inet static
        address 45.94.x.3/24
        gateway 45.94.x.1

    It's a 9950X on a Supermicro board with 2x 40G NICs, nothing exotic.

    I've tried:

    • Using the vendors provided prefix
    • Using our prefix
    • With and without LACP on the host
    • Reinstalling the host and VM
    • Swapping IPs between the host and VM
    • Bridging the 2nd 40G interface to vmbr1 and giving that to the VM
    • Praying
    • A CT container
    • virtio and Intel E1000 NICs
    • Changing MTU
    • Spoofing the host MAC's unused interface for the VMs NIC

    Other boxes in this rack have VMs running fine. What could I be missing?

    sysctl net.ipv4.ip_forward=1

    Already tried this, it didn't work and isn't needed as I'm not forwarding

    @Falzo said:

    @mw said:

    @Falzo said:

    routed setup

    This works, the VM now has networking.

    guess you owe me $100 then.

    Wish it was a valid solution

    @roblowend said:
    I had the EXACT same issue. Are you using TempleOS on the VM per chance?

    i can't tell if you're being serious or not

    @nszerver said:
    If you give VPS to Gateway server IP address?
    so in proxmox:

    VM IP: 45.94.x.5/32 Gateway: 45.94.x.2 Nameserver 1.1.1.1 So did you try it?

    This sounds like the routed configuration Falzo mentioned, if so, it gives the VM networking but won't work in our use case

    The provider is busy looking into it so I'm hoping they come up with a fix

    In Routed a try is worth

    Routed works, but isn't something I can use. I need to understand what I'm doing wrong that makes on-link not work

    /24 change to /32?

    VM IP: 45.94.x.3/32 Gateway: 45.94.x.1 Nameserver 1.1.1.1

    Will give this a shot once I get the box back from the provider

  • FalzoFalzo Member
    edited April 6

    @mw said:

    @Falzo said:

    routed setup

    This works, the VM now has networking.

    guess you owe me $100 then

    Wish it was a valid solution

    Mate, you requested a fix to get your VM networking. Are you backing out now and making your way a condition now?

    The way this subnet is routed to your VM by the provider might be different. Depends on a lot of factors and the providers setup. Seen this before, highly likely nothing you can fix from within.

    You can probably set up the full subnet minus one on the additional bridge and work with that.

    Now send me my money.

    Thanked by 10Andreix fluffernutter magicvpn babywhale kaz050457 six adly rooted tentor mrTom
  • skorupionskorupion Member, Host Rep
    edited April 6

    How does your ip a and ip r look like on the host itself?

    Edit: And on the VM itself

    Thanked by 1mw
  • TimboJonesTimboJones Member

    @wadhah said:
    have you tried turning it on and off?

    I'll take that 100$ in crypto thanks

    That would result in a failed solution 100% of the time. You meant, "have you tried turning it off and on?"

    Thanked by 2wadhah mrTom
  • TimboJonesTimboJones Member

    @skorupion said:
    How does your ip a and ip r look like on the host itself?

    Edit: And on the VM itself

    I was just going to ask. I put all my interface configuration above the IP configuration. Suspicious of a missing "Auto" line for his bridged nic.

  • nanankcorneringnanankcornering
    edited April 7

    an idea:

    just configure the nic as an pci-e passthrough. better performance, better latency, better throughput, no CPU overhead on host.

    coolest thing: nic pci-e passthrough can be used in multiple vms too.

    https://forum.proxmox.com/threads/enabling-sr-iov-for-intel-nic-x550-t2-on-proxmox-6.56677/

  • babywhalebabywhale Member

    looks like Falzo fixed it, time to pay the man.

  • AlexgammaAlexgamma Member
    edited April 7

    @babywhale said:
    looks like Falzo fixed it, time to pay the man.

    True, it should work as Falzo got it working using vmbr2.
    But was OP’s goal just to get connectivity, or to use native bridging (via vmbr0)? What was the original intention here?

  • mwmw Member

    Almost 48 hours and I still don't have the box back from the provider. Once I get it back I'll continue trying to fix it one suggestion at a time

  • AndreixAndreix Member, Host Rep

    @mw said:
    Almost 48 hours and I still don't have the box back from the provider. Once I get it back I'll continue trying to fix it one suggestion at a time

    No idea who the provider is, but at this time I'd probably change it.

  • nullnotherenullnothere Member

    @Andreix said: No idea who the provider is

    I would think it is RoyaleHosting - they also commented in the thread that they're helping out trying to identify the root cause.

    Thanked by 1Andreix
  • mwmw Member

    @Andreix said:

    @mw said:
    Almost 48 hours and I still don't have the box back from the provider. Once I get it back I'll continue trying to fix it one suggestion at a time

    No idea who the provider is, but at this time I'd probably change it.

    at a point in time i would have agreed. instead what we did is just diversify so we can tolerate these types of situations without it impacting the business. their SLA should have me covered for the invoice cost anyway: https://royalehosting.net/legal/service-level-agreement

    Thanked by 1Andreix
  • mwmw Member

    @nullnothere said:

    @Andreix said: No idea who the provider is

    I would think it is RoyaleHosting - they also commented in the thread that they're helping out trying to identify the root cause.

    they have been very cooperative so i'm okay leaving the box with them to let them figure it out for a while. if its taking this long i can only assume its a very tricky situation to understand

    Thanked by 1nullnothere
  • mwmw Member

    @TimboJones said:

    @skorupion said:
    How does your ip a and ip r look like on the host itself?

    Edit: And on the VM itself

    I was just going to ask. I put all my interface configuration above the IP configuration. Suspicious of a missing "Auto" line for his bridged nic.

    this is how we have it on all our VMs, and they work fine. i will test it with auto ens18 once i get the box back though

  • AndreixAndreix Member, Host Rep
    edited April 8

    Well, if you can tolerate the downtime, then it's worth waiting for @RoyaleHosting 's debugging.
    Last time I checked their services they had a beautiful network (latency and throughput).
    Hope they don't use Qupra's fibers which are known to result in major downtime in the middle of the day. :)

  • mwmw Member

    @nanankcornering said:
    an idea:

    just configure the nic as an pci-e passthrough. better performance, better latency, better throughput, no CPU overhead on host.

    coolest thing: nic pci-e passthrough can be used in multiple vms too.

    https://forum.proxmox.com/threads/enabling-sr-iov-for-intel-nic-x550-t2-on-proxmox-6.56677/

    would not work in our case as i need to retain the ability to just migrate VMs around without touching any configuration - but i have bookmarked this for future use, thank you!

  • nullnotherenullnothere Member

    @mw said: if its taking this long i can only assume its a very tricky situation to understand

    Please do post the final findings - at least it'll educate the rest of us on the underlying issue (likewise on the old IPv6 issue which seems to have dropped of your list of to-dos).

    Thanked by 2n1njax PineappleM
  • mwmw Member

    @nanankcornering said:
    an idea:

    just configure the nic as an pci-e passthrough. better performance, better latency, better throughput, no CPU overhead on host.

    coolest thing: nic pci-e passthrough can be used in multiple vms too.

    https://forum.proxmox.com/threads/enabling-sr-iov-for-intel-nic-x550-t2-on-proxmox-6.56677/

    would not work in our case as i need to retain the ability to just migrate VMs around without touching any configuration - but i have bookmarked this for future use, thank you!> @Andreix said:

    Well, if you can tolerate the downtime, then it's worth waiting for @RoyaleHosting 's debugging.
    Last time I checked their services they had a beautiful network (latency and throughput).
    Hope they don't use Qupra's fibers which could result in major downtime in the middle of the day. :)

    yeah i'm giving stan and co the time they need - in the middle of all of this he's still finding time to do all the other work i need him to do so i'm especially patient

    their network is great, we've never had an issue in that respect. its usually just unfortunate shipping circumstances which we mitigate by having servers with several other providers to "offload" to temporarily

    Thanked by 1Andreix
  • mwmw Member

    @nullnothere said:

    @mw said: if its taking this long i can only assume its a very tricky situation to understand

    Please do post the final findings - at least it'll educate the rest of us on the underlying issue (likewise on the old IPv6 issue which seems to have dropped of your list of to-dos).

    i'm going to revisit the IPv6 issue once i have the time and will definitely post the final solution here if its not found by one of you first!

  • ashish168527ashish168527 Member
    edited April 8

    @mw said:
    The VM can ping the host (since they share the same bridge device vmbr0) and the host can ping the VM - whatever is causing the problem is after the physical interface

    Gonna try sniffing with tcpdump

    Please once update the dns servers in proxmox DNS section and then create new vm with static IP and give it a try, i also had kind of same situation with ipv6 fixed it by updating DNS servers in proxmox,

    Use this to remove immutable tag than update dns by gui of proxmox

    chattr -i /etc/resolv.conf

    And let me know if I won that 100$ :p

  • mwmw Member

    @ashish168527 said:

    @mw said:
    The VM can ping the host (since they share the same bridge device vmbr0) and the host can ping the VM - whatever is causing the problem is after the physical interface

    Gonna try sniffing with tcpdump

    Please once update the dns servers in proxmox DNS section and then create new vm with static IP and give it a try, i also had kind of same situation with ipv6 fixed it by updating DNS servers in proxmox,

    Use this to remove immutable tag than update dns by gui of proxmox

    chattr -i /etc/resolv.conf

    we use systemd-resolved and symlink /etc/resolv.conf so we use DoH - out of interest why does IPv6 rely on DNS?

  • ashish168527ashish168527 Member

    @mw said:

    @ashish168527 said:

    @mw said:
    The VM can ping the host (since they share the same bridge device vmbr0) and the host can ping the VM - whatever is causing the problem is after the physical interface

    Gonna try sniffing with tcpdump

    Please once update the dns servers in proxmox DNS section and then create new vm with static IP and give it a try, i also had kind of same situation with ipv6 fixed it by updating DNS servers in proxmox,

    Use this to remove immutable tag than update dns by gui of proxmox

    chattr -i /etc/resolv.conf

    we use systemd-resolved and symlink /etc/resolv.conf so we use DoH - out of interest why does IPv6 rely on DNS?

    https://lowendtalk.com/discussion/204173/proxmox-ipv6-setup-for-vm-not-working#latest

    Thanked by 1mw
  • skorupionskorupion Member, Host Rep

    @mw said:

    @TimboJones said:

    @skorupion said:
    How does your ip a and ip r look like on the host itself?

    Edit: And on the VM itself

    I was just going to ask. I put all my interface configuration above the IP configuration. Suspicious of a missing "Auto" line for his bridged nic.

    this is how we have it on all our VMs, and they work fine. i will test it with auto ens18 once i get the box back though

    Still when you get back the server, and if it's still unfixed, please share the results of these.

  • mwmw Member

    @skorupion said:

    @mw said:

    @TimboJones said:

    @skorupion said:
    How does your ip a and ip r look like on the host itself?

    Edit: And on the VM itself

    I was just going to ask. I put all my interface configuration above the IP configuration. Suspicious of a missing "Auto" line for his bridged nic.

    this is how we have it on all our VMs, and they work fine. i will test it with auto ens18 once i get the box back though

    Still when you get back the server, and if it's still unfixed, please share the results of these.

    Will do, it's still with the provider in their lab

  • TimboJonesTimboJones Member

    @ashish168527 said:

    @mw said:

    @ashish168527 said:

    @mw said:
    The VM can ping the host (since they share the same bridge device vmbr0) and the host can ping the VM - whatever is causing the problem is after the physical interface

    Gonna try sniffing with tcpdump

    Please once update the dns servers in proxmox DNS section and then create new vm with static IP and give it a try, i also had kind of same situation with ipv6 fixed it by updating DNS servers in proxmox,

    Use this to remove immutable tag than update dns by gui of proxmox

    chattr -i /etc/resolv.conf

    we use systemd-resolved and symlink /etc/resolv.conf so we use DoH - out of interest why does IPv6 rely on DNS?

    https://lowendtalk.com/discussion/204173/proxmox-ipv6-setup-for-vm-not-working#latest

    You missed his point. Pinging IPs doesn't work, it's not a DNS issue.

  • mwmw Member

    Feedback from the provider is that this seems to be a hardware compatibility problem. On Monday the NIC is being swapped and I'll update ya'll on what happens

    Thanked by 1maverick
Sign In or Register to comment.