Categories
Networking Server SysAdmin Technology Virtualization

Deploying Proxmox 7 behind a firewall VM

How I run Proxmox with a single IP and MAC address.

IP Addresses

Data Center / ISP

I’ll add this section at a later date. When writing this article and I went to add additional IPs for documenting with screenshots, and to meet the quality that I try to put into each article, I faced high costs that were outside of my server budget for the month. Sorry.

VPN

Depending on what you’re trying to accomplish, you can create your own Site-to-Site between your locations or use a service. A WireGuard configuration file is simple and straightforward, and a typical configuration file from a VPN service looks like this:

[Interface]
PrivateKey = RQPONMLKJIHGFEDCBAZYXWVUTSRQPONMLKJIHGFEDCBA
Address = 1234:5678::123/64, 1.2.3.4/24
DNS = 1.2.3.4, 1234:123:1234:1::53
MTU = 1234

[Peer]
PublicKey = ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQR
Endpoint = wg.example.com:51820
PersistentKeepalive = 25
AllowedIPs = 0.0.0.0/0, ::/0

In the firewall’s configuration page, add the WireGuard package by clicking on SystemPackage ManagerAvailable PackagesSearch termWireGuardSearchInstall:

You can now configure WireGuard by clicking on VPNWireGuard

Using the example configuration from above, first, I will add a tunnel:

Now to add a tunnel peer:

Enable the WireGuard tunnel and then click on the Save button.

Click on Status to ensure that you’re connected.

Click on InterfacesAssignmentsAvailable network ports → the WireGuard interface (in my case it’s tun_wg0) → Add

Then click on the new interface (in my case it’s OPT3).

For the interface, the IP and the Gateway will be the IP address in the configuration file. You can add the gateway by clicking on the Add a new gateway button. When you’re finished, save your settings.

Depending on the popularity of your second IP address, when you look at your logs, you should see multiple attempts at port scans, bots, and people poking at your network (and trying to gain access.)

This is precisely why I’ve warned throughout this article that you should have a complex password and not use a simple password since it’s only a short time before people start poking at servers and services on those servers that you expose to the Internet.

Egress

Change the Outbound NAT Mode so that select Virtual Machines and Containers can use a separate IP address from your single/primary one by clicking on FirewallNATOutboundOutbound NAT ModeHybrid Outbound NATSave

I then define the outbound traffic for the second network that will come from the “second” WAN. I opt for the ideology of explicit is better than implicit, which is why I specify the entirety of the second network will be part of an outbound NAT so long as it’s not trying to reach an RFC 1918 address that’s an alias which I specified above.

Now I add an outbound rule for everything on the “additional” network bridge with Advanced Options. The key points of this rule are:

  • Interface: everything on the “additional” bridge (VMS_ADDITIONAL)
  • Source: VMS_ADDITIONAL net
  • Destination: not RFC 1918
  • Tag: Additional
  • Gateway: WAN_WIREGUARD 1.2.3.4

If your “additional” gateway goes down, you may not want the traffic to originate from your primary IP. There’s two ways to fix this:

  1. Turn off automatic rule creation.
  2. Block tagged traffic with a floating rule on the WAN interface.

To turn off automatic rule creation which omits the gateway:

SystemAdvancedMiscellaneousGateway Monitoring

Then, enable:

  • Flush all states when a gateway goes down
  • Do not create rules when gateway is down

and click on the Save button.

OR

Click on FirewallRulesFloatingAdd

The key points of this rule with Advanced Options are:

  • Action: Block or Reject
  • Interface: WAN
  • Protocol: Any
  • Tagged: Additional

Now you can test to ensure that your Virtual Machine or Container is using its secondary IP via the VPN:

dig -x `curl https://wtfismyip.com/text`

Then, you can stop the VPN service and run this command again, and it should fail. If you see your WAN IP then please re-read these instructions and ensure that everything is set correctly.

Ingress

Remember the Simple HTTP Server that we ran on the WAN? Let’s do that again, but on our “second” WAN courtesy of the VPN.

Click on FirewallNATPort ForwardAdd:

Tunneling DNS

In addition to forcing traffic across your secondary IP via a VPN, don’t forget that you may face DNS Leaking as well. If you have a VPN service that you trust, you can opt for the DNS server(s) found in the WireGuard configuration file. Other candidates to consider using:

First, set your firewall’s DNS servers by clicking on SystemGeneral SetupDNS Server Settings. Ensure that for each of the DNS servers that you add, the gateway is set to your VPN’s gateway. Ensure that DNS Server Override is disabled. When finished, click on the Save button.

Now click on ServicesDNS Resolver

Ensure that the Network Interfaces section is set to All and that the Outgoing Network Interfaces is set to your VPN’s interface (e.g. WAN_WireGuard), enable DNS Query Forwarding, and then click on the Save button.

Now you can test for DNS Leaks by opening a browser in Incognito/Private mode and loading one or more of the following DNS Leak sites:

14 replies on “Deploying Proxmox 7 behind a firewall VM”

Great post. Thank you so much – exactly what I was looking for!
But I’m missing a step-by-step guide to include Traefik 2 to your setup.
Or can’t I simply find it?

Great post. Thanks!
I’m just in the process to replicate your setup.
Nearly everything is working but I can’t get any DNS servers working on OPT2. The traffic is routed over the WireGuard-VPN, all fine.
But it seems that any UDP traffic is blocked and I can’t figure out where to unblock it. Any hints much appreciated! Thank you.

Hi Andreas,

Off the top of my head, maybe I forgot to document the DNS Resolver section, but will have to go re-read my article and look for anything that I might have missed, but am in the middle of bee season (video of some of it: https://LTG.FYI/YouTube) and helping restore multiple ranches, so it may take me a moment.

I’ll e-mail you and try to figure it out with you. Once you and I figure this out, I’ll make appropriate edits to the article or one of us can write here in the comments about what needed to be changed.

Thanks,
Louis

EDIT: I e-mailed you and the e-mail bounced back as an invalid address. Please feel free to reach out again with a way for me to contact you, so that we can try and figure out what the problem is.

Thank you so much for your tutorial!!!

I can’t access the Ingress server from the VMS_Hetzner network, from external it’s working.
Like in your example: “Now I can access the simple server remotely by visiting http://65.109.71.115:8000
I can access the website from my browser, but not from a browser running with any VM on the VMS_Hetzner network.
Any idea which Firewall rule is blocking the access from VMS_Hetzner to WAN_Hetzner?

Hi Loomer, you are most welcome! 🙂

I used the interface’s address for the VMs to access the firewall interface. Please see https://thad.getterman.org/articles/proxmox-7-behind-firewall-vm/5/#live_desktop for an example of a Virtual Machine accessing the firewall’s GUI via the VM interface address to configure it instead of the WAN address.

Best regards,
Louis

EDIT: I e-mailed you and the e-mail bounced back as an invalid address. Please feel free to reach out again with a way for me to contact you.

I am new to proxmox and your article just blew me away.

I just got the hetzner with 15hdd (no nvme) and I asked for KVM to install proxmox directly on the baremetal

But you do some sort of magic.
you boot into recovery and install proxmox in qemu?
I cannot say I follow
I am not sure I follow (and obviously I already broke my setup trying to change the network settings)

Hi Marcin,

I’ll e-mail you so that we can schedule a time for me to have office hours with you; you screen share with me in observation mode so that I can tell you what to click on and help you get up and running.

I’ll make a YouTube video out of it and then add one or more relevant videos to this article.

Talk soon,
Louis T. Getterman IV

Hi,

Thank you very much for sharing this tutorial. It was extremely useful.

My home lab server was running for a few weeks without any problems. However, I have decided to upgrade Proxmox from v7 to v8 this week. I temporarily updated the /etc/network/interfaces file with the configuration in the /etc/network/config-recovery file. The upgrading process ran smoothly, and, in the end, I have updated the /etc/network/interfaces to the original configuration. Everything is working except the fact that in the Proxmox instance, I have no access to the internet. After some debugging, I noticed that, with the upgrade, the default gateway of Promox changed to 172.16.0.1. The output of ip route show is:

default via 172.16.0.1 dev vmbr2 proto kernel onlink
172.16.0.0/24 dev vmbr2 proto kernel scope link src 172.16.0.0
192.168.1.0/24 dev vmbr1 proto kernel scope link src 192.168.1.2

If I delete that and manually define the default gateway to 192.168.1.1 everything works normally. Also, if I comment the line with ‘gateway 172.16.0.1’ in the /etc/network/interfaces, the problem is solved after restarting the networking service. I am not sure if I am supposed to do that, as you had that line in the /etc/networks/config-runtime file

Hi Miguel,
I had the exact same problem.
I can resolve it after a reboot by using the command:

ip route replace default via 192.168.1.1 dev vmbr1 proto kernel onlink

But how can you make this change permanently? What needs to be changed in /etc/network/interfaces?

Has anyone else done the upgrade to v8?
Any guide available?
I’m a little bit scared to do it.

Hey Louis,

First, awesome guide.

However, I am running into some issues when trying to do the same on my Hetzner dedicated server. For some reason, even if I follow all steps, the OPNsense/pfSense VM can’t be reached via the public address.

Leave a Reply

Your email address will not be published. Required fields are marked *