Categories
Networking Server SysAdmin Technology Virtualization

Deploying Proxmox 7 behind a firewall VM

How I run Proxmox with a single IP and MAC address.

Firewall

For the sake of familiarity for you, the reader, I will be demonstrating with pfSense — although many people are moving away from it due to toxicity [1]Hacker News : pfSense CE 2.5.2 [2]/r/HomeLabs : Setting the Record Straight and security concerns.

If possible, I’d urge you to consider moving to OpenWRT, OPNSense, or IPFire.

Download

We’re going to download two ISOs to the local Proxmox node after we SSH in:

  1. pfSense — virtualized firewall responsible for all traffic at the Wide and Local Area Networks (WAN and LAN). Since they gzip the ISO, we’ll pass it through gunzip as we save the ISO:
    wget \
        --output-document=- \
        https://ULR-GOES-HERE \
        | gunzip -c > \
        /var/lib/vz/template/iso/PFSENSE-NAME-GOES-HERE.iso \
        ;
    
  2. Ubuntu Desktop — Since I run desktop configurations inside of Proxmox, I needed this ISO anyways. The installer has a trial mode that provides quick access to a web browser for configuring the bare minimum for the firewall without needing to plow through an install. Beyond the minimal configuration, everything else can then be configured remotely from your computer after you boot into the /etc/network/config-runtime file that was setup above, and will be demonstrated in an upcoming step.
    wget \
        --output-document=/var/lib/vz/template/iso/UBUNTU-NAME-GOES-HERE.iso \
        https://URLS-GOES-HERE
        ;
    

You can confirm that you’ve successfully loaded your ISOs by clicking on Proxmox node → local → ISO Images.

Virtual Machine

Please pay attention to the screenshots in this section. Some of the details are minor and easy to overlook which could result in installation problems or knocking you out of the system until you recover it (which I cover further in this article).

Another point to highlight is that pfSense supports VirtIO, while as other firewall systems may not. You will need to consult their respective documentation to determine whether or not you should load additional drivers or use different models of virtualized hardware.

Create
Network Devices

In the step above, when we created the VM with a network interface, that was the WAN interface that was added. There’s still three remaining interfaces to create, which correspond to the bridges that we created in /etc/network/config-runtime:

  1. Proxmox — communication between the firewall and Proxmox, so that SSH and the Web GUI for Proxmox can be accessed via the firewall.
  2. Single IP — for Virtual Machines and Containers to use.
  3. Optional Additional IP — for Virtual Machines and Containers to use.

Though the interfaces are available, they’re not active since the network configuration hasn’t been applied with a reboot — as a side-note, when editing interfaces from within the Proxmox GUI, it uses ifupdown2 to accomplish this without reboots. With the exception of the second interface, the other three will be set to vmbr0 and disconnected. The second interface will be set to vmbr1 and connected:

Install

Once all interfaces are added, you may proceed to start the Virtual Machine so that you may install the firewall

After completing installation and rebooting into the firewall, you will be asked to assign interface names to the four NICs:

  1. WANvtnet0 (MAC Address: the server’s physical card: ca:fe:41:c0:ff:ee)
  2. LANvtnet1 (MAC Address: auto-generated)
  3. OPT1vtnet2 (MAC Address: auto-generated)
  4. OPT2vtnet3 (MAC Address: auto-generated)

When you’re finished, the firewall will initialize itself, and then display its standard console menu.

Configure

Console

Select “Set interface(s) IP address” and you will be prompted for which interface that you wish to configure. Refer to your network settings in /etc/network/config-runtime but notice that Proxmox is set to 192.168.1.2/29 while as the firewall is set to 192.168.1.1/29, and to use my example settings:

  1. WAN (vtnet0)
    • Address: 65.109.71.115
    • CIDR: 26
    • Netmask: 255.255.255.192
    • Gateway: 65.109.71.65
    • Route: 65.109.71.64
    • Broadcast: 65.109.71.127
    • Network range: 65.109.71.6465.109.71.127
    • DHCP: Not Applicable.
  2. LAN (vtnet1) — Proxmox
    • Address: 192.168.1.1
    • CIDR: 29
    • Netmask: 255.255.255.248
    • Gateway: None.
    • Route: 192.168.1.0
    • Broadcast: 192.168.1.7
    • Network range: 192.168.1.0192.168.1.7
    • DHCP: None.
  3. OPT1 (vtnet2) — Virtual Machines & Containers
    • Address: 172.16.0.1
    • CIDR: 24
    • Netmask: 255.255.255.0
    • Gateway: None.
    • Route: 172.16.0.0
    • Broadcast: 172.16.0.255
    • Network range: 172.16.0.0172.16.0.255
    • DHCP Server Range: 172.16.0.200172.16.0.254
  4. OPT2 (vtnet3)— Virtual Machines & Containers
    • Address: 172.16.1.1
    • CIDR: 24
    • Netmask: 255.255.255.0
    • Gateway: None.
    • Route: 172.16.1.0
    • Broadcast: 172.16.1.255
    • Network range: 172.16.1.0172.16.1.255
    • DHCP Server Range: 172.16.1.200172.16.1.254
Live Desktop

Now that the interfaces are set, set up an ephemeral Virtual Machine for running a web browser to finish configuring the firewall. For the Virtual Machine, set the network device to vmbr1 and inside of the Virtual Machine set the network settings to:

  • Address: 192.168.1.3
  • Netmask: 255.255.255.248
  • Gateway: 192.168.1.1
Quasi-remote

Depending on whether or not you chose to use HTTPS when setting up the firewall interfaces via the console, go to the firewall configuration page at http://192.168.1.1 or https://192.168.1.1

Default credentials:

Username: admin
Password: pfsense

Please ensure that you change your administrator password ASAP.

In the upcoming steps, your firewall’s login page will be temporarily accessible where someone would be able to login if you use a simple (or default) password.

Please ensure that HTTPS is enabled for remote login in an upcoming step.

You will receive a warning since the certificate is self-signed.

Click on Advanced, then click on View Certificate.

Scroll down to the Fingerprints section and copy either the SHA-256 or SHA-1 value (SHA-256 is preferred). In a subsequent step, these values will be used to ensure that you have a secure connection to your firewall since it’s on a self-signed certificate (for now) that you will need to accept in order to communicate to the firewall.

Now on to setting up the firewall for remote access.

I like to create a Web Ports alias that I can use in my firewall rules.

For the meantime, the firewall is still running on a default web port (HTTP or HTTPS), but we’ll switch it over in a subsequent step so that Traefik can run on a Virtual Machine and instead, use these ports. The immediate focus is to get the firewall setup in a manner where we can access it remotely, and more comfortably in our native resolution.

Add a rule on the WAN interface for your Web ports alias.

Apply the new firewall rule that will allow remote access.

Now that the firewall is set up to be accessed remotely, destroy the temporary live desktop:

More (button at the top right corner) → Remove

Shut the firewall down.

SSH into the Proxmox host and copy the network’s runtime configuration into place, which will take effect after reboot:

cp -va \
    /etc/network/config-runtime \
    /etc/network/interfaces \
    ;

After the Firewall’s Virtual Machine has shut down, connect the net0 interface, which will be on the vmbr0 bridge.

Connect the remaining interfaces and change their bridges from vmbr0 to the appropriate bridges:

  • net2vmbr2
  • net3vmbr3

Ensure that the Firewall’s Virtual Machine is set to start when Proxmox boots.

Click on the Proxmox node, and then click on Reboot in the top-right corner.

Remote

A few minutes after rebooting your Proxmox host, you should be able to access the firewall. Similar to the live desktop configuration, you will receive a self-signed certificate warning.

When viewing the certificate, confirm that the fingerprints match the same values from when you looked at them with the live desktop in Proxmox. If they do, accept the certificate.

You should now see the login screen, and be able to login with the username of admin and the password that you set:

Okay, so we’re in! Let’s make sure that we’re on the same page and understand what’s happening:

Internet ↔︎ Proxmox ↔︎ Firewall Virtual Machine

Now to set up port forwarding from the firewall to Proxmox for administrative use: FirewallNATPort Forward. The ports to use are:

  • Proxmox GUI: 8006/TCP
  • Proxmox SSH: 22/TCP
  • Proxmox Mosh: 60,000–61,000/UDP

By adding an associated rule for the NAT rule, you can see the changes appear in the WAN interface.

Administrative Port

To change the port, click on SystemAdvancedAdmin AccessTCP port. This allows me to free up the web ports for use by Traefik, and in the additions section about Tailscale, allows me to remove the need for the firewall to be directly accessible from the Internet.

Interfaces

As you delve further into customizing your firewall’s configuration to suit your needs, make it easier on yourself for rules and elsewhere by renaming the interfaces:

PortInterfacePurpose
vtnet0WAN_HetznerPrimary interface with single, Static IP address.
vtnet1ProxmoxCommunication between Proxmox and virtualized firewall.
vtnet2VMS_HetznerNetwork for Virtual Machines and Containers to use which will route through the single, Static IP.
vtnet3VMS_AdditionalNetwork for Virtual Machines and Containers to use, which will route through a secondary IP or separate network (e.g., Site-to-Site VPN).
Aliases

Many firewall distributions support aliases that can be used in a variety of firewall settings. I like to start with a few basic aliases that help define other settings throughout a firewall:

IP / Network
  • CGNAT
    • 100.64.0.0/10
  • RFC 1918
    • 10.0.0.0/8 — Single class A network
    • 172.16.0.0/12 — 16 contiguous class B networks
    • 192.168.0.0/16 — 256 contiguous class C networks
Ports

13 replies on “Deploying Proxmox 7 behind a firewall VM”

Great post. Thank you so much – exactly what I was looking for!
But I’m missing a step-by-step guide to include Traefik 2 to your setup.
Or can’t I simply find it?

Great post. Thanks!
I’m just in the process to replicate your setup.
Nearly everything is working but I can’t get any DNS servers working on OPT2. The traffic is routed over the WireGuard-VPN, all fine.
But it seems that any UDP traffic is blocked and I can’t figure out where to unblock it. Any hints much appreciated! Thank you.

Hi Andreas,

Off the top of my head, maybe I forgot to document the DNS Resolver section, but will have to go re-read my article and look for anything that I might have missed, but am in the middle of bee season (video of some of it: https://LTG.FYI/YouTube) and helping restore multiple ranches, so it may take me a moment.

I’ll e-mail you and try to figure it out with you. Once you and I figure this out, I’ll make appropriate edits to the article or one of us can write here in the comments about what needed to be changed.

Thanks,
Louis

EDIT: I e-mailed you and the e-mail bounced back as an invalid address. Please feel free to reach out again with a way for me to contact you, so that we can try and figure out what the problem is.

Thank you so much for your tutorial!!!

I can’t access the Ingress server from the VMS_Hetzner network, from external it’s working.
Like in your example: “Now I can access the simple server remotely by visiting http://65.109.71.115:8000
I can access the website from my browser, but not from a browser running with any VM on the VMS_Hetzner network.
Any idea which Firewall rule is blocking the access from VMS_Hetzner to WAN_Hetzner?

Hi Loomer, you are most welcome! 🙂

I used the interface’s address for the VMs to access the firewall interface. Please see https://thad.getterman.org/articles/proxmox-7-behind-firewall-vm/5/#live_desktop for an example of a Virtual Machine accessing the firewall’s GUI via the VM interface address to configure it instead of the WAN address.

Best regards,
Louis

EDIT: I e-mailed you and the e-mail bounced back as an invalid address. Please feel free to reach out again with a way for me to contact you.

I am new to proxmox and your article just blew me away.

I just got the hetzner with 15hdd (no nvme) and I asked for KVM to install proxmox directly on the baremetal

But you do some sort of magic.
you boot into recovery and install proxmox in qemu?
I cannot say I follow
I am not sure I follow (and obviously I already broke my setup trying to change the network settings)

Hi Marcin,

I’ll e-mail you so that we can schedule a time for me to have office hours with you; you screen share with me in observation mode so that I can tell you what to click on and help you get up and running.

I’ll make a YouTube video out of it and then add one or more relevant videos to this article.

Talk soon,
Louis T. Getterman IV

Hi,

Thank you very much for sharing this tutorial. It was extremely useful.

My home lab server was running for a few weeks without any problems. However, I have decided to upgrade Proxmox from v7 to v8 this week. I temporarily updated the /etc/network/interfaces file with the configuration in the /etc/network/config-recovery file. The upgrading process ran smoothly, and, in the end, I have updated the /etc/network/interfaces to the original configuration. Everything is working except the fact that in the Proxmox instance, I have no access to the internet. After some debugging, I noticed that, with the upgrade, the default gateway of Promox changed to 172.16.0.1. The output of ip route show is:

default via 172.16.0.1 dev vmbr2 proto kernel onlink
172.16.0.0/24 dev vmbr2 proto kernel scope link src 172.16.0.0
192.168.1.0/24 dev vmbr1 proto kernel scope link src 192.168.1.2

If I delete that and manually define the default gateway to 192.168.1.1 everything works normally. Also, if I comment the line with ‘gateway 172.16.0.1’ in the /etc/network/interfaces, the problem is solved after restarting the networking service. I am not sure if I am supposed to do that, as you had that line in the /etc/networks/config-runtime file

Has anyone else done the upgrade to v8?
Any guide available?
I’m a little bit scared to do it.

Hey Louis,

First, awesome guide.

However, I am running into some issues when trying to do the same on my Hetzner dedicated server. For some reason, even if I follow all steps, the OPNsense/pfSense VM can’t be reached via the public address.

Leave a Reply

Your email address will not be published. Required fields are marked *