12.0-beta3 pf firewall NAT rule syntax for vnet jail using pf

[Ernie Luzar]

Hello lists;

testing 12.0-beta3 vnet jail that is using pf firewall.
net.inet.ip.forwarding =1 for the vnet jail.
Host is running ipfilter firewall.
The kldload pf.ko pflog.ko command has been issued.
10.0.10.30 is the ip address assigned to the vnet jail in the jail.conf.
Using this nat rule

nat on epair2b from 10.0.0.30/24 to any -> (vge0)

vge0 is the hosts interface facing the public internet and a member of
bridge2 along with member epair2a.

When I do a ping 8.8.8.8 from the vnet jail console I get message
"Time to live exceeded"

The vnet jail pflog shows in and out on epair2b 10.0.10.30 > 8.8.8.8

Thinking the NAT rule is incorrect because the pflog doesn't show the
nated ip address assigned by the isp. OR maybe the nat rule is not
functional in a vnet jail because I found a bug.

Am I missing something here? Help please.

[Kristof Provost]

Post by Ernie Luzar
Hello lists;
testing 12.0-beta3 vnet jail that is using pf firewall.
net.inet.ip.forwarding =1 for the vnet jail.
Host is running ipfilter firewall.
The kldload pf.ko pflog.ko command has been issued.
10.0.10.30 is the ip address assigned to the vnet jail in the
jail.conf.
Using this nat rule
nat on epair2b from 10.0.0.30/24 to any -> (vge0)

Is this rule set on the pf inside the jail?

Post by Ernie Luzar
vge0 is the hosts interface facing the public internet and a member of
bridge2 along with member epair2a.

Is this bridge on the host, so outside the jail?

If so, how can the jail see the vge0 interface?

Best regards,
Kristof

[Ernie Luzar]

Post by Ernie Luzar
Hello lists;
testing 12.0-beta3 vnet jail that is using pf firewall.
net.inet.ip.forwarding =1 for the vnet jail.
Host is running ipfilter firewall.
The kldload pf.ko pflog.ko command has been issued.
10.0.10.30 is the ip address assigned to the vnet jail in the jail.conf.
Using this nat rule
nat on epair2b from 10.0.0.30/24 to any -> (vge0)
Is this rule set on the pf inside the jail?

YES

Post by Ernie Luzar
vge0 is the hosts interface facing the public internet and a member
of bridge2 along with member epair2a.
Is this bridge on the host, so outside the jail?

YES

Post by Ernie Luzar
If so, how can the jail see the vge0 interface?

Through the bridge? I don't really know. Just guessing.

Post by Ernie Luzar
Best regards,
Kristof

I added pass to the pf nat rule so inbound packets that match entry in
state table get passed automatically.

Now using this pf nat rule
nat pass on epair2b from 10.0.0.30/24 to any -> (epair2b)

This is the ifconfig -a on the host after the vnet jail is started.

em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0
mtu 1500
options=81249b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,
VLAN_HWCSUM,LRO,WOL_MAGIC,VLAN_HWFILTER>
ether d0:50:99:93:75:98
inet 10.0.10.2 netmask 0xff000000 broadcast 10.255.255.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vge0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST>
metric 0 mtu 1500
options=3899<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,
WOL_UCAST,WOL_MCAST,WOL_MAGIC>
ether 00:16:36:4e:35:86
hwaddr 10:00:60:21:00:93
inet xx.xx.xx.xx netmask 0xfffff000 broadcast 255.255.255.255
media: Ethernet autoselect (1000baseT <full-duplex,master>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
groups: pflog
bridge2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0
mtu 1500
ether 02:5c:98:6f:9d:0a
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: epair2a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 6 priority 128 path cost 2000
member: vge0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 2 priority 128 path cost 20000
groups: bridge
nd6 options=1<PERFORMNUD>
epair2a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST>
metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:d9:a3:a8:e7:0a
inet6 fe80::d9:a3ff:fea8:e70a%epair2a prefixlen 64 scopeid 0x6
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Here are the pf rules in the vnet jail

oif=epair2b
set block-policy drop
set fail-policy drop
set state-policy if-bound
scrub in on $oif all
set skip on lo0
nat pass on $oif inet from 10.0.0.30/24 to any -> ($oif)
block out log quick on $oif inet proto tcp from any to any port 43
pass log (all) on $oif
pass out quick on $oif all

I test vnet jail by issuing ping 8.8.8.8 and get "time to live exceeded"
message. ping 10.0.10.2 get all lost packets normal message.

Is there some other way to test vnet jails from the host to verify they
are working?

There will come a time when I will need to test vnet jails from the
public internet. Its easy to enable ssh on the vnet jail and then use
some other isp to ssh into the vnet jail. What would be the syntax of
the remote ssh command to do this?

It's my understanding that vnet jails have their own network stack which
means there is no interaction with the hosts network stack. Which also
means there is no vnet firewall interaction with the hosts firewall. Is
this correct?

Since I want all my vnet jails to access the public internet, can their
epair just be added to a single bridge as another member or does each
one need it's own bridge?

How is public internet traffic targeted to an individual vnet jail
running on the host?

Thanks for your help on this.
Ernie Luzar

[Kristof Provost]

Post by Ernie Luzar

Post by Kristof Provost
If so, how can the jail see the vge0 interface?

Through the bridge? I don't really know. Just guessing.

Think of vnet jails as separate machines. There's no mechanism for pf
hosts to exchange that sort of information between machines, so there's
no mechanism for them to exchange that between host and vnet jail.

In this case your nat rule simply won't do anything, because the vge0
interface does not exist in the jail.

Post by Ernie Luzar
I added pass to the pf nat rule so inbound packets that match entry in
state table get passed automatically.
Now using this pf nat rule
nat pass on epair2b from 10.0.0.30/24 to any -> (epair2b)
This is the ifconfig -a on the host after the vnet jail is started.

Your bridge doesn't have an IP address. How do you expect to route
traffic arriving on that interface?

To be frank, you seem to be very confused on general networking
concepts. I'd advise you to study those first, because you're going to
keep struggling until you grasp the fundamentals of how IP works.

Best regards,
Kristof

[Ernie Luzar]

Post by Kristof Provost

Post by Ernie Luzar

Post by Kristof Provost
If so, how can the jail see the vge0 interface?

Through the bridge? I don't really know. Just guessing.

Think of vnet jails as separate machines. There's no mechanism for pf
hosts to exchange that sort of information between machines, so there's
no mechanism for them to exchange that between host and vnet jail.
In this case your nat rule simply won't do anything, because the vge0
interface does not exist in the jail.

Post by Ernie Luzar
I added pass to the pf nat rule so inbound packets that match entry in
state table get passed automatically.
Now using this pf nat rule
nat pass on epair2b from 10.0.0.30/24 to any -> (epair2b)
This is the ifconfig -a on the host after the vnet jail is started.

Your bridge doesn't have an IP address. How do you expect to route
traffic arriving on that interface?
To be frank, you seem to be very confused on general networking
concepts. I'd advise you to study those first, because you're going to
keep struggling until you grasp the fundamentals of how IP works.
Best regards,
Kristof

I am shocked by your reply. For someone who has a prestigious position
as a freebsd developer you should know that this kind of unfriendly
reply is NOT what is expected on Freebsd lists. I find your remark
insulting and belittling. Other Freebsd core members have been removed
for expressing this same type of camouflaged derogatory marks. Shame on
you, you should know better.

The questions are specific to vnet jails with bridge/epair. The model
being employed is what is available from internet documentation as the
Freebsd handbook is void of any vnet info. A person in your position
should already be aware of these facts.

In 12.0 vnet has been upgraded to production status and the pf firewall
repaired to function inside of a vnet jail. These new functions are not
documented any where so of course questions are going to be asked for help.

In all my reading about vnet jails I have never seen an example of the
bridge having a ip address assigned directly to it. Only the epair
assigned to the vnet jail has an ip address.

You can redeem your bad behavior by answering the questions and adding a
complete working vnet jail using pf firewall with bridge/pair to the
12.0 release /usr/share/examples/jails so there will be some
documentation of these new production features available with 12.0
release when its published. You can not just make changes to the system
and not document them.

I'm willing to chalk this up to you having a bad day and I caught the
ricochet. Lets just move forward.