Discussion:
setfib, jails and loopback interfaces
(too old to reply)
Marko Cupać
2017-05-31 08:33:49 UTC
Permalink
Raw Message
Hi,

I'm not subscribed to the list, could you please keep me in CC?

I'm using ezjail as instructed in Handbook, assigning jails
lo1|127.0.0.X,bce0|10.66.66.X addresses, in order to keep jails'
loopback traffic off host's, and in order to be able to keep internal
services on lo1 (such as redis, mongodb, mysql etc.), and external on
bce0 (such as apache, unifi5 etc.).

Recently I got a server with multiple NICs, and I'd like to serve both
LAN and DMZ services from it. I found some information on how to
accomplish that with setfib:

# cat /boot/loader.conf
net.fibs=4
net.add_addr_allfibs=0

# cat /etc/rc.conf
...
cloned_interfaces="lo1"
static_routes="nix nixd"
route_nix="-net 10.66.66.0/24 -interface bce0 -fib 1"
route_nixd="default 10.66.66.254 -fib 1"
...

In this setup, services bound to bce0 interface work fine, but they
can't contact internal services on lo1. I guess it has something to do
with jail routing, but can't figure out what.

Thank you in advance for any hints.
--
Before enlightenment - chop wood, draw water.
After enlightenment - chop wood, draw water.

Marko Cupać
https://www.mimar.rs/
James Gritton
2017-06-02 16:09:24 UTC
Permalink
Raw Message
Post by Marko Cupać
Hi,
I'm not subscribed to the list, could you please keep me in CC?
I'm using ezjail as instructed in Handbook, assigning jails
lo1|127.0.0.X,bce0|10.66.66.X addresses, in order to keep jails'
loopback traffic off host's, and in order to be able to keep internal
services on lo1 (such as redis, mongodb, mysql etc.), and external on
bce0 (such as apache, unifi5 etc.).
Recently I got a server with multiple NICs, and I'd like to serve both
LAN and DMZ services from it. I found some information on how to
# cat /boot/loader.conf
net.fibs=4
net.add_addr_allfibs=0
# cat /etc/rc.conf
...
cloned_interfaces="lo1"
static_routes="nix nixd"
route_nix="-net 10.66.66.0/24 -interface bce0 -fib 1"
route_nixd="default 10.66.66.254 -fib 1"
...
In this setup, services bound to bce0 interface work fine, but they
can't contact internal services on lo1. I guess it has something to do
with jail routing, but can't figure out what.
Thank you in advance for any hints.
I haven't done the lo1 trick before, but I have had jails with addresses
on a different FIB. Note that the jail also has an FIB. You probably
at least want to set the jail's fib to 1 (exec.fib in jail.conf, I
suppose jail_*_fib or whatever in the old rc-based system ezjail still
uses).

The part I'm not sure about is you probably also want to have lo1's
entries in the fib=1 routing table. I don't know the interaction
between cloned_interfaces and fib though - that might take some
exploring in rc, or a word or two from someone who knows that side of
things more than I do.

- Jamie
Marko Cupać
2017-06-05 10:25:30 UTC
Permalink
Raw Message
Guys,

thank you for qjail tip, I'm going to give it a try.

I have solved my current problem by adding appropriate static route in
rc.conf:

route_nix_lo1="-net 127.0.1.0/24 -interface lo1 -fib 1"

Best regards,
--
Before enlightenment - chop wood, draw water.
After enlightenment - chop wood, draw water.

Marko Cupać
https://www.mimar.rs/
Loading...