Discussion:
Fixed ip's for jails
(too old to reply)
Heinz Gies
2017-07-06 18:57:10 UTC
Permalink
Raw Message
Good afternoon everyone,
we are currently contemplating how to handle multi-tenant Jails best to allow ProjectFiFo to administer FreeBSD Jails along with Solaris Zones.

The hurdle we have run into is the following:

With vnet Jails the owner could change the IP, making it impractical for multi-tenancy.

With no-vnet Jails, all the jails would share the same network stack removing a layer of isolation and risking a noisy neighbour problem.

There are a few possible solutions it seems.

Allen suggested using the firewall to restrict the traffic from a vnet in the global zone (host system, not sure how BSD calls it). The top of mind issue with this is that it would block multicast.

An alternative Kevin came up with was putting a jail inside another jail, with the outer jail being a vimage jail and the inner jail using a static (non-vnet IP). This would also mean later on beehive inside a jail would be easier as it could follow the same logic. On the other hand, I am a bit worried about unforeseen consequences of this approach. Also, I am not 100% positive whether the inner jail would use the vnet network stack for it is IP and not the global one.

Thank you for your input.

Cheers,
Heinz


Heinz N. Gies
Project-FiFo
Cloud Orchestration
Web:

project-fifo.net<https://project-fifo.net/>

Docs:

Documentation<https://docs.project-fifo.net/en/latest/index.html>

Tickets:
GPG:

Ticket Tracker<https://project-fifo.atlassian.net/>
452B6F98<https://project-fifo.net/heinz.gpg>
James Gritton
2017-07-07 15:18:47 UTC
Permalink
Raw Message
Post by Heinz Gies
Good afternoon everyone,
we are currently contemplating how to handle multi-tenant Jails best
to allow ProjectFiFo to administer FreeBSD Jails along with Solaris
Zones.
With vnet Jails the owner could change the IP, making it impractical for multi-tenancy.
With no-vnet Jails, all the jails would share the same network stack
removing a layer of isolation and risking a noisy neighbour problem.
There are a few possible solutions it seems.
Allen suggested using the firewall to restrict the traffic from a vnet
in the global zone (host system, not sure how BSD calls it). The top
of mind issue with this is that it would block multicast.
An alternative Kevin came up with was putting a jail inside another
jail, with the outer jail being a vimage jail and the inner jail using
a static (non-vnet IP). This would also mean later on beehive inside a
jail would be easier as it could follow the same logic. On the other
hand, I am a bit worried about unforeseen consequences of this
approach. Also, I am not 100% positive whether the inner jail would
use the vnet network stack for it is IP and not the global one.
Thank you for your input.
I'm for the jail within a jail. Rest assured that the child jail will
be inside the vnet that the parent jail was created with. The outer
jail need not be running anything - just run enough to set up the
network, and set its "persist" parameter. Then it's nothing but an
empty shell to hold the network for the inner jail, with almost no
overhead. The inner jail doesn't need any IP restrictions, which
inherits (but can't change) the parent's addresses. Something like
this:

foo {
vnet;
vnet.interface = foo0;
persist;
}

foo.client {
<normal jail parameters>;
}

As for unforeseen consequences, I don't foresee any ;-).

- Jamie
Heinz Gies
2017-07-09 16:56:20 UTC
Permalink
Raw Message
Thanks a lot Jamie :)
With a bit of experimenting it ended up working pr


Heinz N. Gies
Project-FiFo
Cloud Orchestration
Web:
project-fifo.net <https://project-fifo.net/>
Docs:
Documentation <https://docs.project-fifo.net/en/latest/index.html>
Tickets:
GPG:
Ticket Tracker <https://project-fifo.atlassian.net/>
452B6F98 <https://project-fifo.net/heinz.gpg>
Post by Heinz Gies
Good afternoon everyone,
we are currently contemplating how to handle multi-tenant Jails best
to allow ProjectFiFo to administer FreeBSD Jails along with Solaris
Zones.
With vnet Jails the owner could change the IP, making it impractical for multi-tenancy.
With no-vnet Jails, all the jails would share the same network stack
removing a layer of isolation and risking a noisy neighbour problem.
There are a few possible solutions it seems.
Allen suggested using the firewall to restrict the traffic from a vnet
in the global zone (host system, not sure how BSD calls it). The top
of mind issue with this is that it would block multicast.
An alternative Kevin came up with was putting a jail inside another
jail, with the outer jail being a vimage jail and the inner jail using
a static (non-vnet IP). This would also mean later on beehive inside a
jail would be easier as it could follow the same logic. On the other
hand, I am a bit worried about unforeseen consequences of this
approach. Also, I am not 100% positive whether the inner jail would
use the vnet network stack for it is IP and not the global one.
Thank you for your input.
I'm for the jail within a jail. Rest assured that the child jail will
be inside the vnet that the parent jail was created with. The outer
jail need not be running anything - just run enough to set up the
network, and set its "persist" parameter. Then it's nothing but an
empty shell to hold the network for the inner jail, with almost no
overhead. The inner jail doesn't need any IP restrictions, which
inherits (but can't change) the parent's addresses. Something like
this:

foo {
vnet;
vnet.interface = foo0;
persist;
}

foo.client {
<normal jail parameters>;
}

As for unforeseen consequences, I don't foresee any ;-).

- Jamie
Heinz Gies
2017-07-09 16:59:07 UTC
Permalink
Raw Message
Thanks a lot Jamie :),
With a bit of experimenting it ended up working quite well! I wanted to give some feedback so your advice doesn’t end up on a black hole.

This is what it ended up with, outer jail with vnet inner jail with a inherited NIC.

https://github.com/project-fifo/r-vmadm/pull/20

Cheers,
Heinz


Heinz N. Gies
Project-FiFo
Cloud Orchestration
Web:
project-fifo.net <https://project-fifo.net/>
Docs:
Documentation <https://docs.project-fifo.net/en/latest/index.html>
Tickets:
GPG:
Ticket Tracker <https://project-fifo.atlassian.net/>
452B6F98 <https://project-fifo.net/heinz.gpg>
Post by Heinz Gies
Good afternoon everyone,
we are currently contemplating how to handle multi-tenant Jails best
to allow ProjectFiFo to administer FreeBSD Jails along with Solaris
Zones.
With vnet Jails the owner could change the IP, making it impractical for multi-tenancy.
With no-vnet Jails, all the jails would share the same network stack
removing a layer of isolation and risking a noisy neighbour problem.
There are a few possible solutions it seems.
Allen suggested using the firewall to restrict the traffic from a vnet
in the global zone (host system, not sure how BSD calls it). The top
of mind issue with this is that it would block multicast.
An alternative Kevin came up with was putting a jail inside another
jail, with the outer jail being a vimage jail and the inner jail using
a static (non-vnet IP). This would also mean later on beehive inside a
jail would be easier as it could follow the same logic. On the other
hand, I am a bit worried about unforeseen consequences of this
approach. Also, I am not 100% positive whether the inner jail would
use the vnet network stack for it is IP and not the global one.
Thank you for your input.
I'm for the jail within a jail. Rest assured that the child jail will
be inside the vnet that the parent jail was created with. The outer
jail need not be running anything - just run enough to set up the
network, and set its "persist" parameter. Then it's nothing but an
empty shell to hold the network for the inner jail, with almost no
overhead. The inner jail doesn't need any IP restrictions, which
inherits (but can't change) the parent's addresses. Something like
this:

foo {
vnet;
vnet.interface = foo0;
persist;
}

foo.client {
<normal jail parameters>;
}

As for unforeseen consequences, I don't foresee any ;-).

- Jamie

Loading...