ipsec/strongswan server on docker works on lxd container not

taumeister · 04-18-2018, 04:30 PM

Hi,

I'm trying to setup an container with ipsec vpn on an ubuntu 16.04 LXD Host on Vmware Workstation 14 ( Promiscuous Mode is on )

With docker I can use the ready docker image from here
https://github.com/hwdsl2/docker-ipsec-vpn-server
Unfortunately, I can't do this under LXD containers.

With this tested and working skript ( attached )

I get in the end..

Code:

sysctl: permission denied on key 'kernel.msgmnb'
sysctl: permission denied on key 'kernel.msgmax'
sysctl: permission denied on key 'kernel.shmmax'
sysctl: permission denied on key 'kernel.shmall'

I understand that the isolated container has no right to work on the host kernel, but why does this work with docker?

Does a Docker Container have more rights than an LXD Container?

I tried to :
lxc config set vpn-server raw.lxc lxc.aa_profile=unconfined

but that did not work.

Any suggestions to get such projects run, or better use docker or...docker on lxd??

simosx · 04-19-2018, 05:18 AM

Those kernel.msgmax, etc keys are not per-process keys,
therefore it does not make sense to change them from within a container.
When they are changed, they are applied to the whole kernel, isn't it?

I think it would be better if you adapt the script to run on the host,
make the appropriate kernel settings, and then launch the VPN container for you.

The other option is to split the script into a part that you run on the host (as root),
and the other part you run in the LXD container.

LXD has LXCFS, which is a filesystem that adapts a container's /proc filesystem
so that it makes sense to the container. For example, if you assign one CPU core
to a container, /proc/cpuinfo will show a single core.
I suppose Docker has something similar, but with LXD it should be more pronounced
because with LXD you get machine containers which are closer to a VM (than Docker containers).

taumeister · 04-20-2018, 04:24 PM

Thanks for your reply simosx.

LXC containers are closer to a VM than Docker, that's right, that's why I thought it would be easier to provide a VPN server over LXC.

It's no problem with Docker.

My idea was to move all docker containers into a combined LXC container or at least into several.
I have not yet succeeded with l2tp/ipsec VPN. I guess I'll stay with Docker at VPN. Too bad.
On the weekend I will try to "banish" my combination of owncloud and a website, both with nginx ssl proxy upstream, into an LXC container.
Works as said also super under docker.

Somehow I was hoping to put all my docker containers into a single LXC container...let's see...

PS. Installing the VPN into the LXC host is out of the question for me, that would make no sense to me.
At work I would never think of installing another program or role in our VMWare or Hyper-V hosts.

But anyway, thanks for your help.

simosx · 04-21-2018, 07:57 AM

Perhaps I was not clear enough. The script that you attached above, has the following segment,

Code:

cat >> /etc/sysctl.conf <<EOF

# Added by hwdsl2 VPN script
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = $SHM_MAX
kernel.shmall = $SHM_ALL

net.ipv4.ip_forward = 1
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.$net_iface.send_redirects = 0
net.ipv4.conf.$net_iface.rp_filter = 0

net.core.wmem_max = 12582912
net.core.rmem_max = 12582912
net.ipv4.tcp_rmem = 10240 87380 12582912
net.ipv4.tcp_wmem = 10240 87380 12582912
EOF

These lines (the first four and the last four) instruct the Linux kernel to allocate a bit more kernel memory for networking,
which makes it better when you setup strongswan/IPSec.

Whether these lines (first four and last four) are run in a container or the host,
the effect is the same; there is a single Linux kernel and the changes will be global to the whole server.

Therefore, you can move those 4+4 lines into a separate script, and run that script on the host:

Code:

cat >> /etc/sysctl.conf <<EOF

# Added by hwdsl2 VPN script
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = $SHM_MAX
kernel.shmall = $SHM_ALL

net.core.wmem_max = 12582912
net.core.rmem_max = 12582912
net.ipv4.tcp_rmem = 10240 87380 12582912
net.ipv4.tcp_wmem = 10240 87380 12582912
EOF

An alternative would be to edit the AppArmor rules on the host to allow setting those sysctl values from within a container.
The file is /etc/apparmor.d/abstractions/lxc/container-base and the lines that fail you, are the ones that I commented out.

Code:

  # generated by: lxc-generate-aa-rules.py container-rules.base
#  deny /proc/sys/[^kn]*{,/**} wklx,
#  deny /proc/sys/k[^e]*{,/**} wklx,
#  deny /proc/sys/ke[^r]*{,/**} wklx,
#  deny /proc/sys/ker[^n]*{,/**} wklx,
#  deny /proc/sys/kern[^e]*{,/**} wklx,
#  deny /proc/sys/kerne[^l]*{,/**} wklx,
#  deny /proc/sys/kernel/[^smhd]*{,/**} wklx,
  deny /proc/sys/kernel/d[^o]*{,/**} wklx,
  deny /proc/sys/kernel/do[^m]*{,/**} wklx,
  deny /proc/sys/kernel/dom[^a]*{,/**} wklx,
  deny /proc/sys/kernel/doma[^i]*{,/**} wklx,
  deny /proc/sys/kernel/domai[^n]*{,/**} wklx,
  deny /proc/sys/kernel/domain[^n]*{,/**} wklx,
  deny /proc/sys/kernel/domainn[^a]*{,/**} wklx,
  deny /proc/sys/kernel/domainna[^m]*{,/**} wklx,
  deny /proc/sys/kernel/domainnam[^e]*{,/**} wklx,
  deny /proc/sys/kernel/domainname?*{,/**} wklx,
  deny /proc/sys/kernel/h[^o]*{,/**} wklx,
  deny /proc/sys/kernel/ho[^s]*{,/**} wklx,
  deny /proc/sys/kernel/hos[^t]*{,/**} wklx,
  deny /proc/sys/kernel/host[^n]*{,/**} wklx,
  deny /proc/sys/kernel/hostn[^a]*{,/**} wklx,
  deny /proc/sys/kernel/hostna[^m]*{,/**} wklx,
  deny /proc/sys/kernel/hostnam[^e]*{,/**} wklx,
  deny /proc/sys/kernel/hostname?*{,/**} wklx,
  deny /proc/sys/kernel/m[^s]*{,/**} wklx,
  deny /proc/sys/kernel/ms[^g]*{,/**} wklx,
  deny /proc/sys/kernel/msg*/** wklx,
  deny /proc/sys/kernel/s[^he]*{,/**} wklx,
  deny /proc/sys/kernel/se[^m]*{,/**} wklx,
  deny /proc/sys/kernel/sem*/** wklx,
  deny /proc/sys/kernel/sh[^m]*{,/**} wklx,
  deny /proc/sys/kernel/shm*/** wklx,
  deny /proc/sys/kernel?*{,/**} wklx,
#  deny /proc/sys/n[^e]*{,/**} wklx,
#  deny /proc/sys/ne[^t]*{,/**} wklx,
#  deny /proc/sys/net?*{,/**} wklx,
  deny /sys/[^fdc]*{,/**} wklx,
  deny /sys/c[^l]*{,/**} wklx,
  deny /sys/cl[^a]*{,/**} wklx,
  deny /sys/cla[^s]*{,/**} wklx,
  deny /sys/clas[^s]*{,/**} wklx,
  deny /sys/class/[^n]*{,/**} wklx,
  deny /sys/class/n[^e]*{,/**} wklx,
  deny /sys/class/ne[^t]*{,/**} wklx,
  deny /sys/class/net?*{,/**} wklx,
  deny /sys/class?*{,/**} wklx,
  deny /sys/d[^e]*{,/**} wklx,
  deny /sys/de[^v]*{,/**} wklx,
  deny /sys/dev[^i]*{,/**} wklx,
  deny /sys/devi[^c]*{,/**} wklx,
  deny /sys/devic[^e]*{,/**} wklx,
  deny /sys/device[^s]*{,/**} wklx,
  deny /sys/devices/[^v]*{,/**} wklx,
  deny /sys/devices/v[^i]*{,/**} wklx,
  deny /sys/devices/vi[^r]*{,/**} wklx,
  deny /sys/devices/vir[^t]*{,/**} wklx,
  deny /sys/devices/virt[^u]*{,/**} wklx,
  deny /sys/devices/virtu[^a]*{,/**} wklx,
  deny /sys/devices/virtua[^l]*{,/**} wklx,
  deny /sys/devices/virtual/[^n]*{,/**} wklx,
  deny /sys/devices/virtual/n[^e]*{,/**} wklx,
  deny /sys/devices/virtual/ne[^t]*{,/**} wklx,
  deny /sys/devices/virtual/net?*{,/**} wklx,
  deny /sys/devices/virtual?*{,/**} wklx,
  deny /sys/devices?*{,/**} wklx,
  deny /sys/f[^s]*{,/**} wklx,
  deny /sys/fs/[^c]*{,/**} wklx,
  deny /sys/fs/c[^g]*{,/**} wklx,
  deny /sys/fs/cg[^r]*{,/**} wklx,
  deny /sys/fs/cgr[^o]*{,/**} wklx,
  deny /sys/fs/cgro[^u]*{,/**} wklx,
  deny /sys/fs/cgrou[^p]*{,/**} wklx,
  deny /sys/fs/cgroup?*{,/**} wklx,
  deny /sys/fs?*{,/**} wklx,

After you make the change, you need to reboot in order for the new rules to take effect.

Personally, I suggest not to touch the AppArmor rules but rather do what I described earlier.
That is, create a separate script to apply the special sysctl settings directly on the host.