Page tree

Welcome to FreeSoftwareServers Confluence Wiki

Skip to end of metadata
Go to start of metadata

Disclaimer: This is not something I could find documentation of others doing and people on /r/zfs didn't seem to enthused. I just set this up today so I'll need to give it some time before approving.

https://www.reddit.com/r/zfs/comments/gn2ubp/using_zfs_inside_docker_container/

Note: This requires certain Kernel Modules be loaded in the host. I have yet to test this on a new system, but what I basically did was install software and then remove it, but load the modules.

Like so:

apt install -y zfsutils-linux
apt install -y nfs-kernel-server
apt purge -y zfsutils-linux nfs-kernel-server

I'll have to fine tune/test on a new system sometime.

So there are two methods to go about ensuring the userland application has the expected/proper kernel modules

  • Simple, which is just using the same Ubuntu version for the docker-image as host. This has the tradeoff of not having the latest and greatest, but also guaranteed ability to drop container and manage on host.
    • This does require configuration outside the docker-container unfortunately to get the kernel modules loaded.
  • Complicated, have container inject Kernel Modules, this can get you the latest/greatest etc. But, two things, it's more complicated and if there was any issue, you may have issues using host.
    • IMO, you should/would be able to spin up the OS the container is based on and mount the storage pool that way, but its not as straight forward.
    • Not sure stability of having host vs container load kernel modules
    • Doesn't require host configuration outside container

I do want to practice the kernel loading method sometime, but my main goal was dockerizing my fileserver configs/setup and so I decided to just tag the container w/ the hosts OS version number.

Notes: You'll need a working smb.conf and exports for NFS. This allows newer versions of ZFS, you may need to upgrade your pool to get access to these features! (Enable all features using 'zpool upgrade'.)

root@fileserver:/# zpool status
  pool: raid-z
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(5) for details.
  scan: resilvered 885M in 0 days 00:01:00 with 0 errors on Wed May 20 06:46:39 2020
config:

        NAME                        STATE     READ WRITE CKSUM
        raid-z                      ONLINE       0     0     0
          raidz1-0                  ONLINE       0     0     0
            wwn-0x5000c5008b208ae2  ONLINE       0     0     0
            sde                     ONLINE       0     0     0
            sdd                     ONLINE       0     0     0
            sdc                     ONLINE       0     0     0

errors: No known data errors
root@fileserver:/# zpool upgrade
This system supports ZFS pool feature flags.

All pools are formatted using feature flags.


Some supported features are not enabled on the following pools. Once a
feature is enabled the pool may become incompatible with software
that does not support the feature. See zpool-features(5) for details.

POOL  FEATURE
---------------
raid-z
      encryption
      project_quota
      device_removal
      obsolete_counts
      zpool_checkpoint
      spacemap_v2
      allocation_classes
      resilver_defer
      bookmark_v2

root@fileserver:/# zpool upgrade raid-z
This system supports ZFS pool feature flags.

Enabled the following features on 'raid-z':
  encryption
  project_quota
  device_removal
  obsolete_counts
  zpool_checkpoint
  spacemap_v2
  allocation_classes
  resilver_defer
  bookmark_v2


Docker-Compose:

Make sure to adjust "POOL_NAME" and also, run "modprobe_setup.sh" once.

Caveat: I haven't figured out best way to work around this but /dev/disk/by-id doesn't exist in container so HDD's are mounted as /dev/sd# which in not truly unique.

WD=/opt/fileserver
mkdir -p $WD/{mnt,setup,smb_conf,nfs_conf}
cd $WD/setup

Run Once to ensure K.M. loaded at boot.

cat << 'FOE' >modprobe_setup.sh
#!/bin/bash
cat << 'EOF' >>/etc/modules
zfs
nfsd
EOF
FOE
chmod +x modprobe_setup.sh
./modprobe_setup.sh
cat << 'EOF' >Dockerfile
#FROM ubuntu:${VER} #Not working ATM, debugging.
FROM ubuntu:bionic

RUN apt update \
     && apt install -y zfsutils-linux \
     && apt install -y nfs-kernel-server \
     && apt install -y samba 

COPY docker-*.sh /usr/local/bin/
ENTRYPOINT ["docker-entrypoint.sh"]
CMD []
EOF
cat << 'EOF' >docker-compose.yaml
version: '3.7'

services:
 fileserver:
    container_name: ${CONTAINER_NAME}
    hostname: ${CONTAINER_NAME}
    privileged: true
    build:
      context: .
      args:
        VER: ${VER}
    ports:
      - "137:137/udp"
      - "138:138/udp"
      - "139:139/tcp"
      - "445:445/tcp"
      - '2049:2049'
    volumes:
      - type: bind
        read_only: true
        source: /opt/fileserver/smb_conf/smb.conf
        target: /etc/samba/smb.conf
      - type: bind
        read_only: true
        source: /opt/fileserver/nfs_conf/exports
        target: /etc/exports
    environment:
      - 'TZ=${TZ}'
      - 'POOL_NAME=${POOL_NAME}'
EOF
chmod +x docker-compose.yaml
cd $WD/setup
cat << 'EOF'>.env
POOL_NAME=raid-z
CONTAINER_NAME=fileserver
TZ=American/Whitehorse
EOF
chmod +x .env
cat << 'EOF' >docker-entrypoint.sh
#!/bin/bash

zpool import -d /dev/ -f ${POOL_NAME}
zfs mount -a
/etc/init.d/smbd start
mkdir -p /run/sendsigs.omit.d/
/etc/init.d/rpcbind start
/etc/init.d/nfs-common start
/etc/init.d/nfs-kernel-server start
sleep infinity
EOF
chmod +x docker-entrypoint.sh
WD=/opt/fileserver/setup
cat << EOF >$WD/fileserver.service.setup.sh
cat << EOL >/lib/systemd/system/fileserver.service
[Unit]
Description=fileserver_Docker
Requires=docker.service network-online.target

[Service]

Restart=on-abnormal
ExecStartPre=/bin/sleep 30
ExecStart=/usr/bin/docker-compose --project-name fileserver --project-directory $WD -f $WD/docker-compose.yaml up
ExecStop=/usr/bin/docker-compose --project-name fileserver --project-directory $WD -f $WD/docker-compose.yaml stop

[Install]
WantedBy=multi-user.target
EOL
systemctl enable fileserver
systemctl restart fileserver
systemctl status fileserver
EOF
chmod +x $WD/fileserver.service.setup.sh
$WD/fileserver.service.setup.sh

Regular/Good Start Logs: (Note: Error regarding can't import is OK, it's already imported)

cannot import 'raid-z': a pool with that name already exists
use the form 'zpool import <pool | id> <newpool>' to give it a new name
 * Starting SMB/CIFS daemon smbd
   ...done.
 * Starting RPC port mapper daemon rpcbind
   ...done.
 * Starting NFS common utilities
   ...done.
 * Exporting directories for NFS kernel daemon...
   ...done.
 * Starting NFS kernel daemon
   ...done.

BashRC Alias:

alias fscli="docker exec -it fileserver /bin/bash"

function zpool(){
        command /usr/bin/docker exec -it fileserver zpool "$@"
}
function zfs(){
        command /usr/bin/docker exec -it fileserver zfs "$@"
}
source ~/.bashrc
  • No labels