--- layout: post title: ZFS mirroring with a restricted SSH account & zxfer date: 2015-12-26 tags: ["backups","freebsd","General","unix","ZFS","zxfer"] --- As part of my transition from using a combination of Linux and FreeBSD for our home servers to being exclusively FreeBSD, I wanted to update how I did backups from my public server, bree, to the internal storage server, rivendell. Previously, I had done this with a home grown script which used rsync to transfer updates to the storage server overnight. This solution worked just fine, but was not the most efficient (see: [rsync.net: ZFS Replication to the cloud is finally here-and it's fast](http://arstechnica.com/information-technology/2015/12/rsync-net-zfs-replication-to-the-cloud-is-finally-here-and-its-fast/)). While I didn't intend to replicate to rsync.net I wanted to leverage ZFS since I am now going FreeBSD to FreeBSD. There are numerous [articles](https://zpool.org/zfs-snapshots-and-remote-replication/) about using [zxfer](https://github.com/allanjude/zxfer) to perform backups but there was one big hiccup that I couldn't get over. Quoting the man page: zxfer -dFkPv -o copies=2,compression=lzjb -T root@192.168.123.1 -R storage backup01/pools Having to open up the `root` account on my storage server, no matter how I restricted it to IP address, keys, whatever, makes me really uncomfortable and a show-stopper for me. But I thought I could do better. I have limited experience using restricted-shells to limit access to servers before and I knew that ZFS allows for delegating permissions to non-root users so I decided to give it a shot. TL;DR: It can work. The configuration had a few phases to it: 1. Create a new restricted user account on my backup server and configure the commands that `zxfer` needs access to in the restricted shell 2. Create the destination zfs filesystem to receive the mirror and configure the delegated permissions for the backup user 3. Set up access to the backup server from the source server via SSH 4. Make slight modification to `zxfer` to allow it to run `zfs` command from the `PATH` instead of hardcoding the path in the script ## Setting up the restricted user I created a new user on the backup system named `zbackup` that would be my restricted user for receiving the backups. The goal was for this user to be as limited as possible. It should only be allowed to run the commands necessary for `zxfer` to do its job. I landed on using `rzsh` as the restricted shell as it was the first one I got working with the correct environment. I set up a directory to hold binaries that the `zbackup` user was allowed to use. root@storage$ mkdir /usr/local/restricted_bin root@storage$ ln -s /sbin/zfs /usr/local/restricted_bin/zfs root@storage$ ln -s /usr/bin/uname /usr/local/restricted_bin/uname I then set up the `.zshenv` file for the `zbackup` user to restrict the user to that directory for executables. export PATH=/usr/local/restricted_bin ## Setting up the destination zfs filesystem I already had a zfs filesystem that was devoted to backups so I made a new zfs filesystem underneath it to hold these new backups and be a point where I could set delegation points for permissions. Then, through trial and error, I figured out all the permissions I had to delegate to the `zbackup` user on the filesystem to allow `zxfer` to work root@storage$ zfs create nas/backup/bree-zxfer root@storage$ chown zbackup:zbackup /nas/backup/bree-zxfer root@storage$ zfs allow -u zbackup atime,canmount,casesensitivity,checksum,compression,copies,create, dedup,destroy,exec,filesystem_count,filesystem_limit,jailed,logbias,mount, normalization,quota,readonly,receive,recordsize,redundant_metadata, refquota,refreservation,reservation,setuid,sharenfs,sharesmb,snapdir, snapshot_count,snapshot_limit,sync,userprop,utf8only,volmode nas/backup/bree-zxfer (I figured out the list of actions and properties that I needed to delegate by having `zxfer` dump the `zfs create` command it was trying to run on the backup system when it failed.) _Update:_ I forgot 1 thing that is critical to making this work. You need to ensure that non-root users are allowed to mount filesystems. This can be accomplished by adding the following line to your `/etc/sysctl.conf` and rebooting: vfs.usermount=1 ## Remote access to the backup server Nothing fancy here. On my source server, I created a new SSH keypair for the `root` user (no problem with running the source `zfs` command as `root`). I then copied the public half of that key to the `authorized_keys` file of the `zbackup` user on the backup server. At this point, I could ssh from my source server to the backup server as the `zbackup` user. But when logged in to the backup server, the only commands that could be run are those in the `/usr/local/restricted_bin` directory (`zfs` and `uname`). ## Tweak zxfer script to remove hard coded path in zfs commands One of the limitations (intentional) of a restricted shell is that the restricted user is not allowed to specify a full pathname for any commands. Only commands located in their `PATH` can be run. Unfortunately, while the `zbackup` user has the `zfs` command in their `PATH`, it is referenced as `/sbin/zfs` in the `zxfer` script. To work around this, I modified the `zxfer` script to not use the path of `zfs` directly and assume that `zfs` will be in the path. This was only in 2 places of the script. If you do a quick search for `/sbin/zfs` you will find them. ## Moment of truth! After all this, I was now able to run any number of commands to mirror my source servers `zfs` filesystems (with snapshots) to my backup server. root@source$ zxfer -dFPv -T zbackup@storage -N zroot/git nas/backup/bree-zxfer root@source$ zxfer -dFPv -T zbackup@storage -R zroot/var nas/backup/bree-zxfer And best of all, the storage server does not have SSH enabled for root. Success.