TrueNAS’s snapshot replication functionality is useful for implementing off-site backups. I currently use this functionality to perform a two-way cross-backup of two TrueNAS installations. Unfortunately, the current version of TrueNAS doesn’t seem to provide many options for building strong protections against local compromise (e.g. ransomware attacks that target TrueNAS) having the ability to compromise backups.

The core requirement I have for my personal data is: total compromise of a single TrueNAS machine should not be able to compromise previous backups stored on the other TrueNAS machine.

In practice, I belive this means:

  • Snapshots must be pulled by the backup machine, rather than pushed by the target.
    • This prevents a potentially compromised target from compromising backups on the backup machine.
  • The credentials used by the backup machine must not have privileges that exceed what is required (i.e. no write access on the target).
    • This prevents a potentially compromised backup machine from compromising data on the target.

TrueNAS offers functionality for implementing the former and it is built into the web UI.

However, the default snapshot replication functionality offered by TrueNAS typically requires the use of a privileged user on the target (presumably in order to run zfs commands). The best built-in mitigation offered seems to be to use an otherwise-unprivileged user that has sudo access limited to the zfs command. Unfortunately, still seems too broad of a privilege, since it doesn’t preclude write-access.

The good news is that TrueNAS does seem to bless the use of zfs allow commands to implement finer-grained controls (it is mentioned in the tooltip for the “Use Sudo For ZFS Commands” option), despite the warnings to the contrary shown when you open a shell. The bad news is that there doesn’t seem to be much clear guidance in the TrueNAS manuals on how to implement this.

Here is how I ended up implementing low-trust snapshot replication:

Create an unprivileged backup user on the target:

  • It should not be a member of any groups, or have an SMB account.
  • It should have a real shell (i.e. not nologin).
  • It should have a home directory somewhere (in order to store a ~/.ssh/authorized_keys file).
  • sudo access should not be allowed (not even for zfs).

Grant the backup user privileges to send snapshots:

With an admin account, open a shell and use the following command:

sudo zfs allow <user> send,hold

Add backup credentials on the backup machine:

  1. On the backup machine, navigate to Credentials > Backup Credentials.
  2. Create an SSH keypair.
    • Add the public key to the list of Authorized Keys for the backup user on the target.
  3. Create a new SSH connection.
    • Change Setup Method to manual.
    • Populate the fields, using the backup user on the target as the Username.
    • Select the Private Key created previously.
    • Populate the SSH host key of the target.
      • This is the output of cat /etc/ssh/ssh_host_*.pub on the target.
      • If the network between the backup machine and target is trusted, the “Discover Remote Host Key” button can also be used for convenience.
  4. Save.

Finally, when setting up a replication task on the backup machine, use the SSH connection established previously. Use “On a different system” for the source location (i.e. a PULL configuration), and do not accept the use of sudo for running ZFS commands.