diff --git a/docs/Getting Started/Arch Linux/Root on ZFS/0-overview.rst b/docs/Getting Started/Arch Linux/Root on ZFS/0-overview.rst index 938c522..aca8b0b 100644 --- a/docs/Getting Started/Arch Linux/Root on ZFS/0-overview.rst +++ b/docs/Getting Started/Arch Linux/Root on ZFS/0-overview.rst @@ -62,12 +62,12 @@ Dataset layout | | | | | datasets", "persistent datasets"; also | | | | | | include /var/lib, /srv, ... | +---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ -| bpool/sys/BOOT/default | noauto | legacy /boot | no | noauto is used to switch BE. because of | +| bpool/sys/BOOT/default | noauto | /boot | no | noauto is used to switch BE. because of | | | | | | noauto, must use fstab to mount | +---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ | rpool/sys/ROOT/default | noauto | / | no | mounted by initrd zfs hook | +---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ -| bpool/sys/BOOT/be1 | noauto | legacy /boot | no | see bpool/sys/BOOT/default | +| bpool/sys/BOOT/be1 | noauto | /boot | no | see bpool/sys/BOOT/default | +---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ | rpool/sys/ROOT/be1 | noauto | / | no | see rpool/sys/ROOT/default | +---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ @@ -120,3 +120,32 @@ Encryption As enabling Secure Boot is device specific, this is not covered in detail. + +Booting with disk failure +~~~~~~~~~~~~~~~~~~~~~~~~~ + +This guide is written with disk failure in mind. + +If disks used in Root on ZFS pool failed, but +sufficient redundancy for both root pool and boot pool +still exists, the system will still boot normally. + +Swap partition on the failed disk will fail to mount, +after an 1m30s timeout. + +This feature is useful for use cases such +as an unattended remote server. + +Example: + + - System has disks ``n>1`` + + - Installed with mirrored setup + + - Mirrored setup can tolerate up to ``n-1`` disk failures + + - Disconnect one or more disks, keep at least + one disk connected + + - System still boots, but fails to mount swap and + EFI partition diff --git a/docs/Getting Started/Arch Linux/Root on ZFS/1-preparation.rst b/docs/Getting Started/Arch Linux/Root on ZFS/1-preparation.rst index b95041b..0d369ec 100644 --- a/docs/Getting Started/Arch Linux/Root on ZFS/1-preparation.rst +++ b/docs/Getting Started/Arch Linux/Root on ZFS/1-preparation.rst @@ -73,7 +73,7 @@ Preparation ls /dev/disk/by-id/* If using virtio as disk bus, use - ``/dev/disk/by-path/*`` or ``/dev/vd*``. + ``/dev/disk/by-path/*``. Declare disk array:: diff --git a/docs/Getting Started/Arch Linux/Root on ZFS/2-system-installation.rst b/docs/Getting Started/Arch Linux/Root on ZFS/2-system-installation.rst index 84a6eed..8c33a88 100644 --- a/docs/Getting Started/Arch Linux/Root on ZFS/2-system-installation.rst +++ b/docs/Getting Started/Arch Linux/Root on ZFS/2-system-installation.rst @@ -169,12 +169,11 @@ System Installation zfs create -o canmount=off -o mountpoint=none bpool_$INST_UUID/$INST_ID/BOOT zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID/ROOT zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID/DATA - zfs create -o mountpoint=legacy -o canmount=noauto bpool_$INST_UUID/$INST_ID/BOOT/default + zfs create -o mountpoint=/boot -o canmount=noauto bpool_$INST_UUID/$INST_ID/BOOT/default zfs create -o mountpoint=/ -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/default zfs create -o mountpoint=/ -o canmount=noauto rpool_$INST_UUID/$INST_ID/ROOT/default zfs mount rpool_$INST_UUID/$INST_ID/ROOT/default - mkdir /mnt/boot - mount -t zfs bpool_$INST_UUID/$INST_ID/BOOT/default /mnt/boot + zfs mount bpool_$INST_UUID/$INST_ID/BOOT/default for i in {usr,var,var/lib}; do zfs create -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/default/$i diff --git a/docs/Getting Started/Arch Linux/Root on ZFS/3-system-configuration.rst b/docs/Getting Started/Arch Linux/Root on ZFS/3-system-configuration.rst index 22e7906..5d3bbff 100644 --- a/docs/Getting Started/Arch Linux/Root on ZFS/3-system-configuration.rst +++ b/docs/Getting Started/Arch Linux/Root on ZFS/3-system-configuration.rst @@ -11,21 +11,9 @@ System Configuration echo GRUB_CMDLINE_LINUX=\"zfs_import_dir=${INST_PRIMARY_DISK%/*}\" >> /mnt/etc/default/grub -#. Generate list of datasets for `zfs-mount-generator - `__ to mount them at boot:: - - # tab-separated zfs properties - # see /etc/zfs/zed.d/history_event-zfs-list-cacher.sh - export \ - PROPS="name,mountpoint,canmount,atime,relatime,devices,exec\ - ,readonly,setuid,nbmand,encroot,keylocation" - mkdir -p /mnt/etc/zfs/zfs-list.cache - zfs list -H -t filesystem -o $PROPS -r rpool_$INST_UUID > /mnt/etc/zfs/zfs-list.cache/rpool_$INST_UUID - sed -Ei "s|/mnt/?|/|" /mnt/etc/zfs/zfs-list.cache/* - #. Generate fstab:: - echo bpool_$INST_UUID/$INST_ID/BOOT/default /boot zfs rw,xattr,posixacl 0 0 >> /mnt/etc/fstab + genfstab -U /mnt | sed 's;zfs[[:space:]]*;zfs zfsutil,;g' | grep "zfs zfsutil" >> /mnt/etc/fstab for i in ${DISK}; do echo UUID=$(blkid -s UUID -o value ${i}-part1) /boot/efis/${i##*/}-part1 vfat \ x-systemd.idle-timeout=1min,x-systemd.automount,noauto,umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab @@ -108,7 +96,7 @@ System Configuration systemctl enable zfs-import-scan.service zfs-import.target zfs-zed zfs.target --root=/mnt systemctl disable zfs-mount --root=/mnt - At boot, datasets on rpool are mounted with ``zfs-mount-generator``, + At boot, datasets on rpool are mounted with ``/etc/fstab``, which can control the mounting process more precisely than ``zfs-mount.service``. #. Chroot:: diff --git a/docs/Getting Started/Arch Linux/Root on ZFS/4-optional-configuration.rst b/docs/Getting Started/Arch Linux/Root on ZFS/4-optional-configuration.rst index 6b85568..5ef3be4 100644 --- a/docs/Getting Started/Arch Linux/Root on ZFS/4-optional-configuration.rst +++ b/docs/Getting Started/Arch Linux/Root on ZFS/4-optional-configuration.rst @@ -227,7 +227,7 @@ Persistent swap and hibernation resume=/dev/mapper/${INST_SWAPMAPPER}\"" \ >> /etc/default/grub - Note that hibernation might not work with discrete graphics or + Note that hibernation might not work with discrete graphics, virtio graphics or AMD APU integrated graphics. This is not specific to this guide. Computer must resume from a continuous swap space, resume diff --git a/docs/Getting Started/Fedora/Root on ZFS/0-overview.rst b/docs/Getting Started/Fedora/Root on ZFS/0-overview.rst index b66df47..dce8863 100644 --- a/docs/Getting Started/Fedora/Root on ZFS/0-overview.rst +++ b/docs/Getting Started/Fedora/Root on ZFS/0-overview.rst @@ -63,12 +63,12 @@ Dataset layout | | | | | datasets", "persistent datasets"; also | | | | | | include /var/lib, /srv, ... | +---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ -| bpool/sys/BOOT/default | noauto | legacy /boot | no | noauto is used to switch BE. because of | +| bpool/sys/BOOT/default | noauto | /boot | no | noauto is used to switch BE. because of | | | | | | noauto, must use fstab to mount | +---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ | rpool/sys/ROOT/default | noauto | / | no | mounted by initrd zfs hook | +---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ -| bpool/sys/BOOT/be1 | noauto | legacy /boot | no | see bpool/sys/BOOT/default | +| bpool/sys/BOOT/be1 | noauto | /boot | no | see bpool/sys/BOOT/default | +---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ | rpool/sys/ROOT/be1 | noauto | / | no | see rpool/sys/ROOT/default | +---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ @@ -110,3 +110,32 @@ Encryption which should be sufficient for most purposes. Secure Boot is not supported out-of-the-box due to ZFS module. + +Booting with disk failure +~~~~~~~~~~~~~~~~~~~~~~~~~ + +This guide is written with disk failure in mind. + +If disks used in Root on ZFS pool failed, but +sufficient redundancy for both root pool and boot pool +still exists, the system will still boot normally. + +Swap partition on the failed disk will fail to mount, +after an 1m30s timeout. + +This feature is useful for use cases such +as an unattended remote server. + +Example: + + - System has disks ``n>1`` + + - Installed with mirrored setup + + - Mirrored setup can tolerate up to ``n-1`` disk failures + + - Disconnect one or more disks, keep at least + one disk connected + + - System still boots, but fails to mount swap and + EFI partition diff --git a/docs/Getting Started/Fedora/Root on ZFS/1-preparation.rst b/docs/Getting Started/Fedora/Root on ZFS/1-preparation.rst index 41a1d0f..4f9c8bf 100644 --- a/docs/Getting Started/Fedora/Root on ZFS/1-preparation.rst +++ b/docs/Getting Started/Fedora/Root on ZFS/1-preparation.rst @@ -71,7 +71,7 @@ Preparation ls /dev/disk/by-id/* If using virtio as disk bus, use - ``/dev/disk/by-path/*`` or ``/dev/vd*``. + ``/dev/disk/by-path/*``. Declare disk array:: diff --git a/docs/Getting Started/Fedora/Root on ZFS/2-system-installation.rst b/docs/Getting Started/Fedora/Root on ZFS/2-system-installation.rst index e2c1aaa..7257b5e 100644 --- a/docs/Getting Started/Fedora/Root on ZFS/2-system-installation.rst +++ b/docs/Getting Started/Fedora/Root on ZFS/2-system-installation.rst @@ -178,12 +178,11 @@ System Installation zfs create -o canmount=off -o mountpoint=none bpool_$INST_UUID/$INST_ID/BOOT zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID/ROOT zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID/DATA - zfs create -o mountpoint=legacy -o canmount=noauto bpool_$INST_UUID/$INST_ID/BOOT/default + zfs create -o mountpoint=/boot -o canmount=noauto bpool_$INST_UUID/$INST_ID/BOOT/default zfs create -o mountpoint=/ -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/default zfs create -o mountpoint=/ -o canmount=noauto rpool_$INST_UUID/$INST_ID/ROOT/default zfs mount rpool_$INST_UUID/$INST_ID/ROOT/default - mkdir /mnt/boot - mount -t zfs bpool_$INST_UUID/$INST_ID/BOOT/default /mnt/boot + zfs mount bpool_$INST_UUID/$INST_ID/BOOT/default for i in {usr,var,var/lib}; do zfs create -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/default/$i diff --git a/docs/Getting Started/Fedora/Root on ZFS/3-system-configuration.rst b/docs/Getting Started/Fedora/Root on ZFS/3-system-configuration.rst index c484094..e150de7 100644 --- a/docs/Getting Started/Fedora/Root on ZFS/3-system-configuration.rst +++ b/docs/Getting Started/Fedora/Root on ZFS/3-system-configuration.rst @@ -6,21 +6,9 @@ System Configuration .. contents:: Table of Contents :local: -#. Generate list of datasets for `zfs-mount-generator - `__ to mount them at boot:: - - # tab-separated zfs properties - # see /etc/zfs/zed.d/history_event-zfs-list-cacher.sh - export \ - PROPS="name,mountpoint,canmount,atime,relatime,devices,exec\ - ,readonly,setuid,nbmand,encroot,keylocation" - mkdir -p /mnt/etc/zfs/zfs-list.cache - zfs list -H -t filesystem -o $PROPS -r rpool_$INST_UUID > /mnt/etc/zfs/zfs-list.cache/rpool_$INST_UUID - sed -Ei "s|/mnt/?|/|" /mnt/etc/zfs/zfs-list.cache/* - #. Generate fstab:: - echo bpool_$INST_UUID/$INST_ID/BOOT/default /boot zfs rw,xattr,posixacl 0 0 >> /mnt/etc/fstab + genfstab -U /mnt | sed 's;zfs[[:space:]]*;zfs zfsutil,;g' | grep "zfs zfsutil" >> /mnt/etc/fstab for i in ${DISK}; do echo UUID=$(blkid -s UUID -o value ${i}-part1) /boot/efis/${i##*/}-part1 vfat \ x-systemd.idle-timeout=1min,x-systemd.automount,noauto,umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab @@ -78,7 +66,7 @@ System Configuration systemctl enable zfs-import-scan.service zfs-import.target zfs-zed zfs.target --root=/mnt systemctl disable zfs-mount --root=/mnt - At boot, datasets on rpool are mounted with ``zfs-mount-generator``, + At boot, datasets on rpool are mounted with ``/etc/fstab``, which can control the mounting process more precisely than ``zfs-mount.service``. diff --git a/docs/Getting Started/NixOS/Root on ZFS.rst b/docs/Getting Started/NixOS/Root on ZFS.rst new file mode 100644 index 0000000..72f0de8 --- /dev/null +++ b/docs/Getting Started/NixOS/Root on ZFS.rst @@ -0,0 +1,11 @@ +NixOS Root on ZFS +======================================= +`Start here `__. + +Contents +-------- +.. toctree:: + :maxdepth: 2 + :glob: + + Root on ZFS/* diff --git a/docs/Getting Started/NixOS/Root on ZFS/0-overview.rst b/docs/Getting Started/NixOS/Root on ZFS/0-overview.rst new file mode 100644 index 0000000..75598b7 --- /dev/null +++ b/docs/Getting Started/NixOS/Root on ZFS/0-overview.rst @@ -0,0 +1,142 @@ +.. highlight:: sh + +Overview +====================== +This document describes how to install NixOS with ZFS as root +file system. + +Caution +~~~~~~~ +- This guide wipes entire physical disks. Back up existing data. +- `GRUB does not and + will not work on 4Kn drive with legacy (BIOS) booting. + `__ + +Partition layout +~~~~~~~~~~~~~~~~ + +GUID partition table (GPT) is used. +EFI system partition will be referred to as **ESP** in this document. + ++----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+ +| Name | legacy boot | ESP | Boot pool | swap | root pool | remaining space | ++======================+======================+=======================+======================+=====================+=======================+=================+ +| File system | | vfat | ZFS | swap | ZFS | | ++----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+ +| Size | 1M | 2G | 4G | depends on RAM size | | | ++----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+ +| Optional encryption | | *Secure Boot* | | plain dm-crypt | ZFS native encryption | | +| | | | | | | | ++----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+ +| Partition no. | 5 | 1 | 2 | 4 | 3 | | ++----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+ +| Mount point | | | /boot | | / | | +| | | /boot/efis/disk-part1 | | | | | ++----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+ + +Dataset layout +~~~~~~~~~~~~~~ + +The dataset layout used in this guide follows stardard +mutable file positions (``/var``, ``/etc``, ...), but can +still be modified to `a immutable root `__ +after installation. + ++---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ +| Dataset | canmount | mountpoint | container | notes | ++===========================+======================+======================+=====================================+===========================================+ +| bpool | off | /boot | contains sys | | ++---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ +| rpool | off | / | contains sys | | ++---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ +| bpool/sys | off | none | contains BOOT | | ++---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ +| rpool/sys | off | none | contains ROOT | sys is encryptionroot | ++---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ +| bpool/sys/BOOT | off | none | contains boot environments | | ++---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ +| rpool/sys/ROOT | off | none | contains boot environments | | ++---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ +| rpool/sys/DATA | off | none | contains placeholder "default" | | ++---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ +| rpool/sys/DATA/default | off | / | contains user datasets | child datsets inherits mountpoint | ++---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ +| rpool/sys/DATA/local | off | / | contains /nix datasets | child datsets inherits mountpoint | ++---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ +| rpool/sys/DATA/default/ | on | /home (inherited) | no | | +| home | | | | user datasets, also called "shared | +| | | | | datasets", "persistent datasets"; also | +| | | | | include /var/lib, /srv, ... | ++---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ +| bpool/sys/BOOT/default | noauto | /boot | no | noauto is used to switch BE. because of | +| | | | | noauto, must use fstab to mount | ++---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ +| rpool/sys/ROOT/default | noauto | / | no | mounted by initrd zfs hook | ++---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ + +Encryption +~~~~~~~~~~ + +- Swap + + Swap is always encrypted. By default, swap is encrypted + with plain dm-crypt with key generated from ``/dev/urandom`` + at every boot. Swap content does not persist between reboots. + +- Root pool + + ZFS native encryption can be optionally enabled for ``rpool/sys`` + and child datasets. + + User should be aware that, ZFS native encryption does not + encrypt some metadata of the datasets. + ZFS native encryption also does not change master key when ``zfs change-key`` is invoked. + Therefore, you should wipe the disk when password is compromised to protect confidentiality. + See `zfs-load-key.8 `__ + and `zfs-change-key.8 `__ + for more information regarding ZFS native encryption. + + Encryption is enabled at dataset creation and can not be disabled later. + +- Boot pool + + Boot pool can not be encrypted. + +- Bootloader + + Bootloader can not be encrypted. + + However, with Secure Boot, bootloader + can be verified by motherboard firmware to be untempered, + which should be sufficient for most purposes. + + Secure Boot is not supported out-of-the-box due to ZFS module. + +Booting with disk failure +~~~~~~~~~~~~~~~~~~~~~~~~~ + +This guide is written with disk failure in mind. + +If disks used in Root on ZFS pool failed, but +sufficient redundancy for both root pool and boot pool +still exists, the system will still boot normally. + +Swap partition on the failed disk will fail to mount, +after an 1m30s timeout. + +This feature is useful for use cases such +as an unattended remote server. + +Example: + + - System has disks ``n>1`` + + - Installed with mirrored setup + + - Mirrored setup can tolerate up to ``n-1`` disk failures + + - Disconnect one or more disks, keep at least + one disk connected + + - System still boots, but fails to mount swap and + EFI partition diff --git a/docs/Getting Started/NixOS/Root on ZFS/1-preparation.rst b/docs/Getting Started/NixOS/Root on ZFS/1-preparation.rst new file mode 100644 index 0000000..ba5240a --- /dev/null +++ b/docs/Getting Started/NixOS/Root on ZFS/1-preparation.rst @@ -0,0 +1,109 @@ +.. highlight:: sh + +Preparation +====================== + +.. contents:: Table of Contents + :local: + +#. Download `Minimal ISO image + `__ and boot from it. + +#. Connect to network. See `NixOS manual `__. + +#. SSH server is enabled by default. To connect, set root password with:: + + sudo passwd + +#. Connect from another computer:: + + ssh root@192.168.1.19 + +#. Unique pool suffix. ZFS expects pool names to be + unique, therefore it's recommended to create + pools with a unique suffix:: + + INST_UUID=$(dd if=/dev/urandom bs=1 count=100 2>/dev/null | tr -dc 'a-z0-9' | cut -c-6) + +#. Identify this installation in ZFS filesystem path:: + + INST_ID=nixos + +#. Root on ZFS configuration file name:: + + INST_CONFIG_FILE='zfs.nix' + +#. Target disk + + List available disks with:: + + ls /dev/disk/by-id/* + + If using virtio as disk bus, use + ``/dev/disk/by-path/*``. + + Declare disk array:: + + DISK='/dev/disk/by-id/ata-FOO /dev/disk/by-id/nvme-BAR' + + For single disk installation, use:: + + DISK='/dev/disk/by-id/disk1' + +#. Choose a primary disk. This disk will be used + for primary EFI partition, default to + first disk in the array:: + + INST_PRIMARY_DISK=$(echo $DISK | cut -f1 -d\ ) + +#. Set vdev topology, possible values are: + + - (not set, single disk or striped; no redundancy) + - mirror + - raidz1 + - raidz2 + - raidz3 + + :: + + INST_VDEV= + + This will create a single vdev with the topology of your choice. + It is also possible to manually create a pool with multiple vdevs, such as:: + + zpool create --options \ + poolName \ + mirror sda sdb \ + raidz2 sdc ... \ + raidz3 sde ... \ + spare sdf ... + + Notice the cost of parity when using RAID-Z. See + `here `__ + and `here `__. + + Refer to `zpoolconcepts `__ + and `zpool-create `__ + man pages for details. + +#. Set partition size: + + Set ESP size:: + + INST_PARTSIZE_ESP=2 # in GB + + Set boot pool size. To avoid running out of space while using + boot environments, the minimum is 4GB. Adjust the size if you + intend to use multiple kernel/distros:: + + INST_PARTSIZE_BPOOL=4 + + Set swap size. It's `recommended `__ + to setup a swap partition. If you intend to use hibernation, + the minimum should be no less than RAM size. Skip if swap is not needed:: + + INST_PARTSIZE_SWAP=8 + + Root pool size, use all remaining disk space if not set:: + + INST_PARTSIZE_RPOOL= diff --git a/docs/Getting Started/NixOS/Root on ZFS/2-system-configuration.rst b/docs/Getting Started/NixOS/Root on ZFS/2-system-configuration.rst new file mode 100644 index 0000000..c0b26c7 --- /dev/null +++ b/docs/Getting Started/NixOS/Root on ZFS/2-system-configuration.rst @@ -0,0 +1,356 @@ +.. highlight:: sh + +System Configuration +====================== + +.. contents:: Table of Contents + :local: + +#. Optional: wipe solid-state drives with the generic tool + `blkdiscard `__, + to clean previous partition tables and improve performance. + + All content will be irrevocably destroyed:: + + for i in ${DISK}; do + blkdiscard -f $i & + done + wait + + This is a quick operation and should be completed under one + minute. + + For other device specific methods, see + `Memory cell clearing `__ + +#. Partition the disks. + See `Overview <0-overview.html>`__ for details:: + + for i in ${DISK}; do + sgdisk --zap-all $i + sgdisk -n1:1M:+${INST_PARTSIZE_ESP}G -t1:EF00 $i + sgdisk -n2:0:+${INST_PARTSIZE_BPOOL}G -t2:BE00 $i + if [ "${INST_PARTSIZE_SWAP}" != "" ]; then + sgdisk -n4:0:+${INST_PARTSIZE_SWAP}G -t4:8200 $i + fi + if [ "${INST_PARTSIZE_RPOOL}" = "" ]; then + sgdisk -n3:0:0 -t3:BF00 $i + else + sgdisk -n3:0:+${INST_PARTSIZE_RPOOL}G -t3:BF00 $i + fi + sgdisk -a1 -n5:24K:+1000K -t5:EF02 $i + done + +#. Create boot pool:: + + zpool create \ + -d -o feature@async_destroy=enabled \ + -o feature@bookmarks=enabled \ + -o feature@embedded_data=enabled \ + -o feature@empty_bpobj=enabled \ + -o feature@enabled_txg=enabled \ + -o feature@extensible_dataset=enabled \ + -o feature@filesystem_limits=enabled \ + -o feature@hole_birth=enabled \ + -o feature@large_blocks=enabled \ + -o feature@lz4_compress=enabled \ + -o feature@spacemap_histogram=enabled \ + -o ashift=12 \ + -o autotrim=on \ + -O acltype=posixacl \ + -O canmount=off \ + -O compression=lz4 \ + -O devices=off \ + -O normalization=formD \ + -O relatime=on \ + -O xattr=sa \ + -O mountpoint=/boot \ + -R /mnt \ + bpool_$INST_UUID \ + $INST_VDEV \ + $(for i in ${DISK}; do + printf "$i-part2 "; + done) + + You should not need to customize any of the options for the boot pool. + + GRUB does not support all of the zpool features. See ``spa_feature_names`` + in `grub-core/fs/zfs/zfs.c + `__. + This step creates a separate boot pool for ``/boot`` with the features + limited to only those that GRUB supports, allowing the root pool to use + any/all features. + + Features enabled with ``-o compatibility=grub2`` can be seen + `here `__. + +#. Create root pool:: + + zpool create \ + -o ashift=12 \ + -o autotrim=on \ + -R /mnt \ + -O acltype=posixacl \ + -O canmount=off \ + -O compression=zstd \ + -O dnodesize=auto \ + -O normalization=formD \ + -O relatime=on \ + -O xattr=sa \ + -O mountpoint=/ \ + rpool_$INST_UUID \ + $INST_VDEV \ + $(for i in ${DISK}; do + printf "$i-part3 "; + done) + + **Notes:** + + - The use of ``ashift=12`` is recommended here because many drives + today have 4 KiB (or larger) physical sectors, even though they + present 512 B logical sectors. Also, a future replacement drive may + have 4 KiB physical sectors (in which case ``ashift=12`` is desirable) + or 4 KiB logical sectors (in which case ``ashift=12`` is required). + - Setting ``-O acltype=posixacl`` enables POSIX ACLs globally. If you + do not want this, remove that option, but later add + ``-o acltype=posixacl`` (note: lowercase “o”) to the ``zfs create`` + for ``/var/log``, as `journald requires ACLs + `__ + - Setting ``normalization=formD`` eliminates some corner cases relating + to UTF-8 filename normalization. It also implies ``utf8only=on``, + which means that only UTF-8 filenames are allowed. If you care to + support non-UTF-8 filenames, do not use this option. For a discussion + of why requiring UTF-8 filenames may be a bad idea, see `The problems + with enforced UTF-8 only filenames + `__. + - ``recordsize`` is unset (leaving it at the default of 128 KiB). If you + want to tune it (e.g. ``-o recordsize=1M``), see `these + `__ `various + `__ `blog + `__ + `posts + `__. + - Setting ``relatime=on`` is a middle ground between classic POSIX + ``atime`` behavior (with its significant performance impact) and + ``atime=off`` (which provides the best performance by completely + disabling atime updates). Since Linux 2.6.30, ``relatime`` has been + the default for other filesystems. See `RedHat’s documentation + `__ + for further information. + - Setting ``xattr=sa`` `vastly improves the performance of extended + attributes + `__. + Inside ZFS, extended attributes are used to implement POSIX ACLs. + Extended attributes can also be used by user-space applications. + `They are used by some desktop GUI applications. + `__ + `They can be used by Samba to store Windows ACLs and DOS attributes; + they are required for a Samba Active Directory domain controller. + `__ + Note that ``xattr=sa`` is `Linux-specific + `__. If you move your + ``xattr=sa`` pool to another OpenZFS implementation besides ZFS-on-Linux, + extended attributes will not be readable (though your data will be). If + portability of extended attributes is important to you, omit the + ``-O xattr=sa`` above. Even if you do not want ``xattr=sa`` for the whole + pool, it is probably fine to use it for ``/var/log``. + - Make sure to include the ``-part3`` portion of the drive path. If you + forget that, you are specifying the whole disk, which ZFS will then + re-partition, and you will lose the bootloader partition(s). + +#. This section implements dataset layout as described in `overview <0-overview.html>`__. + + Create root system container: + + - Unencrypted:: + + zfs create \ + -o canmount=off \ + -o mountpoint=none \ + rpool_$INST_UUID/$INST_ID + + - Encrypted: + + Pick a strong password. Once compromised, changing password will not keep your + data safe. See ``zfs-change-key(8)`` for more info:: + + zfs create \ + -o canmount=off \ + -o mountpoint=none \ + -o encryption=aes-256-gcm \ + -o keylocation=prompt \ + -o keyformat=passphrase \ + rpool_$INST_UUID/$INST_ID + + Create other system datasets:: + + zfs create -o canmount=off -o mountpoint=none bpool_$INST_UUID/$INST_ID + zfs create -o canmount=off -o mountpoint=none bpool_$INST_UUID/$INST_ID/BOOT + zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID/ROOT + zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID/DATA + zfs create -o mountpoint=/boot -o canmount=noauto bpool_$INST_UUID/$INST_ID/BOOT/default + zfs create -o mountpoint=/ -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/default + zfs create -o mountpoint=/ -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/local + zfs create -o mountpoint=/ -o canmount=noauto rpool_$INST_UUID/$INST_ID/ROOT/default + zfs mount rpool_$INST_UUID/$INST_ID/ROOT/default + zfs mount bpool_$INST_UUID/$INST_ID/BOOT/default + for i in {usr,var,var/lib}; + do + zfs create -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/default/$i + done + for i in {home,root,srv,usr/local,var/log,var/spool}; + do + zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/$i + done + chmod 750 /mnt/root + for i in {nix,}; do + zfs create -o canmount=on -o mountpoint=/$i rpool_$INST_UUID/$INST_ID/DATA/local/$i + done + + Datasets for immutable root filesystem:: + + zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/state + for i in {/etc/nixos,/etc/cryptkey.d}; do + mkdir -p /mnt/state/$i /mnt/$i + mount -o bind /mnt/state/$i /mnt/$i + done + zfs create -o mountpoint=/ -o canmount=noauto rpool_$INST_UUID/$INST_ID/ROOT/empty + zfs snapshot rpool_$INST_UUID/$INST_ID/ROOT/empty@start + +#. Format and mount ESP:: + + for i in ${DISK}; do + mkfs.vfat -n EFI ${i}-part1 + mkdir -p /mnt/boot/efis/${i##*/}-part1 + mount -t vfat ${i}-part1 /mnt/boot/efis/${i##*/}-part1 + done + +#. Create optional user data datasets to omit data from rollback:: + + zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/games + zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/www + # for GNOME + zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/AccountsService + # for Docker + zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/docker + # for NFS + zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/nfs + # for LXC + zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/lxc + # for LibVirt + zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/libvirt + ##other application + # zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/$name + + Add other datasets when needed, such as PostgreSQL. + +#. Generate initial NixOS system configuration:: + + nixos-generate-config --root /mnt + + This command will generate two files, ``configuration.nix`` + and ``hardware-configuration-zfs.nix``, which will be the starting point + of configuring the system. + +#. Edit config file to import ZFS options:: + + sed -i "s|./hardware-configuration.nix|./hardware-configuration-zfs.nix ./${INST_CONFIG_FILE}|g" /mnt/etc/nixos/configuration.nix + # backup, prevent being overwritten by nixos-generate-config + mv /mnt/etc/nixos/hardware-configuration.nix /mnt/etc/nixos/hardware-configuration-zfs.nix + +#. ZFS options:: + + tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} < /mnt/state/etc/machine-id + tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <>/mnt/etc/nixos/${INST_CONFIG_FILE} + done + tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <>/mnt/etc/nixos/${INST_CONFIG_FILE} + done + tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <`__ section if +no optional configuration is needed. + +Supply password with SSH +~~~~~~~~~~~~~~~~~~~~~~~~ + +Note: if you choose to encrypt boot pool, where decryption is handled +by GRUB, as described in the next section, configuration performed +in this section will have no effect. + +This example uses DHCP:: + + mkdir -p /mnt/etc/state/ssh/ + ssh-keygen -t ed25519 -N "" -f /mnt/state/etc/ssh/ssh_host_initrd_ed25519_key + tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <> /root/.profile + ''; + }; + }; + EOF + +Encrypt boot pool +~~~~~~~~~~~~~~~~~~~ +Note: This will disable password with SSH. The password previously set for +root pool will be replaced by keyfile, embedded in initrd. + +#. Add package:: + + tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} < /mnt/root/bpool_$INST_UUID-${INST_ID}-pre-luks + +#. Unmount EFI partition:: + + for i in ${DISK}; do + umount /mnt/boot/efis/${i##*/}-part1 + done + +#. Destroy boot pool:: + + zpool destroy bpool_$INST_UUID + +#. Create LUKS containers:: + + for i in ${DISK}; do + cryptsetup luksFormat -q --type luks1 --key-file /mnt/etc/cryptkey.d/bpool_$INST_UUID-key-luks $i-part2 + echo $LUKS_PWD | cryptsetup luksAddKey --key-file /mnt/etc/cryptkey.d/bpool_$INST_UUID-key-luks $i-part2 + cryptsetup open ${i}-part2 ${i##*/}-part2-luks-bpool_$INST_UUID --key-file /mnt/etc/cryptkey.d/bpool_$INST_UUID-key-luks + tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <`__. diff --git a/docs/Getting Started/NixOS/Root on ZFS/4-system-installation.rst b/docs/Getting Started/NixOS/Root on ZFS/4-system-installation.rst new file mode 100644 index 0000000..d01dc0b --- /dev/null +++ b/docs/Getting Started/NixOS/Root on ZFS/4-system-installation.rst @@ -0,0 +1,176 @@ +.. highlight:: sh + +System Installation +====================== + +.. contents:: Table of Contents + :local: + + +Additional configuration +~~~~~~~~~~~~~~~~~~~~~~~~~ + +As NixOS configuration is declarative, post-installation tasks, +such as user accounts and package selection, can all be done by +specifing them in configuration. See `NixOS manual `__ +for details. + +For timezone, hostname, networking, keyboard layout, etc, +see ``/mnt/etc/nixos/configuration.nix``. + +Set root password +----------------- + +This optional step is an example +of declaratively configuring the system. + +#. Generate password hash:: + + mkpasswd -m SHA-512 -s + #output: $6$DeHnzc + +#. Declare `initialHashedPassword + `__ + for root user:: + + tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<-'EOF' + users.users.root.initialHashedPassword = "$6$DeHnzc"; + EOF + +System installation +~~~~~~~~~~~~~~~~~~~ + +#. Finalize the config file:: + + tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <`__. + +Save mutable data to alternative path +------------------------------------- + +Before enabling purging on root dataset, we need to back up +essential mutable data first, such as host SSH key and network connections. +Below are some tips. + +- Some programs support specifying another + location for mutable data, such as + Wireguard:: + + networking.wireguard.interfaces.wg0.privateKeyFile = "/state/etc/wireguard/wg0"; + +- For programs without a configurable data path, + `environment.etc `__ + may be used:: + + environment.etc = { + "ssh/ssh_host_rsa_key".source = "/state/etc/ssh/ssh_host_rsa_key"; + } + +- systemd’s tmpfiles.d rules are also an option:: + + systemd.tmpfiles.rules = [ + "L /var/lib/bluetooth - - - - /state/var/lib/bluetooth" + ]; + +- Bind mount:: + + for i in {/etc/nixos,/etc/cryptkey.d}; do + mkdir -p /state/$i /$i + mount -o bind /state/$i /$i + done + nixos-generate-config --show-hardware-config + +Boot from empty root file system +-------------------------------- + +After backing up mutable data, you can try switching to +an empty dataset as root file system. + +#. Check current root file system:: + + ROOT_FS=$(df --output=source /|tail -n1) + # rpool/ROOT/default + +#. Set empty file system as root:: + + sed -i "s,${ROOT_FS},${ROOT_FS%/*}/empty,g" /etc/nixos/hardware-configuration-zfs.nix + +#. Apply changes and reboot:: + + nixos-rebuild boot + reboot + +#. If everything went fine, add the output of the following command to configuration:: + + ROOT_FS=$(df --output=source /|tail -n1) + cat <``. + +#. On another computer, generate rescue image with:: + + pacman -S --needed mtools libisoburn grub + grub-install + grub-mkrescue -o grub-rescue.img + dd if=grub-rescue.img of=/dev/your-usb-stick + + Boot computer from the rescue media. + Both legacy and EFI mode are supported. + + Or `download generated GRUB rescue image `__. + +#. List available disks with ``ls`` command:: + + grub> ls (hd # press tab + Possible devices are: + + hd0 hd1 hd2 hd3 + +#. List partitions by pressing tab key: + + .. code-block:: text + + grub> ls (hd0 # press tab + Possible partitions are: + + Device hd0: No known filesystem detected - Sector size 512B - Total size 20971520KiB + Partition hd0,gpt1: Filesystem type fat - Label `EFI', UUID 0DF5-3A76 - Partition start at 1024KiB - Total size 1048576KiB + Partition hd0,gpt2: No known filesystem detected - Partition start at 1049600KiB - Total size 4194304KiB + + - If boot pool is encrypted: + + Unlock it with ``cryptomount``:: + + grub> insmod luks + grub> cryptomount hd0,gpt2 + Attempting to decrypt master key... + Enter passphrase for hd0,gpt2 (af5a240e13e24483acf02600d61e0f36): + Slot 1 opened + + Unlocked LUKS container is ``(crypto0)``: + + .. code-block:: text + + grub> ls (crypto0) + Device crypto0: Filesystem type zfs - Label `bpool_ip3tdb' - Last modification + time 2021-05-03 12:14:08 Monday, UUID f14d7bdf89fe21fb - Sector size 512B - + Total size 4192256KiB + + - If boot pool is not encrypted: + + .. code-block:: text + + grub> ls (hd0,gpt2) + Device hd0,gpt2: Filesystem type zfs - Label `bpool_ip3tdb' - Last modification + time 2021-05-03 12:14:08 Monday, UUID f14d7bdf89fe21fb - Sector size 512B - + Total size 4192256KiB + +#. List boot environments nested inside ``bpool/$INST_ID/BOOT``:: + + grub> ls (crypto0)/sys/BOOT + @/ default/ be0/ + +#. Instruct GRUB to load configuration from ``be0`` boot environment:: + + grub> prefix=(crypto0)/sys/BOOT/be0/@/grub + grub> configfile $prefix/grub.cfg + +#. GRUB menu should now appear. + +#. After entering system, reinstall GRUB. + +Switch GRUB prefix when disk fails +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you are using LUKS encrypted boot pool with multiple disks, +the primary disk failed, GRUB will fail to load configuration. + +If there's still enough redundancy for the boot pool, try fix +GRUB with the following method: + +#. Ensure ``Slot 1 opened`` message + is shown + + .. code-block:: text + + Welcome to GRUB! + + error: no such cryptodisk found. + Attempting to decrypt master key... + Enter passphrase for hd0,gpt2 (c0987ea1a51049e9b3056622804de62a): + Slot 1 opened + error: disk `cryptouuid/47ed1b7eb0014bc9a70aede3d8714faf' not found. + Entering rescue mode... + grub rescue> + + If ``error: access denied.`` is shown, + try re-enter password with:: + + grub rescue> cryptomount hd0,gpt2 + +#. Check prefix:: + + grub rescue > set + # prefix=(cryptouuid/47ed1b7eb0014bc9a70aede3d8714faf)/sys/BOOT/be0@/grub + # root=cryptouuid/47ed1b7eb0014bc9a70aede3d8714faf + +#. Set correct ``prefix`` and ``root`` by replacing + ``cryptouuid/UUID`` with ``crypto0``:: + + grub rescue> prefix=(crypto0)/sys/BOOT/default@/grub + grub rescue> root=crypto0 + +#. Boot GRUB:: + + grub rescue> insmod normal + grub rescue> normal + + GRUB should then boot normally. + +#. After entering system, edit ``/etc/fstab`` to promote + one backup to ``/boot/efi``. + +#. Make the change to ``prefix`` and ``root`` + permanent by `reinstalling GRUB <#grub-installation>`__. diff --git a/docs/Getting Started/NixOS/index.rst b/docs/Getting Started/NixOS/index.rst new file mode 100644 index 0000000..2f44f7b --- /dev/null +++ b/docs/Getting Started/NixOS/index.rst @@ -0,0 +1,64 @@ +.. highlight:: sh + +NixOS +===== + +Contents +-------- +.. toctree:: + :maxdepth: 1 + :glob: + + * + +Support +------- +Reach out to the community using the :ref:`mailing_lists` or IRC at +`#zfsonlinux `__ on `Libera Chat +`__. + +If you have a bug report or feature request +related to this HOWTO, please `file a new issue and mention @ne9z +`__. + +Installation +------------ + +Note: this is for installing ZFS on an existing +NixOS installation. To use ZFS as root file system, +see below. + +Live image ships with ZFS support by default. + +#. Import separate configration file for ZFS options:: + + vim /etc/nixos/configuration.nix + ##add './zfs.nix' to 'imports' + # imports = [ ./zfs.nix ]; + +#. Configure ZFS options:: + + tee -a /etc/nixos/zfs.nix <`__. + +.. toctree:: + :maxdepth: 1 + :glob: + + Root on ZFS/* diff --git a/docs/Getting Started/RHEL-based distro/RHEL 8-based distro Root on ZFS/0-overview.rst b/docs/Getting Started/RHEL-based distro/RHEL 8-based distro Root on ZFS/0-overview.rst index b660ab7..d9ea513 100644 --- a/docs/Getting Started/RHEL-based distro/RHEL 8-based distro Root on ZFS/0-overview.rst +++ b/docs/Getting Started/RHEL-based distro/RHEL 8-based distro Root on ZFS/0-overview.rst @@ -63,12 +63,12 @@ Dataset layout | | | | | datasets", "persistent datasets"; also | | | | | | include /var/lib, /srv, ... | +---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ -| bpool/sys/BOOT/default | noauto | legacy /boot | no | noauto is used to switch BE. because of | +| bpool/sys/BOOT/default | noauto | /boot | no | noauto is used to switch BE. because of | | | | | | noauto, must use fstab to mount | +---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ | rpool/sys/ROOT/default | noauto | / | no | mounted by initrd zfs hook | +---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ -| bpool/sys/BOOT/be1 | noauto | legacy /boot | no | see bpool/sys/BOOT/default | +| bpool/sys/BOOT/be1 | noauto | /boot | no | see bpool/sys/BOOT/default | +---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ | rpool/sys/ROOT/be1 | noauto | / | no | see rpool/sys/ROOT/default | +---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+ @@ -110,3 +110,32 @@ Encryption which should be sufficient for most purposes. Secure Boot is not supported out-of-the-box due to ZFS module. + +Booting with disk failure +~~~~~~~~~~~~~~~~~~~~~~~~~ + +This guide is written with disk failure in mind. + +If disks used in Root on ZFS pool failed, but +sufficient redundancy for both root pool and boot pool +still exists, the system will still boot normally. + +Swap partition on the failed disk will fail to mount, +after an 1m30s timeout. + +This feature is useful for use cases such +as an unattended remote server. + +Example: + + - System has disks ``n>1`` + + - Installed with mirrored setup + + - Mirrored setup can tolerate up to ``n-1`` disk failures + + - Disconnect one or more disks, keep at least + one disk connected + + - System still boots, but fails to mount swap and + EFI partition diff --git a/docs/Getting Started/RHEL-based distro/RHEL 8-based distro Root on ZFS/1-preparation.rst b/docs/Getting Started/RHEL-based distro/RHEL 8-based distro Root on ZFS/1-preparation.rst index 6ed1c5e..e08fe66 100644 --- a/docs/Getting Started/RHEL-based distro/RHEL 8-based distro Root on ZFS/1-preparation.rst +++ b/docs/Getting Started/RHEL-based distro/RHEL 8-based distro Root on ZFS/1-preparation.rst @@ -77,7 +77,7 @@ Preparation ls /dev/disk/by-id/* If using virtio as disk bus, use - ``/dev/disk/by-path/*`` or ``/dev/vd*``. + ``/dev/disk/by-path/*``. Declare disk array:: diff --git a/docs/Getting Started/RHEL-based distro/RHEL 8-based distro Root on ZFS/2-system-installation.rst b/docs/Getting Started/RHEL-based distro/RHEL 8-based distro Root on ZFS/2-system-installation.rst index e2f64a0..ab5ecc1 100644 --- a/docs/Getting Started/RHEL-based distro/RHEL 8-based distro Root on ZFS/2-system-installation.rst +++ b/docs/Getting Started/RHEL-based distro/RHEL 8-based distro Root on ZFS/2-system-installation.rst @@ -188,12 +188,11 @@ System Installation zfs create -o canmount=off -o mountpoint=none bpool_$INST_UUID/$INST_ID/BOOT zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID/ROOT zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID/DATA - zfs create -o mountpoint=legacy -o canmount=noauto bpool_$INST_UUID/$INST_ID/BOOT/default + zfs create -o mountpoint=/boot -o canmount=noauto bpool_$INST_UUID/$INST_ID/BOOT/default zfs create -o mountpoint=/ -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/default zfs create -o mountpoint=/ -o canmount=noauto rpool_$INST_UUID/$INST_ID/ROOT/default zfs mount rpool_$INST_UUID/$INST_ID/ROOT/default - mkdir /mnt/boot - mount -t zfs bpool_$INST_UUID/$INST_ID/BOOT/default /mnt/boot + zfs mount bpool_$INST_UUID/$INST_ID/BOOT/default for i in {usr,var,var/lib}; do zfs create -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/default/$i diff --git a/docs/Getting Started/RHEL-based distro/RHEL 8-based distro Root on ZFS/3-system-configuration.rst b/docs/Getting Started/RHEL-based distro/RHEL 8-based distro Root on ZFS/3-system-configuration.rst index 3e4d1de..6357e8c 100644 --- a/docs/Getting Started/RHEL-based distro/RHEL 8-based distro Root on ZFS/3-system-configuration.rst +++ b/docs/Getting Started/RHEL-based distro/RHEL 8-based distro Root on ZFS/3-system-configuration.rst @@ -6,21 +6,9 @@ System Configuration .. contents:: Table of Contents :local: -#. Generate list of datasets for `zfs-mount-generator - `__ to mount them at boot:: - - # tab-separated zfs properties - # see /etc/zfs/zed.d/history_event-zfs-list-cacher.sh - export \ - PROPS="name,mountpoint,canmount,atime,relatime,devices,exec\ - ,readonly,setuid,nbmand,encroot,keylocation" - mkdir -p /mnt/etc/zfs/zfs-list.cache - zfs list -H -t filesystem -o $PROPS -r rpool_$INST_UUID > /mnt/etc/zfs/zfs-list.cache/rpool_$INST_UUID - sed -Ei "s|/mnt/?|/|" /mnt/etc/zfs/zfs-list.cache/* - #. Generate fstab:: - echo bpool_$INST_UUID/$INST_ID/BOOT/default /boot zfs rw,xattr,posixacl 0 0 >> /mnt/etc/fstab + genfstab -U /mnt | sed 's;zfs[[:space:]]*;zfs zfsutil,;g' | grep "zfs zfsutil" >> /mnt/etc/fstab for i in ${DISK}; do echo UUID=$(blkid -s UUID -o value ${i}-part1) /boot/efis/${i##*/}-part1 vfat \ x-systemd.idle-timeout=1min,x-systemd.automount,noauto,umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab @@ -73,7 +61,7 @@ System Configuration systemctl enable zfs-import-scan.service zfs-import.target zfs-zed zfs.target --root=/mnt systemctl disable zfs-mount --root=/mnt - At boot, datasets on rpool are mounted with ``zfs-mount-generator``, + At boot, datasets on rpool are mounted with ``/etc/fstab``, which can control the mounting process more precisely than ``zfs-mount.service``. #. By default SSH server is enabled, allowing root login by password, diff --git a/docs/Getting Started/index.rst b/docs/Getting Started/index.rst index 40c8f28..380117c 100644 --- a/docs/Getting Started/index.rst +++ b/docs/Getting Started/index.rst @@ -16,7 +16,7 @@ documentation `__ Fedora/index FreeBSD Gentoo - NixOS + NixOS/index openSUSE/index RHEL-based distro/index Ubuntu/index