jacobpascual/openzfs-docs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Multiple fixes for Root on ZFS guide (#191)

Browse Source 
* Let user know that SELinux will be re-enabled after reboot

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* compat with future releases: use zfs-dkms and newer repo

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* rm unused releasever option

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* let user aware of the ignorable errors

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* detailed explanations for errors during grub-menu generation

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* Build ZFS dkms module in installed system

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* switch to dkms package for better compatibility with kernels

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* add direct download links for live iso

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* rm zfs-fuse before install; mirrorlist

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* reformat notes

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* rm netconfig - networkmanager is enabled by default

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* load kernel module in live;

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* rm encrypted bpool: untested

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* use u=r,go= permission on key file

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* fix typo

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* use bash shell

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* suggest clean the disks

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* add grub-menu auto update

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* monitor kernel-core pkg

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* copyright 2021

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* fix kernel var detection

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* read-only cache file

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* replace zfs-mount.service with zfs-mount-generator

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* notes for mount and POSIX-compliant

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* hard-code kernel version

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* fix chroot variable

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* fix grub cfg

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* fix grub

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* missing comment

Signed-off-by: Maurice Zhou <jasper@apvc.uk>

* comments

Signed-off-by: Maurice Zhou <jasper@apvc.uk>
This commit is contained in:
ne9z
2021-08-28 20:07:35 +00:00
committed by GitHub
parent 5faf8ed9a7
commit f40d19a2c5
18 changed files with 363 additions and 258 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/Getting Started/Arch Linux/2-zfs-dkms.rst
View File

@@ -25,6 +25,8 @@ Installation
#. Check kernel variant::
INST_LINVAR=$(sed 's|.*linux|linux|' /proc/cmdline | sed 's|.img||g' | awk '{ print $1 }')
#for live image
#INST_LINVAR=linux
#. Check kernel version::
@@ -42,7 +44,7 @@ Installation
#. Install zfs-dkms::
pacman -Sy --needed zfs-dkms glibc
pacman -Sy --needed --noconfirm zfs-dkms glibc
If pacman output contains the following error message,
then the kernel needs a `downgrade <#zfs-dkms-compatible-kernel>`__,

32
docs/Getting Started/Arch Linux/Root on ZFS/1-preparation.rst
View File

@@ -18,25 +18,27 @@ Preparation
systemctl start sshd
#. Connect from another computer
and enter a bash shell::
#. Connect from another computer::
ssh root@192.168.1.19
and, most important, enter a bash shell::
bash
This guide is untested with the default shell ``zsh`` in live environment.
#. Expand live root filesystem::
mount -o remount,size=2G /run/archiso/cowspace/
#. `Add archzfs repo <../0-archzfs-repo.html>`__.
#. Install prebuilt ZFS package, corresponding to
`live image kernel version <https://archlinux.org/download/>`__::
#. `Install zfs-dkms in live environment <../2-zfs-dkms.html#installation>`__.
LIVE_ZFS_PKG="zfs-linux-2.1.0_5.13.6.arch1.1-1-x86_64.pkg.tar.zst"
LIVE_ZFS_UTILS="zfs-utils-2.1.0-2-x86_64.pkg.tar.zst"
LIVE_ZFS_MIRROR="https://mirror.sum7.eu/archlinux/archzfs"
pacman -U --noconfirm ${LIVE_ZFS_MIRROR}/archzfs/x86_64/${LIVE_ZFS_UTILS} || \
pacman -U --noconfirm ${LIVE_ZFS_MIRROR}/archive_archzfs/${LIVE_ZFS_UTILS}
pacman -U --noconfirm ${LIVE_ZFS_MIRROR}/archzfs/x86_64/${LIVE_ZFS_PKG} || \
pacman -U --noconfirm ${LIVE_ZFS_MIRROR}/archive_archzfs/${LIVE_ZFS_PKG}
modprobe zfs
#. Load zfs kernel module::
modprobe zfs
#. Kernel variant
@@ -75,17 +77,17 @@ Preparation
Declare disk array::
DISK=(/dev/disk/by-id/ata-FOO /dev/disk/by-id/nvme-BAR)
DISK='/dev/disk/by-id/ata-FOO /dev/disk/by-id/nvme-BAR'
For single disk installation, use::
DISK=(/dev/disk/by-id/disk1)
DISK='/dev/disk/by-id/disk1'
#. Choose a primary disk. This disk will be used
for primary EFI partition and hibernation, default to
first disk in the array::
INST_PRIMARY_DISK=${DISK[0]}
INST_PRIMARY_DISK=$(echo $DISK | cut -f1 -d\ )
If disk path contains colon ``:``, this disk
can not be used for hibernation. ``encrypt`` mkinitcpio

27
docs/Getting Started/Arch Linux/Root on ZFS/2-system-installation.rst
View File

@@ -6,10 +6,27 @@ System Installation
.. contents:: Table of Contents
:local:
#. Optional: wipe solid-state drives with the generic tool
`blkdiscard <https://utcc.utoronto.ca/~cks/space/blog/linux/ErasingSSDsWithBlkdiscard>`__,
to clean previous partition tables and improve performance.
All content will be irrevocably destroyed::
for i in ${DISK}; do
blkdiscard -f $i &
done
wait
This is a quick operation and should be completed under one
minute.
For other device specific methods, see
`Memory cell clearing <https://wiki.archlinux.org/title/Solid_state_drive/Memory_cell_clearing>`__
#. Partition the disks.
See `Overview <0-overview.html>`__ for details::
for i in ${DISK[@]}; do
for i in ${DISK}; do
sgdisk --zap-all $i
sgdisk -n1:1M:+${INST_PARTSIZE_ESP}G -t1:EF00 $i
sgdisk -n2:0:+${INST_PARTSIZE_BPOOL}G -t2:BE00 $i
@@ -41,7 +58,7 @@ System Installation
-R /mnt \
bpool_$INST_UUID \
$INST_VDEV \
$(for i in ${DISK[@]}; do
$(for i in ${DISK}; do
printf "$i-part2 ";
done)
@@ -73,7 +90,7 @@ System Installation
-O mountpoint=/ \
rpool_$INST_UUID \
$INST_VDEV \
$(for i in ${DISK[@]}; do
$(for i in ${DISK}; do
printf "$i-part3 ";
done)
@@ -170,7 +187,7 @@ System Installation
#. Format and mount ESP::
for i in ${DISK[@]}; do
for i in ${DISK}; do
mkfs.vfat -n EFI ${i}-part1
mkdir -p /mnt/boot/efis/${i##*/}-part1
mount -t vfat ${i}-part1 /mnt/boot/efis/${i##*/}-part1
@@ -211,7 +228,7 @@ System Installation
#. Install kernel. Download from archive if kernel is not available::
if [ ${INST_LINVER} == \
if [ ${INST_LINVER} = \
$(pacman -Si ${INST_LINVAR} | grep Version | awk '{ print $3 }') ]; then
pacstrap /mnt ${INST_LINVAR}
else

15
docs/Getting Started/Arch Linux/Root on ZFS/3-system-configuration.rst
View File

@@ -26,14 +26,14 @@ System Configuration
#. Generate fstab::
echo bpool_$INST_UUID/$INST_ID/BOOT/default /boot zfs rw,xattr,posixacl 0 0 >> /mnt/etc/fstab
for i in ${DISK[@]}; do
for i in ${DISK}; do
echo UUID=$(blkid -s UUID -o value ${i}-part1) /boot/efis/${i##*/}-part1 vfat \
x-systemd.idle-timeout=1min,x-systemd.automount,noauto,umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab
done
echo UUID=$(blkid -s UUID -o value ${INST_PRIMARY_DISK}-part1) /boot/efi vfat \
x-systemd.idle-timeout=1min,x-systemd.automount,noauto,umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab
if [ "${INST_PARTSIZE_SWAP}" != "" ]; then
for i in ${DISK[@]}; do
for i in ${DISK}; do
echo ${i##*/}-part4-swap ${i}-part4 /dev/urandom swap,cipher=aes-cbc-essiv:sha256,size=256,discard >> /mnt/etc/crypttab
echo /dev/mapper/${i##*/}-part4-swap none swap defaults 0 0 >> /mnt/etc/fstab
done
@@ -105,7 +105,11 @@ System Configuration
#. Enable ZFS services::
systemctl enable zfs-import-scan.service zfs-import.target zfs-mount zfs-zed zfs.target --root=/mnt
systemctl enable zfs-import-scan.service zfs-import.target zfs-zed zfs.target --root=/mnt
systemctl disable zfs-mount --root=/mnt
At boot, datasets on rpool are mounted with ``zfs-mount-generator``,
which can control the mounting process more precisely than ``zfs-mount.service``.
#. Chroot::
@@ -113,10 +117,9 @@ System Configuration
INST_LINVAR=$INST_LINVAR
INST_UUID=$INST_UUID
INST_ID=$INST_ID
INST_VDEV=$INST_VDEV" > /mnt/root/chroot
echo DISK=\($(for i in ${DISK[@]}; do printf "$i "; done)\) >> /mnt/root/chroot
INST_VDEV=$INST_VDEV
DISK=$DISK" > /mnt/root/chroot
arch-chroot /mnt bash --login
cd ~
#. Source variables::

12
docs/Getting Started/Arch Linux/Root on ZFS/4-optional-configuration.rst
View File

@@ -82,6 +82,7 @@ root pool will be replaced by keyfile, embedded in initrd.
chmod 700 /etc/cryptkey.d/
dd bs=32 count=1 if=/dev/urandom of=/etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs
dd bs=32 count=1 if=/dev/urandom of=/etc/cryptkey.d/bpool_$INST_UUID-key-luks
chmod u=r,go= /etc/cryptkey.d/*
#. Backup boot pool::
@@ -92,7 +93,7 @@ root pool will be replaced by keyfile, embedded in initrd.
umount /boot/efi
for i in ${DISK[@]}; do
for i in ${DISK}; do
umount /boot/efis/${i##*/}-part1
done
@@ -102,7 +103,7 @@ root pool will be replaced by keyfile, embedded in initrd.
#. Create LUKS containers::
for i in ${DISK[@]}; do
for i in ${DISK}; do
cryptsetup luksFormat -q --type luks1 --key-file /etc/cryptkey.d/bpool_$INST_UUID-key-luks $i-part2
echo $LUKS_PWD | cryptsetup luksAddKey --key-file /etc/cryptkey.d/bpool_$INST_UUID-key-luks $i-part2
cryptsetup open ${i}-part2 ${i##*/}-part2-luks-bpool_$INST_UUID --key-file /etc/cryptkey.d/bpool_$INST_UUID-key-luks
@@ -132,7 +133,7 @@ root pool will be replaced by keyfile, embedded in initrd.
-O mountpoint=/boot \
bpool_$INST_UUID \
$INST_VDEV \
$(for i in ${DISK[@]}; do
$(for i in ${DISK}; do
printf "/dev/mapper/${i##*/}-part2-luks-bpool_$INST_UUID ";
done)
@@ -146,7 +147,7 @@ root pool will be replaced by keyfile, embedded in initrd.
mount /boot
mount /boot/efi
for i in ${DISK[@]}; do
for i in ${DISK}; do
mount /boot/efis/${i##*/}-part1
done
@@ -206,7 +207,8 @@ Persistent swap and hibernation
# create key and format partition as LUKS container
dd bs=32 count=1 if=/dev/urandom of=${INST_SWAPKEY};
cryptsetup luksFormat -q --type luks2 --key-file ${INST_SWAPKEY} ${INST_PRIMARY_DISK}-part4;
chmod u=r,go= /etc/cryptkey.d/*
cryptsetup luksFormat -q --type luks2 --key-file ${INST_SWAPKEY} ${INST_PRIMARY_DISK}-part4
cryptsetup luksOpen ${INST_PRIMARY_DISK}-part4 ${INST_SWAPMAPPER} --key-file ${INST_SWAPKEY}
# initialize swap space

26
docs/Getting Started/Arch Linux/Root on ZFS/5-bootloader.rst
View File

@@ -48,6 +48,10 @@ Install GRUB
#. Generate initrd::
rm -f /etc/zfs/zpool.cache
touch /etc/zfs/zpool.cache
chmod a-w /etc/zfs/zpool.cache
chattr +i /etc/zfs/zpool.cache
mkinitcpio -P
#. Create GRUB boot directory, in ESP and boot pool::
@@ -63,7 +67,7 @@ Install GRUB
#. If using legacy booting, install GRUB to every disk::
for i in ${DISK[@]}; do
for i in ${DISK}; do
grub-install --boot-directory /boot/efi/EFI/arch --target=i386-pc $i
done
@@ -71,7 +75,7 @@ Install GRUB
grub-install --boot-directory /boot/efi/EFI/arch --efi-directory /boot/efi/
grub-install --boot-directory /boot/efi/EFI/arch --efi-directory /boot/efi/ --removable
for i in ${DISK[@]}; do
for i in ${DISK}; do
efibootmgr -cgp 1 -l "\EFI\arch\grubx64.efi" \
-L "arch-${i##*/}" -d ${i}
done
@@ -120,6 +124,24 @@ Finish Installation
reboot
Post installaion
~~~~~~~~~~~~~~~~
#. If you have other data pools, generate list of datasets for `zfs-mount-generator
<https://manpages.ubuntu.com/manpages/focal/man8/zfs-mount-generator.8.html>`__ to mount them at boot::
DATA_POOL='tank0 tank1'
# tab-separated zfs properties
# see /etc/zfs/zed.d/history_event-zfs-list-cacher.sh
export \
PROPS="name,mountpoint,canmount,atime,relatime,devices,exec\
,readonly,setuid,nbmand,encroot,keylocation"
for i in $DATA_POOL; do
zfs list -H -t filesystem -o $PROPS -r $i > /etc/zfs/zfs-list.cache/$i
done
#. After reboot, consider adding a normal user::
myUser=UserName

88
docs/Getting Started/Arch Linux/Root on ZFS/snippets/43_archiso.txt
View File

@@ -1,55 +1,99 @@
#!/bin/sh
# mountpoint of vfat-formatted partition
ESP_MNT=/boot/efi
# path to iso files relative to the partition
ISO_REL=/iso
# absolute path to iso files
ISO_PATH=${ESP_MNT}/${ISO_REL}
# df command needs warm up due to systemd mount-on-demand
ls $ISO_PATH 1> /dev/null
# vfat partition UUID
ESP_UUID=$(blkid -s UUID -o value $(df --output=source ${ISO_PATH} | tail -n +2))
cat <<EOF
submenu 'archiso' {
rmmod tpm
insmod search_fs_uuid
set isorootuuid=$ESP_UUID
search --fs-uuid --no-floppy --set=isopart \$isorootuuid
set isopath=$ISO_REL
configfile \$prefix/archiso.cfg
submenu 'Boot from Live ISO' {
#if tpm module is loaded, grub might fail to setup loop
#error message: out of memory
#rmmod tpm
EOF
# limit detected number of ISOs, too many
# lines might crush grub
ISO_NUM=0
# for archlinux
for isofile in $ISO_PATH/archlinux-*.iso; do
if [ "$ISO_NUM" -gt 300 ]; then break; fi
isoname=${isofile##*/}
cat <<EOF
submenu "$isoname" {
insmod search_fs_uuid
set isorootuuid=$ESP_UUID
search --fs-uuid --no-floppy --set=isopart \$isorootuuid
set isopath=$ISO_REL
loopback loop0 (\$isopart)\$isopath/$isoname
set root=(loop0)
menuentry "Arch Linux install medium" {
loopback loop0 (\$isopart)\$isopath/$isoname
linux (loop0)/arch/boot/x86_64/vmlinuz-linux \\
linux /arch/boot/x86_64/vmlinuz-linux \\
earlymodules=loop img_dev=/dev/disk/by-uuid/\$isorootuuid img_loop=\$isopath/$isoname
initrd (loop0)/arch/boot/intel-ucode.img
initrd (loop0)/arch/boot/amd-ucode.img
initrd (loop0)/arch/boot/x86_64/initramfs-linux.img
initrd /arch/boot/intel-ucode.img
initrd /arch/boot/amd-ucode.img
initrd /arch/boot/x86_64/initramfs-linux.img
}
menuentry "Arch Linux install medium, Copy to RAM" {
loopback loop0 (\$isopart)\$isopath/$isoname
linux (loop0)/arch/boot/x86_64/vmlinuz-linux \\
linux /arch/boot/x86_64/vmlinuz-linux \\
earlymodules=loop img_dev=/dev/disk/by-uuid/\$isorootuuid img_loop=\$isopath/$isoname \\
copytoram
initrd (loop0)/arch/boot/intel-ucode.img
initrd (loop0)/arch/boot/amd-ucode.img
initrd (loop0)/arch/boot/x86_64/initramfs-linux.img
initrd /arch/boot/intel-ucode.img
initrd /arch/boot/amd-ucode.img
initrd /arch/boot/x86_64/initramfs-linux.img
}
menuentry "Arch Linux install medium with speech" {
loopback loop0 (\$isopart)\$isopath/$isoname
linux (loop0)/arch/boot/x86_64/vmlinuz-linux \\
linux /arch/boot/x86_64/vmlinuz-linux \\
earlymodules=loop img_dev=/dev/disk/by-uuid/\$isorootuuid img_loop=\$isopath/$isoname \\
accessibility=on
initrd (loop0)/arch/boot/intel-ucode.img
initrd (loop0)/arch/boot/amd-ucode.img
initrd (loop0)/arch/boot/x86_64/initramfs-linux.img
initrd /arch/boot/intel-ucode.img
initrd /arch/boot/amd-ucode.img
initrd /arch/boot/x86_64/initramfs-linux.img
}
}
EOF
ISO_NUM=$(( $ISO_NUM + 1 ))
done
# for ubuntu
for isofile in $ISO_PATH/ubuntu-*.iso; do
if [ "$ISO_NUM" -gt 300 ]; then break; fi
isoname=${isofile##*/}
cat <<EOF
submenu "$isoname" {
insmod search_fs_uuid
set isorootuuid=$ESP_UUID
search --fs-uuid --no-floppy --set=isopart \$isorootuuid
set isopath=$ISO_REL
loopback loop0 (\$isopart)\$isopath/$isoname
set root=(loop0)
menuentry "Ubuntu" {
linux /casper/vmlinuz \\
boot=casper iso-scan/filename=\$isopath/$isoname
initrd /casper/initrd
}
menuentry "Ubuntu, Copy to RAM" {
linux /casper/vmlinuz \\
boot=casper iso-scan/filename=\$isopath/$isoname \\
toram
initrd /casper/initrd
}
}
EOF
ISO_NUM=$(( $ISO_NUM + 1 ))
done
cat <<EOF
}
EOF

15
docs/Getting Started/Fedora/Root on ZFS/1-preparation.rst
View File

@@ -6,10 +6,13 @@ Preparation
.. contents:: Table of Contents
:local:
#. Disable Secure Boot. ZFS modules can not be loaded if Secure Boot is enabled.
#. Download a variant of Fedora 34 live image
and boot from it.
#. Disable Secure Boot. ZFS modules can not be loaded of Secure Boot is enabled.
- `Fedora Workstation (GNOME) <https://download.fedoraproject.org/pub/fedora/linux/releases/34/Workstation/x86_64/iso/>`__
- `Fedora Spins (Xfce, i3, ...) <https://download.fedoraproject.org/pub/fedora/linux/releases/34/Spins/x86_64/iso/>`__
#. Set root password or ``/root/authorized_keys``.
#. Start SSH server::
@@ -20,10 +23,12 @@ Preparation
ssh root@192.168.1.19
#. Set SELinux to persmissive::
#. Temporarily set SELinux to permissive in live environment::
setenforce 0
SELinux will be enabled on the installed system.
#. Install ``kernel-devel``::
source /etc/os-release
@@ -70,17 +75,17 @@ Preparation
Declare disk array::
DISK=(/dev/disk/by-id/ata-FOO /dev/disk/by-id/nvme-BAR)
DISK='/dev/disk/by-id/ata-FOO /dev/disk/by-id/nvme-BAR'
For single disk installation, use::
DISK=(/dev/disk/by-id/disk1)
DISK='/dev/disk/by-id/disk1'
#. Choose a primary disk. This disk will be used
for primary EFI partition and hibernation, default to
first disk in the array::
INST_PRIMARY_DISK=${DISK[0]}
INST_PRIMARY_DISK=$(echo $DISK | cut -f1 -d\ )
#. Set vdev topology, possible values are:

29
docs/Getting Started/Fedora/Root on ZFS/2-system-installation.rst
View File

@@ -6,10 +6,27 @@ System Installation
.. contents:: Table of Contents
:local:
#. Optional: wipe solid-state drives with the generic tool
`blkdiscard <https://utcc.utoronto.ca/~cks/space/blog/linux/ErasingSSDsWithBlkdiscard>`__,
to clean previous partition tables and improve performance.
All content will be irrevocably destroyed::
for i in ${DISK}; do
blkdiscard -f $i &
done
wait
This is a quick operation and should be completed under one
minute.
For other device specific methods, see
`Memory cell clearing <https://wiki.archlinux.org/title/Solid_state_drive/Memory_cell_clearing>`__
#. Partition the disks.
See `Overview <0-overview.html>`__ for details::
for i in ${DISK[@]}; do
for i in ${DISK}; do
sgdisk --zap-all $i
sgdisk -n1:1M:+${INST_PARTSIZE_ESP}G -t1:EF00 $i
sgdisk -n2:0:+${INST_PARTSIZE_BPOOL}G -t2:BE00 $i
@@ -41,7 +58,7 @@ System Installation
-R /mnt \
bpool_$INST_UUID \
$INST_VDEV \
$(for i in ${DISK[@]}; do
$(for i in ${DISK}; do
printf "$i-part2 ";
done)
@@ -73,7 +90,7 @@ System Installation
-O mountpoint=/ \
rpool_$INST_UUID \
$INST_VDEV \
$(for i in ${DISK[@]}; do
$(for i in ${DISK}; do
printf "$i-part3 ";
done)
@@ -179,7 +196,7 @@ System Installation
#. Format and mount ESP::
for i in ${DISK[@]}; do
for i in ${DISK}; do
mkfs.vfat -n EFI ${i}-part1
mkdir -p /mnt/boot/efis/${i##*/}-part1
mount -t vfat ${i}-part1 /mnt/boot/efis/${i##*/}-part1
@@ -212,8 +229,8 @@ System Installation
dnf --installroot=/mnt --releasever=${INST_FEDORA_VER} -y install \
https://zfsonlinux.org/fedora/zfs-release.fc${INST_FEDORA_VER}.noarch.rpm \
@core grub2-efi-x64 grub2-pc-modules grub2-efi-x64-modules shim-x64 efibootmgr cryptsetup \
kernel kernel-devel
kernel kernel-devel python3-dnf-plugin-post-transaction-actions
#. Install ZFS::
dnf --installroot=/mnt --releasever=${INST_FEDORA_VER} -y install zfs zfs-dracut
dnf --installroot=/mnt -y install zfs zfs-dracut

33
docs/Getting Started/Fedora/Root on ZFS/3-system-configuration.rst
View File

@@ -21,14 +21,14 @@ System Configuration
#. Generate fstab::
echo bpool_$INST_UUID/$INST_ID/BOOT/default /boot zfs rw,xattr,posixacl 0 0 >> /mnt/etc/fstab
for i in ${DISK[@]}; do
for i in ${DISK}; do
echo UUID=$(blkid -s UUID -o value ${i}-part1) /boot/efis/${i##*/}-part1 vfat \
x-systemd.idle-timeout=1min,x-systemd.automount,noauto,umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab
done
echo UUID=$(blkid -s UUID -o value ${INST_PRIMARY_DISK}-part1) /boot/efi vfat \
x-systemd.idle-timeout=1min,x-systemd.automount,noauto,umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab
if [ "${INST_PARTSIZE_SWAP}" != "" ]; then
for i in ${DISK[@]}; do
for i in ${DISK}; do
echo ${i##*/}-part4-swap ${i}-part4 /dev/urandom swap,cipher=aes-cbc-essiv:sha256,size=256,discard >> /mnt/etc/crypttab
echo /dev/mapper/${i##*/}-part4-swap none swap x-systemd.requires=cryptsetup.target,defaults 0 0 >> /mnt/etc/fstab
done
@@ -43,24 +43,6 @@ System Configuration
echo 'add_dracutmodules+=" zfs "' > /mnt/etc/dracut.conf.d/zfs.conf
#. Enable DHCP on all ethernet ports::
tee /mnt/etc/systemd/network/20-default.network <<EOF
[Match]
Name=en*
Name=eth*
[Network]
DHCP=yes
EOF
systemctl enable systemd-networkd systemd-resolved --root=/mnt
Customize this file if the system is not using wired DHCP network.
See `Network Configuration <https://wiki.archlinux.org/index.php/Network_configuration>`__.
Alternatively, configure ``NetworkManager``.
#. Enable timezone sync::
hwclock --systohc
@@ -93,7 +75,12 @@ System Configuration
#. Enable ZFS services::
systemctl enable zfs-import-scan.service zfs-import.target zfs-mount zfs-zed zfs.target --root=/mnt
systemctl enable zfs-import-scan.service zfs-import.target zfs-zed zfs.target --root=/mnt
systemctl disable zfs-mount --root=/mnt
At boot, datasets on rpool are mounted with ``zfs-mount-generator``,
which can control the mounting process more precisely than ``zfs-mount.service``.
#. By default SSH server is enabled, allowing root login by password,
disable SSH server::
@@ -108,8 +95,8 @@ System Configuration
INST_UUID=$INST_UUID
INST_ID=$INST_ID
unalias -a
INST_VDEV=$INST_VDEV" > /mnt/root/chroot
echo DISK=\($(for i in ${DISK[@]}; do printf "$i "; done)\) >> /mnt/root/chroot
INST_VDEV=$INST_VDEV
DISK=$DISK" > /mnt/root/chroot
arch-chroot /mnt bash --login
#. Source variables::

140
docs/Getting Started/Fedora/Root on ZFS/4-optional-configuration.rst
View File

@@ -18,143 +18,3 @@ instance of an operating system.
`bieaz <https://gitlab.com/m_zhou/bieaz/-/releases/>`__ can
be installed to manage boot environments. Download and install
prebuilt rpm file.
Encrypt boot pool
~~~~~~~~~~~~~~~~~~~
**WARNING**: Encrypting boot pool may cause significant boot time increases.
In test installation, GRUB took nearly 2 minutes to decrypt LUKS container.
#. LUKS password::
LUKS_PWD=secure-passwd
You will need to enter the same password for
each disk at boot. As root pool key is
protected by this password, the previous warning
about password strength still apply.
Double-check password here. Complete reinstallation is
needed if entered wrong.
#. Create encryption keys::
mkdir /etc/cryptkey.d/
chmod 700 /etc/cryptkey.d/
dd bs=32 count=1 if=/dev/urandom of=/etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs
dd bs=32 count=1 if=/dev/urandom of=/etc/cryptkey.d/bpool_$INST_UUID-key-luks
#. Backup boot pool::
zfs snapshot -r bpool_$INST_UUID/$INST_ID@pre-luks
zfs send -Rv bpool_$INST_UUID/$INST_ID@pre-luks > /root/bpool_$INST_UUID-${INST_ID}-pre-luks
#. Unmount EFI partition::
umount /boot/efi
for i in ${DISK[@]}; do
umount /boot/efis/${i##*/}-part1
done
#. Destroy boot pool::
zpool destroy bpool_$INST_UUID
#. Create LUKS containers::
for i in ${DISK[@]}; do
cryptsetup luksFormat -q --type luks1 --key-file /etc/cryptkey.d/bpool_$INST_UUID-key-luks $i-part2
echo $LUKS_PWD | cryptsetup luksAddKey --key-file /etc/cryptkey.d/bpool_$INST_UUID-key-luks $i-part2
cryptsetup open ${i}-part2 ${i##*/}-part2-luks-bpool_$INST_UUID --key-file /etc/cryptkey.d/bpool_$INST_UUID-key-luks
echo ${i##*/}-part2-luks-bpool_$INST_UUID ${i}-part2 /etc/cryptkey.d/bpool_$INST_UUID-key-luks discard >> /etc/crypttab
done
GRUB 2.06 still does not have complete support for LUKS2, LUKS1
is used instead.
#. Embed key file in initrd::
echo "install_items+=\" \
/etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs \
/etc/cryptkey.d/bpool_$INST_UUID-key-luks \"" \
> /etc/dracut.conf.d/rpool_$INST_UUID-${INST_ID}-key-zfs.conf
#. Recreate boot pool with mappers as vdev::
zpool create \
-o compatibility=grub2 \
-o ashift=12 \
-o autotrim=on \
-O acltype=posixacl \
-O canmount=off \
-O compression=lz4 \
-O devices=off \
-O normalization=formD \
-O relatime=on \
-O xattr=sa \
-O mountpoint=/boot \
bpool_$INST_UUID \
$INST_VDEV \
$(for i in ${DISK[@]}; do
printf "/dev/mapper/${i##*/}-part2-luks-bpool_$INST_UUID ";
done)
#. Restore boot pool backup::
zfs recv bpool_${INST_UUID}/${INST_ID} < /root/bpool_$INST_UUID-${INST_ID}-pre-luks
rm /root/bpool_$INST_UUID-${INST_ID}-pre-luks
#. Mount boot dataset and EFI partitions::
mount /boot
mount /boot/efi
for i in ${DISK[@]}; do
mount /boot/efis/${i##*/}-part1
done
#. As keys are stored in initrd,
set secure permissions for ``/boot``::
chmod 700 /boot
#. Change root pool password to key file::
zfs change-key -l \
-o keylocation=file:///etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs \
-o keyformat=raw \
rpool_$INST_UUID/$INST_ID
#. Enable GRUB cryptodisk::
echo "GRUB_ENABLE_CRYPTODISK=y" >> /etc/default/grub
#. Import bpool service::
tee /etc/systemd/system/zfs-import-bpool-mapper.service <<EOF
[Unit]
Description=Import encrypted boot pool
Documentation=man:zpool(8)
DefaultDependencies=no
Requires=systemd-udev-settle.service
After=cryptsetup.target
Before=boot.mount
ConditionPathIsDirectory=/sys/module/zfs
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/sbin/zpool import -aNd /dev/mapper
[Install]
WantedBy=zfs-import.target
EOF
systemctl enable zfs-import-bpool-mapper.service
#. **Important**: Back up root dataset key ``/etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs``
to a secure location.
In the possible event of LUKS container corruption,
data on root set will only be available
with this key.

49
docs/Getting Started/Fedora/Root on ZFS/5-bootloader.rst
View File

@@ -72,13 +72,13 @@ Install GRUB
#. If using legacy booting, install GRUB to every disk::
for i in ${DISK[@]}; do
for i in ${DISK}; do
grub2-install --boot-directory /boot/efi/EFI/fedora --target=i386-pc $i
done
#. If using EFI::
for i in ${DISK[@]}; do
for i in ${DISK}; do
efibootmgr -cgp 1 -l "\EFI\fedora\shimx64.efi" \
-L "fedora-${i##*/}" -d ${i}
done
@@ -99,6 +99,33 @@ Install GRUB
cp -r $ESP_MIRROR/EFI $i
done
#. Automatically regenerate GRUB menu on kernel update::
tee /etc/dnf/plugins/post-transaction-actions.d/00-update-grub-menu-for-kernel.action <<EOF >/dev/null
# kernel-core package contains vmlinuz and initramfs
# change package name if non-standard kernel is used
kernel-core:in:/usr/local/sbin/update-grub-menu.sh
kernel-core:out:/usr/local/sbin/update-grub-menu.sh
EOF
tee /usr/local/sbin/update-grub-menu.sh <<-'EOF' >/dev/null
#!/bin/sh
export PATH=$PATH:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
export ZPOOL_VDEV_NAME_PATH=YES
source /etc/os-release
grub2-mkconfig -o /boot/efi/EFI/${ID}/grub.cfg
cp /boot/efi/EFI/${ID}/grub.cfg /boot/efi/EFI/${ID}/grub2/grub.cfg
cp /boot/efi/EFI/${ID}/grub.cfg /boot/grub2/grub.cfg
ESP_MIRROR=$(mktemp -d)
cp -r /boot/efi/EFI $ESP_MIRROR
for i in /boot/efis/*; do
cp -r $ESP_MIRROR/EFI $i
done
rm -rf $ESP_MIRROR
EOF
chmod +x /usr/local/sbin/update-grub-menu.sh
#. Notes for GRUB on Fedora
To support Secure Boot, GRUB has been heavily modified by Fedora,
@@ -146,6 +173,24 @@ Finish Installation
reboot
Post installaion
~~~~~~~~~~~~~~~~
#. If you have other data pools, generate list of datasets for `zfs-mount-generator
<https://manpages.ubuntu.com/manpages/focal/man8/zfs-mount-generator.8.html>`__ to mount them at boot::
DATA_POOL='tank0 tank1'
# tab-separated zfs properties
# see /etc/zfs/zed.d/history_event-zfs-list-cacher.sh
export \
PROPS="name,mountpoint,canmount,atime,relatime,devices,exec\
,readonly,setuid,nbmand,encroot,keylocation"
for i in $DATA_POOL; do
zfs list -H -t filesystem -o $PROPS -r $i > /etc/zfs/zfs-list.cache/$i
done
#. After reboot, consider adding a normal user::
myUser=UserName

6
docs/Getting Started/Fedora/index.rst
View File

@@ -16,6 +16,12 @@ Note: this is for installing ZFS on an existing Fedora
installation. To use ZFS as root file system,
see below.
#. If ``zfs-fuse`` from official Fedora repo is installed,
remove it first. It is not maintained and should not be used
under any circumstance::
dnf remove -y zfs-fuse
#. Add ZFS repo::
dnf install -y https://zfsonlinux.org/fedora/zfs-release$(rpm -E %dist).noarch.rpm

20
docs/Getting Started/RHEL-based distro/RHEL 8-based distro Root on ZFS/1-preparation.rst
View File

@@ -6,10 +6,10 @@ Preparation
.. contents:: Table of Contents
:local:
#. Disable Secure Boot. ZFS modules can not be loaded if Secure Boot is enabled.
#. Download a variant of `Rocky Linux 8.4 Live
ISO <https://dl.rockylinux.org/pub/rocky/8.4/Live/x86_64/>`__ and boot from it.
#. Disable Secure Boot. ZFS modules can not be loaded of Secure Boot is enabled.
#. Set root password or ``/root/authorized_keys``.
#. Start SSH server::
@@ -20,10 +20,20 @@ Preparation
ssh root@192.168.1.19
#. Set SELinux to permissive::
#. Temporarily set SELinux to permissive in live environment::
setenforce 0
SELinux will be enabled on the installed system.
#. Optional: If mirror speed is slow, you can manually pick a fixed mirror
from `mirrorlist <https://mirrors.rockylinux.org/mirrormanager/mirrors>`__
and apply it::
sed -i 's|^mirrorlist=|#mirrorlist=|g' /etc/yum.repos.d/*
sed -i 's|^#baseurl=|baseurl=|g' /etc/yum.repos.d/*
sed -i 's|dl.rockylinux.org/$contentdir|mirrors.sjtug.sjtu.edu.cn/rocky|g' /etc/yum.repos.d/*
#. Add ZFS repo::
source /etc/os-release
@@ -71,17 +81,17 @@ Preparation
Declare disk array::
DISK=(/dev/disk/by-id/ata-FOO /dev/disk/by-id/nvme-BAR)
DISK='/dev/disk/by-id/ata-FOO /dev/disk/by-id/nvme-BAR'
For single disk installation, use::
DISK=(/dev/disk/by-id/disk1)
DISK='/dev/disk/by-id/disk1'
#. Choose a primary disk. This disk will be used
for primary EFI partition, default to
first disk in the array::
INST_PRIMARY_DISK=${DISK[0]}
INST_PRIMARY_DISK=$(echo $DISK | cut -f1 -d\ )
#. Set vdev topology, possible values are:

41
docs/Getting Started/RHEL-based distro/RHEL 8-based distro Root on ZFS/2-system-installation.rst
View File

@@ -6,10 +6,27 @@ System Installation
.. contents:: Table of Contents
:local:
#. Optional: wipe solid-state drives with the generic tool
`blkdiscard <https://utcc.utoronto.ca/~cks/space/blog/linux/ErasingSSDsWithBlkdiscard>`__,
to clean previous partition tables and improve performance.
All content will be irrevocably destroyed::
for i in ${DISK}; do
blkdiscard $i &
done
wait
This is a quick operation and should be completed under one
minute.
For other device specific methods, see
`Memory cell clearing <https://wiki.archlinux.org/title/Solid_state_drive/Memory_cell_clearing>`__
#. Partition the disks.
See `Overview <0-overview.html>`__ for details::
for i in ${DISK[@]}; do
for i in ${DISK}; do
sgdisk --zap-all $i
sgdisk -n1:1M:+${INST_PARTSIZE_ESP}G -t1:EF00 $i
sgdisk -n2:0:+${INST_PARTSIZE_BPOOL}G -t2:BE00 $i
@@ -51,7 +68,7 @@ System Installation
-R /mnt \
bpool_$INST_UUID \
$INST_VDEV \
$(for i in ${DISK[@]}; do
$(for i in ${DISK}; do
printf "$i-part2 ";
done)
@@ -83,7 +100,7 @@ System Installation
-O mountpoint=/ \
rpool_$INST_UUID \
$INST_VDEV \
$(for i in ${DISK[@]}; do
$(for i in ${DISK}; do
printf "$i-part3 ";
done)
@@ -189,7 +206,7 @@ System Installation
#. Format and mount ESP::
for i in ${DISK[@]}; do
for i in ${DISK}; do
mkfs.vfat -n EFI ${i}-part1
mkdir -p /mnt/boot/efis/${i##*/}-part1
mount -t vfat ${i}-part1 /mnt/boot/efis/${i##*/}-part1
@@ -220,15 +237,13 @@ System Installation
#. Install base packages::
dnf --installroot=/mnt --releasever=${INST_RHEL_VER} -y install \
${RHEL_ZFS_REPO} @core epel-release grub2-efi-x64 grub2-pc-modules grub2-efi-x64-modules shim-x64 efibootmgr
dnf config-manager --installroot=/mnt --disable zfs
dnf config-manager --installroot=/mnt --enable zfs-kmod
${RHEL_ZFS_REPO} @core epel-release grub2-efi-x64 grub2-pc-modules \
grub2-efi-x64-modules shim-x64 efibootmgr \
kernel kernel-devel python3-dnf-plugin-post-transaction-actions
dnf install --installroot=/mnt -y zfs zfs-dracut
If speed is slow, you can manually pick a fixed mirror
from `mirrorlist <https://mirrors.rockylinux.org/mirrormanager/mirrors>`__
and apply it::
#. Update zfs repo if a newer release is available::
sed -i 's|^mirrorlist=|#mirrorlist=|g' /etc/yum.repos.d/*
sed -i 's|^#baseurl=|baseurl=|g' /etc/yum.repos.d/*
sed -i 's|dl.rockylinux.org/$contentdir|mirrors.sjtug.sjtu.edu.cn/rocky|g' /etc/yum.repos.d/*
source /mnt/etc/os-release
RHEL_ZFS_REPO_NEW=https://zfsonlinux.org/epel/zfs-release.el${VERSION_ID/./_}.noarch.rpm
dnf install --installroot=/mnt -y $RHEL_ZFS_REPO_NEW || true

22
docs/Getting Started/RHEL-based distro/RHEL 8-based distro Root on ZFS/3-system-configuration.rst
View File

@@ -21,14 +21,14 @@ System Configuration
#. Generate fstab::
echo bpool_$INST_UUID/$INST_ID/BOOT/default /boot zfs rw,xattr,posixacl 0 0 >> /mnt/etc/fstab
for i in ${DISK[@]}; do
for i in ${DISK}; do
echo UUID=$(blkid -s UUID -o value ${i}-part1) /boot/efis/${i##*/}-part1 vfat \
x-systemd.idle-timeout=1min,x-systemd.automount,noauto,umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab
done
echo UUID=$(blkid -s UUID -o value ${INST_PRIMARY_DISK}-part1) /boot/efi vfat \
x-systemd.idle-timeout=1min,x-systemd.automount,noauto,umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab
if [ "${INST_PARTSIZE_SWAP}" != "" ]; then
for i in ${DISK[@]}; do
for i in ${DISK}; do
echo ${i##*/}-part4-swap ${i}-part4 /dev/urandom swap,cipher=aes-cbc-essiv:sha256,size=256,discard >> /mnt/etc/crypttab
echo /dev/mapper/${i##*/}-part4-swap none swap x-systemd.requires=cryptsetup.target,defaults 0 0 >> /mnt/etc/fstab
done
@@ -70,7 +70,11 @@ System Configuration
#. Enable ZFS services::
systemctl enable zfs-import-scan.service zfs-import.target zfs-mount zfs-zed zfs.target --root=/mnt
systemctl enable zfs-import-scan.service zfs-import.target zfs-zed zfs.target --root=/mnt
systemctl disable zfs-mount --root=/mnt
At boot, datasets on rpool are mounted with ``zfs-mount-generator``,
which can control the mounting process more precisely than ``zfs-mount.service``.
#. By default SSH server is enabled, allowing root login by password,
disable SSH server::
@@ -85,9 +89,10 @@ System Configuration
INST_UUID=$INST_UUID
INST_ID=$INST_ID
unalias -a
TERM=xterm
INST_VDEV=$INST_VDEV
TERM=xterm" > /mnt/root/chroot
echo DISK=\($(for i in ${DISK[@]}; do printf "$i "; done)\) >> /mnt/root/chroot
INST_VDEV=$INST_VDEV
DISK=$DISK" > /mnt/root/chroot
arch-chroot /mnt bash --login
#. Source variables::
@@ -101,3 +106,10 @@ System Configuration
#. Set root password::
passwd
#. Build ZFS modules::
ls -1 /lib/modules \
| while read kernel_version; do
dkms autoinstall -k $kernel_version
done

60
docs/Getting Started/RHEL-based distro/RHEL 8-based distro Root on ZFS/5-bootloader.rst
View File

@@ -72,13 +72,13 @@ Install GRUB
#. If using legacy booting, install GRUB to every disk::
for i in ${DISK[@]}; do
for i in ${DISK}; do
grub2-install --boot-directory /boot/efi/EFI/rocky --target=i386-pc $i
done
#. If using EFI::
for i in ${DISK[@]}; do
for i in ${DISK}; do
efibootmgr -cgp 1 -l "\EFI\rocky\shimx64.efi" \
-L "rocky-${i##*/}" -d ${i}
done
@@ -101,6 +101,17 @@ Install GRUB
cp /boot/efi/EFI/rocky/grub.cfg /boot/efi/EFI/rocky/grub2/grub.cfg
cp /boot/efi/EFI/rocky/grub.cfg /boot/grub2/grub.cfg
The following errors may be safely ignored:
- ``device-mapper: reload ioctl on osprober-linux-sda2 (253:0) failed: Device or resource busy``
This is caused by os-prober probing OS on the partitions used by ZFS,
harmless but os-prober can be disabled by::
echo GRUB_DISABLE_OS_PROBER=true >> /etc/default/grub
- ``/usr/sbin/grub2-probe: error: ../grub-core/kern/fs.c:120:unknown filesystem.``
This is fixed by /etc/grub.d/09_fix_root_on_zfs
#. For both legacy and EFI booting: mirror ESP content::
ESP_MIRROR=$(mktemp -d)
@@ -110,6 +121,33 @@ Install GRUB
cp -r $ESP_MIRROR/EFI $i
done
#. Automatically regenerate GRUB menu on kernel update::
tee /etc/dnf/plugins/post-transaction-actions.d/00-update-grub-menu-for-kernel.action <<EOF >/dev/null
# kernel-core package contains vmlinuz and initramfs
# change package name if non-standard kernel is used
kernel-core:in:/usr/local/sbin/update-grub-menu.sh
kernel-core:out:/usr/local/sbin/update-grub-menu.sh
EOF
tee /usr/local/sbin/update-grub-menu.sh <<-'EOF' >/dev/null
#!/bin/sh
export PATH=$PATH:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
export ZPOOL_VDEV_NAME_PATH=YES
source /etc/os-release
grub2-mkconfig -o /boot/efi/EFI/${ID}/grub.cfg
cp /boot/efi/EFI/${ID}/grub.cfg /boot/efi/EFI/${ID}/grub2/grub.cfg
cp /boot/efi/EFI/${ID}/grub.cfg /boot/grub2/grub.cfg
ESP_MIRROR=$(mktemp -d)
cp -r /boot/efi/EFI $ESP_MIRROR
for i in /boot/efis/*; do
cp -r $ESP_MIRROR/EFI $i
done
rm -rf $ESP_MIRROR
EOF
chmod +x /usr/local/sbin/update-grub-menu.sh
#. Notes for GRUB on RHEL
To support Secure Boot, GRUB has been heavily modified by Fedora,
@@ -157,6 +195,24 @@ Finish Installation
reboot
Post installaion
~~~~~~~~~~~~~~~~
#. If you have other data pools, generate list of datasets for `zfs-mount-generator
<https://manpages.ubuntu.com/manpages/focal/man8/zfs-mount-generator.8.html>`__ to mount them at boot::
DATA_POOL='tank0 tank1'
# tab-separated zfs properties
# see /etc/zfs/zed.d/history_event-zfs-list-cacher.sh
export \
PROPS="name,mountpoint,canmount,atime,relatime,devices,exec\
,readonly,setuid,nbmand,encroot,keylocation"
for i in $DATA_POOL; do
zfs list -H -t filesystem -o $PROPS -r $i > /etc/zfs/zfs-list.cache/$i
done
#. After reboot, consider adding a normal user::
myUser=UserName

2
docs/conf.py
View File

@@ -22,7 +22,7 @@ import sphinx_rtd_theme
# -- Project information -----------------------------------------------------
project = u'OpenZFS'
copyright = u'2020, OpenZFS'
copyright = u'2021, OpenZFS'
author = u'OpenZFS'
# The short X.Y version