Updates and cleanup for Arch, RHEL, NixOS and Fedora
Signed-off-by: Maurice Zhou <ja@apvc.uk>
This commit is contained in:
Maurice Zhou
committed by
George Melikov
George Melikov
parent
5777295f0a
commit
2766cb7197
@@ -1,49 +0,0 @@
|
|||||||
.. highlight:: sh
|
|
||||||
|
|
||||||
Live image
|
|
||||||
============
|
|
||||||
|
|
||||||
Latest live image might contain a kernel incompatible with
|
|
||||||
ZFS. Check the compatibility with the following procedure.
|
|
||||||
|
|
||||||
#. Choose a mirror::
|
|
||||||
|
|
||||||
https://archlinux.org/mirrorlist/all/
|
|
||||||
https://gitea.artixlinux.org/packagesA/artix-mirrorlist/src/branch/master/trunk/mirrorlist
|
|
||||||
|
|
||||||
#. Check the build date of the
|
|
||||||
latest Arch Linux live image::
|
|
||||||
|
|
||||||
https://mirrors.dotsrc.org/archlinux/iso/latest/
|
|
||||||
https://mirrors.dotsrc.org/artix-linux/iso/
|
|
||||||
# archlinux-2021.01.01-x86_64.iso
|
|
||||||
|
|
||||||
#. Check the kernel version of the live image::
|
|
||||||
|
|
||||||
https://archive.archlinux.org/repos/2021/01/01/core/os/x86_64
|
|
||||||
https://archive.artixlinux.org/repos/2021/01/01/system/os/x86_64
|
|
||||||
# linux-5.10.3.arch1-1-x86_64.pkg.tar.zst
|
|
||||||
|
|
||||||
#. Check latest zfs-dkms package version::
|
|
||||||
|
|
||||||
https://archzfs.com/archzfs/x86_64/
|
|
||||||
# zfs-dkms-2.0.1-1-x86_64.pkg.tar.zst
|
|
||||||
# zfs-linux-2.0.1_5.10.10.arch1.1-1-x86_64.pkg.tar.zst
|
|
||||||
|
|
||||||
#. Visit OpenZFS release page::
|
|
||||||
|
|
||||||
curl -L https://github.com/openzfs/zfs/raw/zfs-2.0.1/META \
|
|
||||||
| grep Linux
|
|
||||||
# Linux-Maximum: 5.10
|
|
||||||
# Linux-Minimum: 3.10
|
|
||||||
|
|
||||||
- If compatible, download the latest live image::
|
|
||||||
|
|
||||||
https://mirrors.dotsrc.org/archlinux/iso/latest/archlinux-2021.01.01-x86_64.iso
|
|
||||||
https://mirrors.dotsrc.org/artix-linux/iso/artix-base-openrc-20210101-x86_64.iso
|
|
||||||
|
|
||||||
- If not compatible, use an older live image and verify that it contains
|
|
||||||
a supported kernel using the above method::
|
|
||||||
|
|
||||||
https://mirrors.dotsrc.org/archlinux/iso/
|
|
||||||
https://iso.artixlinux.org/archived-isos.php
|
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
Arch Linux Root on ZFS
|
Arch Linux Root on ZFS
|
||||||
======================
|
======================
|
||||||
`Start here <Root%20on%20ZFS/0-overview.html>`__.
|
`Start here <Root%20on%20ZFS/1-preparation.html>`__.
|
||||||
|
|
||||||
Contents
|
Contents
|
||||||
--------
|
--------
|
||||||
|
|||||||
@@ -1,151 +0,0 @@
|
|||||||
.. highlight:: sh
|
|
||||||
|
|
||||||
Overview
|
|
||||||
======================
|
|
||||||
This document describes how to install Arch Linux with ZFS as root
|
|
||||||
file system.
|
|
||||||
|
|
||||||
Caution
|
|
||||||
~~~~~~~
|
|
||||||
|
|
||||||
- This guide wipes entire physical disks. Back up existing data.
|
|
||||||
- `GRUB does not and
|
|
||||||
will not work on 4Kn drive with legacy (BIOS) booting.
|
|
||||||
<http://savannah.gnu.org/bugs/?46700>`__
|
|
||||||
|
|
||||||
Partition layout
|
|
||||||
~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
GUID partition table (GPT) is used.
|
|
||||||
EFI system partition will be referred to as **ESP** in this document.
|
|
||||||
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
| Name | legacy boot | ESP | Boot pool | swap | root pool | remaining space |
|
|
||||||
+======================+======================+=======================+======================+=====================+=======================+=================+
|
|
||||||
| File system | | vfat | ZFS | swap | ZFS | |
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
| Size | 1M | 4G, or 1G w/o ISO | 4G | depends on RAM size | | |
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
| Optional encryption | | *Secure Boot* | luks 1 | plain dm-crypt or | ZFS native encryption | |
|
|
||||||
| | | | | luks2 | | |
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
| Partition no. | 5 | 1 | 2 | 4 | 3 | |
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
| Mount point | | /boot/efi | /boot | | / | |
|
|
||||||
| | | /boot/efis/disk-part1 | | | | |
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
|
|
||||||
Dataset layout
|
|
||||||
~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| Dataset | canmount | mountpoint | container | notes |
|
|
||||||
+===========================+======================+======================+=====================================+===========================================+
|
|
||||||
| bpool | off | /boot | contains sys | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool | off | / | contains sys | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| bpool/sys | off | none | contains BOOT | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys | off | none | contains ROOT | sys is encryptionroot |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| bpool/sys/BOOT | off | none | contains boot environments | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/ROOT | off | none | contains boot environments | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/DATA | off | none | contains placeholder "default" | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/DATA/default | off | / | contains user datasets | child datsets inherits mountpoint |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/DATA/default/ | on | /home (inherited) | no | |
|
|
||||||
| home | | | | user datasets, also called "shared |
|
|
||||||
| | | | | datasets", "persistent datasets"; also |
|
|
||||||
| | | | | include /var/lib, /srv, ... |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| bpool/sys/BOOT/default | noauto | /boot | no | noauto is used to switch BE. because of |
|
|
||||||
| | | | | noauto, must use fstab to mount |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/ROOT/default | noauto | / | no | mounted by initrd zfs hook |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| bpool/sys/BOOT/be1 | noauto | /boot | no | see bpool/sys/BOOT/default |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/ROOT/be1 | noauto | / | no | see rpool/sys/ROOT/default |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
|
|
||||||
Encryption
|
|
||||||
~~~~~~~~~~
|
|
||||||
|
|
||||||
- Swap
|
|
||||||
|
|
||||||
Swap is always encrypted. By default, swap is encrypted
|
|
||||||
with plain dm-crypt with key generated from ``/dev/urandom``
|
|
||||||
at every boot. Swap content does not persist between reboots.
|
|
||||||
|
|
||||||
LUKS2-encrypted persistent swap can be
|
|
||||||
enabled after encrypting both boot pool and root pool, see below.
|
|
||||||
|
|
||||||
With persistent swap, hibernation (suspend-to-disk) can be enabled.
|
|
||||||
|
|
||||||
- Root pool
|
|
||||||
|
|
||||||
ZFS native encryption can be optionally enabled for ``rpool/sys``
|
|
||||||
and child datasets.
|
|
||||||
|
|
||||||
User should be aware that, ZFS native encryption does not
|
|
||||||
encrypt some metadata of the datasets.
|
|
||||||
ZFS native encryption also does not change master key when ``zfs change-key`` is invoked.
|
|
||||||
Therefore, you should wipe the disk when password is compromised to protect confidentiality.
|
|
||||||
See `zfs-load-key.8 <https://openzfs.github.io/openzfs-docs/man/8/zfs-load-key.8.html>`__
|
|
||||||
and `zfs-change-key.8 <https://openzfs.github.io/openzfs-docs/man/8/zfs-change-key.8.html>`__
|
|
||||||
for more information regarding ZFS native encryption.
|
|
||||||
|
|
||||||
Encryption is enabled at dataset creation and can not be disabled later.
|
|
||||||
Password can be supplied via SSH.
|
|
||||||
|
|
||||||
- Boot pool
|
|
||||||
|
|
||||||
After encrypting root pool, boot pool can also be encrypted with LUKS1.
|
|
||||||
This protects initrd from attacks and also protects key material in initrd.
|
|
||||||
|
|
||||||
Password must be interactively entered at boot in GRUB. This disables
|
|
||||||
password with SSH.
|
|
||||||
|
|
||||||
- Bootloader
|
|
||||||
|
|
||||||
Bootloader can not be encrypted.
|
|
||||||
|
|
||||||
However, with Secure Boot, bootloader
|
|
||||||
can be verified by motherboard firmware to be untempered,
|
|
||||||
which should be sufficient for most purposes.
|
|
||||||
|
|
||||||
As enabling Secure Boot is device specific, this is not
|
|
||||||
covered in detail.
|
|
||||||
|
|
||||||
Booting with disk failure
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
This guide is written with disk failure in mind.
|
|
||||||
|
|
||||||
If disks used in Root on ZFS pool failed, but
|
|
||||||
sufficient redundancy for both root pool and boot pool
|
|
||||||
still exists, the system will still boot normally.
|
|
||||||
|
|
||||||
Swap partition on the failed disk will fail to mount,
|
|
||||||
after an 1m30s timeout.
|
|
||||||
|
|
||||||
This feature is useful for use cases such
|
|
||||||
as an unattended remote server.
|
|
||||||
|
|
||||||
Example:
|
|
||||||
|
|
||||||
- System has disks ``n>1``
|
|
||||||
|
|
||||||
- Installed with mirrored setup
|
|
||||||
|
|
||||||
- Mirrored setup can tolerate up to ``n-1`` disk failures
|
|
||||||
|
|
||||||
- Disconnect one or more disks, keep at least
|
|
||||||
one disk connected
|
|
||||||
|
|
||||||
- System still boots, but fails to mount swap and
|
|
||||||
EFI partition
|
|
||||||
@@ -6,88 +6,21 @@ Preparation
|
|||||||
.. contents:: Table of Contents
|
.. contents:: Table of Contents
|
||||||
:local:
|
:local:
|
||||||
|
|
||||||
#. Download live ISO compatible with ZFS:
|
#. Disable Secure Boot. ZFS modules can not be loaded if Secure Boot is enabled.
|
||||||
|
#. Download latest `Arch Linux live image <https://archlinux.org/download/>`__ and boot from it.
|
||||||
#. Check zfs-dkms version
|
#. Connect to the Internet.
|
||||||
|
|
||||||
Visit `archzfs repo <https://archzfs.com/archzfs/x86_64/>`__,
|
|
||||||
search for ``zfs-dkms`` in page::
|
|
||||||
|
|
||||||
zfs-dkms-2.1.2-1-x86_64.pkg.tar.zst
|
|
||||||
|
|
||||||
Ignore unstable variants such as zfs-dkms-git or zfs-dkms-rc.
|
|
||||||
In this case, version number is 2.1.2, between two ``-``.
|
|
||||||
|
|
||||||
#. Check supported kernel version. Plug the version number into the
|
|
||||||
following link::
|
|
||||||
|
|
||||||
https://github.com/openzfs/zfs/blob/zfs-2.1.2/META
|
|
||||||
|
|
||||||
Visit the page, the supported max kernel version is::
|
|
||||||
|
|
||||||
Linux-Maximum: 5.15
|
|
||||||
|
|
||||||
#. Visit `Arch Linux releases page
|
|
||||||
<https://archlinux.org/releng/releases/>`__ and download a
|
|
||||||
supported version. Note that minor kernel releases does not
|
|
||||||
affect ZFS compatibility, i. e., both ``5.15.5`` and ``5.15.15``
|
|
||||||
are supported by ZFS with ``Linux-Maximum: 5.15``.
|
|
||||||
|
|
||||||
#. Follow `installation guide on Arch wiki <https://wiki.archlinux.org/title/Installation_guide>`__
|
|
||||||
up to **Update the system clock**.
|
|
||||||
|
|
||||||
#. Set root password or ``/root/.ssh/authorized_keys``.
|
#. Set root password or ``/root/.ssh/authorized_keys``.
|
||||||
#. Start SSH server::
|
#. Start SSH server::
|
||||||
|
|
||||||
systemctl start sshd
|
systemctl restart sshd
|
||||||
|
|
||||||
#. Connect from another computer::
|
#. Connect from another computer::
|
||||||
|
|
||||||
ssh root@192.168.1.19
|
ssh root@192.168.1.19
|
||||||
|
|
||||||
and, most important, enter a bash shell::
|
#. Use bash shell. Other shell not tested::
|
||||||
|
|
||||||
bash
|
bash
|
||||||
|
|
||||||
This guide is untested with the default shell ``zsh`` in live environment.
|
|
||||||
|
|
||||||
#. Expand live root filesystem::
|
|
||||||
|
|
||||||
mount -o remount,size=2G /run/archiso/cowspace/
|
|
||||||
|
|
||||||
#. `Add archzfs repo <../0-archzfs-repo.html>`__.
|
|
||||||
|
|
||||||
#. `Install zfs-dkms in live environment <../2-zfs-dkms.html#installation>`__.
|
|
||||||
|
|
||||||
#. Load zfs kernel module::
|
|
||||||
|
|
||||||
modprobe zfs
|
|
||||||
|
|
||||||
#. Kernel variant
|
|
||||||
|
|
||||||
Store the kernel variant in a variable.
|
|
||||||
Available variants in official repo are:
|
|
||||||
|
|
||||||
- linux
|
|
||||||
- linux-lts
|
|
||||||
- linux-zen
|
|
||||||
- linux-hardened
|
|
||||||
|
|
||||||
::
|
|
||||||
|
|
||||||
INST_LINVAR='linux'
|
|
||||||
|
|
||||||
``linux-hardened`` does not support hibernation.
|
|
||||||
|
|
||||||
#. Unique pool suffix. ZFS expects pool names to be
|
|
||||||
unique, therefore it's recommended to create
|
|
||||||
pools with a unique suffix::
|
|
||||||
|
|
||||||
INST_UUID=$(dd if=/dev/urandom bs=1 count=100 2>/dev/null | tr -dc 'a-z0-9' | cut -c-6)
|
|
||||||
|
|
||||||
#. Identify this installation in ZFS filesystem path::
|
|
||||||
|
|
||||||
INST_ID=arch
|
|
||||||
|
|
||||||
#. Target disk
|
#. Target disk
|
||||||
|
|
||||||
@@ -95,8 +28,7 @@ Preparation
|
|||||||
|
|
||||||
ls /dev/disk/by-id/*
|
ls /dev/disk/by-id/*
|
||||||
|
|
||||||
If using virtio as disk bus, use
|
If using virtio as disk bus, use ``/dev/disk/by-path/*``.
|
||||||
``/dev/disk/by-path/*``.
|
|
||||||
|
|
||||||
Declare disk array::
|
Declare disk array::
|
||||||
|
|
||||||
@@ -106,64 +38,8 @@ Preparation
|
|||||||
|
|
||||||
DISK='/dev/disk/by-id/disk1'
|
DISK='/dev/disk/by-id/disk1'
|
||||||
|
|
||||||
#. Choose a primary disk. This disk will be used
|
|
||||||
for primary EFI partition and hibernation, default to
|
|
||||||
first disk in the array::
|
|
||||||
|
|
||||||
INST_PRIMARY_DISK=$(echo $DISK | cut -f1 -d\ )
|
|
||||||
|
|
||||||
If disk path contains colon ``:``, this disk
|
|
||||||
can not be used for hibernation. ``encrypt`` mkinitcpio
|
|
||||||
hook treats ``:`` as argument separator without a means to
|
|
||||||
escape this character.
|
|
||||||
|
|
||||||
#. Set vdev topology, possible values are:
|
|
||||||
|
|
||||||
- (not set, single disk or striped; no redundancy)
|
|
||||||
- mirror
|
|
||||||
- raidz1
|
|
||||||
- raidz2
|
|
||||||
- raidz3
|
|
||||||
|
|
||||||
::
|
|
||||||
|
|
||||||
INST_VDEV=
|
|
||||||
|
|
||||||
This will create a single vdev with the topology of your choice.
|
|
||||||
It is also possible to manually create a pool with multiple vdevs, such as::
|
|
||||||
|
|
||||||
zpool create --options \
|
|
||||||
poolName \
|
|
||||||
mirror sda sdb \
|
|
||||||
raidz2 sdc ... \
|
|
||||||
raidz3 sde ... \
|
|
||||||
spare sdf ...
|
|
||||||
|
|
||||||
Notice the cost of parity when using RAID-Z. See
|
|
||||||
`here <https://www.delphix.com/blog/delphix-engineering/zfs-raidz-stripe-width-or-how-i-learned-stop-worrying-and-love-raidz>`__
|
|
||||||
and `here <https://docs.google.com/spreadsheets/d/1tf4qx1aMJp8Lo_R6gpT689wTjHv6CGVElrPqTA0w_ZY/>`__.
|
|
||||||
|
|
||||||
For boot pool, which must be readable by GRUB, mirrored vdev should always be used for maximum redundancy.
|
|
||||||
This guide will use mirrored bpool for multi-disk setup.
|
|
||||||
|
|
||||||
Refer to `zpoolconcepts <https://openzfs.github.io/openzfs-docs/man/7/zpoolconcepts.7.html>`__
|
|
||||||
and `zpool-create <https://openzfs.github.io/openzfs-docs/man/8/zpool-create.8.html>`__
|
|
||||||
man pages for details.
|
|
||||||
|
|
||||||
#. Set partition size:
|
#. Set partition size:
|
||||||
|
|
||||||
Set ESP size. ESP contains Live ISO for recovery,
|
|
||||||
as described in `optional configurations <4-optional-configuration.html>`__::
|
|
||||||
|
|
||||||
INST_PARTSIZE_ESP=4 # in GB
|
|
||||||
#INST_PARTSIZE_ESP=1 # if local recovery is not needed
|
|
||||||
|
|
||||||
Set boot pool size. To avoid running out of space while using
|
|
||||||
boot environments, the minimum is 4GB. Adjust the size if you
|
|
||||||
intend to use multiple kernel/distros::
|
|
||||||
|
|
||||||
INST_PARTSIZE_BPOOL=4
|
|
||||||
|
|
||||||
Set swap size. It's `recommended <https://chrisdown.name/2018/01/02/in-defence-of-swap.html>`__
|
Set swap size. It's `recommended <https://chrisdown.name/2018/01/02/in-defence-of-swap.html>`__
|
||||||
to setup a swap partition. If you intend to use hibernation,
|
to setup a swap partition. If you intend to use hibernation,
|
||||||
the minimum should be no less than RAM size. Skip if swap is not needed::
|
the minimum should be no less than RAM size. Skip if swap is not needed::
|
||||||
@@ -173,3 +49,46 @@ Preparation
|
|||||||
Root pool size, use all remaining disk space if not set::
|
Root pool size, use all remaining disk space if not set::
|
||||||
|
|
||||||
INST_PARTSIZE_RPOOL=
|
INST_PARTSIZE_RPOOL=
|
||||||
|
|
||||||
|
#. Check kernel version::
|
||||||
|
|
||||||
|
uname -r
|
||||||
|
#5.18.7-arch1-1
|
||||||
|
|
||||||
|
#. Add ZFS repo::
|
||||||
|
|
||||||
|
curl -L https://archzfs.com/archzfs.gpg | pacman-key -a -
|
||||||
|
pacman-key --lsign-key $(curl -L https://git.io/JsfVS)
|
||||||
|
curl -L https://git.io/Jsfw2 > /etc/pacman.d/mirrorlist-archzfs
|
||||||
|
|
||||||
|
tee -a /etc/pacman.conf <<- 'EOF'
|
||||||
|
|
||||||
|
#[archzfs-testing]
|
||||||
|
#Include = /etc/pacman.d/mirrorlist-archzfs
|
||||||
|
|
||||||
|
[archzfs]
|
||||||
|
Include = /etc/pacman.d/mirrorlist-archzfs
|
||||||
|
EOF
|
||||||
|
|
||||||
|
#. Find a ZFS package compatible with the kernel:
|
||||||
|
|
||||||
|
Search kernel version string (e.g. 5.18.7) in both pages:
|
||||||
|
|
||||||
|
* https://archzfs.com/archive_archzfs/
|
||||||
|
* https://archzfs.com/archzfs/x86_64/
|
||||||
|
|
||||||
|
Result: https/.../archive_archzfs/zfs-linux-2.1.5_5.18.7.arch1.1-1-x86_64.pkg.tar.zst
|
||||||
|
|
||||||
|
#. Find compatible zfs-utils package:
|
||||||
|
|
||||||
|
Search ZFS version string (e.g. 2.1.5) in both pages above.
|
||||||
|
|
||||||
|
Result: https/.../archzfs/x86_64/zfs-utils-2.1.5-2-x86_64.pkg.tar.zst
|
||||||
|
|
||||||
|
#. Download both then install::
|
||||||
|
|
||||||
|
pacman -U link-to-zfs.zst link-to-utils.zst
|
||||||
|
|
||||||
|
#. Load kernel modules::
|
||||||
|
|
||||||
|
modprobe zfs
|
||||||
|
|||||||
@@ -6,46 +6,29 @@ System Installation
|
|||||||
.. contents:: Table of Contents
|
.. contents:: Table of Contents
|
||||||
:local:
|
:local:
|
||||||
|
|
||||||
#. Optional: wipe solid-state drives with the generic tool
|
#. Partition the disks::
|
||||||
`blkdiscard <https://utcc.utoronto.ca/~cks/space/blog/linux/ErasingSSDsWithBlkdiscard>`__,
|
|
||||||
to clean previous partition tables and improve performance.
|
|
||||||
|
|
||||||
All content will be irrevocably destroyed::
|
|
||||||
|
|
||||||
for i in ${DISK}; do
|
|
||||||
blkdiscard -f $i &
|
|
||||||
done
|
|
||||||
wait
|
|
||||||
|
|
||||||
This is a quick operation and should be completed under one
|
|
||||||
minute.
|
|
||||||
|
|
||||||
For other device specific methods, see
|
|
||||||
`Memory cell clearing <https://wiki.archlinux.org/title/Solid_state_drive/Memory_cell_clearing>`__
|
|
||||||
|
|
||||||
#. Partition the disks.
|
|
||||||
See `Overview <0-overview.html>`__ for details::
|
|
||||||
|
|
||||||
for i in ${DISK}; do
|
for i in ${DISK}; do
|
||||||
|
|
||||||
sgdisk --zap-all $i
|
sgdisk --zap-all $i
|
||||||
sgdisk -n1:1M:+${INST_PARTSIZE_ESP}G -t1:EF00 $i
|
|
||||||
sgdisk -n2:0:+${INST_PARTSIZE_BPOOL}G -t2:BE00 $i
|
sgdisk -n1:1M:+1G -t1:EF00 $i
|
||||||
if [ "${INST_PARTSIZE_SWAP}" != "" ]; then
|
|
||||||
sgdisk -n4:0:+${INST_PARTSIZE_SWAP}G -t4:8200 $i
|
sgdisk -n2:0:+4G -t2:BE00 $i
|
||||||
fi
|
|
||||||
if [ "${INST_PARTSIZE_RPOOL}" = "" ]; then
|
test -z $INST_PARTSIZE_SWAP || sgdisk -n4:0:+${INST_PARTSIZE_SWAP}G -t4:8200 $i
|
||||||
|
|
||||||
|
if test -z $INST_PARTSIZE_RPOOL; then
|
||||||
sgdisk -n3:0:0 -t3:BF00 $i
|
sgdisk -n3:0:0 -t3:BF00 $i
|
||||||
else
|
else
|
||||||
sgdisk -n3:0:+${INST_PARTSIZE_RPOOL}G -t3:BF00 $i
|
sgdisk -n3:0:+${INST_PARTSIZE_RPOOL}G -t3:BF00 $i
|
||||||
fi
|
fi
|
||||||
|
|
||||||
sgdisk -a1 -n5:24K:+1000K -t5:EF02 $i
|
sgdisk -a1 -n5:24K:+1000K -t5:EF02 $i
|
||||||
done
|
done
|
||||||
|
|
||||||
#. Create boot pool::
|
#. Create boot pool::
|
||||||
|
|
||||||
disk_num=0; for i in $DISK; do disk_num=$(( $disk_num + 1 )); done
|
|
||||||
if [ $disk_num -gt 1 ]; then INST_VDEV_BPOOL=mirror; fi
|
|
||||||
|
|
||||||
zpool create \
|
zpool create \
|
||||||
-o compatibility=grub2 \
|
-o compatibility=grub2 \
|
||||||
-o ashift=12 \
|
-o ashift=12 \
|
||||||
@@ -59,12 +42,14 @@ System Installation
|
|||||||
-O xattr=sa \
|
-O xattr=sa \
|
||||||
-O mountpoint=/boot \
|
-O mountpoint=/boot \
|
||||||
-R /mnt \
|
-R /mnt \
|
||||||
bpool_$INST_UUID \
|
bpool \
|
||||||
$INST_VDEV_BPOOL \
|
mirror \
|
||||||
$(for i in ${DISK}; do
|
$(for i in ${DISK}; do
|
||||||
printf "$i-part2 ";
|
printf "$i-part2 ";
|
||||||
done)
|
done)
|
||||||
|
|
||||||
|
If not using a multi-disk setup, remove ``mirror``.
|
||||||
|
|
||||||
You should not need to customize any of the options for the boot pool.
|
You should not need to customize any of the options for the boot pool.
|
||||||
|
|
||||||
GRUB does not support all of the zpool features. See ``spa_feature_names``
|
GRUB does not support all of the zpool features. See ``spa_feature_names``
|
||||||
@@ -91,101 +76,49 @@ System Installation
|
|||||||
-O relatime=on \
|
-O relatime=on \
|
||||||
-O xattr=sa \
|
-O xattr=sa \
|
||||||
-O mountpoint=/ \
|
-O mountpoint=/ \
|
||||||
rpool_$INST_UUID \
|
rpool \
|
||||||
$INST_VDEV \
|
mirror \
|
||||||
$(for i in ${DISK}; do
|
$(for i in ${DISK}; do
|
||||||
printf "$i-part3 ";
|
printf "$i-part3 ";
|
||||||
done)
|
done)
|
||||||
|
|
||||||
**Notes:**
|
If not using a multi-disk setup, remove ``mirror``.
|
||||||
|
|
||||||
- The use of ``ashift=12`` is recommended here because many drives
|
#. This section implements dataset layout as described in `overview <1-preparation.html>`__.
|
||||||
today have 4 KiB (or larger) physical sectors, even though they
|
|
||||||
present 512 B logical sectors. Also, a future replacement drive may
|
|
||||||
have 4 KiB physical sectors (in which case ``ashift=12`` is desirable)
|
|
||||||
or 4 KiB logical sectors (in which case ``ashift=12`` is required).
|
|
||||||
- Setting ``-O acltype=posixacl`` enables POSIX ACLs globally. If you
|
|
||||||
do not want this, remove that option, but later add
|
|
||||||
``-o acltype=posixacl`` (note: lowercase “o”) to the ``zfs create``
|
|
||||||
for ``/var/log``, as `journald requires ACLs
|
|
||||||
<https://askubuntu.com/questions/970886/journalctl-says-failed-to-search-journal-acl-operation-not-supported>`__
|
|
||||||
- Setting ``normalization=formD`` eliminates some corner cases relating
|
|
||||||
to UTF-8 filename normalization. It also implies ``utf8only=on``,
|
|
||||||
which means that only UTF-8 filenames are allowed. If you care to
|
|
||||||
support non-UTF-8 filenames, do not use this option. For a discussion
|
|
||||||
of why requiring UTF-8 filenames may be a bad idea, see `The problems
|
|
||||||
with enforced UTF-8 only filenames
|
|
||||||
<http://utcc.utoronto.ca/~cks/space/blog/linux/ForcedUTF8Filenames>`__.
|
|
||||||
- ``recordsize`` is unset (leaving it at the default of 128 KiB). If you
|
|
||||||
want to tune it (e.g. ``-o recordsize=1M``), see `these
|
|
||||||
<https://jrs-s.net/2019/04/03/on-zfs-recordsize/>`__ `various
|
|
||||||
<http://blog.programster.org/zfs-record-size>`__ `blog
|
|
||||||
<https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSFileRecordsizeGrowth>`__
|
|
||||||
`posts
|
|
||||||
<https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSRecordsizeAndCompression>`__.
|
|
||||||
- Setting ``relatime=on`` is a middle ground between classic POSIX
|
|
||||||
``atime`` behavior (with its significant performance impact) and
|
|
||||||
``atime=off`` (which provides the best performance by completely
|
|
||||||
disabling atime updates). Since Linux 2.6.30, ``relatime`` has been
|
|
||||||
the default for other filesystems. See `RedHat’s documentation
|
|
||||||
<https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/power_management_guide/relatime>`__
|
|
||||||
for further information.
|
|
||||||
- Setting ``xattr=sa`` `vastly improves the performance of extended
|
|
||||||
attributes
|
|
||||||
<https://github.com/zfsonlinux/zfs/commit/82a37189aac955c81a59a5ecc3400475adb56355>`__.
|
|
||||||
Inside ZFS, extended attributes are used to implement POSIX ACLs.
|
|
||||||
Extended attributes can also be used by user-space applications.
|
|
||||||
`They are used by some desktop GUI applications.
|
|
||||||
<https://en.wikipedia.org/wiki/Extended_file_attributes#Linux>`__
|
|
||||||
`They can be used by Samba to store Windows ACLs and DOS attributes;
|
|
||||||
they are required for a Samba Active Directory domain controller.
|
|
||||||
<https://wiki.samba.org/index.php/Setting_up_a_Share_Using_Windows_ACLs>`__
|
|
||||||
Note that ``xattr=sa`` is `Linux-specific
|
|
||||||
<https://openzfs.org/wiki/Platform_code_differences>`__. If you move your
|
|
||||||
``xattr=sa`` pool to another OpenZFS implementation besides ZFS-on-Linux,
|
|
||||||
extended attributes will not be readable (though your data will be). If
|
|
||||||
portability of extended attributes is important to you, omit the
|
|
||||||
``-O xattr=sa`` above. Even if you do not want ``xattr=sa`` for the whole
|
|
||||||
pool, it is probably fine to use it for ``/var/log``.
|
|
||||||
- Make sure to include the ``-part3`` portion of the drive path. If you
|
|
||||||
forget that, you are specifying the whole disk, which ZFS will then
|
|
||||||
re-partition, and you will lose the bootloader partition(s).
|
|
||||||
|
|
||||||
#. This section implements dataset layout as described in `overview <0-overview.html>`__.
|
|
||||||
|
|
||||||
Create root system container:
|
Create root system container:
|
||||||
|
|
||||||
- Unencrypted::
|
- Unencrypted::
|
||||||
|
|
||||||
zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID
|
zfs create \
|
||||||
|
-o canmount=off \
|
||||||
|
-o mountpoint=none \
|
||||||
|
rpool/archlinux
|
||||||
|
|
||||||
- Encrypted:
|
- Encrypted:
|
||||||
|
|
||||||
Pick a strong password. Once compromised, changing password will not keep your
|
Pick a strong password. Once compromised, changing password will not keep your
|
||||||
data safe. See ``zfs-change-key(8)`` for more info::
|
data safe. See ``zfs-change-key(8)`` for more info::
|
||||||
|
|
||||||
zfs create -o canmount=off -o mountpoint=none -o encryption=on -o keylocation=prompt -o keyformat=passphrase rpool_$INST_UUID/$INST_ID
|
zfs create \
|
||||||
|
-o canmount=off \
|
||||||
|
-o mountpoint=none \
|
||||||
|
-o encryption=on \
|
||||||
|
-o keylocation=prompt \
|
||||||
|
-o keyformat=passphrase \
|
||||||
|
rpool/archlinux
|
||||||
|
|
||||||
Create other system datasets::
|
Create system datasets::
|
||||||
|
|
||||||
zfs create -o canmount=off -o mountpoint=none bpool_$INST_UUID/$INST_ID
|
zfs create -o canmount=on -o mountpoint=/ rpool/archlinux/root
|
||||||
zfs create -o canmount=off -o mountpoint=none bpool_$INST_UUID/$INST_ID/BOOT
|
zfs create -o canmount=on -o mountpoint=/home rpool/archlinux/home
|
||||||
zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID/ROOT
|
zfs create -o canmount=off -o mountpoint=/var rpool/archlinux/var
|
||||||
zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID/DATA
|
zfs create -o canmount=on rpool/archlinux/var/lib
|
||||||
zfs create -o mountpoint=/boot -o canmount=noauto bpool_$INST_UUID/$INST_ID/BOOT/default
|
zfs create -o canmount=on rpool/archlinux/var/log
|
||||||
zfs create -o mountpoint=/ -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/default
|
|
||||||
zfs create -o mountpoint=/ -o canmount=noauto rpool_$INST_UUID/$INST_ID/ROOT/default
|
Create boot dataset::
|
||||||
zfs mount rpool_$INST_UUID/$INST_ID/ROOT/default
|
|
||||||
zfs mount bpool_$INST_UUID/$INST_ID/BOOT/default
|
zfs create -o canmount=on -o mountpoint=/boot bpool/archlinux
|
||||||
for i in {usr,var,var/lib};
|
|
||||||
do
|
|
||||||
zfs create -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/default/$i
|
|
||||||
done
|
|
||||||
for i in {home,root,srv,usr/local,var/log,var/spool};
|
|
||||||
do
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/$i
|
|
||||||
done
|
|
||||||
chmod 750 /mnt/root
|
|
||||||
|
|
||||||
#. Format and mount ESP::
|
#. Format and mount ESP::
|
||||||
|
|
||||||
@@ -196,65 +129,15 @@ System Installation
|
|||||||
done
|
done
|
||||||
|
|
||||||
mkdir -p /mnt/boot/efi
|
mkdir -p /mnt/boot/efi
|
||||||
mount -t vfat ${INST_PRIMARY_DISK}-part1 /mnt/boot/efi
|
mount -t vfat $(echo $DISK | cut -f1 -d\ )-part1 /mnt/boot/efi
|
||||||
|
|
||||||
#. Create separate user dataset at ``/home/User``, dateset name can be
|
#. Install packages::
|
||||||
changed later::
|
|
||||||
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/home/User
|
|
||||||
|
|
||||||
If needed, snapshot, rollback and other related permissions can be
|
|
||||||
delegated to the user later.
|
|
||||||
|
|
||||||
#. Create optional program data datasets to omit data from rollback::
|
|
||||||
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/games
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/www
|
|
||||||
# for GNOME
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/AccountsService
|
|
||||||
# for Docker
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/docker
|
|
||||||
# for NFS
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/nfs
|
|
||||||
# for LXC
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/lxc
|
|
||||||
# for LibVirt
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/libvirt
|
|
||||||
##other application
|
|
||||||
# zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/$name
|
|
||||||
|
|
||||||
Add other datasets when needed, such as PostgreSQL.
|
|
||||||
|
|
||||||
#. Install base packages::
|
|
||||||
|
|
||||||
pacstrap /mnt base vi mandoc grub efibootmgr mkinitcpio
|
pacstrap /mnt base vi mandoc grub efibootmgr mkinitcpio
|
||||||
|
|
||||||
#. Check compatible kernel version::
|
CompatibleVer=$(pacman -Si zfs-linux | grep 'Depends On' | sed "s|.*linux=||" | awk '{ print $1 }')
|
||||||
|
pacstrap -U /mnt https://archive.archlinux.org/packages/l/linux/linux-${CompatibleVer}-x86_64.pkg.tar.zst
|
||||||
|
|
||||||
INST_LINVER=$(pacman -Si zfs-${INST_LINVAR} \
|
pacstrap /mnt zfs-linux zfs-utils
|
||||||
| grep 'Depends On' \
|
|
||||||
| sed "s|.*${INST_LINVAR}=||" \
|
|
||||||
| awk '{ print $1 }')
|
|
||||||
|
|
||||||
#. Install kernel. Download from archive if kernel is not available::
|
|
||||||
|
|
||||||
if [ ${INST_LINVER} = \
|
|
||||||
$(pacman -Si ${INST_LINVAR} | grep Version | awk '{ print $3 }') ]; then
|
|
||||||
pacstrap /mnt ${INST_LINVAR}
|
|
||||||
else
|
|
||||||
pacstrap -U /mnt \
|
|
||||||
https://archive.archlinux.org/packages/l/${INST_LINVAR}/${INST_LINVAR}-${INST_LINVER}-x86_64.pkg.tar.zst
|
|
||||||
fi
|
|
||||||
|
|
||||||
Ignore ``error: command failed to execute correctly``.
|
|
||||||
|
|
||||||
#. Install archzfs package::
|
|
||||||
|
|
||||||
pacstrap /mnt zfs-$INST_LINVAR zfs-utils
|
|
||||||
|
|
||||||
#. Install firmware::
|
|
||||||
|
|
||||||
pacstrap /mnt linux-firmware intel-ucode amd-ucode
|
pacstrap /mnt linux-firmware intel-ucode amd-ucode
|
||||||
|
|
||||||
#. For other optional packages,
|
|
||||||
see `ArchWiki <https://wiki.archlinux.org/index.php/Installation_guide#Installation>`__.
|
|
||||||
|
|||||||
@@ -6,121 +6,63 @@ System Configuration
|
|||||||
.. contents:: Table of Contents
|
.. contents:: Table of Contents
|
||||||
:local:
|
:local:
|
||||||
|
|
||||||
#. Set `mkinitcpio zfs hook scan path
|
|
||||||
<https://github.com/archzfs/archzfs/blob/master/src/zfs-utils/zfs-utils.initcpio.install>`__::
|
|
||||||
|
|
||||||
echo GRUB_CMDLINE_LINUX=\"zfs_import_dir=${INST_PRIMARY_DISK%/*}\" >> /mnt/etc/default/grub
|
|
||||||
|
|
||||||
#. Generate fstab::
|
#. Generate fstab::
|
||||||
|
|
||||||
genfstab -U /mnt | sed 's;zfs[[:space:]]*;zfs zfsutil,;g' | grep "zfs zfsutil" >> /mnt/etc/fstab
|
mkdir -p /mnt/etc/
|
||||||
for i in ${DISK}; do
|
for i in ${DISK}; do
|
||||||
echo UUID=$(blkid -s UUID -o value ${i}-part1) /boot/efis/${i##*/}-part1 vfat \
|
echo UUID=$(blkid -s UUID -o value ${i}-part1) /boot/efis/${i##*/}-part1 vfat \
|
||||||
x-systemd.idle-timeout=1min,x-systemd.automount,noauto,umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab
|
umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab
|
||||||
done
|
done
|
||||||
echo UUID=$(blkid -s UUID -o value ${INST_PRIMARY_DISK}-part1) /boot/efi vfat \
|
echo $(echo $DISK | cut -f1 -d\ )-part1 /boot/efi vfat \
|
||||||
x-systemd.idle-timeout=1min,x-systemd.automount,noauto,umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab
|
noauto,umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab
|
||||||
if [ "${INST_PARTSIZE_SWAP}" != "" ]; then
|
|
||||||
for i in ${DISK}; do
|
|
||||||
echo ${i##*/}-part4-swap ${i}-part4 /dev/urandom swap,cipher=aes-cbc-essiv:sha256,size=256,discard >> /mnt/etc/crypttab
|
|
||||||
echo /dev/mapper/${i##*/}-part4-swap none swap defaults 0 0 >> /mnt/etc/fstab
|
|
||||||
done
|
|
||||||
fi
|
|
||||||
|
|
||||||
By default, systemd will halt boot process if any entry in ``/etc/fstab`` fails
|
|
||||||
to mount. This is unnecessary for mirrored EFI boot partitions.
|
|
||||||
With the above mount options, systemd will skip mounting them at boot,
|
|
||||||
only mount them on demand when accessed.
|
|
||||||
|
|
||||||
#. Configure mkinitcpio::
|
#. Configure mkinitcpio::
|
||||||
|
|
||||||
mv /mnt/etc/mkinitcpio.conf /mnt/etc/mkinitcpio.conf.original
|
mv /mnt/etc/mkinitcpio.conf /mnt/etc/mkinitcpio.conf.original
|
||||||
tee /mnt/etc/mkinitcpio.conf <<EOF
|
tee /mnt/etc/mkinitcpio.conf <<EOF
|
||||||
HOOKS=(base udev autodetect modconf block keyboard zfs filesystems)
|
HOOKS=(base udev autodetect modconf block keyboard zfs filesystems)
|
||||||
EOF
|
|
||||||
|
|
||||||
For more information on mkinitcpio configuration,
|
|
||||||
such as support for other keyboard layouts, see
|
|
||||||
`wiki article <https://wiki.archlinux.org/title/mkinitcpio>`__.
|
|
||||||
|
|
||||||
#. Enable DHCP on all ethernet ports::
|
|
||||||
|
|
||||||
tee /mnt/etc/systemd/network/20-default.network <<EOF
|
|
||||||
|
|
||||||
[Match]
|
|
||||||
Name=en*
|
|
||||||
Name=eth*
|
|
||||||
|
|
||||||
[Network]
|
|
||||||
DHCP=yes
|
|
||||||
EOF
|
EOF
|
||||||
systemctl enable systemd-networkd systemd-resolved --root=/mnt
|
|
||||||
|
|
||||||
Customize this file if the system is not using wired DHCP network.
|
#. Enable internet time synchronisation::
|
||||||
See `Network Configuration <https://wiki.archlinux.org/index.php/Network_configuration>`__.
|
|
||||||
|
|
||||||
Alternatively, install a network manager such as
|
hwclock --systohc
|
||||||
``NetworkManager``.
|
systemctl enable systemd-timesyncd --root=/mnt
|
||||||
|
|
||||||
#. Enable internet time sync::
|
#. Set locale, keymap, timezone, hostname and root password::
|
||||||
|
|
||||||
hwclock --systohc
|
|
||||||
systemctl enable systemd-timesyncd --root=/mnt
|
|
||||||
|
|
||||||
#. Interactively set locale, keymap, timezone, hostname and root password::
|
|
||||||
|
|
||||||
rm -f /mnt/etc/localtime
|
rm -f /mnt/etc/localtime
|
||||||
systemd-firstboot --root=/mnt --force --prompt --root-password=PASSWORD
|
systemd-firstboot --root=/mnt --prompt --force
|
||||||
|
|
||||||
This can be non-interactive, see man page for details::
|
|
||||||
|
|
||||||
rm -f /mnt/etc/localtime
|
|
||||||
systemd-firstboot --root=/mnt --force \
|
|
||||||
--locale="en_US.UTF-8" --locale-messages="en_US.UTF-8" \
|
|
||||||
--keymap=us --timezone="Europe/Berlin" --hostname=myHost \
|
|
||||||
--root-password=PASSWORD --root-shell=/bin/bash
|
|
||||||
|
|
||||||
``systemd-firstboot`` has bugs for setting root password, reset it here::
|
|
||||||
|
|
||||||
arch-chroot /mnt passwd
|
|
||||||
|
|
||||||
#. Generate host id::
|
#. Generate host id::
|
||||||
|
|
||||||
zgenhostid -f -o /mnt/etc/hostid
|
zgenhostid -f -o /mnt/etc/hostid
|
||||||
|
|
||||||
#. Ignore kernel updates::
|
|
||||||
|
|
||||||
sed -i 's/#IgnorePkg/IgnorePkg/' /mnt/etc/pacman.conf
|
|
||||||
sed -i "/^IgnorePkg/ s/$/ ${INST_LINVAR} ${INST_LINVAR}-headers zfs-${INST_LINVAR} zfs-utils/" /mnt/etc/pacman.conf
|
|
||||||
|
|
||||||
Kernel will be updated manually. See `here <../1-zfs-linux.html#update-kernel>`__.
|
|
||||||
|
|
||||||
#. Enable ZFS services::
|
#. Enable ZFS services::
|
||||||
|
|
||||||
systemctl enable zfs-import-scan.service zfs-import.target zfs-zed zfs.target --root=/mnt
|
systemctl enable zfs-import-scan.service zfs-mount zfs-import.target zfs-zed zfs.target --root=/mnt
|
||||||
systemctl disable zfs-mount --root=/mnt
|
|
||||||
|
#. Add archzfs repo::
|
||||||
|
|
||||||
|
curl -L https://archzfs.com/archzfs.gpg | pacman-key -a - --gpgdir /mnt/etc/pacman.d/gnupg
|
||||||
|
pacman-key --lsign-key --gpgdir /mnt/etc/pacman.d/gnupg $(curl -L https://git.io/JsfVS)
|
||||||
|
curl -L https://git.io/Jsfw2 > /mnt/etc/pacman.d/mirrorlist-archzfs
|
||||||
|
|
||||||
|
tee -a /mnt/etc/pacman.conf <<- 'EOF'
|
||||||
|
|
||||||
|
#[archzfs-testing]
|
||||||
|
#Include = /etc/pacman.d/mirrorlist-archzfs
|
||||||
|
|
||||||
|
[archzfs]
|
||||||
|
Include = /etc/pacman.d/mirrorlist-archzfs
|
||||||
|
EOF
|
||||||
|
|
||||||
At boot, datasets on rpool are mounted with ``/etc/fstab``,
|
|
||||||
which can control the mounting process more precisely than ``zfs-mount.service``.
|
|
||||||
|
|
||||||
#. Chroot::
|
#. Chroot::
|
||||||
|
|
||||||
echo "INST_PRIMARY_DISK=$INST_PRIMARY_DISK
|
|
||||||
INST_LINVAR=$INST_LINVAR
|
|
||||||
INST_UUID=$INST_UUID
|
|
||||||
INST_ID=$INST_ID
|
|
||||||
INST_VDEV=$INST_VDEV
|
|
||||||
DISK=\"$DISK\"" > /mnt/root/chroot
|
|
||||||
history -w /mnt/home/sys-install-pre-chroot.txt
|
history -w /mnt/home/sys-install-pre-chroot.txt
|
||||||
arch-chroot /mnt bash --login
|
arch-chroot /mnt /usr/bin/env DISK=$DISK bash
|
||||||
|
|
||||||
#. Source variables::
|
#. Generate locales::
|
||||||
|
|
||||||
source /root/chroot
|
|
||||||
|
|
||||||
#. Apply locales, change if needed::
|
|
||||||
|
|
||||||
echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen
|
echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen
|
||||||
locale-gen
|
locale-gen
|
||||||
|
|
||||||
#. `Add archzfs repo <../0-archzfs-repo.html>`__.
|
|
||||||
|
|||||||
@@ -1,316 +0,0 @@
|
|||||||
.. highlight:: sh
|
|
||||||
|
|
||||||
Optional Configuration
|
|
||||||
======================
|
|
||||||
|
|
||||||
.. contents:: Table of Contents
|
|
||||||
:local:
|
|
||||||
|
|
||||||
Skip to `bootloader <5-bootloader.html>`__ section if
|
|
||||||
no optional configuration is needed.
|
|
||||||
|
|
||||||
Boot environment manager
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
A boot environment is a dataset which contains a bootable
|
|
||||||
instance of an operating system. Within the context of this installation,
|
|
||||||
boot environments can be created on-the-fly to preserve root file system
|
|
||||||
states before pacman transactions.
|
|
||||||
|
|
||||||
Install an AUR helper of choice then install ``rozb3-pac`` from AUR
|
|
||||||
for pacman integration::
|
|
||||||
|
|
||||||
pacman -S --needed git base-devel sudo
|
|
||||||
echo 'nobody ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/00_nobody
|
|
||||||
su - nobody -s /bin/bash
|
|
||||||
mkdir /tmp/build
|
|
||||||
export HOME=/tmp/build
|
|
||||||
cd
|
|
||||||
git clone https://aur.archlinux.org/paru-bin.git
|
|
||||||
cd paru-bin
|
|
||||||
makepkg -si
|
|
||||||
paru -S rozb3-pac
|
|
||||||
exit
|
|
||||||
rm /etc/sudoers.d/00_nobody
|
|
||||||
|
|
||||||
If multi-disk setup is used, enable multi-disk
|
|
||||||
support inside ``/etc/bieaz.cfg``.
|
|
||||||
|
|
||||||
Supply password with SSH
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
#. Install mkinitcpio tools::
|
|
||||||
|
|
||||||
pacman -S mkinitcpio-netconf mkinitcpio-dropbear openssh
|
|
||||||
|
|
||||||
#. Store public keys in ``/etc/dropbear/root_key``::
|
|
||||||
|
|
||||||
vi /etc/dropbear/root_key
|
|
||||||
|
|
||||||
#. Edit mkinitcpio::
|
|
||||||
|
|
||||||
tee /etc/mkinitcpio.conf <<- 'EOF'
|
|
||||||
HOOKS=(base udev autodetect modconf block keyboard netconf dropbear zfsencryptssh zfs filesystems)
|
|
||||||
EOF
|
|
||||||
|
|
||||||
#. Add ``ip=`` to kernel command line::
|
|
||||||
|
|
||||||
# example DHCP
|
|
||||||
echo 'GRUB_CMDLINE_LINUX="ip=::::::dhcp"' >> /etc/default/grub
|
|
||||||
|
|
||||||
Details for ``ip=`` can be found at
|
|
||||||
`here <https://www.kernel.org/doc/html/latest/admin-guide/nfs/nfsroot.html#kernel-command-line>`__.
|
|
||||||
|
|
||||||
#. Generate host keys::
|
|
||||||
|
|
||||||
ssh-keygen -Am pem
|
|
||||||
dropbearconvert openssh dropbear /etc/ssh/ssh_host_ed25519_key /etc/dropbear/dropbear_ed25519_host_key
|
|
||||||
|
|
||||||
`mkinitcpio-dropbear
|
|
||||||
<https://archlinux.org/packages/community/any/mkinitcpio-dropbear/>`__
|
|
||||||
lacks support for converting ed25519 host key,
|
|
||||||
`see this pull request
|
|
||||||
<https://github.com/grazzolini/mkinitcpio-dropbear/pull/13>`__.
|
|
||||||
|
|
||||||
Encrypt boot pool
|
|
||||||
~~~~~~~~~~~~~~~~~~~
|
|
||||||
Note: This will disable password with SSH. The password previously set for
|
|
||||||
root pool will be replaced by keyfile, embedded in initrd.
|
|
||||||
|
|
||||||
#. LUKS password::
|
|
||||||
|
|
||||||
LUKS_PWD=secure-passwd
|
|
||||||
|
|
||||||
You will need to enter the same password for
|
|
||||||
each disk at boot. As root pool key is
|
|
||||||
protected by this password, the previous warning
|
|
||||||
about password strength still apply.
|
|
||||||
|
|
||||||
Double-check password here. Complete reinstallation is
|
|
||||||
needed if entered wrong.
|
|
||||||
|
|
||||||
#. Create encryption keys::
|
|
||||||
|
|
||||||
mkdir /etc/cryptkey.d/
|
|
||||||
chmod 700 /etc/cryptkey.d/
|
|
||||||
dd bs=32 count=1 if=/dev/urandom of=/etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs
|
|
||||||
dd bs=32 count=1 if=/dev/urandom of=/etc/cryptkey.d/bpool_$INST_UUID-key-luks
|
|
||||||
chmod u=r,go= /etc/cryptkey.d/*
|
|
||||||
|
|
||||||
#. Backup boot pool::
|
|
||||||
|
|
||||||
zfs snapshot -r bpool_$INST_UUID/$INST_ID@pre-luks
|
|
||||||
zfs send -Rv bpool_$INST_UUID/$INST_ID@pre-luks > /root/bpool_$INST_UUID-${INST_ID}-pre-luks
|
|
||||||
|
|
||||||
#. Unmount EFI partition::
|
|
||||||
|
|
||||||
umount /boot/efi
|
|
||||||
|
|
||||||
for i in ${DISK}; do
|
|
||||||
umount /boot/efis/${i##*/}-part1
|
|
||||||
done
|
|
||||||
|
|
||||||
#. Destroy boot pool::
|
|
||||||
|
|
||||||
zpool destroy bpool_$INST_UUID
|
|
||||||
|
|
||||||
#. Create LUKS containers::
|
|
||||||
|
|
||||||
for i in ${DISK}; do
|
|
||||||
cryptsetup luksFormat -q --type luks2 --key-file /etc/cryptkey.d/bpool_$INST_UUID-key-luks $i-part2
|
|
||||||
echo $LUKS_PWD | cryptsetup luksAddKey --pbkdf pbkdf2 --key-file /etc/cryptkey.d/bpool_$INST_UUID-key-luks $i-part2
|
|
||||||
cryptsetup open ${i}-part2 ${i##*/}-part2-luks-bpool_$INST_UUID --key-file /etc/cryptkey.d/bpool_$INST_UUID-key-luks
|
|
||||||
echo ${i##*/}-part2-luks-bpool_$INST_UUID ${i}-part2 /etc/cryptkey.d/bpool_$INST_UUID-key-luks discard >> /etc/crypttab
|
|
||||||
done
|
|
||||||
|
|
||||||
In GRUB 2.06, only the PBKDF2 key derivation function
|
|
||||||
is supported, thus PBKDF2 is used
|
|
||||||
for passphrase key slot. Other slots are not affected.
|
|
||||||
|
|
||||||
#. Embed key file in initrd::
|
|
||||||
|
|
||||||
echo 'FILES=(/etc/cryptkey.d/* )' >> /etc/mkinitcpio.conf
|
|
||||||
|
|
||||||
#. Recreate boot pool with mappers as vdev::
|
|
||||||
|
|
||||||
disk_num=0; for i in $DISK; do disk_num=$(( $disk_num + 1 )); done
|
|
||||||
if [ $disk_num -gt 1 ]; then INST_VDEV_BPOOL=mirror; fi
|
|
||||||
|
|
||||||
|
|
||||||
zpool create \
|
|
||||||
-o compatibility=grub2 \
|
|
||||||
-o ashift=12 \
|
|
||||||
-o autotrim=on \
|
|
||||||
-O acltype=posixacl \
|
|
||||||
-O canmount=off \
|
|
||||||
-O compression=lz4 \
|
|
||||||
-O devices=off \
|
|
||||||
-O normalization=formD \
|
|
||||||
-O relatime=on \
|
|
||||||
-O xattr=sa \
|
|
||||||
-O mountpoint=/boot \
|
|
||||||
bpool_$INST_UUID \
|
|
||||||
$INST_VDEV_BPOOL \
|
|
||||||
$(for i in ${DISK}; do
|
|
||||||
printf "/dev/mapper/${i##*/}-part2-luks-bpool_$INST_UUID ";
|
|
||||||
done)
|
|
||||||
|
|
||||||
#. Restore boot pool backup::
|
|
||||||
|
|
||||||
zfs recv bpool_${INST_UUID}/${INST_ID} < /root/bpool_$INST_UUID-${INST_ID}-pre-luks
|
|
||||||
rm /root/bpool_$INST_UUID-${INST_ID}-pre-luks
|
|
||||||
|
|
||||||
#. Mount boot dataset and EFI partitions::
|
|
||||||
|
|
||||||
mount /boot
|
|
||||||
mount /boot/efi
|
|
||||||
|
|
||||||
for i in ${DISK}; do
|
|
||||||
mount /boot/efis/${i##*/}-part1
|
|
||||||
done
|
|
||||||
|
|
||||||
#. As keys are stored in initrd,
|
|
||||||
set secure permissions for ``/boot``::
|
|
||||||
|
|
||||||
chmod 700 /boot
|
|
||||||
|
|
||||||
#. Change root pool password to key file::
|
|
||||||
|
|
||||||
zfs change-key -l \
|
|
||||||
-o keylocation=file:///etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs \
|
|
||||||
-o keyformat=raw \
|
|
||||||
rpool_$INST_UUID/$INST_ID
|
|
||||||
|
|
||||||
#. Import encrypted boot pool from ``/dev/mapper``::
|
|
||||||
|
|
||||||
curl -L https://git.io/Jsfwj > /etc/systemd/system/zfs-import-bpool-mapper.service
|
|
||||||
systemctl enable zfs-import-bpool-mapper.service
|
|
||||||
|
|
||||||
#. Remove ``zfsencryptssh`` hook.
|
|
||||||
Encrypted boot pool is incompatible with
|
|
||||||
password by SSH::
|
|
||||||
|
|
||||||
sed -i 's|zfsencryptssh||g' /etc/mkinitcpio.conf
|
|
||||||
|
|
||||||
If ``zfsencryptssh`` is not removed, initrd will
|
|
||||||
stuck at ``fail to load key material`` and fail to boot.
|
|
||||||
|
|
||||||
#. Enable GRUB cryptodisk::
|
|
||||||
|
|
||||||
echo "GRUB_ENABLE_CRYPTODISK=y" >> /etc/default/grub
|
|
||||||
#. Let GRUB decrypt all LUKS containers on boot::
|
|
||||||
|
|
||||||
tee -a /etc/grub.d/09_bpool_luks2-decryption <<FOE
|
|
||||||
#!/bin/sh
|
|
||||||
cat <<EOF
|
|
||||||
insmod luks2
|
|
||||||
insmod pbkdf2
|
|
||||||
insmod part_gpt
|
|
||||||
insmod gcry_rijndael
|
|
||||||
insmod gcry_sha256
|
|
||||||
insmod cryptodisk
|
|
||||||
cryptomount hd0,gpt2
|
|
||||||
EOF
|
|
||||||
FOE
|
|
||||||
|
|
||||||
chmod +x /etc/grub.d/09_bpool_luks2-decryption
|
|
||||||
|
|
||||||
#. **Important**: Back up root dataset key ``/etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs``
|
|
||||||
to a secure location.
|
|
||||||
|
|
||||||
In the possible event of LUKS container corruption,
|
|
||||||
data on root set will only be available
|
|
||||||
with this key.
|
|
||||||
|
|
||||||
Persistent swap and hibernation
|
|
||||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
||||||
#. Optional: enable persistent swap partition. By default
|
|
||||||
encryption key of swap partition is discarded on reboot::
|
|
||||||
|
|
||||||
INST_SWAPKEY=/etc/cryptkey.d/${INST_PRIMARY_DISK##*/}-part4-key-luks-swap
|
|
||||||
INST_SWAPMAPPER=${INST_PRIMARY_DISK##*/}-part4-luks-swap
|
|
||||||
|
|
||||||
# fstab
|
|
||||||
# remove all existing swap entries
|
|
||||||
sed -i '/ none swap defaults 0 0/d' /etc/fstab
|
|
||||||
# add single swap entry for LUKS encrypted swap partition
|
|
||||||
echo "/dev/mapper/${INST_SWAPMAPPER} none swap defaults 0 0" >> /etc/fstab
|
|
||||||
|
|
||||||
# comment out entry in crypttab
|
|
||||||
sed -i "s|^${INST_PRIMARY_DISK##*/}-part4-swap|#${INST_PRIMARY_DISK##*/}-part4-swap|" /etc/crypttab
|
|
||||||
|
|
||||||
# create key and format partition as LUKS container
|
|
||||||
dd bs=32 count=1 if=/dev/urandom of=${INST_SWAPKEY};
|
|
||||||
chmod u=r,go= /etc/cryptkey.d/*
|
|
||||||
cryptsetup luksFormat -q --type luks2 --key-file ${INST_SWAPKEY} ${INST_PRIMARY_DISK}-part4
|
|
||||||
cryptsetup luksOpen ${INST_PRIMARY_DISK}-part4 ${INST_SWAPMAPPER} --key-file ${INST_SWAPKEY}
|
|
||||||
|
|
||||||
# initialize swap space
|
|
||||||
mkswap /dev/mapper/${INST_SWAPMAPPER}
|
|
||||||
|
|
||||||
#. Optional: after enabling persistent swap partition,
|
|
||||||
enable hibernation::
|
|
||||||
|
|
||||||
# add hook in initrd
|
|
||||||
sed -i 's| zfs | encrypt resume zfs |' /etc/mkinitcpio.conf
|
|
||||||
# add kernel cmdline to decrypt swap in initrd
|
|
||||||
echo "GRUB_CMDLINE_LINUX=\" \
|
|
||||||
zfs_import_dir=${INST_PRIMARY_DISK%/*} \
|
|
||||||
cryptdevice=PARTUUID=$(blkid -s PARTUUID -o value ${INST_PRIMARY_DISK}-part4):${INST_SWAPMAPPER}:allow-discards \
|
|
||||||
cryptkey=rootfs:${INST_SWAPKEY} \
|
|
||||||
resume=/dev/mapper/${INST_SWAPMAPPER}\"" \
|
|
||||||
>> /etc/default/grub
|
|
||||||
|
|
||||||
Note that hibernation might not work with discrete graphics, virtio graphics or
|
|
||||||
AMD APU integrated graphics. This is not specific to this guide.
|
|
||||||
|
|
||||||
Computer must resume from a continuous swap space, resume
|
|
||||||
from multiple swap partitions is not supported.
|
|
||||||
|
|
||||||
``encrypt`` hook can only decrypt one container at boot.
|
|
||||||
``sd-encrypt`` can decrypt multiple devices but is
|
|
||||||
not compatible with ``zfs`` hook.
|
|
||||||
|
|
||||||
Do not touch anything on disk while the computer is
|
|
||||||
in hibernation, see `kernel documentation
|
|
||||||
<https://www.kernel.org/doc/html/latest/power/swsusp.html>`__.
|
|
||||||
|
|
||||||
Boot Live ISO with GRUB
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
GRUB `can be configured <https://wiki.archlinux.org/title/Multiboot_USB_drive>`__ to boot ISO file directly.
|
|
||||||
|
|
||||||
In this section, we will download Live ISO to ESP and configure GRUB to
|
|
||||||
boot from it. This enables system recovery and re-installation.
|
|
||||||
|
|
||||||
#. Download Live iso to EFI system partition::
|
|
||||||
|
|
||||||
mkdir /boot/efi/iso
|
|
||||||
cd /boot/efi/iso
|
|
||||||
# select a mirror and disk version
|
|
||||||
# curl -O https://mirrors.ocf.berkeley.edu/archlinux/iso/2021.11.01/archlinux-2021.11.01-x86_64.iso
|
|
||||||
curl -O https://archlinux.org/iso/2021.11.01/archlinux-2021.11.01-x86_64.iso.sig
|
|
||||||
gpg --auto-key-retrieve --verify archlinux-2021.11.01-x86_64.iso.sig
|
|
||||||
|
|
||||||
Additionally you can build your own live image
|
|
||||||
with `archiso package <https://gitlab.archlinux.org/archlinux/archiso>`__.
|
|
||||||
|
|
||||||
GRUB supports verifying checksum.
|
|
||||||
See `manual page
|
|
||||||
<https://www.gnu.org/software/grub/manual/grub/html_node/Command_002dline-and-menu-entry-commands.html#Command_002dline-and-menu-entry-commands>`__
|
|
||||||
for details.
|
|
||||||
|
|
||||||
#. Add custom GRUB entry for ``/boot/efi/iso/archlinux-*.iso``::
|
|
||||||
|
|
||||||
curl -L https://git.io/Jsfr3 > /etc/grub.d/43_archiso
|
|
||||||
chmod +x /etc/grub.d/43_archiso
|
|
||||||
|
|
||||||
You can also boot Live ISO for other distros, see `glim
|
|
||||||
<https://github.com/thias/glim/tree/master/grub2>`__
|
|
||||||
configurations.
|
|
||||||
|
|
||||||
ISO is not mirrored to other devices due to its size.
|
|
||||||
Change ``$ESP_MNT`` to adapt to other ESP.
|
|
||||||
|
|
||||||
#. Generate ``grub.cfg`` in the next step. If a new file
|
|
||||||
has been added later, regenerate ``grub.cfg``.
|
|
||||||
@@ -25,13 +25,6 @@ Workarounds have to be applied.
|
|||||||
echo 'export ZPOOL_VDEV_NAME_PATH=YES' >> /etc/profile.d/zpool_vdev_name_path.sh
|
echo 'export ZPOOL_VDEV_NAME_PATH=YES' >> /etc/profile.d/zpool_vdev_name_path.sh
|
||||||
source /etc/profile.d/zpool_vdev_name_path.sh
|
source /etc/profile.d/zpool_vdev_name_path.sh
|
||||||
|
|
||||||
Note that ``sudo`` will not read ``/etc/profile`` and will
|
|
||||||
not pass variables in parent shell. Consider setting the following
|
|
||||||
in ``/etc/sudoers``::
|
|
||||||
|
|
||||||
pacman -S --noconfirm --needed sudo
|
|
||||||
echo 'Defaults env_keep += "ZPOOL_VDEV_NAME_PATH"' >> /etc/sudoers
|
|
||||||
|
|
||||||
#. Pool name missing
|
#. Pool name missing
|
||||||
|
|
||||||
See `this bug report <https://savannah.gnu.org/bugs/?59614>`__.
|
See `this bug report <https://savannah.gnu.org/bugs/?59614>`__.
|
||||||
@@ -43,60 +36,51 @@ Workarounds have to be applied.
|
|||||||
|
|
||||||
sed -i "s|rpool=.*|rpool=\`zdb -l \${GRUB_DEVICE} \| grep -E '[[:blank:]]name' \| cut -d\\\' -f 2\`|" /etc/grub.d/10_linux
|
sed -i "s|rpool=.*|rpool=\`zdb -l \${GRUB_DEVICE} \| grep -E '[[:blank:]]name' \| cut -d\\\' -f 2\`|" /etc/grub.d/10_linux
|
||||||
|
|
||||||
|
Caution: this fix must be applied after every GRUB update and before generating the menu.
|
||||||
|
|
||||||
Install GRUB
|
Install GRUB
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
|
#. Create empty cache file and generate initrd::
|
||||||
#. Generate initrd::
|
|
||||||
|
|
||||||
rm -f /etc/zfs/zpool.cache
|
rm -f /etc/zfs/zpool.cache
|
||||||
touch /etc/zfs/zpool.cache
|
touch /etc/zfs/zpool.cache
|
||||||
chmod a-w /etc/zfs/zpool.cache
|
chmod a-w /etc/zfs/zpool.cache
|
||||||
chattr +i /etc/zfs/zpool.cache
|
chattr +i /etc/zfs/zpool.cache
|
||||||
|
|
||||||
mkinitcpio -P
|
mkinitcpio -P
|
||||||
|
|
||||||
#. Create GRUB boot directory, in ESP and boot pool::
|
|
||||||
|
|
||||||
mkdir -p /boot/efi/EFI/arch
|
|
||||||
mkdir -p /boot/grub
|
|
||||||
|
|
||||||
Boot environment-specific configuration (kernel, etc)
|
|
||||||
is stored in ``/boot/grub/grub.cfg``, enabling rollback.
|
|
||||||
|
|
||||||
#. When in doubt, install both legacy boot
|
|
||||||
and EFI.
|
|
||||||
|
|
||||||
#. If using legacy booting, install GRUB to every disk::
|
#. If using legacy booting, install GRUB to every disk::
|
||||||
|
|
||||||
for i in ${DISK}; do
|
for i in ${DISK}; do
|
||||||
grub-install --boot-directory /boot/efi/EFI/arch --target=i386-pc $i
|
grub-install --target=i386-pc $i
|
||||||
done
|
done
|
||||||
|
|
||||||
#. If using EFI::
|
#. If using EFI::
|
||||||
|
|
||||||
grub-install --boot-directory /boot/efi/EFI/arch --efi-directory /boot/efi/
|
grub-install --target x86_64-efi
|
||||||
grub-install --boot-directory /boot/efi/EFI/arch --efi-directory /boot/efi/ --removable
|
grub-install --target x86_64-efi --removable
|
||||||
for i in ${DISK}; do
|
for i in ${DISK}; do
|
||||||
efibootmgr -cgp 1 -l "\EFI\arch\grubx64.efi" \
|
efibootmgr -cgp 1 -l "\EFI\arch\grubx64.efi" \
|
||||||
-L "arch-${i##*/}" -d ${i}
|
-L "arch-${i##*/}" -d ${i}
|
||||||
done
|
done
|
||||||
|
|
||||||
#. Generate GRUB Menu::
|
#. Generate GRUB Menu:
|
||||||
|
|
||||||
grub-mkconfig -o /boot/efi/EFI/arch/grub/grub.cfg
|
Generate menu::
|
||||||
cp /boot/efi/EFI/arch/grub/grub.cfg /boot/grub/grub.cfg
|
|
||||||
|
echo GRUB_CMDLINE_LINUX=\"zfs_import_dir=/dev/disk/by-id/\" >> /etc/default/grub
|
||||||
|
grub-mkconfig -o /boot/grub/grub.cfg
|
||||||
|
cp /boot/grub/grub.cfg /boot/efi/EFI/arch/
|
||||||
|
|
||||||
#. For both legacy and EFI booting: mirror ESP content::
|
#. For both legacy and EFI booting: mirror ESP content::
|
||||||
|
|
||||||
ESP_MIRROR=$(mktemp -d)
|
ESP_MIRROR=$(mktemp -d)
|
||||||
|
unalias -a
|
||||||
cp -r /boot/efi/EFI $ESP_MIRROR
|
cp -r /boot/efi/EFI $ESP_MIRROR
|
||||||
for i in /boot/efis/*; do
|
for i in /boot/efis/*; do
|
||||||
cp -r $ESP_MIRROR/EFI $i
|
cp -r $ESP_MIRROR/EFI $i
|
||||||
done
|
done
|
||||||
|
rm -rf $ESP_MIRROR
|
||||||
Enable Secure Boot
|
|
||||||
----------------------------
|
|
||||||
|
|
||||||
This is optional. `See Arch Wiki article <https://wiki.archlinux.org/title/Secure_Boot>`__.
|
|
||||||
|
|
||||||
Finish Installation
|
Finish Installation
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
@@ -105,82 +89,11 @@ Finish Installation
|
|||||||
|
|
||||||
exit
|
exit
|
||||||
|
|
||||||
#. Take a snapshot of the clean installation for future use::
|
|
||||||
|
|
||||||
zfs snapshot -r rpool_$INST_UUID/$INST_ID@install
|
|
||||||
zfs snapshot -r bpool_$INST_UUID/$INST_ID@install
|
|
||||||
|
|
||||||
#. Unmount EFI system partition::
|
|
||||||
|
|
||||||
umount /mnt/boot/efi
|
|
||||||
umount /mnt/boot/efis/*
|
|
||||||
|
|
||||||
#. Export pools::
|
#. Export pools::
|
||||||
|
|
||||||
zpool export bpool_$INST_UUID
|
umount -Rl /mnt
|
||||||
zpool export rpool_$INST_UUID
|
zpool export -a
|
||||||
|
|
||||||
#. Reboot::
|
#. Reboot::
|
||||||
|
|
||||||
reboot
|
reboot
|
||||||
|
|
||||||
Post installaion
|
|
||||||
~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
#. If you have other data pools, generate list of datasets for `zfs-mount-generator
|
|
||||||
<https://manpages.ubuntu.com/manpages/focal/man8/zfs-mount-generator.8.html>`__ to mount them at boot::
|
|
||||||
|
|
||||||
DATA_POOL='tank0 tank1'
|
|
||||||
|
|
||||||
# tab-separated zfs properties
|
|
||||||
# see /etc/zfs/zed.d/history_event-zfs-list-cacher.sh
|
|
||||||
export \
|
|
||||||
PROPS="name,mountpoint,canmount,atime,relatime,devices,exec\
|
|
||||||
,readonly,setuid,nbmand,encroot,keylocation"
|
|
||||||
|
|
||||||
for i in $DATA_POOL; do
|
|
||||||
zfs list -H -t filesystem -o $PROPS -r $i > /etc/zfs/zfs-list.cache/$i
|
|
||||||
done
|
|
||||||
|
|
||||||
#. After reboot, consider adding a normal user::
|
|
||||||
|
|
||||||
# with root permissions
|
|
||||||
sudo -i
|
|
||||||
|
|
||||||
# store user name in a variable
|
|
||||||
myUser=UserName
|
|
||||||
|
|
||||||
# rename default `User` to new user name
|
|
||||||
zfs rename $(df --output=source /home | tail -n +2)/User $(df --output=source /home | tail -n +2)/${myUser}
|
|
||||||
|
|
||||||
# update entry in fstab
|
|
||||||
sed -i "s|/home/User|/home/${myUser}|g" /etc/fstab
|
|
||||||
|
|
||||||
# add user
|
|
||||||
useradd --no-create-home --user-group --home-dir /home/${myUser} --comment 'My Name' ${myUser}
|
|
||||||
|
|
||||||
# delegate snapshot and destroy permissions of the home dataset to
|
|
||||||
# new user
|
|
||||||
zfs allow -u ${myUser} mount,snapshot,destroy $(df --output=source /home | tail -n +2)/${myUser}
|
|
||||||
|
|
||||||
# fix permissions
|
|
||||||
chown --recursive ${myUser}:${myUser} /home/${myUser}
|
|
||||||
chmod 700 /home/${myUser}
|
|
||||||
|
|
||||||
# set new password for user
|
|
||||||
passwd ${myUser}
|
|
||||||
|
|
||||||
Set up cron job to snapshot user home everyday::
|
|
||||||
|
|
||||||
pacman -S cronie
|
|
||||||
systemctl enable --now cronie
|
|
||||||
crontab -eu ${myUser}
|
|
||||||
#@daily zfs snap $(df --output=source /home/${myUser} | tail -n +2)@$(dd if=/dev/urandom of=/dev/stdout bs=1 count=100 2>/dev/null |tr -dc 'a-z0-9' | cut -c-6)
|
|
||||||
zfs list -t snapshot -S creation $(df --output=source /home/${myUser} | tail -n +2)
|
|
||||||
|
|
||||||
Install package groups::
|
|
||||||
|
|
||||||
pacman -Sg # query package groups
|
|
||||||
pacman -S 'gnome'
|
|
||||||
pacman -S 'plasma'
|
|
||||||
|
|
||||||
|
|||||||
@@ -1,211 +0,0 @@
|
|||||||
.. highlight:: sh
|
|
||||||
|
|
||||||
Recovery
|
|
||||||
======================
|
|
||||||
|
|
||||||
.. contents:: Table of Contents
|
|
||||||
:local:
|
|
||||||
|
|
||||||
GRUB Tips
|
|
||||||
-------------
|
|
||||||
|
|
||||||
Boot from GRUB rescue
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
If bootloader file is damaged, it's still possible
|
|
||||||
to boot computer with GRUB rescue image.
|
|
||||||
|
|
||||||
This section is also applicable if you are in
|
|
||||||
``grub rescue>``.
|
|
||||||
|
|
||||||
#. On another computer, generate rescue image with::
|
|
||||||
|
|
||||||
pacman -S --needed mtools libisoburn grub
|
|
||||||
grub-install
|
|
||||||
grub-mkrescue -o grub-rescue.img
|
|
||||||
dd if=grub-rescue.img of=/dev/your-usb-stick
|
|
||||||
|
|
||||||
Boot computer from the rescue media.
|
|
||||||
Both legacy and EFI mode are supported.
|
|
||||||
|
|
||||||
Or `download generated GRUB rescue image <https://gitlab.com/m_zhou/bieaz/uploads/e0847a8675cda4317ea7f48abb1d9f10/grub-rescue-2.06.img.7z>`__.
|
|
||||||
|
|
||||||
#. List available disks with ``ls`` command::
|
|
||||||
|
|
||||||
grub> ls (hd # press tab
|
|
||||||
Possible devices are:
|
|
||||||
|
|
||||||
hd0 hd1 hd2 hd3
|
|
||||||
|
|
||||||
#. List partitions by pressing tab key:
|
|
||||||
|
|
||||||
.. code-block:: text
|
|
||||||
|
|
||||||
grub> ls (hd0 # press tab
|
|
||||||
Possible partitions are:
|
|
||||||
|
|
||||||
Device hd0: No known filesystem detected - Sector size 512B - Total size 20971520KiB
|
|
||||||
Partition hd0,gpt1: Filesystem type fat - Label `EFI', UUID 0DF5-3A76 - Partition start at 1024KiB - Total size 1048576KiB
|
|
||||||
Partition hd0,gpt2: No known filesystem detected - Partition start at 1049600KiB - Total size 4194304KiB
|
|
||||||
|
|
||||||
- If boot pool is encrypted:
|
|
||||||
|
|
||||||
Unlock it with ``cryptomount``::
|
|
||||||
|
|
||||||
grub> insmod luks
|
|
||||||
grub> cryptomount hd0,gpt2
|
|
||||||
Attempting to decrypt master key...
|
|
||||||
Enter passphrase for hd0,gpt2 (af5a240e13e24483acf02600d61e0f36):
|
|
||||||
Slot 1 opened
|
|
||||||
|
|
||||||
Unlocked LUKS container is ``(crypto0)``:
|
|
||||||
|
|
||||||
.. code-block:: text
|
|
||||||
|
|
||||||
grub> ls (crypto0)
|
|
||||||
Device crypto0: Filesystem type zfs - Label `bpool_ip3tdb' - Last modification
|
|
||||||
time 2021-05-03 12:14:08 Monday, UUID f14d7bdf89fe21fb - Sector size 512B -
|
|
||||||
Total size 4192256KiB
|
|
||||||
|
|
||||||
- If boot pool is not encrypted:
|
|
||||||
|
|
||||||
.. code-block:: text
|
|
||||||
|
|
||||||
grub> ls (hd0,gpt2)
|
|
||||||
Device hd0,gpt2: Filesystem type zfs - Label `bpool_ip3tdb' - Last modification
|
|
||||||
time 2021-05-03 12:14:08 Monday, UUID f14d7bdf89fe21fb - Sector size 512B -
|
|
||||||
Total size 4192256KiB
|
|
||||||
|
|
||||||
#. List boot environments nested inside ``bpool/$INST_ID/BOOT``::
|
|
||||||
|
|
||||||
grub> ls (crypto0)/sys/BOOT
|
|
||||||
@/ default/ be0/
|
|
||||||
|
|
||||||
#. Instruct GRUB to load configuration from ``be0`` boot environment::
|
|
||||||
|
|
||||||
grub> prefix=(crypto0)/sys/BOOT/be0/@/grub
|
|
||||||
grub> configfile $prefix/grub.cfg
|
|
||||||
|
|
||||||
#. GRUB menu should now appear.
|
|
||||||
|
|
||||||
#. After entering system, `reinstall GRUB <#grub-installation>`__.
|
|
||||||
|
|
||||||
Switch GRUB prefix when disk fails
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
If you are using LUKS encrypted boot pool with multiple disks,
|
|
||||||
the primary disk failed, GRUB will fail to load configuration.
|
|
||||||
|
|
||||||
If there's still enough redundancy for the boot pool, try fix
|
|
||||||
GRUB with the following method:
|
|
||||||
|
|
||||||
#. Ensure ``Slot 1 opened`` message
|
|
||||||
is shown
|
|
||||||
|
|
||||||
.. code-block:: text
|
|
||||||
|
|
||||||
Welcome to GRUB!
|
|
||||||
|
|
||||||
error: no such cryptodisk found.
|
|
||||||
Attempting to decrypt master key...
|
|
||||||
Enter passphrase for hd0,gpt2 (c0987ea1a51049e9b3056622804de62a):
|
|
||||||
Slot 1 opened
|
|
||||||
error: disk `cryptouuid/47ed1b7eb0014bc9a70aede3d8714faf' not found.
|
|
||||||
Entering rescue mode...
|
|
||||||
grub rescue>
|
|
||||||
|
|
||||||
If ``error: access denied.`` is shown,
|
|
||||||
try re-enter password with::
|
|
||||||
|
|
||||||
grub rescue> cryptomount hd0,gpt2
|
|
||||||
|
|
||||||
#. Check prefix::
|
|
||||||
|
|
||||||
grub rescue > set
|
|
||||||
# prefix=(cryptouuid/47ed1b7eb0014bc9a70aede3d8714faf)/sys/BOOT/be0@/grub
|
|
||||||
# root=cryptouuid/47ed1b7eb0014bc9a70aede3d8714faf
|
|
||||||
|
|
||||||
#. Set correct ``prefix`` and ``root`` by replacing
|
|
||||||
``cryptouuid/UUID`` with ``crypto0``::
|
|
||||||
|
|
||||||
grub rescue> prefix=(crypto0)/sys/BOOT/default@/grub
|
|
||||||
grub rescue> root=crypto0
|
|
||||||
|
|
||||||
#. Boot GRUB::
|
|
||||||
|
|
||||||
grub rescue> insmod normal
|
|
||||||
grub rescue> normal
|
|
||||||
|
|
||||||
GRUB should then boot normally.
|
|
||||||
|
|
||||||
#. After entering system, edit ``/etc/fstab`` to promote
|
|
||||||
one backup to ``/boot/efi``.
|
|
||||||
|
|
||||||
#. Make the change to ``prefix`` and ``root``
|
|
||||||
permanent by `reinstalling GRUB <#grub-installation>`__.
|
|
||||||
|
|
||||||
Access system in chroot
|
|
||||||
-----------------------
|
|
||||||
|
|
||||||
#. Go through `preparation <1-preparation.html>`__.
|
|
||||||
|
|
||||||
#. Import and unlock root and boot pool::
|
|
||||||
|
|
||||||
zpool import -NR /mnt rpool_$INST_UUID
|
|
||||||
zpool import -NR /mnt bpool_$INST_UUID
|
|
||||||
|
|
||||||
If using password::
|
|
||||||
|
|
||||||
zfs load-key rpool_$INST_UUID/$INST_ID
|
|
||||||
|
|
||||||
If using keyfile::
|
|
||||||
|
|
||||||
zfs load-key -L file:///path/to/keyfile rpool_$INST_UUID/$INST_ID
|
|
||||||
|
|
||||||
#. Find the current boot environment::
|
|
||||||
|
|
||||||
zfs list
|
|
||||||
BE=default
|
|
||||||
|
|
||||||
#. Mount root filesystem::
|
|
||||||
|
|
||||||
zfs mount rpool_$INST_UUID/$INST_ID/ROOT/$BE
|
|
||||||
|
|
||||||
#. chroot into the system::
|
|
||||||
|
|
||||||
arch-chroot /mnt /bin/bash --login
|
|
||||||
zfs mount -a
|
|
||||||
mount -a
|
|
||||||
|
|
||||||
#. Finish rescue. See `finish installation <5-bootloader.html#finish-installation>`__.
|
|
||||||
|
|
||||||
Backup and migrate existing installation
|
|
||||||
----------------------------------------
|
|
||||||
With the help of `zfs send
|
|
||||||
<https://openzfs.github.io/openzfs-docs/man/8/zfs-send.8.html>`__
|
|
||||||
it is relatively easy to perform a system backup and migration.
|
|
||||||
|
|
||||||
#. Create a snapshot of root file system::
|
|
||||||
|
|
||||||
zfs snapshot -r rpool/arch@backup
|
|
||||||
zfs snapshot -r bpool/arch@backup
|
|
||||||
|
|
||||||
#. Save snapshot to a file or pipe to SSH::
|
|
||||||
|
|
||||||
zfs send --options rpool/arch@backup > /backup/arch-rpool
|
|
||||||
zfs send --options bpool/arch@backup > /backup/arch-bpool
|
|
||||||
|
|
||||||
#. Re-create partitions and root/boot
|
|
||||||
pool on target system.
|
|
||||||
|
|
||||||
#. Restore backup::
|
|
||||||
|
|
||||||
zfs recv rpool_new/arch < /backup/arch-rpool
|
|
||||||
zfs recv bpool_new/arch < /backup/arch-bpool
|
|
||||||
|
|
||||||
#. Chroot and reinstall bootloader.
|
|
||||||
|
|
||||||
#. Update pool name in ``/etc/fstab``, ``/boot/grub/grub.cfg``
|
|
||||||
and ``/etc/zfs/zfs-list.cache/*``.
|
|
||||||
|
|
||||||
#. Update device name, etc, in ``/etc/fstab`` and ``/etc/crypttab``.
|
|
||||||
@@ -1,102 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
|
|
||||||
# mountpoint of vfat-formatted partition
|
|
||||||
ESP_MNT=/boot/efi
|
|
||||||
|
|
||||||
# path to iso files relative to the partition
|
|
||||||
ISO_REL=/iso
|
|
||||||
|
|
||||||
# absolute path to iso files
|
|
||||||
ISO_PATH=${ESP_MNT}/${ISO_REL}
|
|
||||||
|
|
||||||
# df command needs warm up due to systemd mount-on-demand
|
|
||||||
ls $ISO_PATH 1> /dev/null
|
|
||||||
|
|
||||||
# vfat partition UUID
|
|
||||||
ESP_UUID=$(blkid -s UUID -o value $(df --output=source ${ISO_PATH} | tail -n +2))
|
|
||||||
|
|
||||||
cat <<EOF
|
|
||||||
submenu 'Boot from Live ISO' {
|
|
||||||
#if tpm module is loaded, grub might fail to setup loop
|
|
||||||
#error message: out of memory
|
|
||||||
#rmmod tpm
|
|
||||||
EOF
|
|
||||||
|
|
||||||
# limit detected number of ISOs, too many
|
|
||||||
# lines might crush grub
|
|
||||||
ISO_NUM=0
|
|
||||||
|
|
||||||
# for archlinux
|
|
||||||
for isofile in $ISO_PATH/archlinux-*.iso; do
|
|
||||||
if [ "$ISO_NUM" -gt 300 ]; then break; fi
|
|
||||||
isoname=${isofile##*/}
|
|
||||||
cat <<EOF
|
|
||||||
submenu "$isoname" {
|
|
||||||
insmod search_fs_uuid
|
|
||||||
set isorootuuid=$ESP_UUID
|
|
||||||
search --fs-uuid --no-floppy --set=isopart \$isorootuuid
|
|
||||||
set isopath=$ISO_REL
|
|
||||||
loopback loop0 (\$isopart)\$isopath/$isoname
|
|
||||||
set root=(loop0)
|
|
||||||
menuentry "Arch Linux install medium" {
|
|
||||||
linux /arch/boot/x86_64/vmlinuz-linux \\
|
|
||||||
earlymodules=loop img_dev=/dev/disk/by-uuid/\$isorootuuid img_loop=\$isopath/$isoname
|
|
||||||
initrd /arch/boot/intel-ucode.img
|
|
||||||
initrd /arch/boot/amd-ucode.img
|
|
||||||
initrd /arch/boot/x86_64/initramfs-linux.img
|
|
||||||
}
|
|
||||||
menuentry "Arch Linux install medium, Copy to RAM" {
|
|
||||||
linux /arch/boot/x86_64/vmlinuz-linux \\
|
|
||||||
earlymodules=loop img_dev=/dev/disk/by-uuid/\$isorootuuid img_loop=\$isopath/$isoname \\
|
|
||||||
copytoram
|
|
||||||
initrd /arch/boot/intel-ucode.img
|
|
||||||
initrd /arch/boot/amd-ucode.img
|
|
||||||
initrd /arch/boot/x86_64/initramfs-linux.img
|
|
||||||
}
|
|
||||||
menuentry "Arch Linux install medium with speech" {
|
|
||||||
linux /arch/boot/x86_64/vmlinuz-linux \\
|
|
||||||
earlymodules=loop img_dev=/dev/disk/by-uuid/\$isorootuuid img_loop=\$isopath/$isoname \\
|
|
||||||
accessibility=on
|
|
||||||
initrd /arch/boot/intel-ucode.img
|
|
||||||
initrd /arch/boot/amd-ucode.img
|
|
||||||
initrd /arch/boot/x86_64/initramfs-linux.img
|
|
||||||
}
|
|
||||||
}
|
|
||||||
EOF
|
|
||||||
ISO_NUM=$(( $ISO_NUM + 1 ))
|
|
||||||
done
|
|
||||||
|
|
||||||
# for ubuntu
|
|
||||||
for isofile in $ISO_PATH/ubuntu-*.iso; do
|
|
||||||
if [ "$ISO_NUM" -gt 300 ]; then break; fi
|
|
||||||
isoname=${isofile##*/}
|
|
||||||
cat <<EOF
|
|
||||||
submenu "$isoname" {
|
|
||||||
insmod search_fs_uuid
|
|
||||||
set isorootuuid=$ESP_UUID
|
|
||||||
search --fs-uuid --no-floppy --set=isopart \$isorootuuid
|
|
||||||
set isopath=$ISO_REL
|
|
||||||
loopback loop0 (\$isopart)\$isopath/$isoname
|
|
||||||
set root=(loop0)
|
|
||||||
menuentry "Ubuntu" {
|
|
||||||
linux /casper/vmlinuz \\
|
|
||||||
boot=casper iso-scan/filename=\$isopath/$isoname
|
|
||||||
initrd /casper/initrd
|
|
||||||
}
|
|
||||||
menuentry "Ubuntu, Copy to RAM" {
|
|
||||||
linux /casper/vmlinuz \\
|
|
||||||
boot=casper iso-scan/filename=\$isopath/$isoname \\
|
|
||||||
toram
|
|
||||||
initrd /casper/initrd
|
|
||||||
}
|
|
||||||
}
|
|
||||||
EOF
|
|
||||||
ISO_NUM=$(( $ISO_NUM + 1 ))
|
|
||||||
done
|
|
||||||
|
|
||||||
cat <<EOF
|
|
||||||
}
|
|
||||||
EOF
|
|
||||||
|
|
||||||
# archiso kernel cmdline from
|
|
||||||
# https://gitlab.archlinux.org/archlinux/archiso/-/tree/master/configs/releng/efiboot/loader/entries
|
|
||||||
@@ -1,16 +0,0 @@
|
|||||||
[Unit]
|
|
||||||
Description=Import encrypted boot pool
|
|
||||||
Documentation=man:zpool(8)
|
|
||||||
DefaultDependencies=no
|
|
||||||
Requires=systemd-udev-settle.service
|
|
||||||
After=cryptsetup.target
|
|
||||||
Before=boot.mount
|
|
||||||
ConditionPathIsDirectory=/sys/module/zfs
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
Type=oneshot
|
|
||||||
RemainAfterExit=yes
|
|
||||||
ExecStart=/usr/bin/zpool import -aNd /dev/mapper
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=zfs-import.target
|
|
||||||
@@ -52,7 +52,7 @@ Root on ZFS
|
|||||||
ZFS can be used as root file system for Arch Linux.
|
ZFS can be used as root file system for Arch Linux.
|
||||||
An installation guide is available.
|
An installation guide is available.
|
||||||
|
|
||||||
`Start here <Root%20on%20ZFS/0-overview.html>`__.
|
`Start here <Root%20on%20ZFS/1-preparation.html>`__.
|
||||||
|
|
||||||
.. toctree::
|
.. toctree::
|
||||||
:maxdepth: 1
|
:maxdepth: 1
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
Fedora Root on ZFS
|
Fedora Root on ZFS
|
||||||
======================
|
======================
|
||||||
`Start here <Root%20on%20ZFS/0-overview.html>`__.
|
`Start here <Root%20on%20ZFS/1-preparation.html>`__.
|
||||||
|
|
||||||
Contents
|
Contents
|
||||||
--------
|
--------
|
||||||
|
|||||||
@@ -1,141 +0,0 @@
|
|||||||
.. highlight:: sh
|
|
||||||
|
|
||||||
Overview
|
|
||||||
======================
|
|
||||||
This document describes how to install Fedora with ZFS as root
|
|
||||||
file system.
|
|
||||||
|
|
||||||
Caution
|
|
||||||
~~~~~~~
|
|
||||||
- With less than 4GB RAM, DKMS might fail to build
|
|
||||||
in live environment.
|
|
||||||
- This guide wipes entire physical disks. Back up existing data.
|
|
||||||
- `GRUB does not and
|
|
||||||
will not work on 4Kn drive with legacy (BIOS) booting.
|
|
||||||
<http://savannah.gnu.org/bugs/?46700>`__
|
|
||||||
|
|
||||||
Partition layout
|
|
||||||
~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
GUID partition table (GPT) is used.
|
|
||||||
EFI system partition will be referred to as **ESP** in this document.
|
|
||||||
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
| Name | legacy boot | ESP | Boot pool | swap | root pool | remaining space |
|
|
||||||
+======================+======================+=======================+======================+=====================+=======================+=================+
|
|
||||||
| File system | | vfat | ZFS | swap | ZFS | |
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
| Size | 1M | 2G | 4G | depends on RAM size | | |
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
| Optional encryption | | *Secure Boot* | | plain dm-crypt | ZFS native encryption | |
|
|
||||||
| | | | | | | |
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
| Partition no. | 5 | 1 | 2 | 4 | 3 | |
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
| Mount point | | /boot/efi | /boot | | / | |
|
|
||||||
| | | /boot/efis/disk-part1 | | | | |
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
|
|
||||||
Dataset layout
|
|
||||||
~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| Dataset | canmount | mountpoint | container | notes |
|
|
||||||
+===========================+======================+======================+=====================================+===========================================+
|
|
||||||
| bpool | off | /boot | contains sys | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool | off | / | contains sys | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| bpool/sys | off | none | contains BOOT | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys | off | none | contains ROOT | sys is encryptionroot |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| bpool/sys/BOOT | off | none | contains boot environments | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/ROOT | off | none | contains boot environments | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/DATA | off | none | contains placeholder "default" | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/DATA/default | off | / | contains user datasets | child datsets inherits mountpoint |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/DATA/default/ | on | /home (inherited) | no | |
|
|
||||||
| home | | | | user datasets, also called "shared |
|
|
||||||
| | | | | datasets", "persistent datasets"; also |
|
|
||||||
| | | | | include /var/lib, /srv, ... |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| bpool/sys/BOOT/default | noauto | /boot | no | noauto is used to switch BE. because of |
|
|
||||||
| | | | | noauto, must use fstab to mount |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/ROOT/default | noauto | / | no | mounted by initrd zfs hook |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| bpool/sys/BOOT/be1 | noauto | /boot | no | see bpool/sys/BOOT/default |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/ROOT/be1 | noauto | / | no | see rpool/sys/ROOT/default |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
|
|
||||||
Encryption
|
|
||||||
~~~~~~~~~~
|
|
||||||
|
|
||||||
- Swap
|
|
||||||
|
|
||||||
Swap is always encrypted. By default, swap is encrypted
|
|
||||||
with plain dm-crypt with key generated from ``/dev/urandom``
|
|
||||||
at every boot. Swap content does not persist between reboots.
|
|
||||||
|
|
||||||
- Root pool
|
|
||||||
|
|
||||||
ZFS native encryption can be optionally enabled for ``rpool/sys``
|
|
||||||
and child datasets.
|
|
||||||
|
|
||||||
User should be aware that, ZFS native encryption does not
|
|
||||||
encrypt some metadata of the datasets.
|
|
||||||
ZFS native encryption also does not change master key when ``zfs change-key`` is invoked.
|
|
||||||
Therefore, you should wipe the disk when password is compromised to protect confidentiality.
|
|
||||||
See `zfs-load-key.8 <https://openzfs.github.io/openzfs-docs/man/8/zfs-load-key.8.html>`__
|
|
||||||
and `zfs-change-key.8 <https://openzfs.github.io/openzfs-docs/man/8/zfs-change-key.8.html>`__
|
|
||||||
for more information regarding ZFS native encryption.
|
|
||||||
|
|
||||||
Encryption is enabled at dataset creation and can not be disabled later.
|
|
||||||
|
|
||||||
- Boot pool
|
|
||||||
|
|
||||||
Boot pool can not be encrypted.
|
|
||||||
|
|
||||||
- Bootloader
|
|
||||||
|
|
||||||
Bootloader can not be encrypted.
|
|
||||||
|
|
||||||
However, with Secure Boot, bootloader
|
|
||||||
can be verified by motherboard firmware to be untempered,
|
|
||||||
which should be sufficient for most purposes.
|
|
||||||
|
|
||||||
Secure Boot is not supported out-of-the-box due to ZFS module.
|
|
||||||
|
|
||||||
Booting with disk failure
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
This guide is written with disk failure in mind.
|
|
||||||
|
|
||||||
If disks used in Root on ZFS pool failed, but
|
|
||||||
sufficient redundancy for both root pool and boot pool
|
|
||||||
still exists, the system will still boot normally.
|
|
||||||
|
|
||||||
Swap partition on the failed disk will fail to mount,
|
|
||||||
after an 1m30s timeout.
|
|
||||||
|
|
||||||
This feature is useful for use cases such
|
|
||||||
as an unattended remote server.
|
|
||||||
|
|
||||||
Example:
|
|
||||||
|
|
||||||
- System has disks ``n>1``
|
|
||||||
|
|
||||||
- Installed with mirrored setup
|
|
||||||
|
|
||||||
- Mirrored setup can tolerate up to ``n-1`` disk failures
|
|
||||||
|
|
||||||
- Disconnect one or more disks, keep at least
|
|
||||||
one disk connected
|
|
||||||
|
|
||||||
- System still boots, but fails to mount swap and
|
|
||||||
EFI partition
|
|
||||||
@@ -7,75 +7,26 @@ Preparation
|
|||||||
:local:
|
:local:
|
||||||
|
|
||||||
#. Disable Secure Boot. ZFS modules can not be loaded if Secure Boot is enabled.
|
#. Disable Secure Boot. ZFS modules can not be loaded if Secure Boot is enabled.
|
||||||
#. Download a variant of Fedora 36 live image
|
#. Download a variant of `Fedora live image
|
||||||
and boot from it.
|
<https://download.fedoraproject.org/pub/fedora/linux/releases/>`__ and boot from it.
|
||||||
|
#. Connect to the Internet.
|
||||||
- `Fedora Workstation (GNOME) <https://download.fedoraproject.org/pub/fedora/linux/releases/36/Workstation/x86_64/iso/>`__
|
|
||||||
- `Fedora Spins (Xfce, i3, ...) <https://download.fedoraproject.org/pub/fedora/linux/releases/36/Spins/x86_64/iso/>`__
|
|
||||||
|
|
||||||
#. Set root password or ``/root/.ssh/authorized_keys``.
|
#. Set root password or ``/root/.ssh/authorized_keys``.
|
||||||
#. Start SSH server::
|
#. Start SSH server::
|
||||||
|
|
||||||
echo PermitRootLogin yes >> /etc/ssh/sshd_config
|
echo PermitRootLogin yes >> /etc/ssh/sshd_config
|
||||||
systemctl start sshd
|
systemctl restart sshd
|
||||||
|
|
||||||
#. Connect from another computer::
|
#. Connect from another computer::
|
||||||
|
|
||||||
ssh root@192.168.1.19
|
ssh root@192.168.1.19
|
||||||
|
|
||||||
#. Temporarily set SELinux to permissive in live environment::
|
|
||||||
|
|
||||||
setenforce 0
|
|
||||||
|
|
||||||
SELinux will be enabled on the installed system.
|
|
||||||
|
|
||||||
#. Install ``kernel-devel``::
|
|
||||||
|
|
||||||
source /etc/os-release
|
|
||||||
dnf install -y https://dl.fedoraproject.org/pub/fedora/linux/releases/${VERSION_ID}/Everything/x86_64/os/Packages/k/kernel-devel-$(uname -r).rpm
|
|
||||||
|
|
||||||
#. Add ZFS repo::
|
|
||||||
|
|
||||||
dnf install -y https://zfsonlinux.org/fedora/zfs-release.fc${VERSION_ID}.noarch.rpm
|
|
||||||
|
|
||||||
#. If zfs-fuse from official Fedora repo is installed, remove it first. It is not maintained and should not be used under any circumstance::
|
|
||||||
|
|
||||||
rpm -e --nodeps zfs-fuse
|
|
||||||
|
|
||||||
#. Install ZFS packages::
|
|
||||||
|
|
||||||
dnf install -y zfs
|
|
||||||
|
|
||||||
#. Load kernel modules::
|
|
||||||
|
|
||||||
modprobe zfs
|
|
||||||
|
|
||||||
#. Install helper script and partition tool::
|
|
||||||
|
|
||||||
dnf install -y arch-install-scripts gdisk dosfstools
|
|
||||||
|
|
||||||
#. Target Fedora version::
|
|
||||||
|
|
||||||
INST_FEDORA_VER=${VERSION_ID}
|
|
||||||
|
|
||||||
#. Unique pool suffix. ZFS expects pool names to be
|
|
||||||
unique, therefore it's recommended to create
|
|
||||||
pools with a unique suffix::
|
|
||||||
|
|
||||||
INST_UUID=$(dd if=/dev/urandom bs=1 count=100 2>/dev/null | tr -dc 'a-z0-9' | cut -c-6)
|
|
||||||
|
|
||||||
#. Identify this installation in ZFS filesystem path::
|
|
||||||
|
|
||||||
INST_ID=fedora
|
|
||||||
|
|
||||||
#. Target disk
|
#. Target disk
|
||||||
|
|
||||||
List available disks with::
|
List available disks with::
|
||||||
|
|
||||||
ls /dev/disk/by-id/*
|
ls /dev/disk/by-id/*
|
||||||
|
|
||||||
If using virtio as disk bus, use
|
If using virtio as disk bus, use ``/dev/disk/by-path/*``.
|
||||||
``/dev/disk/by-path/*``.
|
|
||||||
|
|
||||||
Declare disk array::
|
Declare disk array::
|
||||||
|
|
||||||
@@ -85,57 +36,8 @@ Preparation
|
|||||||
|
|
||||||
DISK='/dev/disk/by-id/disk1'
|
DISK='/dev/disk/by-id/disk1'
|
||||||
|
|
||||||
#. Choose a primary disk. This disk will be used
|
|
||||||
for primary EFI partition and hibernation, default to
|
|
||||||
first disk in the array::
|
|
||||||
|
|
||||||
INST_PRIMARY_DISK=$(echo $DISK | cut -f1 -d\ )
|
|
||||||
|
|
||||||
#. Set vdev topology, possible values are:
|
|
||||||
|
|
||||||
- (not set, single disk or striped; no redundancy)
|
|
||||||
- mirror
|
|
||||||
- raidz1
|
|
||||||
- raidz2
|
|
||||||
- raidz3
|
|
||||||
|
|
||||||
::
|
|
||||||
|
|
||||||
INST_VDEV=
|
|
||||||
|
|
||||||
This will create a single vdev with the topology of your choice.
|
|
||||||
It is also possible to manually create a pool with multiple vdevs, such as::
|
|
||||||
|
|
||||||
zpool create --options \
|
|
||||||
poolName \
|
|
||||||
mirror sda sdb \
|
|
||||||
raidz2 sdc ... \
|
|
||||||
raidz3 sde ... \
|
|
||||||
spare sdf ...
|
|
||||||
|
|
||||||
Notice the cost of parity when using RAID-Z. See
|
|
||||||
`here <https://www.delphix.com/blog/delphix-engineering/zfs-raidz-stripe-width-or-how-i-learned-stop-worrying-and-love-raidz>`__
|
|
||||||
and `here <https://docs.google.com/spreadsheets/d/1tf4qx1aMJp8Lo_R6gpT689wTjHv6CGVElrPqTA0w_ZY/>`__.
|
|
||||||
|
|
||||||
For boot pool, which must be readable by GRUB, mirrored vdev should always be used for maximum redundancy.
|
|
||||||
This guide will use mirrored bpool for multi-disk setup.
|
|
||||||
|
|
||||||
Refer to `zpoolconcepts <https://openzfs.github.io/openzfs-docs/man/7/zpoolconcepts.7.html>`__
|
|
||||||
and `zpool-create <https://openzfs.github.io/openzfs-docs/man/8/zpool-create.8.html>`__
|
|
||||||
man pages for details.
|
|
||||||
|
|
||||||
#. Set partition size:
|
#. Set partition size:
|
||||||
|
|
||||||
Set ESP size::
|
|
||||||
|
|
||||||
INST_PARTSIZE_ESP=2 # in GB
|
|
||||||
|
|
||||||
Set boot pool size. To avoid running out of space while using
|
|
||||||
boot environments, the minimum is 4GB. Adjust the size if you
|
|
||||||
intend to use multiple kernel/distros::
|
|
||||||
|
|
||||||
INST_PARTSIZE_BPOOL=4
|
|
||||||
|
|
||||||
Set swap size. It's `recommended <https://chrisdown.name/2018/01/02/in-defence-of-swap.html>`__
|
Set swap size. It's `recommended <https://chrisdown.name/2018/01/02/in-defence-of-swap.html>`__
|
||||||
to setup a swap partition. If you intend to use hibernation,
|
to setup a swap partition. If you intend to use hibernation,
|
||||||
the minimum should be no less than RAM size. Skip if swap is not needed::
|
the minimum should be no less than RAM size. Skip if swap is not needed::
|
||||||
@@ -145,3 +47,31 @@ Preparation
|
|||||||
Root pool size, use all remaining disk space if not set::
|
Root pool size, use all remaining disk space if not set::
|
||||||
|
|
||||||
INST_PARTSIZE_RPOOL=
|
INST_PARTSIZE_RPOOL=
|
||||||
|
|
||||||
|
#. Temporarily set SELinux to permissive in live environment::
|
||||||
|
|
||||||
|
setenforce 0
|
||||||
|
|
||||||
|
SELinux will be enabled on the installed system.
|
||||||
|
|
||||||
|
#. Add ZFS repo::
|
||||||
|
|
||||||
|
dnf install -y https://zfsonlinux.org/fedora/zfs-release-fedora-2-1.noarch.rpm
|
||||||
|
|
||||||
|
#. Check available repos::
|
||||||
|
|
||||||
|
dnf repolist --all
|
||||||
|
|
||||||
|
#. Install ZFS packages::
|
||||||
|
|
||||||
|
rpm -e --nodeps zfs-fuse
|
||||||
|
dnf install -y https://dl.fedoraproject.org/pub/fedora/linux/releases/$(source /etc/os-release; echo $VERSION_ID)/Everything/x86_64/os/Packages/k/kernel-devel-$(uname -r).rpm
|
||||||
|
dnf install -y zfs
|
||||||
|
|
||||||
|
#. Load kernel modules::
|
||||||
|
|
||||||
|
modprobe zfs
|
||||||
|
|
||||||
|
#. Install partition tool::
|
||||||
|
|
||||||
|
dnf install -y gdisk dosfstools
|
||||||
|
|||||||
@@ -6,48 +6,29 @@ System Installation
|
|||||||
.. contents:: Table of Contents
|
.. contents:: Table of Contents
|
||||||
:local:
|
:local:
|
||||||
|
|
||||||
#. Optional: wipe solid-state drives with the generic tool
|
#. Partition the disks::
|
||||||
`blkdiscard <https://utcc.utoronto.ca/~cks/space/blog/linux/ErasingSSDsWithBlkdiscard>`__,
|
|
||||||
to clean previous partition tables and improve performance.
|
|
||||||
|
|
||||||
All content will be irrevocably destroyed::
|
|
||||||
|
|
||||||
for i in ${DISK}; do
|
|
||||||
blkdiscard -f $i &
|
|
||||||
done
|
|
||||||
wait
|
|
||||||
|
|
||||||
This is a quick operation and should be completed under one
|
|
||||||
minute.
|
|
||||||
|
|
||||||
For other device specific methods, see
|
|
||||||
`Memory cell clearing <https://wiki.archlinux.org/title/Solid_state_drive/Memory_cell_clearing>`__
|
|
||||||
|
|
||||||
#. Partition the disks.
|
|
||||||
See `Overview <0-overview.html>`__ for details::
|
|
||||||
|
|
||||||
for i in ${DISK}; do
|
for i in ${DISK}; do
|
||||||
|
|
||||||
sgdisk --zap-all $i
|
sgdisk --zap-all $i
|
||||||
sgdisk -n1:1M:+${INST_PARTSIZE_ESP}G -t1:EF00 $i
|
|
||||||
sgdisk -n2:0:+${INST_PARTSIZE_BPOOL}G -t2:BE00 $i
|
sgdisk -n1:1M:+1G -t1:EF00 $i
|
||||||
if [ "${INST_PARTSIZE_SWAP}" != "" ]; then
|
|
||||||
sgdisk -n4:0:+${INST_PARTSIZE_SWAP}G -t4:8200 $i
|
sgdisk -n2:0:+4G -t2:BE00 $i
|
||||||
fi
|
|
||||||
if [ "${INST_PARTSIZE_RPOOL}" = "" ]; then
|
test -z $INST_PARTSIZE_SWAP || sgdisk -n4:0:+${INST_PARTSIZE_SWAP}G -t4:8200 $i
|
||||||
|
|
||||||
|
if test -z $INST_PARTSIZE_RPOOL; then
|
||||||
sgdisk -n3:0:0 -t3:BF00 $i
|
sgdisk -n3:0:0 -t3:BF00 $i
|
||||||
else
|
else
|
||||||
sgdisk -n3:0:+${INST_PARTSIZE_RPOOL}G -t3:BF00 $i
|
sgdisk -n3:0:+${INST_PARTSIZE_RPOOL}G -t3:BF00 $i
|
||||||
fi
|
fi
|
||||||
|
|
||||||
sgdisk -a1 -n5:24K:+1000K -t5:EF02 $i
|
sgdisk -a1 -n5:24K:+1000K -t5:EF02 $i
|
||||||
done
|
done
|
||||||
|
|
||||||
#. Create boot pool::
|
#. Create boot pool::
|
||||||
|
|
||||||
|
|
||||||
disk_num=0; for i in $DISK; do disk_num=$(( $disk_num + 1 )); done
|
|
||||||
if [ $disk_num -gt 1 ]; then INST_VDEV_BPOOL=mirror; fi
|
|
||||||
|
|
||||||
|
|
||||||
zpool create \
|
zpool create \
|
||||||
-o compatibility=grub2 \
|
-o compatibility=grub2 \
|
||||||
-o ashift=12 \
|
-o ashift=12 \
|
||||||
@@ -61,12 +42,14 @@ System Installation
|
|||||||
-O xattr=sa \
|
-O xattr=sa \
|
||||||
-O mountpoint=/boot \
|
-O mountpoint=/boot \
|
||||||
-R /mnt \
|
-R /mnt \
|
||||||
bpool_$INST_UUID \
|
bpool \
|
||||||
$INST_VDEV_BPOOL \
|
mirror \
|
||||||
$(for i in ${DISK}; do
|
$(for i in ${DISK}; do
|
||||||
printf "$i-part2 ";
|
printf "$i-part2 ";
|
||||||
done)
|
done)
|
||||||
|
|
||||||
|
If not using a multi-disk setup, remove ``mirror``.
|
||||||
|
|
||||||
You should not need to customize any of the options for the boot pool.
|
You should not need to customize any of the options for the boot pool.
|
||||||
|
|
||||||
GRUB does not support all of the zpool features. See ``spa_feature_names``
|
GRUB does not support all of the zpool features. See ``spa_feature_names``
|
||||||
@@ -93,67 +76,15 @@ System Installation
|
|||||||
-O relatime=on \
|
-O relatime=on \
|
||||||
-O xattr=sa \
|
-O xattr=sa \
|
||||||
-O mountpoint=/ \
|
-O mountpoint=/ \
|
||||||
rpool_$INST_UUID \
|
rpool \
|
||||||
$INST_VDEV \
|
mirror \
|
||||||
$(for i in ${DISK}; do
|
$(for i in ${DISK}; do
|
||||||
printf "$i-part3 ";
|
printf "$i-part3 ";
|
||||||
done)
|
done)
|
||||||
|
|
||||||
**Notes:**
|
If not using a multi-disk setup, remove ``mirror``.
|
||||||
|
|
||||||
- The use of ``ashift=12`` is recommended here because many drives
|
#. This section implements dataset layout as described in `overview <1-preparation.html>`__.
|
||||||
today have 4 KiB (or larger) physical sectors, even though they
|
|
||||||
present 512 B logical sectors. Also, a future replacement drive may
|
|
||||||
have 4 KiB physical sectors (in which case ``ashift=12`` is desirable)
|
|
||||||
or 4 KiB logical sectors (in which case ``ashift=12`` is required).
|
|
||||||
- Setting ``-O acltype=posixacl`` enables POSIX ACLs globally. If you
|
|
||||||
do not want this, remove that option, but later add
|
|
||||||
``-o acltype=posixacl`` (note: lowercase “o”) to the ``zfs create``
|
|
||||||
for ``/var/log``, as `journald requires ACLs
|
|
||||||
<https://askubuntu.com/questions/970886/journalctl-says-failed-to-search-journal-acl-operation-not-supported>`__
|
|
||||||
- Setting ``normalization=formD`` eliminates some corner cases relating
|
|
||||||
to UTF-8 filename normalization. It also implies ``utf8only=on``,
|
|
||||||
which means that only UTF-8 filenames are allowed. If you care to
|
|
||||||
support non-UTF-8 filenames, do not use this option. For a discussion
|
|
||||||
of why requiring UTF-8 filenames may be a bad idea, see `The problems
|
|
||||||
with enforced UTF-8 only filenames
|
|
||||||
<http://utcc.utoronto.ca/~cks/space/blog/linux/ForcedUTF8Filenames>`__.
|
|
||||||
- ``recordsize`` is unset (leaving it at the default of 128 KiB). If you
|
|
||||||
want to tune it (e.g. ``-o recordsize=1M``), see `these
|
|
||||||
<https://jrs-s.net/2019/04/03/on-zfs-recordsize/>`__ `various
|
|
||||||
<http://blog.programster.org/zfs-record-size>`__ `blog
|
|
||||||
<https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSFileRecordsizeGrowth>`__
|
|
||||||
`posts
|
|
||||||
<https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSRecordsizeAndCompression>`__.
|
|
||||||
- Setting ``relatime=on`` is a middle ground between classic POSIX
|
|
||||||
``atime`` behavior (with its significant performance impact) and
|
|
||||||
``atime=off`` (which provides the best performance by completely
|
|
||||||
disabling atime updates). Since Linux 2.6.30, ``relatime`` has been
|
|
||||||
the default for other filesystems. See `RedHat’s documentation
|
|
||||||
<https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/power_management_guide/relatime>`__
|
|
||||||
for further information.
|
|
||||||
- Setting ``xattr=sa`` `vastly improves the performance of extended
|
|
||||||
attributes
|
|
||||||
<https://github.com/zfsonlinux/zfs/commit/82a37189aac955c81a59a5ecc3400475adb56355>`__.
|
|
||||||
Inside ZFS, extended attributes are used to implement POSIX ACLs.
|
|
||||||
Extended attributes can also be used by user-space applications.
|
|
||||||
`They are used by some desktop GUI applications.
|
|
||||||
<https://en.wikipedia.org/wiki/Extended_file_attributes#Linux>`__
|
|
||||||
`They can be used by Samba to store Windows ACLs and DOS attributes;
|
|
||||||
they are required for a Samba Active Directory domain controller.
|
|
||||||
<https://wiki.samba.org/index.php/Setting_up_a_Share_Using_Windows_ACLs>`__
|
|
||||||
Note that ``xattr=sa`` is `Linux-specific
|
|
||||||
<https://openzfs.org/wiki/Platform_code_differences>`__. If you move your
|
|
||||||
``xattr=sa`` pool to another OpenZFS implementation besides ZFS-on-Linux,
|
|
||||||
extended attributes will not be readable (though your data will be). If
|
|
||||||
portability of extended attributes is important to you, omit the
|
|
||||||
``-O xattr=sa`` above. Even if you do not want ``xattr=sa`` for the whole
|
|
||||||
pool, it is probably fine to use it for ``/var/log``.
|
|
||||||
- Make sure to include the ``-part3`` portion of the drive path. If you
|
|
||||||
forget that, you are specifying the whole disk, which ZFS will then
|
|
||||||
re-partition, and you will lose the bootloader partition(s).
|
|
||||||
|
|
||||||
#. This section implements dataset layout as described in `overview <0-overview.html>`__.
|
|
||||||
|
|
||||||
Create root system container:
|
Create root system container:
|
||||||
|
|
||||||
@@ -162,7 +93,7 @@ System Installation
|
|||||||
zfs create \
|
zfs create \
|
||||||
-o canmount=off \
|
-o canmount=off \
|
||||||
-o mountpoint=none \
|
-o mountpoint=none \
|
||||||
rpool_$INST_UUID/$INST_ID
|
rpool/redhat
|
||||||
|
|
||||||
- Encrypted:
|
- Encrypted:
|
||||||
|
|
||||||
@@ -175,28 +106,19 @@ System Installation
|
|||||||
-o encryption=on \
|
-o encryption=on \
|
||||||
-o keylocation=prompt \
|
-o keylocation=prompt \
|
||||||
-o keyformat=passphrase \
|
-o keyformat=passphrase \
|
||||||
rpool_$INST_UUID/$INST_ID
|
rpool/redhat
|
||||||
|
|
||||||
Create other system datasets::
|
Create system datasets::
|
||||||
|
|
||||||
zfs create -o canmount=off -o mountpoint=none bpool_$INST_UUID/$INST_ID
|
zfs create -o canmount=on -o mountpoint=/ rpool/redhat/root
|
||||||
zfs create -o canmount=off -o mountpoint=none bpool_$INST_UUID/$INST_ID/BOOT
|
zfs create -o canmount=on -o mountpoint=/home rpool/redhat/home
|
||||||
zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID/ROOT
|
zfs create -o canmount=off -o mountpoint=/var rpool/redhat/var
|
||||||
zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID/DATA
|
zfs create -o canmount=on rpool/redhat/var/lib
|
||||||
zfs create -o mountpoint=/boot -o canmount=noauto bpool_$INST_UUID/$INST_ID/BOOT/default
|
zfs create -o canmount=on rpool/redhat/var/log
|
||||||
zfs create -o mountpoint=/ -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/default
|
|
||||||
zfs create -o mountpoint=/ -o canmount=noauto rpool_$INST_UUID/$INST_ID/ROOT/default
|
Create boot dataset::
|
||||||
zfs mount rpool_$INST_UUID/$INST_ID/ROOT/default
|
|
||||||
zfs mount bpool_$INST_UUID/$INST_ID/BOOT/default
|
zfs create -o canmount=on -o mountpoint=/boot bpool/redhat
|
||||||
for i in {usr,var,var/lib};
|
|
||||||
do
|
|
||||||
zfs create -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/default/$i
|
|
||||||
done
|
|
||||||
for i in {home,root,srv,usr/local,var/log,var/spool};
|
|
||||||
do
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/$i
|
|
||||||
done
|
|
||||||
chmod 750 /mnt/root
|
|
||||||
|
|
||||||
#. Format and mount ESP::
|
#. Format and mount ESP::
|
||||||
|
|
||||||
@@ -207,50 +129,15 @@ System Installation
|
|||||||
done
|
done
|
||||||
|
|
||||||
mkdir -p /mnt/boot/efi
|
mkdir -p /mnt/boot/efi
|
||||||
mount -t vfat ${INST_PRIMARY_DISK}-part1 /mnt/boot/efi
|
mount -t vfat $(echo $DISK | cut -f1 -d\ )-part1 /mnt/boot/efi
|
||||||
|
|
||||||
#. Create separate user dataset at ``/home/User``, dateset name can be
|
#. Install packages::
|
||||||
changed later::
|
|
||||||
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/home/User
|
dnf --installroot=/mnt --releasever=$(source /etc/os-release ; echo $VERSION_ID) -y install \
|
||||||
|
@core grub2-efi-x64 grub2-pc-modules grub2-efi-x64-modules shim-x64 efibootmgr kernel \
|
||||||
|
kernel-devel
|
||||||
|
|
||||||
If needed, snapshot, rollback and other related permissions can be
|
dnf --installroot=/mnt --releasever=$(source /etc/os-release ; echo $VERSION_ID) -y install \
|
||||||
delegated to the user later.
|
https://zfsonlinux.org/fedora/zfs-release-fedora-2-1.noarch.rpm
|
||||||
|
|
||||||
#. Create optional program data datasets to omit data from rollback::
|
dnf --installroot=/mnt --releasever=$(source /etc/os-release ; echo $VERSION_ID) -y install zfs zfs-dracut
|
||||||
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/games
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/www
|
|
||||||
# for GNOME
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/AccountsService
|
|
||||||
# for Docker
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/docker
|
|
||||||
# for NFS
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/nfs
|
|
||||||
# for LXC
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/lxc
|
|
||||||
# for LibVirt
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/libvirt
|
|
||||||
##other application
|
|
||||||
# zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/$name
|
|
||||||
|
|
||||||
Add other datasets when needed, such as PostgreSQL.
|
|
||||||
|
|
||||||
#. Install base packages::
|
|
||||||
|
|
||||||
dnf --installroot=/mnt --releasever=${INST_FEDORA_VER} -y install \
|
|
||||||
https://zfsonlinux.org/fedora/zfs-release.fc${INST_FEDORA_VER}.noarch.rpm \
|
|
||||||
@core grub2-efi-x64 grub2-pc-modules grub2-efi-x64-modules shim-x64 efibootmgr cryptsetup \
|
|
||||||
kernel kernel-devel python3-dnf-plugin-post-transaction-actions
|
|
||||||
|
|
||||||
#. Install ZFS::
|
|
||||||
|
|
||||||
dnf --installroot=/mnt -y install zfs zfs-dracut
|
|
||||||
|
|
||||||
#. Optional: enable boot environment support and dnf integration::
|
|
||||||
|
|
||||||
dnf --installroot=/mnt copr enable -y m0p/bieaz
|
|
||||||
dnf --installroot=/mnt install -y bieaz python3-dnf-plugin-rozb3
|
|
||||||
|
|
||||||
If multi-disk setup is used, enable multi-disk
|
|
||||||
support inside ``/mnt/etc/bieaz.cfg``.
|
|
||||||
|
|||||||
@@ -8,24 +8,13 @@ System Configuration
|
|||||||
|
|
||||||
#. Generate fstab::
|
#. Generate fstab::
|
||||||
|
|
||||||
genfstab -U /mnt | sed 's;zfs[[:space:]]*;zfs zfsutil,;g' | grep "zfs zfsutil" >> /mnt/etc/fstab
|
mkdir -p /mnt/etc/
|
||||||
for i in ${DISK}; do
|
for i in ${DISK}; do
|
||||||
echo UUID=$(blkid -s UUID -o value ${i}-part1) /boot/efis/${i##*/}-part1 vfat \
|
echo UUID=$(blkid -s UUID -o value ${i}-part1) /boot/efis/${i##*/}-part1 vfat \
|
||||||
x-systemd.idle-timeout=1min,x-systemd.automount,noauto,umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab
|
umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab
|
||||||
done
|
done
|
||||||
echo UUID=$(blkid -s UUID -o value ${INST_PRIMARY_DISK}-part1) /boot/efi vfat \
|
echo $(echo $DISK | cut -f1 -d\ )-part1 /boot/efi vfat \
|
||||||
x-systemd.idle-timeout=1min,x-systemd.automount,noauto,umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab
|
noauto,umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab
|
||||||
if [ "${INST_PARTSIZE_SWAP}" != "" ]; then
|
|
||||||
for i in ${DISK}; do
|
|
||||||
echo ${i##*/}-part4-swap ${i}-part4 /dev/urandom swap,cipher=aes-cbc-essiv:sha256,size=256,discard >> /mnt/etc/crypttab
|
|
||||||
echo /dev/mapper/${i##*/}-part4-swap none swap x-systemd.requires=cryptsetup.target,defaults 0 0 >> /mnt/etc/fstab
|
|
||||||
done
|
|
||||||
fi
|
|
||||||
|
|
||||||
By default, systemd will halt boot process if any entry in ``/etc/fstab`` fails
|
|
||||||
to mount. This is unnecessary for mirrored EFI boot partitions.
|
|
||||||
With the above mount options, systemd will skip mounting them at boot,
|
|
||||||
only mount them on demand when accessed.
|
|
||||||
|
|
||||||
#. Configure dracut::
|
#. Configure dracut::
|
||||||
|
|
||||||
@@ -37,25 +26,10 @@ System Configuration
|
|||||||
echo 'forced_drivers+=" mpt3sas "' >> /mnt/etc/dracut.conf.d/zfs.conf
|
echo 'forced_drivers+=" mpt3sas "' >> /mnt/etc/dracut.conf.d/zfs.conf
|
||||||
fi
|
fi
|
||||||
|
|
||||||
#. Enable timezone sync::
|
#. Set locale, keymap, timezone, hostname and root password::
|
||||||
|
|
||||||
hwclock --systohc
|
|
||||||
systemctl enable systemd-timesyncd --root=/mnt
|
|
||||||
|
|
||||||
#. Interactively set locale, keymap, timezone, hostname and root password::
|
|
||||||
|
|
||||||
rm -f /mnt/etc/localtime
|
rm -f /mnt/etc/localtime
|
||||||
systemd-firstboot --root=/mnt --force --prompt --root-password=PASSWORD
|
systemd-firstboot --root=/mnt --prompt --root-password=PASSWORD --force
|
||||||
|
|
||||||
This can be non-interactive, see man page for details::
|
|
||||||
|
|
||||||
rm -f /mnt/etc/localtime
|
|
||||||
systemd-firstboot --root=/mnt --force \
|
|
||||||
--locale="en_US.UTF-8" --locale-messages="en_US.UTF-8" \
|
|
||||||
--keymap=us --timezone="Europe/Berlin" --hostname=myHost \
|
|
||||||
--root-password=PASSWORD --root-shell=/bin/bash
|
|
||||||
|
|
||||||
``systemd-firstboot`` have bugs, root password is set below.
|
|
||||||
|
|
||||||
#. Generate host id::
|
#. Generate host id::
|
||||||
|
|
||||||
@@ -65,16 +39,9 @@ System Configuration
|
|||||||
|
|
||||||
dnf --installroot=/mnt install -y glibc-minimal-langpack glibc-langpack-en
|
dnf --installroot=/mnt install -y glibc-minimal-langpack glibc-langpack-en
|
||||||
|
|
||||||
Program will show errors if not installed.
|
|
||||||
|
|
||||||
#. Enable ZFS services::
|
#. Enable ZFS services::
|
||||||
|
|
||||||
systemctl enable zfs-import-scan.service zfs-import.target zfs-zed zfs.target --root=/mnt
|
systemctl enable zfs-import-scan.service zfs-mount zfs-import.target zfs-zed zfs.target --root=/mnt
|
||||||
systemctl disable zfs-mount --root=/mnt
|
|
||||||
|
|
||||||
At boot, datasets on rpool are mounted with ``/etc/fstab``,
|
|
||||||
which can control the mounting process more precisely than ``zfs-mount.service``.
|
|
||||||
|
|
||||||
|
|
||||||
#. By default SSH server is enabled, allowing root login by password,
|
#. By default SSH server is enabled, allowing root login by password,
|
||||||
disable SSH server::
|
disable SSH server::
|
||||||
@@ -84,22 +51,13 @@ System Configuration
|
|||||||
|
|
||||||
#. Chroot::
|
#. Chroot::
|
||||||
|
|
||||||
echo "INST_PRIMARY_DISK=$INST_PRIMARY_DISK
|
m='/dev /proc /sys'
|
||||||
INST_LINVAR=$INST_LINVAR
|
for i in $m; do mount --rbind $i /mnt/$i; done
|
||||||
INST_UUID=$INST_UUID
|
|
||||||
INST_ID=$INST_ID
|
|
||||||
unalias -a
|
|
||||||
INST_VDEV=$INST_VDEV
|
|
||||||
DISK=\"$DISK\"" > /mnt/root/chroot
|
|
||||||
#backup bash inputs up to this point
|
|
||||||
history -w /mnt/home/sys-install-pre-chroot.txt
|
history -w /mnt/home/sys-install-pre-chroot.txt
|
||||||
arch-chroot /mnt bash --login
|
chroot /mnt /usr/bin/env DISK=$DISK bash --login
|
||||||
|
|
||||||
#. Source variables::
|
#. For SELinux, relabel filesystem on reboot::
|
||||||
|
|
||||||
source /root/chroot
|
|
||||||
|
|
||||||
#. For SELinux, relabel filesystem on next boot::
|
|
||||||
|
|
||||||
fixfiles -F onboot
|
fixfiles -F onboot
|
||||||
|
|
||||||
|
|||||||
@@ -36,6 +36,8 @@ Workarounds have to be applied.
|
|||||||
|
|
||||||
sed -i "s|rpool=.*|rpool=\`zdb -l \${GRUB_DEVICE} \| grep -E '[[:blank:]]name' \| cut -d\\\' -f 2\`|" /etc/grub.d/10_linux
|
sed -i "s|rpool=.*|rpool=\`zdb -l \${GRUB_DEVICE} \| grep -E '[[:blank:]]name' \| cut -d\\\' -f 2\`|" /etc/grub.d/10_linux
|
||||||
|
|
||||||
|
Caution: this fix must be applied after every GRUB update and before generating the menu.
|
||||||
|
|
||||||
Install GRUB
|
Install GRUB
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
@@ -43,37 +45,26 @@ Install GRUB
|
|||||||
|
|
||||||
echo 'filesystems+=" virtio_blk "' >> /etc/dracut.conf.d/fs.conf
|
echo 'filesystems+=" virtio_blk "' >> /etc/dracut.conf.d/fs.conf
|
||||||
|
|
||||||
#. Generate initrd::
|
#. Create empty cache file and generate initrd::
|
||||||
|
|
||||||
rm -f /etc/zfs/zpool.cache
|
rm -f /etc/zfs/zpool.cache
|
||||||
touch /etc/zfs/zpool.cache
|
touch /etc/zfs/zpool.cache
|
||||||
chmod a-w /etc/zfs/zpool.cache
|
chmod a-w /etc/zfs/zpool.cache
|
||||||
chattr +i /etc/zfs/zpool.cache
|
chattr +i /etc/zfs/zpool.cache
|
||||||
|
|
||||||
for directory in /lib/modules/*; do
|
for directory in /lib/modules/*; do
|
||||||
kernel_version=$(basename $directory)
|
kernel_version=$(basename $directory)
|
||||||
dracut --force --kver $kernel_version
|
dracut --force --kver $kernel_version
|
||||||
done
|
done
|
||||||
|
|
||||||
#. Disable BLS::
|
#. Load ZFS modules and disable BLS::
|
||||||
|
|
||||||
echo "GRUB_ENABLE_BLSCFG=false" >> /etc/default/grub
|
echo 'GRUB_ENABLE_BLSCFG=false' >> /etc/default/grub
|
||||||
|
|
||||||
#. Create GRUB boot directory, in ESP and boot pool::
|
|
||||||
|
|
||||||
mkdir -p /boot/efi/EFI/fedora # EFI GRUB dir
|
|
||||||
mkdir -p /boot/efi/EFI/fedora/grub2 # legacy GRUB dir
|
|
||||||
mkdir -p /boot/grub2
|
|
||||||
|
|
||||||
Boot environment-specific configuration (kernel, etc)
|
|
||||||
is stored in ``/boot/grub2/grub.cfg``, enabling rollback.
|
|
||||||
|
|
||||||
#. When in doubt, install both legacy boot
|
|
||||||
and EFI.
|
|
||||||
|
|
||||||
#. If using legacy booting, install GRUB to every disk::
|
#. If using legacy booting, install GRUB to every disk::
|
||||||
|
|
||||||
for i in ${DISK}; do
|
for i in ${DISK}; do
|
||||||
grub2-install --boot-directory /boot/efi/EFI/fedora --target=i386-pc $i
|
grub2-install --target=i386-pc $i
|
||||||
done
|
done
|
||||||
|
|
||||||
#. If using EFI::
|
#. If using EFI::
|
||||||
@@ -82,13 +73,14 @@ Install GRUB
|
|||||||
efibootmgr -cgp 1 -l "\EFI\fedora\shimx64.efi" \
|
efibootmgr -cgp 1 -l "\EFI\fedora\shimx64.efi" \
|
||||||
-L "fedora-${i##*/}" -d ${i}
|
-L "fedora-${i##*/}" -d ${i}
|
||||||
done
|
done
|
||||||
cp -r /usr/lib/grub/x86_64-efi/ /boot/efi/EFI/fedora
|
cp -r /usr/lib/grub/x86_64-efi/ /boot/efi/EFI/fedora/
|
||||||
|
|
||||||
#. Generate GRUB Menu::
|
#. Generate GRUB Menu:
|
||||||
|
|
||||||
grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg
|
Generate menu::
|
||||||
cp /boot/efi/EFI/fedora/grub.cfg /boot/efi/EFI/fedora/grub2/grub.cfg
|
|
||||||
cp /boot/efi/EFI/fedora/grub.cfg /boot/grub2/grub.cfg
|
grub2-mkconfig -o /boot/grub2/grub.cfg
|
||||||
|
cp /boot/grub2/grub.cfg /boot/efi/EFI/fedora/
|
||||||
|
|
||||||
#. For both legacy and EFI booting: mirror ESP content::
|
#. For both legacy and EFI booting: mirror ESP content::
|
||||||
|
|
||||||
@@ -98,55 +90,14 @@ Install GRUB
|
|||||||
for i in /boot/efis/*; do
|
for i in /boot/efis/*; do
|
||||||
cp -r $ESP_MIRROR/EFI $i
|
cp -r $ESP_MIRROR/EFI $i
|
||||||
done
|
done
|
||||||
|
rm -rf $ESP_MIRROR
|
||||||
|
|
||||||
#. Automatically regenerate GRUB menu on kernel update::
|
#. Notes for GRUB on RHEL
|
||||||
|
|
||||||
tee /etc/dnf/plugins/post-transaction-actions.d/00-update-grub-menu-for-kernel.action <<EOF >/dev/null
|
|
||||||
# kernel-core package contains vmlinuz and initramfs
|
|
||||||
# change package name if non-standard kernel is used
|
|
||||||
kernel-core:in:/usr/local/sbin/update-grub-menu.sh
|
|
||||||
kernel-core:out:/usr/local/sbin/update-grub-menu.sh
|
|
||||||
EOF
|
|
||||||
|
|
||||||
tee /usr/local/sbin/update-grub-menu.sh <<-'EOF' >/dev/null
|
|
||||||
#!/bin/sh
|
|
||||||
export PATH=$PATH:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
|
|
||||||
export ZPOOL_VDEV_NAME_PATH=YES
|
|
||||||
source /etc/os-release
|
|
||||||
grub2-mkconfig -o /boot/efi/EFI/${ID}/grub.cfg
|
|
||||||
cp /boot/efi/EFI/${ID}/grub.cfg /boot/efi/EFI/${ID}/grub2/grub.cfg
|
|
||||||
cp /boot/efi/EFI/${ID}/grub.cfg /boot/grub2/grub.cfg
|
|
||||||
ESP_MIRROR=$(mktemp -d)
|
|
||||||
cp -r /boot/efi/EFI $ESP_MIRROR
|
|
||||||
for i in /boot/efis/*; do
|
|
||||||
cp -r $ESP_MIRROR/EFI $i
|
|
||||||
done
|
|
||||||
rm -rf $ESP_MIRROR
|
|
||||||
EOF
|
|
||||||
|
|
||||||
chmod +x /usr/local/sbin/update-grub-menu.sh
|
|
||||||
|
|
||||||
#. Notes for GRUB on Fedora
|
|
||||||
|
|
||||||
To support Secure Boot, GRUB has been heavily modified by Fedora,
|
|
||||||
namely:
|
|
||||||
|
|
||||||
- ``grub2-install`` is `disabled for UEFI <https://bugzilla.redhat.com/show_bug.cgi?id=1917213>`__
|
|
||||||
- Only a static, signed version of bootloader is copied to EFI system partition
|
|
||||||
- This signed bootloader does not have built-in support for either ZFS or LUKS containers
|
|
||||||
- This signed bootloader only loads configuration from ``/boot/efi/EFI/fedora/grub.cfg``
|
|
||||||
|
|
||||||
Unrelated to Secure Boot, GRUB has also been modified to provide optional
|
|
||||||
support for `systemd bootloader specification (bls) <https://systemd.io/BOOT_LOADER_SPECIFICATION/>`__.
|
|
||||||
Currently ``blscfg.mod`` is incompatible with root on ZFS.
|
|
||||||
|
|
||||||
As bls is disabled, you will need to regenerate GRUB menu after each kernel upgrade.
|
As bls is disabled, you will need to regenerate GRUB menu after each kernel upgrade.
|
||||||
Or else the new kernel will not be recognized and system will boot the old kernel
|
Or else the new kernel will not be recognized and system will boot the old kernel
|
||||||
on reboot.
|
on reboot.
|
||||||
|
|
||||||
Also see `Fedora docs for GRUB
|
|
||||||
<https://docs.fedoraproject.org/en-US/fedora/rawhide/system-administrators-guide/kernel-module-driver-configuration/Working_with_the_GRUB_2_Boot_Loader/>`__.
|
|
||||||
|
|
||||||
Finish Installation
|
Finish Installation
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
@@ -154,85 +105,25 @@ Finish Installation
|
|||||||
|
|
||||||
exit
|
exit
|
||||||
|
|
||||||
#. Take a snapshot of the clean installation for future use::
|
|
||||||
|
|
||||||
zfs snapshot -r rpool_$INST_UUID/$INST_ID@install
|
|
||||||
zfs snapshot -r bpool_$INST_UUID/$INST_ID@install
|
|
||||||
|
|
||||||
#. Unmount EFI system partition::
|
|
||||||
|
|
||||||
umount /mnt/boot/efi
|
|
||||||
umount /mnt/boot/efis/*
|
|
||||||
|
|
||||||
#. Export pools::
|
#. Export pools::
|
||||||
|
|
||||||
zpool export bpool_$INST_UUID
|
umount -Rl /mnt
|
||||||
zpool export rpool_$INST_UUID
|
zpool export -a
|
||||||
|
|
||||||
#. Reboot::
|
#. Reboot::
|
||||||
|
|
||||||
reboot
|
reboot
|
||||||
|
|
||||||
|
#. On first reboot, the boot process will fail, with failure messages such
|
||||||
|
as "You are in Emergency Mode...Press Ctrl-D to continue".
|
||||||
|
|
||||||
|
Wait for the computer to automatically reboot and the problem will be resolved.
|
||||||
|
|
||||||
Post installaion
|
Post installaion
|
||||||
~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~
|
||||||
|
#. Install package groups::
|
||||||
#. If you have other data pools, generate list of datasets for `zfs-mount-generator
|
|
||||||
<https://manpages.ubuntu.com/manpages/focal/man8/zfs-mount-generator.8.html>`__ to mount them at boot::
|
|
||||||
|
|
||||||
DATA_POOL='tank0 tank1'
|
|
||||||
|
|
||||||
# tab-separated zfs properties
|
|
||||||
# see /etc/zfs/zed.d/history_event-zfs-list-cacher.sh
|
|
||||||
export \
|
|
||||||
PROPS="name,mountpoint,canmount,atime,relatime,devices,exec\
|
|
||||||
,readonly,setuid,nbmand,encroot,keylocation"
|
|
||||||
|
|
||||||
for i in $DATA_POOL; do
|
|
||||||
zfs list -H -t filesystem -o $PROPS -r $i > /etc/zfs/zfs-list.cache/$i
|
|
||||||
done
|
|
||||||
|
|
||||||
#. After reboot, consider adding a normal user::
|
|
||||||
|
|
||||||
# with root permissions
|
|
||||||
sudo -i
|
|
||||||
|
|
||||||
# store user name in a variable
|
|
||||||
myUser=UserName
|
|
||||||
|
|
||||||
# rename default `User` to new user name
|
|
||||||
zfs rename $(df --output=source /home | tail -n +2)/User $(df --output=source /home | tail -n +2)/${myUser}
|
|
||||||
|
|
||||||
# update entry in fstab
|
|
||||||
sed -i "s|/home/User|/home/${myUser}|g" /etc/fstab
|
|
||||||
|
|
||||||
# add user
|
|
||||||
useradd --no-create-home --user-group --home-dir /home/${myUser} --comment 'My Name' ${myUser}
|
|
||||||
|
|
||||||
# delegate snapshot and destroy permissions of the home dataset to
|
|
||||||
# new user
|
|
||||||
zfs allow -u ${myUser} mount,snapshot,destroy $(df --output=source /home | tail -n +2)/${myUser}
|
|
||||||
|
|
||||||
# fix permissions
|
|
||||||
chown --recursive ${myUser}:${myUser} /home/${myUser}
|
|
||||||
chmod 700 /home/${myUser}
|
|
||||||
|
|
||||||
# fix selinux context
|
|
||||||
restorecon /home/${myUser}
|
|
||||||
|
|
||||||
# set new password for user
|
|
||||||
passwd ${myUser}
|
|
||||||
|
|
||||||
Set up cron job to snapshot user home everyday::
|
|
||||||
|
|
||||||
dnf install cronie
|
|
||||||
systemctl enable --now crond
|
|
||||||
crontab -eu ${myUser}
|
|
||||||
#@daily /usr/sbin/zfs snap $(df --output=source /home/${myUser} | tail -n +2)@$(dd if=/dev/urandom of=/dev/stdout bs=1 count=100 2>/dev/null |tr -dc 'a-z0-9' | cut -c-6)
|
|
||||||
zfs list -t snapshot -S creation $(df --output=source /home/${myUser} | tail -n +2)
|
|
||||||
|
|
||||||
Install package groups::
|
|
||||||
|
|
||||||
dnf group list --hidden -v # query package groups
|
dnf group list --hidden -v # query package groups
|
||||||
dnf group install 'i3 Desktop'
|
dnf group install @gnome-desktop
|
||||||
dnf group install 'Fedora Workstation' # GNOME
|
|
||||||
dnf group install 'Web Server'
|
#. Add new user, configure swap.
|
||||||
|
|||||||
@@ -1,211 +0,0 @@
|
|||||||
.. highlight:: sh
|
|
||||||
|
|
||||||
Recovery
|
|
||||||
======================
|
|
||||||
|
|
||||||
.. contents:: Table of Contents
|
|
||||||
:local:
|
|
||||||
|
|
||||||
GRUB Tips
|
|
||||||
-------------
|
|
||||||
|
|
||||||
Boot from GRUB rescue
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
If bootloader file is damaged, it's still possible
|
|
||||||
to boot computer with GRUB rescue image.
|
|
||||||
|
|
||||||
This section is also applicable if you are in
|
|
||||||
``grub rescue>``.
|
|
||||||
|
|
||||||
#. On another computer, generate rescue image with::
|
|
||||||
|
|
||||||
pacman -S --needed mtools libisoburn grub
|
|
||||||
grub-install
|
|
||||||
grub-mkrescue -o grub-rescue.img
|
|
||||||
dd if=grub-rescue.img of=/dev/your-usb-stick
|
|
||||||
|
|
||||||
Boot computer from the rescue media.
|
|
||||||
Both legacy and EFI mode are supported.
|
|
||||||
|
|
||||||
Or `download generated GRUB rescue image <https://gitlab.com/m_zhou/bieaz/uploads/e0847a8675cda4317ea7f48abb1d9f10/grub-rescue-2.06.img.7z>`__.
|
|
||||||
|
|
||||||
#. List available disks with ``ls`` command::
|
|
||||||
|
|
||||||
grub> ls (hd # press tab
|
|
||||||
Possible devices are:
|
|
||||||
|
|
||||||
hd0 hd1 hd2 hd3
|
|
||||||
|
|
||||||
#. List partitions by pressing tab key:
|
|
||||||
|
|
||||||
.. code-block:: text
|
|
||||||
|
|
||||||
grub> ls (hd0 # press tab
|
|
||||||
Possible partitions are:
|
|
||||||
|
|
||||||
Device hd0: No known filesystem detected - Sector size 512B - Total size 20971520KiB
|
|
||||||
Partition hd0,gpt1: Filesystem type fat - Label `EFI', UUID 0DF5-3A76 - Partition start at 1024KiB - Total size 1048576KiB
|
|
||||||
Partition hd0,gpt2: No known filesystem detected - Partition start at 1049600KiB - Total size 4194304KiB
|
|
||||||
|
|
||||||
- If boot pool is encrypted:
|
|
||||||
|
|
||||||
Unlock it with ``cryptomount``::
|
|
||||||
|
|
||||||
grub> insmod luks
|
|
||||||
grub> cryptomount hd0,gpt2
|
|
||||||
Attempting to decrypt master key...
|
|
||||||
Enter passphrase for hd0,gpt2 (af5a240e13e24483acf02600d61e0f36):
|
|
||||||
Slot 1 opened
|
|
||||||
|
|
||||||
Unlocked LUKS container is ``(crypto0)``:
|
|
||||||
|
|
||||||
.. code-block:: text
|
|
||||||
|
|
||||||
grub> ls (crypto0)
|
|
||||||
Device crypto0: Filesystem type zfs - Label `bpool_ip3tdb' - Last modification
|
|
||||||
time 2021-05-03 12:14:08 Monday, UUID f14d7bdf89fe21fb - Sector size 512B -
|
|
||||||
Total size 4192256KiB
|
|
||||||
|
|
||||||
- If boot pool is not encrypted:
|
|
||||||
|
|
||||||
.. code-block:: text
|
|
||||||
|
|
||||||
grub> ls (hd0,gpt2)
|
|
||||||
Device hd0,gpt2: Filesystem type zfs - Label `bpool_ip3tdb' - Last modification
|
|
||||||
time 2021-05-03 12:14:08 Monday, UUID f14d7bdf89fe21fb - Sector size 512B -
|
|
||||||
Total size 4192256KiB
|
|
||||||
|
|
||||||
#. List boot environments nested inside ``bpool/$INST_ID/BOOT``::
|
|
||||||
|
|
||||||
grub> ls (crypto0)/sys/BOOT
|
|
||||||
@/ default/ be0/
|
|
||||||
|
|
||||||
#. Instruct GRUB to load configuration from ``be0`` boot environment::
|
|
||||||
|
|
||||||
grub> prefix=(crypto0)/sys/BOOT/be0/@/grub
|
|
||||||
grub> configfile $prefix/grub.cfg
|
|
||||||
|
|
||||||
#. GRUB menu should now appear.
|
|
||||||
|
|
||||||
#. After entering system, `reinstall GRUB <#grub-installation>`__.
|
|
||||||
|
|
||||||
Switch GRUB prefix when disk fails
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
If you are using LUKS encrypted boot pool with multiple disks,
|
|
||||||
the primary disk failed, GRUB will fail to load configuration.
|
|
||||||
|
|
||||||
If there's still enough redundancy for the boot pool, try fix
|
|
||||||
GRUB with the following method:
|
|
||||||
|
|
||||||
#. Ensure ``Slot 1 opened`` message
|
|
||||||
is shown
|
|
||||||
|
|
||||||
.. code-block:: text
|
|
||||||
|
|
||||||
Welcome to GRUB!
|
|
||||||
|
|
||||||
error: no such cryptodisk found.
|
|
||||||
Attempting to decrypt master key...
|
|
||||||
Enter passphrase for hd0,gpt2 (c0987ea1a51049e9b3056622804de62a):
|
|
||||||
Slot 1 opened
|
|
||||||
error: disk `cryptouuid/47ed1b7eb0014bc9a70aede3d8714faf' not found.
|
|
||||||
Entering rescue mode...
|
|
||||||
grub rescue>
|
|
||||||
|
|
||||||
If ``error: access denied.`` is shown,
|
|
||||||
try re-enter password with::
|
|
||||||
|
|
||||||
grub rescue> cryptomount hd0,gpt2
|
|
||||||
|
|
||||||
#. Check prefix::
|
|
||||||
|
|
||||||
grub rescue > set
|
|
||||||
# prefix=(cryptouuid/47ed1b7eb0014bc9a70aede3d8714faf)/sys/BOOT/be0@/grub
|
|
||||||
# root=cryptouuid/47ed1b7eb0014bc9a70aede3d8714faf
|
|
||||||
|
|
||||||
#. Set correct ``prefix`` and ``root`` by replacing
|
|
||||||
``cryptouuid/UUID`` with ``crypto0``::
|
|
||||||
|
|
||||||
grub rescue> prefix=(crypto0)/sys/BOOT/default@/grub
|
|
||||||
grub rescue> root=crypto0
|
|
||||||
|
|
||||||
#. Boot GRUB::
|
|
||||||
|
|
||||||
grub rescue> insmod normal
|
|
||||||
grub rescue> normal
|
|
||||||
|
|
||||||
GRUB should then boot normally.
|
|
||||||
|
|
||||||
#. After entering system, edit ``/etc/fstab`` to promote
|
|
||||||
one backup to ``/boot/efi``.
|
|
||||||
|
|
||||||
#. Make the change to ``prefix`` and ``root``
|
|
||||||
permanent by `reinstalling GRUB <#grub-installation>`__.
|
|
||||||
|
|
||||||
Access system in chroot
|
|
||||||
-----------------------
|
|
||||||
|
|
||||||
#. Go through `preparation <1-preparation.html>`__.
|
|
||||||
|
|
||||||
#. Import and unlock root and boot pool::
|
|
||||||
|
|
||||||
zpool import -NR /mnt rpool_$INST_UUID
|
|
||||||
zpool import -NR /mnt bpool_$INST_UUID
|
|
||||||
|
|
||||||
If using password::
|
|
||||||
|
|
||||||
zfs load-key rpool_$INST_UUID/$INST_ID
|
|
||||||
|
|
||||||
If using keyfile::
|
|
||||||
|
|
||||||
zfs load-key -L file:///path/to/keyfile rpool_$INST_UUID/$INST_ID
|
|
||||||
|
|
||||||
#. Find the current boot environment::
|
|
||||||
|
|
||||||
zfs list
|
|
||||||
BE=default
|
|
||||||
|
|
||||||
#. Mount root filesystem::
|
|
||||||
|
|
||||||
zfs mount rpool_$INST_UUID/$INST_ID/ROOT/$BE
|
|
||||||
|
|
||||||
#. chroot into the system::
|
|
||||||
|
|
||||||
arch-chroot /mnt /bin/bash --login
|
|
||||||
zfs mount -a
|
|
||||||
mount -a
|
|
||||||
|
|
||||||
#. Finish rescue. See `finish installation <#finish-installation>`__.
|
|
||||||
|
|
||||||
Backup and migrate existing installation
|
|
||||||
----------------------------------------
|
|
||||||
With the help of `zfs send
|
|
||||||
<https://openzfs.github.io/openzfs-docs/man/8/zfs-send.8.html>`__
|
|
||||||
it is relatively easy to perform a system backup and migration.
|
|
||||||
|
|
||||||
#. Create a snapshot of root file system::
|
|
||||||
|
|
||||||
zfs snapshot -r rpool/$INST_ID@backup
|
|
||||||
zfs snapshot -r bpool/$INST_ID@backup
|
|
||||||
|
|
||||||
#. Save snapshot to a file or pipe to SSH::
|
|
||||||
|
|
||||||
zfs send --options rpool/$INST_ID@backup > /backup/$INST_ID-rpool
|
|
||||||
zfs send --options bpool/$INST_ID@backup > /backup/$INST_ID-bpool
|
|
||||||
|
|
||||||
#. Re-create partitions and root/boot
|
|
||||||
pool on target system.
|
|
||||||
|
|
||||||
#. Restore backup::
|
|
||||||
|
|
||||||
zfs recv rpool_new/$INST_ID < /backup/$INST_ID-rpool
|
|
||||||
zfs recv bpool_new/$INST_ID < /backup/$INST_ID-bpool
|
|
||||||
|
|
||||||
#. Chroot and reinstall bootloader.
|
|
||||||
|
|
||||||
#. Update pool name in ``/etc/fstab``, ``/boot/grub/grub.cfg``
|
|
||||||
and ``/etc/zfs/zfs-list.cache/*``.
|
|
||||||
|
|
||||||
#. Update device name, etc, in ``/etc/fstab`` and ``/etc/crypttab``.
|
|
||||||
@@ -90,7 +90,7 @@ Root on ZFS
|
|||||||
ZFS can be used as root file system for Fedora.
|
ZFS can be used as root file system for Fedora.
|
||||||
An installation guide is available.
|
An installation guide is available.
|
||||||
|
|
||||||
`Start here <Root%20on%20ZFS/0-overview.html>`__.
|
`Start here <Root%20on%20ZFS/1-preparation.html>`__.
|
||||||
|
|
||||||
.. toctree::
|
.. toctree::
|
||||||
:maxdepth: 1
|
:maxdepth: 1
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
NixOS Root on ZFS
|
NixOS Root on ZFS
|
||||||
=======================================
|
=======================================
|
||||||
`Start here <Root%20on%20ZFS/0-overview.html>`__.
|
`Start here <Root%20on%20ZFS/1-preparation.html>`__.
|
||||||
|
|
||||||
Contents
|
Contents
|
||||||
--------
|
--------
|
||||||
|
|||||||
@@ -1,142 +0,0 @@
|
|||||||
.. highlight:: sh
|
|
||||||
|
|
||||||
Overview
|
|
||||||
======================
|
|
||||||
This document describes how to install NixOS with ZFS as root
|
|
||||||
file system.
|
|
||||||
|
|
||||||
Caution
|
|
||||||
~~~~~~~
|
|
||||||
- This guide wipes entire physical disks. Back up existing data.
|
|
||||||
- `GRUB does not and
|
|
||||||
will not work on 4Kn drive with legacy (BIOS) booting.
|
|
||||||
<http://savannah.gnu.org/bugs/?46700>`__
|
|
||||||
|
|
||||||
Partition layout
|
|
||||||
~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
GUID partition table (GPT) is used.
|
|
||||||
EFI system partition will be referred to as **ESP** in this document.
|
|
||||||
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
| Name | legacy boot | ESP | Boot pool | swap | root pool | remaining space |
|
|
||||||
+======================+======================+=======================+======================+=====================+=======================+=================+
|
|
||||||
| File system | | vfat | ZFS | swap | ZFS | |
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
| Size | 1M | 2G | 4G | depends on RAM size | | |
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
| Optional encryption | | *Secure Boot* | | plain dm-crypt | ZFS native encryption | |
|
|
||||||
| | | | | | | |
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
| Partition no. | 5 | 1 | 2 | 4 | 3 | |
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
| Mount point | | | /boot | | / | |
|
|
||||||
| | | /boot/efis/disk-part1 | | | | |
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
|
|
||||||
Dataset layout
|
|
||||||
~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
The dataset layout used in this guide follows stardard
|
|
||||||
mutable file positions (``/var``, ``/etc``, ...), but can
|
|
||||||
still be modified to `a immutable root <https://grahamc.com/blog/erase-your-darlings>`__
|
|
||||||
after installation.
|
|
||||||
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| Dataset | canmount | mountpoint | container | notes |
|
|
||||||
+===========================+======================+======================+=====================================+===========================================+
|
|
||||||
| bpool | off | /boot | contains sys | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool | off | / | contains sys | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| bpool/sys | off | none | contains BOOT | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys | off | none | contains ROOT | sys is encryptionroot |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| bpool/sys/BOOT | off | none | contains boot environments | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/ROOT | off | none | contains boot environments | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/DATA | off | none | contains placeholder "default" | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/DATA/default | off | / | contains user datasets | child datsets inherits mountpoint |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/DATA/local | off | / | contains /nix datasets | child datsets inherits mountpoint |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/DATA/default/ | on | /home (inherited) | no | |
|
|
||||||
| home | | | | user datasets, also called "shared |
|
|
||||||
| | | | | datasets", "persistent datasets"; also |
|
|
||||||
| | | | | include /var/lib, /srv, ... |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| bpool/sys/BOOT/default | noauto | /boot | no | noauto is used to switch BE. because of |
|
|
||||||
| | | | | noauto, must use fstab to mount |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/ROOT/default | noauto | / | no | mounted by initrd zfs hook |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
|
|
||||||
Encryption
|
|
||||||
~~~~~~~~~~
|
|
||||||
|
|
||||||
- Swap
|
|
||||||
|
|
||||||
Swap is always encrypted. By default, swap is encrypted
|
|
||||||
with plain dm-crypt with key generated from ``/dev/urandom``
|
|
||||||
at every boot. Swap content does not persist between reboots.
|
|
||||||
|
|
||||||
- Root pool
|
|
||||||
|
|
||||||
ZFS native encryption can be optionally enabled for ``rpool/sys``
|
|
||||||
and child datasets.
|
|
||||||
|
|
||||||
User should be aware that, ZFS native encryption does not
|
|
||||||
encrypt some metadata of the datasets.
|
|
||||||
ZFS native encryption also does not change master key when ``zfs change-key`` is invoked.
|
|
||||||
Therefore, you should wipe the disk when password is compromised to protect confidentiality.
|
|
||||||
See `zfs-load-key.8 <https://openzfs.github.io/openzfs-docs/man/8/zfs-load-key.8.html>`__
|
|
||||||
and `zfs-change-key.8 <https://openzfs.github.io/openzfs-docs/man/8/zfs-change-key.8.html>`__
|
|
||||||
for more information regarding ZFS native encryption.
|
|
||||||
|
|
||||||
Encryption is enabled at dataset creation and can not be disabled later.
|
|
||||||
|
|
||||||
- Boot pool
|
|
||||||
|
|
||||||
Boot pool can not be encrypted.
|
|
||||||
|
|
||||||
- Bootloader
|
|
||||||
|
|
||||||
Bootloader can not be encrypted.
|
|
||||||
|
|
||||||
However, with Secure Boot, bootloader
|
|
||||||
can be verified by motherboard firmware to be untempered,
|
|
||||||
which should be sufficient for most purposes.
|
|
||||||
|
|
||||||
Secure Boot is not supported out-of-the-box due to ZFS module.
|
|
||||||
|
|
||||||
Booting with disk failure
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
This guide is written with disk failure in mind.
|
|
||||||
|
|
||||||
If disks used in Root on ZFS pool failed, but
|
|
||||||
sufficient redundancy for both root pool and boot pool
|
|
||||||
still exists, the system will still boot normally.
|
|
||||||
|
|
||||||
Swap partition on the failed disk will fail to mount,
|
|
||||||
after an 1m30s timeout.
|
|
||||||
|
|
||||||
This feature is useful for use cases such
|
|
||||||
as an unattended remote server.
|
|
||||||
|
|
||||||
Example:
|
|
||||||
|
|
||||||
- System has disks ``n>1``
|
|
||||||
|
|
||||||
- Installed with mirrored setup
|
|
||||||
|
|
||||||
- Mirrored setup can tolerate up to ``n-1`` disk failures
|
|
||||||
|
|
||||||
- Disconnect one or more disks, keep at least
|
|
||||||
one disk connected
|
|
||||||
|
|
||||||
- System still boots, but fails to mount swap and
|
|
||||||
EFI partition
|
|
||||||
@@ -6,41 +6,26 @@ Preparation
|
|||||||
.. contents:: Table of Contents
|
.. contents:: Table of Contents
|
||||||
:local:
|
:local:
|
||||||
|
|
||||||
#. Download `Minimal ISO image
|
#. Disable Secure Boot. ZFS modules can not be loaded if Secure Boot is enabled.
|
||||||
<https://channels.nixos.org/nixos-22.05/latest-nixos-minimal-x86_64-linux.iso>`__ and boot from it.
|
#. Download `NixOS Live Image
|
||||||
|
<https://channels.nixos.org/nixos-22.05/latest-nixos-gnome-x86_64-linux.iso>`__ and boot from it.
|
||||||
|
#. Connect to the Internet.
|
||||||
|
#. Set root password or ``/root/.ssh/authorized_keys``.
|
||||||
|
#. Start SSH server::
|
||||||
|
|
||||||
#. Connect to network. See `NixOS manual <https://nixos.org/manual/nixos/stable/index.html#sec-installation-booting>`__.
|
systemctl restart sshd
|
||||||
|
|
||||||
#. SSH server is enabled by default. To connect, set root password with::
|
|
||||||
|
|
||||||
sudo passwd
|
|
||||||
|
|
||||||
#. Connect from another computer::
|
#. Connect from another computer::
|
||||||
|
|
||||||
ssh root@192.168.1.19
|
ssh root@192.168.1.19
|
||||||
|
|
||||||
#. Unique pool suffix. ZFS expects pool names to be
|
|
||||||
unique, therefore it's recommended to create
|
|
||||||
pools with a unique suffix::
|
|
||||||
|
|
||||||
INST_UUID=$(dd if=/dev/urandom bs=1 count=100 2>/dev/null | tr -dc 'a-z0-9' | cut -c-6)
|
|
||||||
|
|
||||||
#. Identify this installation in ZFS filesystem path::
|
|
||||||
|
|
||||||
INST_ID=nixos
|
|
||||||
|
|
||||||
#. Root on ZFS configuration file name::
|
|
||||||
|
|
||||||
INST_CONFIG_FILE='zfs.nix'
|
|
||||||
|
|
||||||
#. Target disk
|
#. Target disk
|
||||||
|
|
||||||
List available disks with::
|
List available disks with::
|
||||||
|
|
||||||
ls /dev/disk/by-id/*
|
ls /dev/disk/by-id/*
|
||||||
|
|
||||||
If using virtio as disk bus, use
|
If using virtio as disk bus, use ``/dev/disk/by-path/*``.
|
||||||
``/dev/disk/by-path/*``.
|
|
||||||
|
|
||||||
Declare disk array::
|
Declare disk array::
|
||||||
|
|
||||||
@@ -50,57 +35,8 @@ Preparation
|
|||||||
|
|
||||||
DISK='/dev/disk/by-id/disk1'
|
DISK='/dev/disk/by-id/disk1'
|
||||||
|
|
||||||
#. Choose a primary disk. This disk will be used
|
|
||||||
for primary EFI partition, default to
|
|
||||||
first disk in the array::
|
|
||||||
|
|
||||||
INST_PRIMARY_DISK=$(echo $DISK | cut -f1 -d\ )
|
|
||||||
|
|
||||||
#. Set vdev topology, possible values are:
|
|
||||||
|
|
||||||
- (not set, single disk or striped; no redundancy)
|
|
||||||
- mirror
|
|
||||||
- raidz1
|
|
||||||
- raidz2
|
|
||||||
- raidz3
|
|
||||||
|
|
||||||
::
|
|
||||||
|
|
||||||
INST_VDEV=
|
|
||||||
|
|
||||||
This will create a single vdev with the topology of your choice.
|
|
||||||
It is also possible to manually create a pool with multiple vdevs, such as::
|
|
||||||
|
|
||||||
zpool create --options \
|
|
||||||
poolName \
|
|
||||||
mirror sda sdb \
|
|
||||||
raidz2 sdc ... \
|
|
||||||
raidz3 sde ... \
|
|
||||||
spare sdf ...
|
|
||||||
|
|
||||||
Notice the cost of parity when using RAID-Z. See
|
|
||||||
`here <https://www.delphix.com/blog/delphix-engineering/zfs-raidz-stripe-width-or-how-i-learned-stop-worrying-and-love-raidz>`__
|
|
||||||
and `here <https://docs.google.com/spreadsheets/d/1tf4qx1aMJp8Lo_R6gpT689wTjHv6CGVElrPqTA0w_ZY/>`__.
|
|
||||||
|
|
||||||
For boot pool, which must be readable by GRUB, mirrored vdev should always be used for maximum redundancy.
|
|
||||||
This guide will use mirrored bpool for multi-disk setup.
|
|
||||||
|
|
||||||
Refer to `zpoolconcepts <https://openzfs.github.io/openzfs-docs/man/7/zpoolconcepts.7.html>`__
|
|
||||||
and `zpool-create <https://openzfs.github.io/openzfs-docs/man/8/zpool-create.8.html>`__
|
|
||||||
man pages for details.
|
|
||||||
|
|
||||||
#. Set partition size:
|
#. Set partition size:
|
||||||
|
|
||||||
Set ESP size::
|
|
||||||
|
|
||||||
INST_PARTSIZE_ESP=2 # in GB
|
|
||||||
|
|
||||||
Set boot pool size. To avoid running out of space while using
|
|
||||||
boot environments, the minimum is 4GB. Adjust the size if you
|
|
||||||
intend to use multiple kernel/distros::
|
|
||||||
|
|
||||||
INST_PARTSIZE_BPOOL=4
|
|
||||||
|
|
||||||
Set swap size. It's `recommended <https://chrisdown.name/2018/01/02/in-defence-of-swap.html>`__
|
Set swap size. It's `recommended <https://chrisdown.name/2018/01/02/in-defence-of-swap.html>`__
|
||||||
to setup a swap partition. If you intend to use hibernation,
|
to setup a swap partition. If you intend to use hibernation,
|
||||||
the minimum should be no less than RAM size. Skip if swap is not needed::
|
the minimum should be no less than RAM size. Skip if swap is not needed::
|
||||||
|
|||||||
@@ -1,362 +0,0 @@
|
|||||||
.. highlight:: sh
|
|
||||||
|
|
||||||
System Configuration
|
|
||||||
======================
|
|
||||||
|
|
||||||
.. contents:: Table of Contents
|
|
||||||
:local:
|
|
||||||
|
|
||||||
#. Optional: wipe solid-state drives with the generic tool
|
|
||||||
`blkdiscard <https://utcc.utoronto.ca/~cks/space/blog/linux/ErasingSSDsWithBlkdiscard>`__,
|
|
||||||
to clean previous partition tables and improve performance.
|
|
||||||
|
|
||||||
All content will be irrevocably destroyed::
|
|
||||||
|
|
||||||
for i in ${DISK}; do
|
|
||||||
blkdiscard -f $i &
|
|
||||||
done
|
|
||||||
wait
|
|
||||||
|
|
||||||
This is a quick operation and should be completed under one
|
|
||||||
minute.
|
|
||||||
|
|
||||||
For other device specific methods, see
|
|
||||||
`Memory cell clearing <https://wiki.archlinux.org/title/Solid_state_drive/Memory_cell_clearing>`__
|
|
||||||
|
|
||||||
#. Partition the disks.
|
|
||||||
See `Overview <0-overview.html>`__ for details::
|
|
||||||
|
|
||||||
for i in ${DISK}; do
|
|
||||||
sgdisk --zap-all $i
|
|
||||||
sgdisk -n1:1M:+${INST_PARTSIZE_ESP}G -t1:EF00 $i
|
|
||||||
sgdisk -n2:0:+${INST_PARTSIZE_BPOOL}G -t2:BE00 $i
|
|
||||||
if [ "${INST_PARTSIZE_SWAP}" != "" ]; then
|
|
||||||
sgdisk -n4:0:+${INST_PARTSIZE_SWAP}G -t4:8200 $i
|
|
||||||
fi
|
|
||||||
if [ "${INST_PARTSIZE_RPOOL}" = "" ]; then
|
|
||||||
sgdisk -n3:0:0 -t3:BF00 $i
|
|
||||||
else
|
|
||||||
sgdisk -n3:0:+${INST_PARTSIZE_RPOOL}G -t3:BF00 $i
|
|
||||||
fi
|
|
||||||
sgdisk -a1 -n5:24K:+1000K -t5:EF02 $i
|
|
||||||
done
|
|
||||||
|
|
||||||
#. Create boot pool::
|
|
||||||
|
|
||||||
|
|
||||||
disk_num=0; for i in $DISK; do disk_num=$(( $disk_num + 1 )); done
|
|
||||||
if [ $disk_num -gt 1 ]; then INST_VDEV_BPOOL=mirror; fi
|
|
||||||
|
|
||||||
|
|
||||||
zpool create \
|
|
||||||
-o compatibility=grub2 \
|
|
||||||
-o ashift=12 \
|
|
||||||
-o autotrim=on \
|
|
||||||
-O acltype=posixacl \
|
|
||||||
-O canmount=off \
|
|
||||||
-O compression=lz4 \
|
|
||||||
-O devices=off \
|
|
||||||
-O normalization=formD \
|
|
||||||
-O relatime=on \
|
|
||||||
-O xattr=sa \
|
|
||||||
-O mountpoint=/boot \
|
|
||||||
-R /mnt \
|
|
||||||
bpool_$INST_UUID \
|
|
||||||
$INST_VDEV_BPOOL \
|
|
||||||
$(for i in ${DISK}; do
|
|
||||||
printf "$i-part2 ";
|
|
||||||
done)
|
|
||||||
|
|
||||||
You should not need to customize any of the options for the boot pool.
|
|
||||||
|
|
||||||
GRUB does not support all of the zpool features. See ``spa_feature_names``
|
|
||||||
in `grub-core/fs/zfs/zfs.c
|
|
||||||
<http://git.savannah.gnu.org/cgit/grub.git/tree/grub-core/fs/zfs/zfs.c#n276>`__.
|
|
||||||
This step creates a separate boot pool for ``/boot`` with the features
|
|
||||||
limited to only those that GRUB supports, allowing the root pool to use
|
|
||||||
any/all features.
|
|
||||||
|
|
||||||
Features enabled with ``-o compatibility=grub2`` can be seen
|
|
||||||
`here <https://github.com/openzfs/zfs/blob/master/cmd/zpool/compatibility.d/grub2>`__.
|
|
||||||
|
|
||||||
#. Create root pool::
|
|
||||||
|
|
||||||
zpool create \
|
|
||||||
-o ashift=12 \
|
|
||||||
-o autotrim=on \
|
|
||||||
-R /mnt \
|
|
||||||
-O acltype=posixacl \
|
|
||||||
-O canmount=off \
|
|
||||||
-O compression=zstd \
|
|
||||||
-O dnodesize=auto \
|
|
||||||
-O normalization=formD \
|
|
||||||
-O relatime=on \
|
|
||||||
-O xattr=sa \
|
|
||||||
-O mountpoint=/ \
|
|
||||||
rpool_$INST_UUID \
|
|
||||||
$INST_VDEV \
|
|
||||||
$(for i in ${DISK}; do
|
|
||||||
printf "$i-part3 ";
|
|
||||||
done)
|
|
||||||
|
|
||||||
**Notes:**
|
|
||||||
|
|
||||||
- The use of ``ashift=12`` is recommended here because many drives
|
|
||||||
today have 4 KiB (or larger) physical sectors, even though they
|
|
||||||
present 512 B logical sectors. Also, a future replacement drive may
|
|
||||||
have 4 KiB physical sectors (in which case ``ashift=12`` is desirable)
|
|
||||||
or 4 KiB logical sectors (in which case ``ashift=12`` is required).
|
|
||||||
- Setting ``-O acltype=posixacl`` enables POSIX ACLs globally. If you
|
|
||||||
do not want this, remove that option, but later add
|
|
||||||
``-o acltype=posixacl`` (note: lowercase “o”) to the ``zfs create``
|
|
||||||
for ``/var/log``, as `journald requires ACLs
|
|
||||||
<https://askubuntu.com/questions/970886/journalctl-says-failed-to-search-journal-acl-operation-not-supported>`__
|
|
||||||
- Setting ``normalization=formD`` eliminates some corner cases relating
|
|
||||||
to UTF-8 filename normalization. It also implies ``utf8only=on``,
|
|
||||||
which means that only UTF-8 filenames are allowed. If you care to
|
|
||||||
support non-UTF-8 filenames, do not use this option. For a discussion
|
|
||||||
of why requiring UTF-8 filenames may be a bad idea, see `The problems
|
|
||||||
with enforced UTF-8 only filenames
|
|
||||||
<http://utcc.utoronto.ca/~cks/space/blog/linux/ForcedUTF8Filenames>`__.
|
|
||||||
- ``recordsize`` is unset (leaving it at the default of 128 KiB). If you
|
|
||||||
want to tune it (e.g. ``-o recordsize=1M``), see `these
|
|
||||||
<https://jrs-s.net/2019/04/03/on-zfs-recordsize/>`__ `various
|
|
||||||
<http://blog.programster.org/zfs-record-size>`__ `blog
|
|
||||||
<https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSFileRecordsizeGrowth>`__
|
|
||||||
`posts
|
|
||||||
<https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSRecordsizeAndCompression>`__.
|
|
||||||
- Setting ``relatime=on`` is a middle ground between classic POSIX
|
|
||||||
``atime`` behavior (with its significant performance impact) and
|
|
||||||
``atime=off`` (which provides the best performance by completely
|
|
||||||
disabling atime updates). Since Linux 2.6.30, ``relatime`` has been
|
|
||||||
the default for other filesystems. See `RedHat’s documentation
|
|
||||||
<https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/power_management_guide/relatime>`__
|
|
||||||
for further information.
|
|
||||||
- Setting ``xattr=sa`` `vastly improves the performance of extended
|
|
||||||
attributes
|
|
||||||
<https://github.com/zfsonlinux/zfs/commit/82a37189aac955c81a59a5ecc3400475adb56355>`__.
|
|
||||||
Inside ZFS, extended attributes are used to implement POSIX ACLs.
|
|
||||||
Extended attributes can also be used by user-space applications.
|
|
||||||
`They are used by some desktop GUI applications.
|
|
||||||
<https://en.wikipedia.org/wiki/Extended_file_attributes#Linux>`__
|
|
||||||
`They can be used by Samba to store Windows ACLs and DOS attributes;
|
|
||||||
they are required for a Samba Active Directory domain controller.
|
|
||||||
<https://wiki.samba.org/index.php/Setting_up_a_Share_Using_Windows_ACLs>`__
|
|
||||||
Note that ``xattr=sa`` is `Linux-specific
|
|
||||||
<https://openzfs.org/wiki/Platform_code_differences>`__. If you move your
|
|
||||||
``xattr=sa`` pool to another OpenZFS implementation besides ZFS-on-Linux,
|
|
||||||
extended attributes will not be readable (though your data will be). If
|
|
||||||
portability of extended attributes is important to you, omit the
|
|
||||||
``-O xattr=sa`` above. Even if you do not want ``xattr=sa`` for the whole
|
|
||||||
pool, it is probably fine to use it for ``/var/log``.
|
|
||||||
- Make sure to include the ``-part3`` portion of the drive path. If you
|
|
||||||
forget that, you are specifying the whole disk, which ZFS will then
|
|
||||||
re-partition, and you will lose the bootloader partition(s).
|
|
||||||
|
|
||||||
#. This section implements dataset layout as described in `overview <0-overview.html>`__.
|
|
||||||
|
|
||||||
Create root system container:
|
|
||||||
|
|
||||||
- Unencrypted::
|
|
||||||
|
|
||||||
zfs create \
|
|
||||||
-o canmount=off \
|
|
||||||
-o mountpoint=none \
|
|
||||||
rpool_$INST_UUID/$INST_ID
|
|
||||||
|
|
||||||
- Encrypted:
|
|
||||||
|
|
||||||
Pick a strong password. Once compromised, changing password will not keep your
|
|
||||||
data safe. See ``zfs-change-key(8)`` for more info::
|
|
||||||
|
|
||||||
zfs create \
|
|
||||||
-o canmount=off \
|
|
||||||
-o mountpoint=none \
|
|
||||||
-o encryption=on \
|
|
||||||
-o keylocation=prompt \
|
|
||||||
-o keyformat=passphrase \
|
|
||||||
rpool_$INST_UUID/$INST_ID
|
|
||||||
|
|
||||||
Create other system datasets::
|
|
||||||
|
|
||||||
zfs create -o canmount=off -o mountpoint=none bpool_$INST_UUID/$INST_ID
|
|
||||||
zfs create -o canmount=off -o mountpoint=none bpool_$INST_UUID/$INST_ID/BOOT
|
|
||||||
zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID/ROOT
|
|
||||||
zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID/DATA
|
|
||||||
zfs create -o mountpoint=/boot -o canmount=noauto bpool_$INST_UUID/$INST_ID/BOOT/default
|
|
||||||
zfs create -o mountpoint=/ -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/default
|
|
||||||
zfs create -o mountpoint=/ -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/local
|
|
||||||
zfs create -o mountpoint=/ -o canmount=noauto rpool_$INST_UUID/$INST_ID/ROOT/default
|
|
||||||
zfs mount rpool_$INST_UUID/$INST_ID/ROOT/default
|
|
||||||
zfs mount bpool_$INST_UUID/$INST_ID/BOOT/default
|
|
||||||
for i in {usr,var,var/lib};
|
|
||||||
do
|
|
||||||
zfs create -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/default/$i
|
|
||||||
done
|
|
||||||
for i in {home,root,srv,usr/local,var/log,var/spool};
|
|
||||||
do
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/$i
|
|
||||||
done
|
|
||||||
chmod 750 /mnt/root
|
|
||||||
for i in {nix,}; do
|
|
||||||
zfs create -o canmount=on -o mountpoint=/$i rpool_$INST_UUID/$INST_ID/DATA/local/$i
|
|
||||||
done
|
|
||||||
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/state
|
|
||||||
for i in {/etc/nixos,/etc/cryptkey.d}; do
|
|
||||||
mkdir -p /mnt/state/$i /mnt/$i
|
|
||||||
mount -o bind /mnt/state/$i /mnt/$i
|
|
||||||
done
|
|
||||||
zfs create -o mountpoint=/ -o canmount=noauto rpool_$INST_UUID/$INST_ID/ROOT/empty
|
|
||||||
zfs snapshot rpool_$INST_UUID/$INST_ID/ROOT/empty@start
|
|
||||||
|
|
||||||
#. Format and mount ESP::
|
|
||||||
|
|
||||||
for i in ${DISK}; do
|
|
||||||
mkfs.vfat -n EFI ${i}-part1
|
|
||||||
mkdir -p /mnt/boot/efis/${i##*/}-part1
|
|
||||||
mount -t vfat ${i}-part1 /mnt/boot/efis/${i##*/}-part1
|
|
||||||
done
|
|
||||||
|
|
||||||
#. Create optional user data datasets to omit data from rollback::
|
|
||||||
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/games
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/www
|
|
||||||
# for GNOME
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/AccountsService
|
|
||||||
# for Docker
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/docker
|
|
||||||
# for NFS
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/nfs
|
|
||||||
# for LXC
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/lxc
|
|
||||||
# for LibVirt
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/libvirt
|
|
||||||
##other application
|
|
||||||
# zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/$name
|
|
||||||
|
|
||||||
Add other datasets when needed, such as PostgreSQL.
|
|
||||||
|
|
||||||
#. Generate initial NixOS system configuration::
|
|
||||||
|
|
||||||
nixos-generate-config --root /mnt
|
|
||||||
|
|
||||||
This command will generate two files, ``configuration.nix``
|
|
||||||
and ``hardware-configuration-zfs.nix``, which will be the starting point
|
|
||||||
of configuring the system.
|
|
||||||
|
|
||||||
#. Edit config file to import ZFS options::
|
|
||||||
|
|
||||||
sed -i "s|./hardware-configuration.nix|./hardware-configuration-zfs.nix ./${INST_CONFIG_FILE}|g" /mnt/etc/nixos/configuration.nix
|
|
||||||
# backup, prevent being overwritten by nixos-generate-config
|
|
||||||
mv /mnt/etc/nixos/hardware-configuration.nix /mnt/etc/nixos/hardware-configuration-zfs.nix
|
|
||||||
|
|
||||||
#. ZFS options::
|
|
||||||
|
|
||||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
|
||||||
{ config, pkgs, ... }:
|
|
||||||
|
|
||||||
{ boot.supportedFilesystems = [ "zfs" ];
|
|
||||||
networking.hostId = "$(head -c 8 /etc/machine-id)";
|
|
||||||
boot.zfs.devNodes = "${INST_PRIMARY_DISK%/*}";
|
|
||||||
EOF
|
|
||||||
|
|
||||||
ZFS datasets should be mounted with ``-o zfsutil`` option::
|
|
||||||
|
|
||||||
sed -i 's|fsType = "zfs";|fsType = "zfs"; options = [ "zfsutil" "X-mount.mkdir" ];|g' \
|
|
||||||
/mnt/etc/nixos/hardware-configuration-zfs.nix
|
|
||||||
|
|
||||||
Allow EFI system partition mounting to fail at boot::
|
|
||||||
|
|
||||||
sed -i 's|fsType = "vfat";|fsType = "vfat"; options = [ "x-systemd.idle-timeout=1min" "x-systemd.automount" "noauto" ];|g' \
|
|
||||||
/mnt/etc/nixos/hardware-configuration-zfs.nix
|
|
||||||
|
|
||||||
Restrict kernel to versions supported by ZFS::
|
|
||||||
|
|
||||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
|
||||||
boot.kernelPackages = config.boot.zfs.package.latestCompatibleLinuxPackages;
|
|
||||||
EOF
|
|
||||||
|
|
||||||
Disable cache::
|
|
||||||
|
|
||||||
mkdir -p /mnt/state/etc/zfs/
|
|
||||||
rm -f /mnt/state/etc/zfs/zpool.cache
|
|
||||||
touch /mnt/state/etc/zfs/zpool.cache
|
|
||||||
chmod a-w /mnt/state/etc/zfs/zpool.cache
|
|
||||||
chattr +i /mnt/state/etc/zfs/zpool.cache
|
|
||||||
|
|
||||||
#. If swap is enabled::
|
|
||||||
|
|
||||||
if [ "${INST_PARTSIZE_SWAP}" != "" ]; then
|
|
||||||
sed -i '/swapDevices/d' /mnt/etc/nixos/hardware-configuration-zfs.nix
|
|
||||||
|
|
||||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
|
||||||
swapDevices = [
|
|
||||||
EOF
|
|
||||||
for i in $DISK; do
|
|
||||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
|
||||||
{ device = "$i-part4"; randomEncryption.enable = true; }
|
|
||||||
EOF
|
|
||||||
done
|
|
||||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
|
||||||
];
|
|
||||||
EOF
|
|
||||||
fi
|
|
||||||
|
|
||||||
#. For immutable root file system, save machine-id and other files::
|
|
||||||
|
|
||||||
mkdir -p /mnt/state/etc/{ssh,zfs}
|
|
||||||
systemd-machine-id-setup --print > /mnt/state/etc/machine-id
|
|
||||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
|
||||||
systemd.services.zfs-mount.enable = false;
|
|
||||||
environment.etc."machine-id".source = "/state/etc/machine-id";
|
|
||||||
environment.etc."aliases".source = "/state/etc/aliases";
|
|
||||||
environment.etc."zfs/zpool.cache".source
|
|
||||||
= "/state/etc/zfs/zpool.cache";
|
|
||||||
boot.loader.efi.efiSysMountPoint = "/boot/efis/${INST_PRIMARY_DISK##*/}-part1";
|
|
||||||
EOF
|
|
||||||
touch /mnt/state/etc/aliases
|
|
||||||
|
|
||||||
#. Configure GRUB boot loader for both legacy boot and UEFI::
|
|
||||||
|
|
||||||
sed -i '/boot.loader/d' /mnt/etc/nixos/configuration.nix
|
|
||||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<-'EOF'
|
|
||||||
boot.loader.efi.canTouchEfiVariables = false;
|
|
||||||
##if UEFI firmware can detect entries
|
|
||||||
#boot.loader.efi.canTouchEfiVariables = true;
|
|
||||||
|
|
||||||
boot.loader = {
|
|
||||||
generationsDir.copyKernels = true;
|
|
||||||
##for problematic UEFI firmware
|
|
||||||
grub.efiInstallAsRemovable = true;
|
|
||||||
grub.enable = true;
|
|
||||||
grub.version = 2;
|
|
||||||
grub.copyKernels = true;
|
|
||||||
grub.efiSupport = true;
|
|
||||||
grub.zfsSupport = true;
|
|
||||||
# for systemd-autofs
|
|
||||||
grub.extraPrepareConfig = ''
|
|
||||||
mkdir -p /boot/efis
|
|
||||||
for i in /boot/efis/*; do mount $i ; done
|
|
||||||
'';
|
|
||||||
grub.extraInstallCommands = ''
|
|
||||||
export ESP_MIRROR=$(mktemp -d -p /tmp)
|
|
||||||
EOF
|
|
||||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
|
||||||
cp -r /boot/efis/${INST_PRIMARY_DISK##*/}-part1/EFI \$ESP_MIRROR
|
|
||||||
EOF
|
|
||||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<-'EOF'
|
|
||||||
for i in /boot/efis/*; do
|
|
||||||
cp -r $ESP_MIRROR/EFI $i
|
|
||||||
done
|
|
||||||
rm -rf $ESP_MIRROR
|
|
||||||
'';
|
|
||||||
grub.devices = [
|
|
||||||
EOF
|
|
||||||
for i in $DISK; do
|
|
||||||
printf " \"$i\"\n" >>/mnt/etc/nixos/${INST_CONFIG_FILE}
|
|
||||||
done
|
|
||||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
|
||||||
];
|
|
||||||
};
|
|
||||||
EOF
|
|
||||||
132
docs/Getting Started/NixOS/Root on ZFS/2-system-installation.rst
Normal file
132
docs/Getting Started/NixOS/Root on ZFS/2-system-installation.rst
Normal file
@@ -0,0 +1,132 @@
|
|||||||
|
.. highlight:: sh
|
||||||
|
|
||||||
|
System Installation
|
||||||
|
======================
|
||||||
|
|
||||||
|
.. contents:: Table of Contents
|
||||||
|
:local:
|
||||||
|
|
||||||
|
#. Partition the disks::
|
||||||
|
|
||||||
|
for i in ${DISK}; do
|
||||||
|
|
||||||
|
sgdisk --zap-all $i
|
||||||
|
|
||||||
|
sgdisk -n1:1M:+1G -t1:EF00 $i
|
||||||
|
|
||||||
|
sgdisk -n2:0:+4G -t2:BE00 $i
|
||||||
|
|
||||||
|
test -z $INST_PARTSIZE_SWAP || sgdisk -n4:0:+${INST_PARTSIZE_SWAP}G -t4:8200 $i
|
||||||
|
|
||||||
|
if test -z $INST_PARTSIZE_RPOOL; then
|
||||||
|
sgdisk -n3:0:0 -t3:BF00 $i
|
||||||
|
else
|
||||||
|
sgdisk -n3:0:+${INST_PARTSIZE_RPOOL}G -t3:BF00 $i
|
||||||
|
fi
|
||||||
|
|
||||||
|
sgdisk -a1 -n5:24K:+1000K -t5:EF02 $i
|
||||||
|
done
|
||||||
|
|
||||||
|
#. Create boot pool::
|
||||||
|
|
||||||
|
zpool create \
|
||||||
|
-o compatibility=grub2 \
|
||||||
|
-o ashift=12 \
|
||||||
|
-o autotrim=on \
|
||||||
|
-O acltype=posixacl \
|
||||||
|
-O canmount=off \
|
||||||
|
-O compression=lz4 \
|
||||||
|
-O devices=off \
|
||||||
|
-O normalization=formD \
|
||||||
|
-O relatime=on \
|
||||||
|
-O xattr=sa \
|
||||||
|
-O mountpoint=/boot \
|
||||||
|
-R /mnt \
|
||||||
|
bpool \
|
||||||
|
mirror \
|
||||||
|
$(for i in ${DISK}; do
|
||||||
|
printf "$i-part2 ";
|
||||||
|
done)
|
||||||
|
|
||||||
|
If not using a multi-disk setup, remove ``mirror``.
|
||||||
|
|
||||||
|
You should not need to customize any of the options for the boot pool.
|
||||||
|
|
||||||
|
GRUB does not support all of the zpool features. See ``spa_feature_names``
|
||||||
|
in `grub-core/fs/zfs/zfs.c
|
||||||
|
<http://git.savannah.gnu.org/cgit/grub.git/tree/grub-core/fs/zfs/zfs.c#n276>`__.
|
||||||
|
This step creates a separate boot pool for ``/boot`` with the features
|
||||||
|
limited to only those that GRUB supports, allowing the root pool to use
|
||||||
|
any/all features.
|
||||||
|
|
||||||
|
Features enabled with ``-o compatibility=grub2`` can be seen
|
||||||
|
`here <https://github.com/openzfs/zfs/blob/master/cmd/zpool/compatibility.d/grub2>`__.
|
||||||
|
|
||||||
|
#. Create root pool::
|
||||||
|
|
||||||
|
zpool create \
|
||||||
|
-o ashift=12 \
|
||||||
|
-o autotrim=on \
|
||||||
|
-R /mnt \
|
||||||
|
-O acltype=posixacl \
|
||||||
|
-O canmount=off \
|
||||||
|
-O compression=zstd \
|
||||||
|
-O dnodesize=auto \
|
||||||
|
-O normalization=formD \
|
||||||
|
-O relatime=on \
|
||||||
|
-O xattr=sa \
|
||||||
|
-O mountpoint=/ \
|
||||||
|
rpool \
|
||||||
|
mirror \
|
||||||
|
$(for i in ${DISK}; do
|
||||||
|
printf "$i-part3 ";
|
||||||
|
done)
|
||||||
|
|
||||||
|
If not using a multi-disk setup, remove ``mirror``.
|
||||||
|
|
||||||
|
#. This section implements dataset layout as described in `overview <1-preparation.html>`__.
|
||||||
|
|
||||||
|
Create root system container:
|
||||||
|
|
||||||
|
- Unencrypted::
|
||||||
|
|
||||||
|
zfs create \
|
||||||
|
-o canmount=off \
|
||||||
|
-o mountpoint=none \
|
||||||
|
rpool/nixos
|
||||||
|
|
||||||
|
- Encrypted:
|
||||||
|
|
||||||
|
Pick a strong password. Once compromised, changing password will not keep your
|
||||||
|
data safe. See ``zfs-change-key(8)`` for more info::
|
||||||
|
|
||||||
|
zfs create \
|
||||||
|
-o canmount=off \
|
||||||
|
-o mountpoint=none \
|
||||||
|
-o encryption=on \
|
||||||
|
-o keylocation=prompt \
|
||||||
|
-o keyformat=passphrase \
|
||||||
|
rpool/nixos
|
||||||
|
|
||||||
|
Create system datasets::
|
||||||
|
|
||||||
|
zfs create -o canmount=on -o mountpoint=/ rpool/nixos/root
|
||||||
|
zfs create -o canmount=on -o mountpoint=/home rpool/nixos/home
|
||||||
|
zfs create -o canmount=off -o mountpoint=/var rpool/nixos/var
|
||||||
|
zfs create -o canmount=on rpool/nixos/var/lib
|
||||||
|
zfs create -o canmount=on rpool/nixos/var/log
|
||||||
|
|
||||||
|
Create boot dataset::
|
||||||
|
|
||||||
|
zfs create -o canmount=on -o mountpoint=/boot bpool/nixos
|
||||||
|
|
||||||
|
#. Format and mount ESP::
|
||||||
|
|
||||||
|
for i in ${DISK}; do
|
||||||
|
mkfs.vfat -n EFI ${i}-part1
|
||||||
|
mkdir -p /mnt/boot/efis/${i##*/}-part1
|
||||||
|
mount -t vfat ${i}-part1 /mnt/boot/efis/${i##*/}-part1
|
||||||
|
done
|
||||||
|
|
||||||
|
mkdir -p /mnt/boot/efi
|
||||||
|
mount -t vfat $(echo $DISK | cut -f1 -d\ )-part1 /mnt/boot/efi
|
||||||
@@ -1,235 +0,0 @@
|
|||||||
.. highlight:: sh
|
|
||||||
|
|
||||||
Optional Configuration
|
|
||||||
======================
|
|
||||||
|
|
||||||
.. contents:: Table of Contents
|
|
||||||
:local:
|
|
||||||
|
|
||||||
Skip to `System Installation <./4-system-installation.html>`__ section if
|
|
||||||
no optional configuration is needed.
|
|
||||||
|
|
||||||
Mail notification for ZFS status
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
For headless applications such as NAS, it is useful to set up mail notification
|
|
||||||
for hardware changes and monitor for scrub results.
|
|
||||||
|
|
||||||
#. Set up an alias for root account::
|
|
||||||
|
|
||||||
tee -a /state/etc/aliases <<EOF
|
|
||||||
root: user@example.com
|
|
||||||
EOF
|
|
||||||
|
|
||||||
#. Set up mail transfer agent, the program that sends email::
|
|
||||||
|
|
||||||
programs.msmtp = {
|
|
||||||
enable = true;
|
|
||||||
setSendmail = true;
|
|
||||||
defaults = {
|
|
||||||
aliases = "/state/etc/aliases";
|
|
||||||
port = 465;
|
|
||||||
tls_trust_file = "/etc/ssl/certs/ca-certificates.crt";
|
|
||||||
tls = "on";
|
|
||||||
auth = "plain";
|
|
||||||
tls_starttls = "off";
|
|
||||||
};
|
|
||||||
accounts = {
|
|
||||||
default = {
|
|
||||||
host = "mail.example.com";
|
|
||||||
# set secure permissions for password file
|
|
||||||
passwordeval = "cat /state/etc/emailpass.txt";
|
|
||||||
user = "user@example.com";
|
|
||||||
from = "user@example.com";
|
|
||||||
};
|
|
||||||
};
|
|
||||||
};
|
|
||||||
|
|
||||||
#. Enable mail notification for ZFS Event Daemon::
|
|
||||||
|
|
||||||
services.zfs.zed.settings = {
|
|
||||||
ZED_EMAIL_ADDR = [ "root" ];
|
|
||||||
ZED_EMAIL_PROG = "${pkgs.msmtp}/bin/msmtp";
|
|
||||||
ZED_EMAIL_OPTS = "@ADDRESS@";
|
|
||||||
ZED_NOTIFY_VERBOSE = true;
|
|
||||||
};
|
|
||||||
# this option does not work
|
|
||||||
services.zfs.zed.enableMail = false;
|
|
||||||
|
|
||||||
Supply password with SSH
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
Note: if you choose to encrypt boot pool, where decryption is handled
|
|
||||||
by GRUB, as described in the next section, configuration performed
|
|
||||||
in this section will have no effect.
|
|
||||||
|
|
||||||
This example uses DHCP::
|
|
||||||
|
|
||||||
mkdir -p /mnt/etc/state/ssh/
|
|
||||||
ssh-keygen -t ed25519 -N "" -f /mnt/state/etc/ssh/ssh_host_initrd_ed25519_key
|
|
||||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
|
||||||
#networking.interfaces.enp1s0.useDHCP = true;
|
|
||||||
boot = {
|
|
||||||
initrd.network = {
|
|
||||||
enable = true;
|
|
||||||
ssh = {
|
|
||||||
enable = true;
|
|
||||||
hostKeys = [ /state/etc/ssh/ssh_host_initrd_ed25519_key ];
|
|
||||||
authorizedKeys = [ "$YOUR_PUBLIC_KEY" ];
|
|
||||||
};
|
|
||||||
postCommands = ''
|
|
||||||
echo "zfs load-key -a; killall zfs" >> /root/.profile
|
|
||||||
'';
|
|
||||||
};
|
|
||||||
};
|
|
||||||
EOF
|
|
||||||
|
|
||||||
Encrypt boot pool
|
|
||||||
~~~~~~~~~~~~~~~~~~~
|
|
||||||
Note: This will disable password with SSH. The password previously set for
|
|
||||||
root pool will be replaced by keyfile, embedded in initrd.
|
|
||||||
|
|
||||||
#. Add package::
|
|
||||||
|
|
||||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
|
||||||
environment.systemPackages = [ pkgs.cryptsetup ];
|
|
||||||
EOF
|
|
||||||
|
|
||||||
#. LUKS password::
|
|
||||||
|
|
||||||
LUKS_PWD=secure-passwd
|
|
||||||
|
|
||||||
You will need to enter the same password for
|
|
||||||
each disk at boot. As root pool key is
|
|
||||||
protected by this password, the previous warning
|
|
||||||
about password strength still apply.
|
|
||||||
|
|
||||||
Double-check password here. Complete reinstallation is
|
|
||||||
needed if entered wrong.
|
|
||||||
|
|
||||||
#. Create encryption keys::
|
|
||||||
|
|
||||||
mkdir -p /mnt/etc/cryptkey.d/
|
|
||||||
chmod 700 /mnt/etc/cryptkey.d/
|
|
||||||
dd bs=32 count=1 if=/dev/urandom of=/mnt/etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs
|
|
||||||
dd bs=32 count=1 if=/dev/urandom of=/mnt/etc/cryptkey.d/bpool_$INST_UUID-key-luks
|
|
||||||
chmod u=r,go= /mnt/etc/cryptkey.d/*
|
|
||||||
|
|
||||||
#. Backup boot pool::
|
|
||||||
|
|
||||||
zfs snapshot -r bpool_$INST_UUID/$INST_ID@pre-luks
|
|
||||||
zfs send -Rv bpool_$INST_UUID/$INST_ID@pre-luks > /mnt/root/bpool_$INST_UUID-${INST_ID}-pre-luks
|
|
||||||
|
|
||||||
#. Unmount EFI partition::
|
|
||||||
|
|
||||||
for i in ${DISK}; do
|
|
||||||
umount /mnt/boot/efis/${i##*/}-part1
|
|
||||||
done
|
|
||||||
|
|
||||||
#. Destroy boot pool::
|
|
||||||
|
|
||||||
zpool destroy bpool_$INST_UUID
|
|
||||||
|
|
||||||
#. Create LUKS containers::
|
|
||||||
|
|
||||||
for i in ${DISK}; do
|
|
||||||
cryptsetup luksFormat -q --type luks1 --key-file /mnt/etc/cryptkey.d/bpool_$INST_UUID-key-luks $i-part2
|
|
||||||
echo $LUKS_PWD | cryptsetup luksAddKey --key-file /mnt/etc/cryptkey.d/bpool_$INST_UUID-key-luks $i-part2
|
|
||||||
cryptsetup open ${i}-part2 ${i##*/}-part2-luks-bpool_$INST_UUID --key-file /mnt/etc/cryptkey.d/bpool_$INST_UUID-key-luks
|
|
||||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
|
||||||
boot.initrd.luks.devices = {
|
|
||||||
"${i##*/}-part2-luks-bpool_$INST_UUID" = {
|
|
||||||
device = "${i}-part2";
|
|
||||||
allowDiscards = true;
|
|
||||||
keyFile = "/etc/cryptkey.d/bpool_$INST_UUID-key-luks";
|
|
||||||
};
|
|
||||||
};
|
|
||||||
EOF
|
|
||||||
done
|
|
||||||
|
|
||||||
GRUB 2.06 still does not have complete support for LUKS2, LUKS1
|
|
||||||
is used instead.
|
|
||||||
|
|
||||||
#. Embed key file in initrd::
|
|
||||||
|
|
||||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
|
||||||
boot.initrd.secrets = {
|
|
||||||
"/etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs" = "/etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs";
|
|
||||||
"/etc/cryptkey.d/bpool_$INST_UUID-key-luks" = "/etc/cryptkey.d/bpool_$INST_UUID-key-luks";
|
|
||||||
};
|
|
||||||
EOF
|
|
||||||
|
|
||||||
#. Recreate boot pool with mappers as vdev::
|
|
||||||
|
|
||||||
disk_num=0; for i in $DISK; do disk_num=$(( $disk_num + 1 )); done
|
|
||||||
if [ $disk_num -gt 1 ]; then INST_VDEV_BPOOL=mirror; fi
|
|
||||||
|
|
||||||
|
|
||||||
zpool create \
|
|
||||||
-d -o feature@async_destroy=enabled \
|
|
||||||
-o feature@bookmarks=enabled \
|
|
||||||
-o feature@embedded_data=enabled \
|
|
||||||
-o feature@empty_bpobj=enabled \
|
|
||||||
-o feature@enabled_txg=enabled \
|
|
||||||
-o feature@extensible_dataset=enabled \
|
|
||||||
-o feature@filesystem_limits=enabled \
|
|
||||||
-o feature@hole_birth=enabled \
|
|
||||||
-o feature@large_blocks=enabled \
|
|
||||||
-o feature@lz4_compress=enabled \
|
|
||||||
-o feature@spacemap_histogram=enabled \
|
|
||||||
-o ashift=12 \
|
|
||||||
-o autotrim=on \
|
|
||||||
-O acltype=posixacl \
|
|
||||||
-O canmount=off \
|
|
||||||
-O compression=lz4 \
|
|
||||||
-O devices=off \
|
|
||||||
-O normalization=formD \
|
|
||||||
-O relatime=on \
|
|
||||||
-O xattr=sa \
|
|
||||||
-O mountpoint=/boot \
|
|
||||||
-R /mnt \
|
|
||||||
bpool_$INST_UUID \
|
|
||||||
$INST_VDEV_BPOOL \
|
|
||||||
$(for i in ${DISK}; do
|
|
||||||
printf "/dev/mapper/${i##*/}-part2-luks-bpool_$INST_UUID ";
|
|
||||||
done)
|
|
||||||
|
|
||||||
#. Restore boot pool backup::
|
|
||||||
|
|
||||||
zfs recv bpool_${INST_UUID}/${INST_ID} < /mnt/root/bpool_$INST_UUID-${INST_ID}-pre-luks
|
|
||||||
rm /mnt/root/bpool_$INST_UUID-${INST_ID}-pre-luks
|
|
||||||
|
|
||||||
#. Mount boot dataset and EFI partitions::
|
|
||||||
|
|
||||||
zfs mount bpool_$INST_UUID/$INST_ID/BOOT/default
|
|
||||||
|
|
||||||
for i in ${DISK}; do
|
|
||||||
mount ${i}-part1 /mnt/boot/efis/${i##*/}-part1
|
|
||||||
done
|
|
||||||
|
|
||||||
#. As keys are stored in initrd,
|
|
||||||
set secure permissions for ``/boot``::
|
|
||||||
|
|
||||||
chmod 700 /mnt/boot
|
|
||||||
|
|
||||||
#. Change root pool password to key file::
|
|
||||||
|
|
||||||
mkdir -p /etc/cryptkey.d/
|
|
||||||
cp /mnt/etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs /etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs
|
|
||||||
zfs change-key -l \
|
|
||||||
-o keylocation=file:///etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs \
|
|
||||||
-o keyformat=raw \
|
|
||||||
rpool_$INST_UUID/$INST_ID
|
|
||||||
|
|
||||||
#. Enable GRUB cryptodisk::
|
|
||||||
|
|
||||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
|
||||||
boot.loader.grub.enableCryptodisk = true;
|
|
||||||
EOF
|
|
||||||
|
|
||||||
#. **Important**: Back up root dataset key ``/etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs``
|
|
||||||
to a secure location.
|
|
||||||
|
|
||||||
In the possible event of LUKS container corruption,
|
|
||||||
data on root set will only be available
|
|
||||||
with this key.
|
|
||||||
@@ -0,0 +1,102 @@
|
|||||||
|
.. highlight:: sh
|
||||||
|
|
||||||
|
System Configuration
|
||||||
|
======================
|
||||||
|
|
||||||
|
.. contents:: Table of Contents
|
||||||
|
:local:
|
||||||
|
|
||||||
|
#. Disable cache, stale cache will prevent system from booting::
|
||||||
|
|
||||||
|
mkdir -p /mnt/state/etc/zfs/
|
||||||
|
rm -f /mnt/state/etc/zfs/zpool.cache
|
||||||
|
touch /mnt/state/etc/zfs/zpool.cache
|
||||||
|
chmod a-w /mnt/state/etc/zfs/zpool.cache
|
||||||
|
chattr +i /mnt/state/etc/zfs/zpool.cache
|
||||||
|
|
||||||
|
#. Generate initial system configuration::
|
||||||
|
|
||||||
|
nixos-generate-config --root /mnt
|
||||||
|
|
||||||
|
#. Import ZFS-specific configuration::
|
||||||
|
|
||||||
|
sed -i "s|./hardware-configuration.nix|./hardware-configuration.nix ./zfs.nix|g" /mnt/etc/nixos/configuration.nix
|
||||||
|
|
||||||
|
#. Configure hostid::
|
||||||
|
|
||||||
|
tee -a /mnt/etc/nixos/zfs.nix <<EOF
|
||||||
|
{ config, pkgs, ... }:
|
||||||
|
|
||||||
|
{ boot.supportedFilesystems = [ "zfs" ];
|
||||||
|
networking.hostId = "$(head -c 8 /etc/machine-id)";
|
||||||
|
boot.kernelPackages = config.boot.zfs.package.latestCompatibleLinuxPackages;
|
||||||
|
EOF
|
||||||
|
|
||||||
|
#. Configure bootloader for both legacy boot and UEFI boot and mirror bootloader::
|
||||||
|
|
||||||
|
sed -i '/boot.loader/d' /mnt/etc/nixos/configuration.nix
|
||||||
|
sed -i '/services.xserver/d' /mnt/etc/nixos/configuration.nix
|
||||||
|
tee -a /mnt/etc/nixos/zfs.nix <<-'EOF'
|
||||||
|
boot.loader.efi.efiSysMountPoint = "/boot/efi";
|
||||||
|
boot.loader.efi.canTouchEfiVariables = false;
|
||||||
|
boot.loader.generationsDir.copyKernels = true;
|
||||||
|
boot.loader.grub.efiInstallAsRemovable = true;
|
||||||
|
boot.loader.grub.enable = true;
|
||||||
|
boot.loader.grub.version = 2;
|
||||||
|
boot.loader.grub.copyKernels = true;
|
||||||
|
boot.loader.grub.efiSupport = true;
|
||||||
|
boot.loader.grub.zfsSupport = true;
|
||||||
|
boot.loader.grub.extraPrepareConfig = ''
|
||||||
|
mkdir -p /boot/efis
|
||||||
|
for i in /boot/efis/*; do mount $i ; done
|
||||||
|
|
||||||
|
mkdir -p /boot/efi
|
||||||
|
mount /boot/efi
|
||||||
|
'';
|
||||||
|
boot.loader.grub.extraInstallCommands = ''
|
||||||
|
ESP_MIRROR=$(mktemp -d)
|
||||||
|
cp -r /boot/efi/EFI $ESP_MIRROR
|
||||||
|
for i in /boot/efis/*; do
|
||||||
|
cp -r $ESP_MIRROR/EFI $i
|
||||||
|
done
|
||||||
|
rm -rf $ESP_MIRROR
|
||||||
|
'';
|
||||||
|
boot.loader.grub.devices = [
|
||||||
|
EOF
|
||||||
|
|
||||||
|
for i in $DISK; do
|
||||||
|
printf " \"$i\"\n" >>/mnt/etc/nixos/zfs.nix
|
||||||
|
done
|
||||||
|
|
||||||
|
tee -a /mnt/etc/nixos/zfs.nix <<EOF
|
||||||
|
];
|
||||||
|
EOF
|
||||||
|
|
||||||
|
#. Mount datasets with zfsutil option::
|
||||||
|
|
||||||
|
sed -i 's|fsType = "zfs";|fsType = "zfs"; options = [ "zfsutil" "X-mount.mkdir" ];|g' \
|
||||||
|
/mnt/etc/nixos/hardware-configuration.nix
|
||||||
|
|
||||||
|
#. Set root password::
|
||||||
|
|
||||||
|
rootPwd=$(mkpasswd -m SHA-512 -s)
|
||||||
|
|
||||||
|
Declare password in configuration::
|
||||||
|
|
||||||
|
tee -a /mnt/etc/nixos/zfs.nix <<EOF
|
||||||
|
users.users.root.initialHashedPassword = "${rootPwd}";
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
|
||||||
|
#. Install system and apply configuration::
|
||||||
|
|
||||||
|
nixos-install -v --show-trace --no-root-passwd --root /mnt
|
||||||
|
|
||||||
|
#. Unmount filesystems::
|
||||||
|
|
||||||
|
umount -Rl /mnt
|
||||||
|
zpool export -a
|
||||||
|
|
||||||
|
#. Reboot::
|
||||||
|
|
||||||
|
reboot
|
||||||
@@ -1,235 +0,0 @@
|
|||||||
.. highlight:: sh
|
|
||||||
|
|
||||||
System Installation
|
|
||||||
======================
|
|
||||||
|
|
||||||
.. contents:: Table of Contents
|
|
||||||
:local:
|
|
||||||
|
|
||||||
|
|
||||||
Additional configuration
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
As NixOS configuration is declarative, post-installation tasks,
|
|
||||||
such as user accounts and package selection, can all be done by
|
|
||||||
specifing them in configuration. See `NixOS manual <https://nixos.org/nixos/manual/>`__
|
|
||||||
for details.
|
|
||||||
|
|
||||||
For timezone, hostname, networking, keyboard layout, etc,
|
|
||||||
see ``/mnt/etc/nixos/configuration.nix``.
|
|
||||||
|
|
||||||
Set root password
|
|
||||||
-----------------
|
|
||||||
|
|
||||||
This optional step is an example
|
|
||||||
of declaratively configuring the system.
|
|
||||||
|
|
||||||
#. Generate password hash::
|
|
||||||
|
|
||||||
INST_ROOT_PASSWD=$(mkpasswd -m SHA-512 -s)
|
|
||||||
|
|
||||||
#. Declare `initialHashedPassword
|
|
||||||
<https://nixos.org/manual/nixos/stable/options.html#opt-users.users._name_.initialHashedPassword>`__
|
|
||||||
for root user::
|
|
||||||
|
|
||||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
|
||||||
users.users.root.initialHashedPassword = "${INST_ROOT_PASSWD}";
|
|
||||||
EOF
|
|
||||||
|
|
||||||
System installation
|
|
||||||
~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
#. Finalize the config file::
|
|
||||||
|
|
||||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
|
||||||
}
|
|
||||||
EOF
|
|
||||||
|
|
||||||
#. Take a snapshot of the clean installation, without state
|
|
||||||
for future use::
|
|
||||||
|
|
||||||
zfs snapshot -r rpool_$INST_UUID/$INST_ID@install_start
|
|
||||||
zfs snapshot -r bpool_$INST_UUID/$INST_ID@install_start
|
|
||||||
|
|
||||||
#. Back up bash inputs::
|
|
||||||
|
|
||||||
history -w /mnt/home/sys-install-pre-chroot.txt
|
|
||||||
|
|
||||||
#. Apply configuration
|
|
||||||
|
|
||||||
If root password hash is not set::
|
|
||||||
|
|
||||||
nixos-install -v --show-trace --root /mnt
|
|
||||||
|
|
||||||
You will be prompted for a new root password.
|
|
||||||
|
|
||||||
If password hash has been set::
|
|
||||||
|
|
||||||
nixos-install -v --show-trace --no-root-passwd --root /mnt
|
|
||||||
|
|
||||||
#. If boot pool encryption is used and installation fails with::
|
|
||||||
|
|
||||||
#mktemp: failed to create directory via template
|
|
||||||
#‘/mnt/tmp.coRUoqzl1P/initrd-secrets.XXXXXXXXXX’: No such file or directory
|
|
||||||
#failed to create initrd secrets: No such file or directory
|
|
||||||
|
|
||||||
This is `a bug <https://github.com/NixOS/nixpkgs/issues/157989>`__.
|
|
||||||
Complete the installation by executing::
|
|
||||||
|
|
||||||
nixos-enter --root /mnt -- nixos-rebuild boot
|
|
||||||
|
|
||||||
Finish installation
|
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
#. Take a snapshot of the clean installation for future use::
|
|
||||||
|
|
||||||
zfs snapshot -r rpool_$INST_UUID/$INST_ID@install
|
|
||||||
zfs snapshot -r bpool_$INST_UUID/$INST_ID@install
|
|
||||||
|
|
||||||
#. Unmount EFI system partition::
|
|
||||||
|
|
||||||
umount /mnt/boot/efis/*
|
|
||||||
|
|
||||||
#. Export pools::
|
|
||||||
|
|
||||||
zpool export bpool_$INST_UUID
|
|
||||||
zpool export rpool_$INST_UUID
|
|
||||||
|
|
||||||
#. Reboot::
|
|
||||||
|
|
||||||
reboot
|
|
||||||
|
|
||||||
Upgrade NixOS
|
|
||||||
~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
Routine updates within the same major version
|
|
||||||
=============================================
|
|
||||||
|
|
||||||
Updates within the same major version, such as from [21.11].001 to
|
|
||||||
[21.11].100, can be done with one of the following commands::
|
|
||||||
|
|
||||||
# take immediate effect
|
|
||||||
nixos-rebuild --upgrade switch
|
|
||||||
|
|
||||||
# update upon reboot
|
|
||||||
nixos-rebuild --upgrade boot
|
|
||||||
|
|
||||||
Upgrade to a newer major version
|
|
||||||
================================
|
|
||||||
|
|
||||||
Upgrading to a newer major version involves switching software
|
|
||||||
distribution channel.
|
|
||||||
|
|
||||||
#. View existing channels, run as root::
|
|
||||||
|
|
||||||
nix-channel --list
|
|
||||||
#nixos https://nixos.org/channels/nixos-21.11
|
|
||||||
|
|
||||||
#this is the major version released around November 2021
|
|
||||||
|
|
||||||
#. View available channels::
|
|
||||||
|
|
||||||
w3m https://hydra.nixos.org/project/nixos
|
|
||||||
|
|
||||||
#. Switch to a newer channel (22.05)::
|
|
||||||
|
|
||||||
nix-channel --add https://nixos.org/channels/nixos-22.05 nixos
|
|
||||||
|
|
||||||
#. In ``/etc/nixos/configuration.nix``::
|
|
||||||
|
|
||||||
system.stateVersion = "22.05";
|
|
||||||
|
|
||||||
If using Home Manager, in ``~/.config/nixpkgs/home.nix``::
|
|
||||||
|
|
||||||
home.stateVersion = "22.05";
|
|
||||||
|
|
||||||
#. Then follow the procedures for updating witin minor versions.
|
|
||||||
|
|
||||||
Immutable root file system
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
This section is optional.
|
|
||||||
|
|
||||||
Often, programs generate mutable files in paths such as
|
|
||||||
``/etc`` and ``/var/lib``. The generated files can be considered a
|
|
||||||
part of the system state.
|
|
||||||
|
|
||||||
This generated state is not declaratively managed
|
|
||||||
by NixOS and can not be reproduced from NixOS configuration.
|
|
||||||
|
|
||||||
To ensure that the system state is fully managed by NixOS and reproducible,
|
|
||||||
we need to periodically purge the system state and let NixOS
|
|
||||||
regenerate root file system from scratch.
|
|
||||||
|
|
||||||
Also see: `Erase your darlings:
|
|
||||||
immutable infrastructure for mutable systems <https://grahamc.com/blog/erase-your-darlings>`__.
|
|
||||||
|
|
||||||
Save mutable data to alternative path
|
|
||||||
-------------------------------------
|
|
||||||
|
|
||||||
Before enabling purging on root dataset, we need to back up
|
|
||||||
essential mutable data first, such as host SSH key and network connections.
|
|
||||||
Below are some tips.
|
|
||||||
|
|
||||||
- Some programs support specifying another
|
|
||||||
location for mutable data, such as
|
|
||||||
Wireguard::
|
|
||||||
|
|
||||||
networking.wireguard.interfaces.wg0.privateKeyFile = "/state/etc/wireguard/wg0";
|
|
||||||
|
|
||||||
- For programs without a configurable data path,
|
|
||||||
`environment.etc <https://nixos.org/manual/nixos/stable/options.html#opt-environment.etc>`__
|
|
||||||
may be used::
|
|
||||||
|
|
||||||
environment.etc = {
|
|
||||||
"ssh/ssh_host_rsa_key".source = "/state/etc/ssh/ssh_host_rsa_key";
|
|
||||||
}
|
|
||||||
|
|
||||||
- systemd’s tmpfiles.d rules are also an option::
|
|
||||||
|
|
||||||
systemd.tmpfiles.rules = [
|
|
||||||
"L /var/lib/bluetooth - - - - /state/var/lib/bluetooth"
|
|
||||||
];
|
|
||||||
|
|
||||||
- Bind mount::
|
|
||||||
|
|
||||||
for i in {/etc/nixos,/etc/cryptkey.d}; do
|
|
||||||
mkdir -p /state/$i /$i
|
|
||||||
mount -o bind /state/$i /$i
|
|
||||||
done
|
|
||||||
nixos-generate-config --show-hardware-config
|
|
||||||
|
|
||||||
Boot from empty root file system
|
|
||||||
--------------------------------
|
|
||||||
|
|
||||||
After backing up mutable data, you can try switching to
|
|
||||||
an empty dataset as root file system.
|
|
||||||
|
|
||||||
#. Check current root file system::
|
|
||||||
|
|
||||||
ROOT_FS=$(df --output=source /|tail -n1)
|
|
||||||
# rpool/ROOT/default
|
|
||||||
|
|
||||||
#. Set empty file system as root::
|
|
||||||
|
|
||||||
sed -i "s,${ROOT_FS},${ROOT_FS%/*}/empty,g" /etc/nixos/hardware-configuration-zfs.nix
|
|
||||||
|
|
||||||
#. Apply changes and reboot::
|
|
||||||
|
|
||||||
nixos-rebuild boot
|
|
||||||
reboot
|
|
||||||
|
|
||||||
#. If everything went fine, add the output of the following command to configuration::
|
|
||||||
|
|
||||||
ROOT_FS=$(df --output=source /|tail -n1)
|
|
||||||
cat <<EOF
|
|
||||||
boot.initrd.postDeviceCommands = ''
|
|
||||||
zpool import -Nf ${ROOT_FS%%/*}
|
|
||||||
zfs rollback -r ${ROOT_FS%/*}/empty@start
|
|
||||||
'';
|
|
||||||
EOF
|
|
||||||
|
|
||||||
#. Apply and reboot::
|
|
||||||
|
|
||||||
nixos-rebuild boot
|
|
||||||
reboot
|
|
||||||
@@ -1,145 +0,0 @@
|
|||||||
.. highlight:: sh
|
|
||||||
|
|
||||||
Recovery
|
|
||||||
======================
|
|
||||||
|
|
||||||
.. contents:: Table of Contents
|
|
||||||
:local:
|
|
||||||
|
|
||||||
GRUB Tips
|
|
||||||
-------------
|
|
||||||
|
|
||||||
Boot from GRUB rescue
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
If bootloader file is damaged, it's still possible
|
|
||||||
to boot computer with GRUB rescue image.
|
|
||||||
|
|
||||||
This section is also applicable if you are in
|
|
||||||
``grub rescue>``.
|
|
||||||
|
|
||||||
#. On another computer, generate rescue image with::
|
|
||||||
|
|
||||||
pacman -S --needed mtools libisoburn grub
|
|
||||||
grub-install
|
|
||||||
grub-mkrescue -o grub-rescue.img
|
|
||||||
dd if=grub-rescue.img of=/dev/your-usb-stick
|
|
||||||
|
|
||||||
Boot computer from the rescue media.
|
|
||||||
Both legacy and EFI mode are supported.
|
|
||||||
|
|
||||||
Or `download generated GRUB rescue image <https://gitlab.com/m_zhou/bieaz/uploads/e0847a8675cda4317ea7f48abb1d9f10/grub-rescue-2.06.img.7z>`__.
|
|
||||||
|
|
||||||
#. List available disks with ``ls`` command::
|
|
||||||
|
|
||||||
grub> ls (hd # press tab
|
|
||||||
Possible devices are:
|
|
||||||
|
|
||||||
hd0 hd1 hd2 hd3
|
|
||||||
|
|
||||||
#. List partitions by pressing tab key:
|
|
||||||
|
|
||||||
.. code-block:: text
|
|
||||||
|
|
||||||
grub> ls (hd0 # press tab
|
|
||||||
Possible partitions are:
|
|
||||||
|
|
||||||
Device hd0: No known filesystem detected - Sector size 512B - Total size 20971520KiB
|
|
||||||
Partition hd0,gpt1: Filesystem type fat - Label `EFI', UUID 0DF5-3A76 - Partition start at 1024KiB - Total size 1048576KiB
|
|
||||||
Partition hd0,gpt2: No known filesystem detected - Partition start at 1049600KiB - Total size 4194304KiB
|
|
||||||
|
|
||||||
- If boot pool is encrypted:
|
|
||||||
|
|
||||||
Unlock it with ``cryptomount``::
|
|
||||||
|
|
||||||
grub> insmod luks
|
|
||||||
grub> cryptomount hd0,gpt2
|
|
||||||
Attempting to decrypt master key...
|
|
||||||
Enter passphrase for hd0,gpt2 (af5a240e13e24483acf02600d61e0f36):
|
|
||||||
Slot 1 opened
|
|
||||||
|
|
||||||
Unlocked LUKS container is ``(crypto0)``:
|
|
||||||
|
|
||||||
.. code-block:: text
|
|
||||||
|
|
||||||
grub> ls (crypto0)
|
|
||||||
Device crypto0: Filesystem type zfs - Label `bpool_ip3tdb' - Last modification
|
|
||||||
time 2021-05-03 12:14:08 Monday, UUID f14d7bdf89fe21fb - Sector size 512B -
|
|
||||||
Total size 4192256KiB
|
|
||||||
|
|
||||||
- If boot pool is not encrypted:
|
|
||||||
|
|
||||||
.. code-block:: text
|
|
||||||
|
|
||||||
grub> ls (hd0,gpt2)
|
|
||||||
Device hd0,gpt2: Filesystem type zfs - Label `bpool_ip3tdb' - Last modification
|
|
||||||
time 2021-05-03 12:14:08 Monday, UUID f14d7bdf89fe21fb - Sector size 512B -
|
|
||||||
Total size 4192256KiB
|
|
||||||
|
|
||||||
#. List boot environments nested inside ``bpool/$INST_ID/BOOT``::
|
|
||||||
|
|
||||||
grub> ls (crypto0)/sys/BOOT
|
|
||||||
@/ default/ be0/
|
|
||||||
|
|
||||||
#. Instruct GRUB to load configuration from ``be0`` boot environment::
|
|
||||||
|
|
||||||
grub> prefix=(crypto0)/sys/BOOT/be0/@/grub
|
|
||||||
grub> configfile $prefix/grub.cfg
|
|
||||||
|
|
||||||
#. GRUB menu should now appear.
|
|
||||||
|
|
||||||
#. After entering system, reinstall GRUB.
|
|
||||||
|
|
||||||
Switch GRUB prefix when disk fails
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
If you are using LUKS encrypted boot pool with multiple disks,
|
|
||||||
the primary disk failed, GRUB will fail to load configuration.
|
|
||||||
|
|
||||||
If there's still enough redundancy for the boot pool, try fix
|
|
||||||
GRUB with the following method:
|
|
||||||
|
|
||||||
#. Ensure ``Slot 1 opened`` message
|
|
||||||
is shown
|
|
||||||
|
|
||||||
.. code-block:: text
|
|
||||||
|
|
||||||
Welcome to GRUB!
|
|
||||||
|
|
||||||
error: no such cryptodisk found.
|
|
||||||
Attempting to decrypt master key...
|
|
||||||
Enter passphrase for hd0,gpt2 (c0987ea1a51049e9b3056622804de62a):
|
|
||||||
Slot 1 opened
|
|
||||||
error: disk `cryptouuid/47ed1b7eb0014bc9a70aede3d8714faf' not found.
|
|
||||||
Entering rescue mode...
|
|
||||||
grub rescue>
|
|
||||||
|
|
||||||
If ``error: access denied.`` is shown,
|
|
||||||
try re-enter password with::
|
|
||||||
|
|
||||||
grub rescue> cryptomount hd0,gpt2
|
|
||||||
|
|
||||||
#. Check prefix::
|
|
||||||
|
|
||||||
grub rescue > set
|
|
||||||
# prefix=(cryptouuid/47ed1b7eb0014bc9a70aede3d8714faf)/sys/BOOT/be0@/grub
|
|
||||||
# root=cryptouuid/47ed1b7eb0014bc9a70aede3d8714faf
|
|
||||||
|
|
||||||
#. Set correct ``prefix`` and ``root`` by replacing
|
|
||||||
``cryptouuid/UUID`` with ``crypto0``::
|
|
||||||
|
|
||||||
grub rescue> prefix=(crypto0)/sys/BOOT/default@/grub
|
|
||||||
grub rescue> root=crypto0
|
|
||||||
|
|
||||||
#. Boot GRUB::
|
|
||||||
|
|
||||||
grub rescue> insmod normal
|
|
||||||
grub rescue> normal
|
|
||||||
|
|
||||||
GRUB should then boot normally.
|
|
||||||
|
|
||||||
#. After entering system, edit ``/etc/fstab`` to promote
|
|
||||||
one backup to ``/boot/efi``.
|
|
||||||
|
|
||||||
#. Make the change to ``prefix`` and ``root``
|
|
||||||
permanent by `reinstalling GRUB <#grub-installation>`__.
|
|
||||||
@@ -55,7 +55,7 @@ Root on ZFS
|
|||||||
ZFS can be used as root file system for NixOS.
|
ZFS can be used as root file system for NixOS.
|
||||||
An installation guide is available.
|
An installation guide is available.
|
||||||
|
|
||||||
`Start here <Root%20on%20ZFS/0-overview.html>`__.
|
`Start here <Root%20on%20ZFS/1-preparation.html>`__.
|
||||||
|
|
||||||
.. toctree::
|
.. toctree::
|
||||||
:maxdepth: 1
|
:maxdepth: 1
|
||||||
|
|||||||
@@ -1,11 +0,0 @@
|
|||||||
Rocky Linux 8 Root on ZFS
|
|
||||||
=======================================
|
|
||||||
`Start here <RHEL%208-based%20distro%20Root%20on%20ZFS/0-overview.html>`__.
|
|
||||||
|
|
||||||
Contents
|
|
||||||
--------
|
|
||||||
.. toctree::
|
|
||||||
:maxdepth: 2
|
|
||||||
:glob:
|
|
||||||
|
|
||||||
RHEL 8-based distro Root on ZFS/*
|
|
||||||
@@ -1,141 +0,0 @@
|
|||||||
.. highlight:: sh
|
|
||||||
|
|
||||||
Overview
|
|
||||||
======================
|
|
||||||
This document describes how to install RHEL 8-based distro with ZFS as root
|
|
||||||
file system.
|
|
||||||
|
|
||||||
Caution
|
|
||||||
~~~~~~~
|
|
||||||
- With less than 4GB RAM, DKMS might fail to build
|
|
||||||
in live environment.
|
|
||||||
- This guide wipes entire physical disks. Back up existing data.
|
|
||||||
- `GRUB does not and
|
|
||||||
will not work on 4Kn drive with legacy (BIOS) booting.
|
|
||||||
<http://savannah.gnu.org/bugs/?46700>`__
|
|
||||||
|
|
||||||
Partition layout
|
|
||||||
~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
GUID partition table (GPT) is used.
|
|
||||||
EFI system partition will be referred to as **ESP** in this document.
|
|
||||||
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
| Name | legacy boot | ESP | Boot pool | swap | root pool | remaining space |
|
|
||||||
+======================+======================+=======================+======================+=====================+=======================+=================+
|
|
||||||
| File system | | vfat | ZFS | swap | ZFS | |
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
| Size | 1M | 2G | 4G | depends on RAM size | | |
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
| Optional encryption | | *Secure Boot* | | plain dm-crypt | ZFS native encryption | |
|
|
||||||
| | | | | | | |
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
| Partition no. | 5 | 1 | 2 | 4 | 3 | |
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
| Mount point | | /boot/efi | /boot | | / | |
|
|
||||||
| | | /boot/efis/disk-part1 | | | | |
|
|
||||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
|
||||||
|
|
||||||
Dataset layout
|
|
||||||
~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| Dataset | canmount | mountpoint | container | notes |
|
|
||||||
+===========================+======================+======================+=====================================+===========================================+
|
|
||||||
| bpool | off | /boot | contains sys | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool | off | / | contains sys | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| bpool/sys | off | none | contains BOOT | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys | off | none | contains ROOT | sys is encryptionroot |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| bpool/sys/BOOT | off | none | contains boot environments | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/ROOT | off | none | contains boot environments | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/DATA | off | none | contains placeholder "default" | |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/DATA/default | off | / | contains user datasets | child datsets inherits mountpoint |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/DATA/default/ | on | /home (inherited) | no | |
|
|
||||||
| home | | | | user datasets, also called "shared |
|
|
||||||
| | | | | datasets", "persistent datasets"; also |
|
|
||||||
| | | | | include /var/lib, /srv, ... |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| bpool/sys/BOOT/default | noauto | /boot | no | noauto is used to switch BE. because of |
|
|
||||||
| | | | | noauto, must use fstab to mount |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/ROOT/default | noauto | / | no | mounted by initrd zfs hook |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| bpool/sys/BOOT/be1 | noauto | /boot | no | see bpool/sys/BOOT/default |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
| rpool/sys/ROOT/be1 | noauto | / | no | see rpool/sys/ROOT/default |
|
|
||||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
|
||||||
|
|
||||||
Encryption
|
|
||||||
~~~~~~~~~~
|
|
||||||
|
|
||||||
- Swap
|
|
||||||
|
|
||||||
Swap is always encrypted. By default, swap is encrypted
|
|
||||||
with plain dm-crypt with key generated from ``/dev/urandom``
|
|
||||||
at every boot. Swap content does not persist between reboots.
|
|
||||||
|
|
||||||
- Root pool
|
|
||||||
|
|
||||||
ZFS native encryption can be optionally enabled for ``rpool/sys``
|
|
||||||
and child datasets.
|
|
||||||
|
|
||||||
User should be aware that, ZFS native encryption does not
|
|
||||||
encrypt some metadata of the datasets.
|
|
||||||
ZFS native encryption also does not change master key when ``zfs change-key`` is invoked.
|
|
||||||
Therefore, you should wipe the disk when password is compromised to protect confidentiality.
|
|
||||||
See `zfs-load-key.8 <https://openzfs.github.io/openzfs-docs/man/8/zfs-load-key.8.html>`__
|
|
||||||
and `zfs-change-key.8 <https://openzfs.github.io/openzfs-docs/man/8/zfs-change-key.8.html>`__
|
|
||||||
for more information regarding ZFS native encryption.
|
|
||||||
|
|
||||||
Encryption is enabled at dataset creation and can not be disabled later.
|
|
||||||
|
|
||||||
- Boot pool
|
|
||||||
|
|
||||||
Boot pool can not be encrypted.
|
|
||||||
|
|
||||||
- Bootloader
|
|
||||||
|
|
||||||
Bootloader can not be encrypted.
|
|
||||||
|
|
||||||
However, with Secure Boot, bootloader
|
|
||||||
can be verified by motherboard firmware to be untempered,
|
|
||||||
which should be sufficient for most purposes.
|
|
||||||
|
|
||||||
Secure Boot is not supported out-of-the-box due to ZFS module.
|
|
||||||
|
|
||||||
Booting with disk failure
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
This guide is written with disk failure in mind.
|
|
||||||
|
|
||||||
If disks used in Root on ZFS pool failed, but
|
|
||||||
sufficient redundancy for both root pool and boot pool
|
|
||||||
still exists, the system will still boot normally.
|
|
||||||
|
|
||||||
Swap partition on the failed disk will fail to mount,
|
|
||||||
after an 1m30s timeout.
|
|
||||||
|
|
||||||
This feature is useful for use cases such
|
|
||||||
as an unattended remote server.
|
|
||||||
|
|
||||||
Example:
|
|
||||||
|
|
||||||
- System has disks ``n>1``
|
|
||||||
|
|
||||||
- Installed with mirrored setup
|
|
||||||
|
|
||||||
- Mirrored setup can tolerate up to ``n-1`` disk failures
|
|
||||||
|
|
||||||
- Disconnect one or more disks, keep at least
|
|
||||||
one disk connected
|
|
||||||
|
|
||||||
- System still boots, but fails to mount swap and
|
|
||||||
EFI partition
|
|
||||||
@@ -1,149 +0,0 @@
|
|||||||
.. highlight:: sh
|
|
||||||
|
|
||||||
Preparation
|
|
||||||
======================
|
|
||||||
|
|
||||||
.. contents:: Table of Contents
|
|
||||||
:local:
|
|
||||||
|
|
||||||
#. Disable Secure Boot. ZFS modules can not be loaded if Secure Boot is enabled.
|
|
||||||
#. Download a variant of `Rocky Linux 8.5 Live
|
|
||||||
ISO <https://dl.rockylinux.org/pub/rocky/8.5/Live/x86_64/>`__ and boot from it.
|
|
||||||
|
|
||||||
#. Set root password or ``/root/.ssh/authorized_keys``.
|
|
||||||
#. Start SSH server::
|
|
||||||
|
|
||||||
echo PermitRootLogin yes >> /etc/ssh/sshd_config
|
|
||||||
systemctl restart sshd
|
|
||||||
|
|
||||||
#. Connect from another computer::
|
|
||||||
|
|
||||||
ssh root@192.168.1.19
|
|
||||||
|
|
||||||
#. Temporarily set SELinux to permissive in live environment::
|
|
||||||
|
|
||||||
setenforce 0
|
|
||||||
|
|
||||||
SELinux will be enabled on the installed system.
|
|
||||||
|
|
||||||
#. Optional: If mirror speed is slow, you can manually pick a fixed mirror
|
|
||||||
from `mirrorlist <https://mirrors.rockylinux.org/mirrormanager/mirrors>`__
|
|
||||||
and apply it::
|
|
||||||
|
|
||||||
sed -i 's|^mirrorlist=|#mirrorlist=|g' /etc/yum.repos.d/*
|
|
||||||
sed -i 's|^#baseurl=|baseurl=|g' /etc/yum.repos.d/*
|
|
||||||
sed -i 's|dl.rockylinux.org/$contentdir|mirrors.sjtug.sjtu.edu.cn/rocky|g' /etc/yum.repos.d/*
|
|
||||||
|
|
||||||
#. Add ZFS repo::
|
|
||||||
|
|
||||||
source /etc/os-release
|
|
||||||
RHEL_ZFS_REPO=https://zfsonlinux.org/epel/zfs-release.el${VERSION_ID/./_}.noarch.rpm
|
|
||||||
dnf install -y $RHEL_ZFS_REPO
|
|
||||||
|
|
||||||
#. Install ZFS packages::
|
|
||||||
|
|
||||||
dnf install -y epel-release
|
|
||||||
dnf config-manager --disable zfs
|
|
||||||
dnf config-manager --enable zfs-kmod
|
|
||||||
dnf install -y zfs
|
|
||||||
|
|
||||||
#. Load kernel modules::
|
|
||||||
|
|
||||||
modprobe zfs
|
|
||||||
|
|
||||||
#. Install helper script and partition tool::
|
|
||||||
|
|
||||||
rpm -ivh --nodeps https://dl.fedoraproject.org/pub/fedora/linux/releases/36/Everything/x86_64/os/Packages/a/arch-install-scripts-24-3.fc36.noarch.rpm
|
|
||||||
dnf install -y gdisk dosfstools
|
|
||||||
|
|
||||||
#. Set RHEL major version::
|
|
||||||
|
|
||||||
INST_RHEL_VER=${VERSION_ID%%.*}
|
|
||||||
|
|
||||||
#. Unique pool suffix. ZFS expects pool names to be
|
|
||||||
unique, therefore it's recommended to create
|
|
||||||
pools with a unique suffix::
|
|
||||||
|
|
||||||
INST_UUID=$(dd if=/dev/urandom bs=1 count=100 2>/dev/null | tr -dc 'a-z0-9' | cut -c-6)
|
|
||||||
|
|
||||||
#. Identify this installation in ZFS filesystem path::
|
|
||||||
|
|
||||||
INST_ID=rhel
|
|
||||||
|
|
||||||
#. Target disk
|
|
||||||
|
|
||||||
List available disks with::
|
|
||||||
|
|
||||||
ls /dev/disk/by-id/*
|
|
||||||
|
|
||||||
If using virtio as disk bus, use
|
|
||||||
``/dev/disk/by-path/*``.
|
|
||||||
|
|
||||||
Declare disk array::
|
|
||||||
|
|
||||||
DISK='/dev/disk/by-id/ata-FOO /dev/disk/by-id/nvme-BAR'
|
|
||||||
|
|
||||||
For single disk installation, use::
|
|
||||||
|
|
||||||
DISK='/dev/disk/by-id/disk1'
|
|
||||||
|
|
||||||
#. Choose a primary disk. This disk will be used
|
|
||||||
for primary EFI partition, default to
|
|
||||||
first disk in the array::
|
|
||||||
|
|
||||||
INST_PRIMARY_DISK=$(echo $DISK | cut -f1 -d\ )
|
|
||||||
|
|
||||||
#. Set vdev topology, possible values are:
|
|
||||||
|
|
||||||
- (not set, single disk or striped; no redundancy)
|
|
||||||
- mirror
|
|
||||||
- raidz1
|
|
||||||
- raidz2
|
|
||||||
- raidz3
|
|
||||||
|
|
||||||
::
|
|
||||||
|
|
||||||
INST_VDEV=
|
|
||||||
|
|
||||||
This will create a single vdev with the topology of your choice.
|
|
||||||
It is also possible to manually create a pool with multiple vdevs, such as::
|
|
||||||
|
|
||||||
zpool create --options \
|
|
||||||
poolName \
|
|
||||||
mirror sda sdb \
|
|
||||||
raidz2 sdc ... \
|
|
||||||
raidz3 sde ... \
|
|
||||||
spare sdf ...
|
|
||||||
|
|
||||||
Notice the cost of parity when using RAID-Z. See
|
|
||||||
`here <https://www.delphix.com/blog/delphix-engineering/zfs-raidz-stripe-width-or-how-i-learned-stop-worrying-and-love-raidz>`__
|
|
||||||
and `here <https://docs.google.com/spreadsheets/d/1tf4qx1aMJp8Lo_R6gpT689wTjHv6CGVElrPqTA0w_ZY/>`__.
|
|
||||||
|
|
||||||
For boot pool, which must be readable by GRUB, mirrored vdev should always be used for maximum redundancy.
|
|
||||||
This guide will use mirrored bpool for multi-disk setup.
|
|
||||||
|
|
||||||
Refer to `zpoolconcepts <https://openzfs.github.io/openzfs-docs/man/7/zpoolconcepts.7.html>`__
|
|
||||||
and `zpool-create <https://openzfs.github.io/openzfs-docs/man/8/zpool-create.8.html>`__
|
|
||||||
man pages for details.
|
|
||||||
|
|
||||||
#. Set partition size:
|
|
||||||
|
|
||||||
Set ESP size::
|
|
||||||
|
|
||||||
INST_PARTSIZE_ESP=2 # in GB
|
|
||||||
|
|
||||||
Set boot pool size. To avoid running out of space while using
|
|
||||||
boot environments, the minimum is 4GB. Adjust the size if you
|
|
||||||
intend to use multiple kernel/distros::
|
|
||||||
|
|
||||||
INST_PARTSIZE_BPOOL=4
|
|
||||||
|
|
||||||
Set swap size. It's `recommended <https://chrisdown.name/2018/01/02/in-defence-of-swap.html>`__
|
|
||||||
to setup a swap partition. If you intend to use hibernation,
|
|
||||||
the minimum should be no less than RAM size. Skip if swap is not needed::
|
|
||||||
|
|
||||||
INST_PARTSIZE_SWAP=8
|
|
||||||
|
|
||||||
Root pool size, use all remaining disk space if not set::
|
|
||||||
|
|
||||||
INST_PARTSIZE_RPOOL=
|
|
||||||
@@ -1,271 +0,0 @@
|
|||||||
.. highlight:: sh
|
|
||||||
|
|
||||||
System Installation
|
|
||||||
======================
|
|
||||||
|
|
||||||
.. contents:: Table of Contents
|
|
||||||
:local:
|
|
||||||
|
|
||||||
#. Optional: wipe solid-state drives with the generic tool
|
|
||||||
`blkdiscard <https://utcc.utoronto.ca/~cks/space/blog/linux/ErasingSSDsWithBlkdiscard>`__,
|
|
||||||
to clean previous partition tables and improve performance.
|
|
||||||
|
|
||||||
All content will be irrevocably destroyed::
|
|
||||||
|
|
||||||
for i in ${DISK}; do
|
|
||||||
blkdiscard $i &
|
|
||||||
done
|
|
||||||
wait
|
|
||||||
|
|
||||||
This is a quick operation and should be completed under one
|
|
||||||
minute.
|
|
||||||
|
|
||||||
For other device specific methods, see
|
|
||||||
`Memory cell clearing <https://wiki.archlinux.org/title/Solid_state_drive/Memory_cell_clearing>`__
|
|
||||||
|
|
||||||
#. Partition the disks.
|
|
||||||
See `Overview <0-overview.html>`__ for details::
|
|
||||||
|
|
||||||
for i in ${DISK}; do
|
|
||||||
sgdisk --zap-all $i
|
|
||||||
sgdisk -n1:1M:+${INST_PARTSIZE_ESP}G -t1:EF00 $i
|
|
||||||
sgdisk -n2:0:+${INST_PARTSIZE_BPOOL}G -t2:BE00 $i
|
|
||||||
if [ "${INST_PARTSIZE_SWAP}" != "" ]; then
|
|
||||||
sgdisk -n4:0:+${INST_PARTSIZE_SWAP}G -t4:8200 $i
|
|
||||||
fi
|
|
||||||
if [ "${INST_PARTSIZE_RPOOL}" = "" ]; then
|
|
||||||
sgdisk -n3:0:0 -t3:BF00 $i
|
|
||||||
else
|
|
||||||
sgdisk -n3:0:+${INST_PARTSIZE_RPOOL}G -t3:BF00 $i
|
|
||||||
fi
|
|
||||||
sgdisk -a1 -n5:24K:+1000K -t5:EF02 $i
|
|
||||||
done
|
|
||||||
|
|
||||||
#. Create boot pool::
|
|
||||||
|
|
||||||
|
|
||||||
disk_num=0; for i in $DISK; do disk_num=$(( $disk_num + 1 )); done
|
|
||||||
if [ $disk_num -gt 1 ]; then INST_VDEV_BPOOL=mirror; fi
|
|
||||||
|
|
||||||
|
|
||||||
zpool create \
|
|
||||||
-d -o feature@async_destroy=enabled \
|
|
||||||
-o feature@bookmarks=enabled \
|
|
||||||
-o feature@embedded_data=enabled \
|
|
||||||
-o feature@empty_bpobj=enabled \
|
|
||||||
-o feature@enabled_txg=enabled \
|
|
||||||
-o feature@extensible_dataset=enabled \
|
|
||||||
-o feature@filesystem_limits=enabled \
|
|
||||||
-o feature@hole_birth=enabled \
|
|
||||||
-o feature@large_blocks=enabled \
|
|
||||||
-o feature@lz4_compress=enabled \
|
|
||||||
-o feature@spacemap_histogram=enabled \
|
|
||||||
-o ashift=12 \
|
|
||||||
-o autotrim=on \
|
|
||||||
-O acltype=posixacl \
|
|
||||||
-O canmount=off \
|
|
||||||
-O compression=lz4 \
|
|
||||||
-O devices=off \
|
|
||||||
-O normalization=formD \
|
|
||||||
-O relatime=on \
|
|
||||||
-O xattr=sa \
|
|
||||||
-O mountpoint=/boot \
|
|
||||||
-R /mnt \
|
|
||||||
bpool_$INST_UUID \
|
|
||||||
$INST_VDEV_BPOOL \
|
|
||||||
$(for i in ${DISK}; do
|
|
||||||
printf "$i-part2 ";
|
|
||||||
done)
|
|
||||||
|
|
||||||
You should not need to customize any of the options for the boot pool.
|
|
||||||
|
|
||||||
GRUB does not support all of the zpool features. See ``spa_feature_names``
|
|
||||||
in `grub-core/fs/zfs/zfs.c
|
|
||||||
<http://git.savannah.gnu.org/cgit/grub.git/tree/grub-core/fs/zfs/zfs.c#n276>`__.
|
|
||||||
This step creates a separate boot pool for ``/boot`` with the features
|
|
||||||
limited to only those that GRUB supports, allowing the root pool to use
|
|
||||||
any/all features.
|
|
||||||
|
|
||||||
Features enabled with ``-o compatibility=grub2`` can be seen
|
|
||||||
`here <https://github.com/openzfs/zfs/blob/master/cmd/zpool/compatibility.d/grub2>`__.
|
|
||||||
|
|
||||||
#. Create root pool::
|
|
||||||
|
|
||||||
zpool create \
|
|
||||||
-o ashift=12 \
|
|
||||||
-o autotrim=on \
|
|
||||||
-R /mnt \
|
|
||||||
-O acltype=posixacl \
|
|
||||||
-O canmount=off \
|
|
||||||
-O compression=zstd \
|
|
||||||
-O dnodesize=auto \
|
|
||||||
-O normalization=formD \
|
|
||||||
-O relatime=on \
|
|
||||||
-O xattr=sa \
|
|
||||||
-O mountpoint=/ \
|
|
||||||
rpool_$INST_UUID \
|
|
||||||
$INST_VDEV \
|
|
||||||
$(for i in ${DISK}; do
|
|
||||||
printf "$i-part3 ";
|
|
||||||
done)
|
|
||||||
|
|
||||||
**Notes:**
|
|
||||||
|
|
||||||
- The use of ``ashift=12`` is recommended here because many drives
|
|
||||||
today have 4 KiB (or larger) physical sectors, even though they
|
|
||||||
present 512 B logical sectors. Also, a future replacement drive may
|
|
||||||
have 4 KiB physical sectors (in which case ``ashift=12`` is desirable)
|
|
||||||
or 4 KiB logical sectors (in which case ``ashift=12`` is required).
|
|
||||||
- Setting ``-O acltype=posixacl`` enables POSIX ACLs globally. If you
|
|
||||||
do not want this, remove that option, but later add
|
|
||||||
``-o acltype=posixacl`` (note: lowercase “o”) to the ``zfs create``
|
|
||||||
for ``/var/log``, as `journald requires ACLs
|
|
||||||
<https://askubuntu.com/questions/970886/journalctl-says-failed-to-search-journal-acl-operation-not-supported>`__
|
|
||||||
- Setting ``normalization=formD`` eliminates some corner cases relating
|
|
||||||
to UTF-8 filename normalization. It also implies ``utf8only=on``,
|
|
||||||
which means that only UTF-8 filenames are allowed. If you care to
|
|
||||||
support non-UTF-8 filenames, do not use this option. For a discussion
|
|
||||||
of why requiring UTF-8 filenames may be a bad idea, see `The problems
|
|
||||||
with enforced UTF-8 only filenames
|
|
||||||
<http://utcc.utoronto.ca/~cks/space/blog/linux/ForcedUTF8Filenames>`__.
|
|
||||||
- ``recordsize`` is unset (leaving it at the default of 128 KiB). If you
|
|
||||||
want to tune it (e.g. ``-o recordsize=1M``), see `these
|
|
||||||
<https://jrs-s.net/2019/04/03/on-zfs-recordsize/>`__ `various
|
|
||||||
<http://blog.programster.org/zfs-record-size>`__ `blog
|
|
||||||
<https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSFileRecordsizeGrowth>`__
|
|
||||||
`posts
|
|
||||||
<https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSRecordsizeAndCompression>`__.
|
|
||||||
- Setting ``relatime=on`` is a middle ground between classic POSIX
|
|
||||||
``atime`` behavior (with its significant performance impact) and
|
|
||||||
``atime=off`` (which provides the best performance by completely
|
|
||||||
disabling atime updates). Since Linux 2.6.30, ``relatime`` has been
|
|
||||||
the default for other filesystems. See `RedHat’s documentation
|
|
||||||
<https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/power_management_guide/relatime>`__
|
|
||||||
for further information.
|
|
||||||
- Setting ``xattr=sa`` `vastly improves the performance of extended
|
|
||||||
attributes
|
|
||||||
<https://github.com/zfsonlinux/zfs/commit/82a37189aac955c81a59a5ecc3400475adb56355>`__.
|
|
||||||
Inside ZFS, extended attributes are used to implement POSIX ACLs.
|
|
||||||
Extended attributes can also be used by user-space applications.
|
|
||||||
`They are used by some desktop GUI applications.
|
|
||||||
<https://en.wikipedia.org/wiki/Extended_file_attributes#Linux>`__
|
|
||||||
`They can be used by Samba to store Windows ACLs and DOS attributes;
|
|
||||||
they are required for a Samba Active Directory domain controller.
|
|
||||||
<https://wiki.samba.org/index.php/Setting_up_a_Share_Using_Windows_ACLs>`__
|
|
||||||
Note that ``xattr=sa`` is `Linux-specific
|
|
||||||
<https://openzfs.org/wiki/Platform_code_differences>`__. If you move your
|
|
||||||
``xattr=sa`` pool to another OpenZFS implementation besides ZFS-on-Linux,
|
|
||||||
extended attributes will not be readable (though your data will be). If
|
|
||||||
portability of extended attributes is important to you, omit the
|
|
||||||
``-O xattr=sa`` above. Even if you do not want ``xattr=sa`` for the whole
|
|
||||||
pool, it is probably fine to use it for ``/var/log``.
|
|
||||||
- Make sure to include the ``-part3`` portion of the drive path. If you
|
|
||||||
forget that, you are specifying the whole disk, which ZFS will then
|
|
||||||
re-partition, and you will lose the bootloader partition(s).
|
|
||||||
|
|
||||||
#. This section implements dataset layout as described in `overview <0-overview.html>`__.
|
|
||||||
|
|
||||||
Create root system container:
|
|
||||||
|
|
||||||
- Unencrypted::
|
|
||||||
|
|
||||||
zfs create \
|
|
||||||
-o canmount=off \
|
|
||||||
-o mountpoint=none \
|
|
||||||
rpool_$INST_UUID/$INST_ID
|
|
||||||
|
|
||||||
- Encrypted:
|
|
||||||
|
|
||||||
Pick a strong password. Once compromised, changing password will not keep your
|
|
||||||
data safe. See ``zfs-change-key(8)`` for more info::
|
|
||||||
|
|
||||||
zfs create \
|
|
||||||
-o canmount=off \
|
|
||||||
-o mountpoint=none \
|
|
||||||
-o encryption=on \
|
|
||||||
-o keylocation=prompt \
|
|
||||||
-o keyformat=passphrase \
|
|
||||||
rpool_$INST_UUID/$INST_ID
|
|
||||||
|
|
||||||
Create other system datasets::
|
|
||||||
|
|
||||||
zfs create -o canmount=off -o mountpoint=none bpool_$INST_UUID/$INST_ID
|
|
||||||
zfs create -o canmount=off -o mountpoint=none bpool_$INST_UUID/$INST_ID/BOOT
|
|
||||||
zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID/ROOT
|
|
||||||
zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID/DATA
|
|
||||||
zfs create -o mountpoint=/boot -o canmount=noauto bpool_$INST_UUID/$INST_ID/BOOT/default
|
|
||||||
zfs create -o mountpoint=/ -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/default
|
|
||||||
zfs create -o mountpoint=/ -o canmount=noauto rpool_$INST_UUID/$INST_ID/ROOT/default
|
|
||||||
zfs mount rpool_$INST_UUID/$INST_ID/ROOT/default
|
|
||||||
zfs mount bpool_$INST_UUID/$INST_ID/BOOT/default
|
|
||||||
for i in {usr,var,var/lib};
|
|
||||||
do
|
|
||||||
zfs create -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/default/$i
|
|
||||||
done
|
|
||||||
for i in {home,root,srv,usr/local,var/log,var/spool};
|
|
||||||
do
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/$i
|
|
||||||
done
|
|
||||||
chmod 750 /mnt/root
|
|
||||||
|
|
||||||
#. Format and mount ESP::
|
|
||||||
|
|
||||||
for i in ${DISK}; do
|
|
||||||
mkfs.vfat -n EFI ${i}-part1
|
|
||||||
mkdir -p /mnt/boot/efis/${i##*/}-part1
|
|
||||||
mount -t vfat ${i}-part1 /mnt/boot/efis/${i##*/}-part1
|
|
||||||
done
|
|
||||||
|
|
||||||
mkdir -p /mnt/boot/efi
|
|
||||||
mount -t vfat ${INST_PRIMARY_DISK}-part1 /mnt/boot/efi
|
|
||||||
|
|
||||||
#. Create separate user dataset at ``/home/User``, dateset name can be
|
|
||||||
changed later::
|
|
||||||
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/home/User
|
|
||||||
|
|
||||||
If needed, snapshot, rollback and other related permissions can be
|
|
||||||
delegated to the user later.
|
|
||||||
|
|
||||||
#. Create optional program data datasets to omit data from rollback::
|
|
||||||
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/games
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/www
|
|
||||||
# for GNOME
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/AccountsService
|
|
||||||
# for Docker
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/docker
|
|
||||||
# for NFS
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/nfs
|
|
||||||
# for LXC
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/lxc
|
|
||||||
# for LibVirt
|
|
||||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/libvirt
|
|
||||||
##other application
|
|
||||||
# zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/$name
|
|
||||||
|
|
||||||
Add other datasets when needed, such as PostgreSQL.
|
|
||||||
|
|
||||||
#. Install base packages::
|
|
||||||
|
|
||||||
dnf --installroot=/mnt --releasever=${INST_RHEL_VER} -y install \
|
|
||||||
${RHEL_ZFS_REPO} @core epel-release grub2-efi-x64 grub2-pc-modules \
|
|
||||||
grub2-efi-x64-modules shim-x64 efibootmgr \
|
|
||||||
kernel kernel-devel python3-dnf-plugin-post-transaction-actions
|
|
||||||
dnf config-manager --installroot=/mnt --disable zfs
|
|
||||||
dnf config-manager --installroot=/mnt --enable zfs-kmod
|
|
||||||
dnf install --installroot=/mnt -y zfs zfs-dracut
|
|
||||||
|
|
||||||
#. Update zfs repo if a newer release is available::
|
|
||||||
|
|
||||||
source /mnt/etc/os-release
|
|
||||||
RHEL_ZFS_REPO_NEW=https://zfsonlinux.org/epel/zfs-release.el${VERSION_ID/./_}.noarch.rpm
|
|
||||||
dnf install --installroot=/mnt -y $RHEL_ZFS_REPO_NEW || true
|
|
||||||
|
|
||||||
#. Optional: enable boot environment support and dnf integration::
|
|
||||||
|
|
||||||
dnf --installroot=/mnt copr enable -y m0p/bieaz
|
|
||||||
dnf --installroot=/mnt install -y bieaz python3-dnf-plugin-rozb3
|
|
||||||
|
|
||||||
If multi-disk setup is used, enable multi-disk
|
|
||||||
support inside ``/mnt/etc/bieaz.cfg``.
|
|
||||||
@@ -1,103 +0,0 @@
|
|||||||
.. highlight:: sh
|
|
||||||
|
|
||||||
System Configuration
|
|
||||||
======================
|
|
||||||
|
|
||||||
.. contents:: Table of Contents
|
|
||||||
:local:
|
|
||||||
|
|
||||||
#. Generate fstab::
|
|
||||||
|
|
||||||
genfstab -U /mnt | sed 's;zfs[[:space:]]*;zfs zfsutil,;g' | grep "zfs zfsutil" >> /mnt/etc/fstab
|
|
||||||
for i in ${DISK}; do
|
|
||||||
echo UUID=$(blkid -s UUID -o value ${i}-part1) /boot/efis/${i##*/}-part1 vfat \
|
|
||||||
x-systemd.idle-timeout=1min,x-systemd.automount,noauto,umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab
|
|
||||||
done
|
|
||||||
echo UUID=$(blkid -s UUID -o value ${INST_PRIMARY_DISK}-part1) /boot/efi vfat \
|
|
||||||
x-systemd.idle-timeout=1min,x-systemd.automount,noauto,umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab
|
|
||||||
if [ "${INST_PARTSIZE_SWAP}" != "" ]; then
|
|
||||||
for i in ${DISK}; do
|
|
||||||
echo ${i##*/}-part4-swap ${i}-part4 /dev/urandom swap,cipher=aes-cbc-essiv:sha256,size=256,discard >> /mnt/etc/crypttab
|
|
||||||
echo /dev/mapper/${i##*/}-part4-swap none swap x-systemd.requires=cryptsetup.target,defaults 0 0 >> /mnt/etc/fstab
|
|
||||||
done
|
|
||||||
fi
|
|
||||||
|
|
||||||
By default, systemd will halt boot process if any entry in ``/etc/fstab`` fails
|
|
||||||
to mount. This is unnecessary for mirrored EFI boot partitions.
|
|
||||||
With the above mount options, systemd will skip mounting them at boot,
|
|
||||||
only mount them on demand when accessed.
|
|
||||||
|
|
||||||
#. Configure dracut::
|
|
||||||
|
|
||||||
echo 'add_dracutmodules+=" zfs "' > /mnt/etc/dracut.conf.d/zfs.conf
|
|
||||||
|
|
||||||
#. Force load mpt3sas module if used::
|
|
||||||
|
|
||||||
if grep mpt3sas /proc/modules; then
|
|
||||||
echo 'forced_drivers+=" mpt3sas "' >> /mnt/etc/dracut.conf.d/zfs.conf
|
|
||||||
fi
|
|
||||||
|
|
||||||
#. Interactively set locale, keymap, timezone, hostname and root password::
|
|
||||||
|
|
||||||
rm -f /mnt/etc/localtime
|
|
||||||
systemd-firstboot --root=/mnt --prompt --root-password=PASSWORD
|
|
||||||
|
|
||||||
This can be non-interactive, see man page for details::
|
|
||||||
|
|
||||||
rm -f /mnt/etc/localtime
|
|
||||||
systemd-firstboot --root=/mnt \
|
|
||||||
--locale="en_US.UTF-8" --locale-messages="en_US.UTF-8" \
|
|
||||||
--keymap=us --timezone="Europe/Berlin" --hostname=myHost \
|
|
||||||
--root-password=PASSWORD
|
|
||||||
|
|
||||||
``systemd-firstboot`` have bugs, root password is set below.
|
|
||||||
|
|
||||||
#. Generate host id::
|
|
||||||
|
|
||||||
zgenhostid -f -o /mnt/etc/hostid
|
|
||||||
|
|
||||||
#. Install locale package, example for English locale::
|
|
||||||
|
|
||||||
dnf --installroot=/mnt install -y glibc-minimal-langpack glibc-langpack-en
|
|
||||||
|
|
||||||
Program will show errors if not installed.
|
|
||||||
|
|
||||||
#. Enable ZFS services::
|
|
||||||
|
|
||||||
systemctl enable zfs-import-scan.service zfs-import.target zfs-zed zfs.target --root=/mnt
|
|
||||||
systemctl disable zfs-mount --root=/mnt
|
|
||||||
|
|
||||||
At boot, datasets on rpool are mounted with ``/etc/fstab``,
|
|
||||||
which can control the mounting process more precisely than ``zfs-mount.service``.
|
|
||||||
|
|
||||||
#. By default SSH server is enabled, allowing root login by password,
|
|
||||||
disable SSH server::
|
|
||||||
|
|
||||||
systemctl disable sshd --root=/mnt
|
|
||||||
systemctl enable firewalld --root=/mnt
|
|
||||||
|
|
||||||
#. Chroot::
|
|
||||||
|
|
||||||
echo "INST_PRIMARY_DISK=$INST_PRIMARY_DISK
|
|
||||||
INST_LINVAR=$INST_LINVAR
|
|
||||||
INST_UUID=$INST_UUID
|
|
||||||
INST_ID=$INST_ID
|
|
||||||
unalias -a
|
|
||||||
TERM=xterm
|
|
||||||
INST_VDEV=$INST_VDEV
|
|
||||||
INST_VDEV=$INST_VDEV
|
|
||||||
DISK=\"$DISK\"" > /mnt/root/chroot
|
|
||||||
history -w /mnt/home/sys-install-pre-chroot.txt
|
|
||||||
arch-chroot /mnt bash --login
|
|
||||||
|
|
||||||
#. Source variables::
|
|
||||||
|
|
||||||
source /root/chroot
|
|
||||||
|
|
||||||
#. For SELinux, relabel filesystem on reboot::
|
|
||||||
|
|
||||||
fixfiles -F onboot
|
|
||||||
|
|
||||||
#. Set root password::
|
|
||||||
|
|
||||||
passwd
|
|
||||||
@@ -1,258 +0,0 @@
|
|||||||
.. highlight:: sh
|
|
||||||
|
|
||||||
Bootloader
|
|
||||||
======================
|
|
||||||
|
|
||||||
.. contents:: Table of Contents
|
|
||||||
:local:
|
|
||||||
|
|
||||||
Apply workarounds
|
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
|
||||||
Currently GRUB has multiple compatibility problems with ZFS,
|
|
||||||
especially with regards to newer ZFS features.
|
|
||||||
Workarounds have to be applied.
|
|
||||||
|
|
||||||
#. grub2-probe fails to get canonical path
|
|
||||||
|
|
||||||
When persistent device names ``/dev/disk/by-id/*`` are used
|
|
||||||
with ZFS, GRUB will fail to resolve the path of the boot pool
|
|
||||||
device. Error::
|
|
||||||
|
|
||||||
# /usr/bin/grub2-probe: error: failed to get canonical path of `/dev/virtio-pci-0000:06:00.0-part3'.
|
|
||||||
|
|
||||||
Solution::
|
|
||||||
|
|
||||||
echo 'export ZPOOL_VDEV_NAME_PATH=YES' >> /etc/profile.d/zpool_vdev_name_path.sh
|
|
||||||
source /etc/profile.d/zpool_vdev_name_path.sh
|
|
||||||
|
|
||||||
#. Pool name missing
|
|
||||||
|
|
||||||
See `this bug report <https://savannah.gnu.org/bugs/?59614>`__.
|
|
||||||
Root pool name is missing from ``root=ZFS=rpool_$INST_UUID/ROOT/default``
|
|
||||||
kernel cmdline in generated ``grub.cfg`` file.
|
|
||||||
|
|
||||||
A workaround is to replace the pool name detection with ``zdb``
|
|
||||||
command::
|
|
||||||
|
|
||||||
sed -i "s|rpool=.*|rpool=\`zdb -l \${GRUB_DEVICE} \| grep -E '[[:blank:]]name' \| cut -d\\\' -f 2\`|" /etc/grub.d/10_linux
|
|
||||||
|
|
||||||
Install GRUB
|
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
#. If using virtio disk, add driver to initrd::
|
|
||||||
|
|
||||||
echo 'filesystems+=" virtio_blk "' >> /etc/dracut.conf.d/fs.conf
|
|
||||||
|
|
||||||
#. Generate initrd::
|
|
||||||
|
|
||||||
rm -f /etc/zfs/zpool.cache
|
|
||||||
touch /etc/zfs/zpool.cache
|
|
||||||
chmod a-w /etc/zfs/zpool.cache
|
|
||||||
chattr +i /etc/zfs/zpool.cache
|
|
||||||
for directory in /lib/modules/*; do
|
|
||||||
kernel_version=$(basename $directory)
|
|
||||||
dracut --force --kver $kernel_version
|
|
||||||
done
|
|
||||||
|
|
||||||
#. Load ZFS modules and disable BLS::
|
|
||||||
|
|
||||||
echo 'GRUB_ENABLE_BLSCFG=false' >> /etc/default/grub
|
|
||||||
|
|
||||||
#. Create GRUB boot directory, in ESP and boot pool::
|
|
||||||
|
|
||||||
mkdir -p /boot/efi/EFI/rocky # EFI GRUB dir
|
|
||||||
mkdir -p /boot/efi/EFI/rocky/grub2 # legacy GRUB dir
|
|
||||||
mkdir -p /boot/grub2
|
|
||||||
|
|
||||||
Boot environment-specific configuration (kernel, etc)
|
|
||||||
is stored in ``/boot/grub2/grub.cfg``, enabling rollback.
|
|
||||||
|
|
||||||
#. When in doubt, install both legacy boot
|
|
||||||
and EFI.
|
|
||||||
|
|
||||||
#. If using legacy booting, install GRUB to every disk::
|
|
||||||
|
|
||||||
for i in ${DISK}; do
|
|
||||||
grub2-install --boot-directory /boot/efi/EFI/rocky --target=i386-pc $i
|
|
||||||
done
|
|
||||||
|
|
||||||
#. If using EFI::
|
|
||||||
|
|
||||||
for i in ${DISK}; do
|
|
||||||
efibootmgr -cgp 1 -l "\EFI\rocky\shimx64.efi" \
|
|
||||||
-L "rocky-${i##*/}" -d ${i}
|
|
||||||
done
|
|
||||||
cp -r /usr/lib/grub/x86_64-efi/ /boot/efi/EFI/rocky
|
|
||||||
|
|
||||||
#. Generate GRUB Menu:
|
|
||||||
|
|
||||||
Apply workaround::
|
|
||||||
|
|
||||||
tee /etc/grub.d/09_fix_root_on_zfs <<EOF
|
|
||||||
#!/bin/sh
|
|
||||||
echo 'insmod zfs'
|
|
||||||
echo 'set root=(hd0,gpt2)'
|
|
||||||
EOF
|
|
||||||
chmod +x /etc/grub.d/09_fix_root_on_zfs
|
|
||||||
|
|
||||||
Generate menu::
|
|
||||||
|
|
||||||
grub2-mkconfig -o /boot/efi/EFI/rocky/grub.cfg
|
|
||||||
cp /boot/efi/EFI/rocky/grub.cfg /boot/efi/EFI/rocky/grub2/grub.cfg
|
|
||||||
cp /boot/efi/EFI/rocky/grub.cfg /boot/grub2/grub.cfg
|
|
||||||
|
|
||||||
The following errors may be safely ignored:
|
|
||||||
|
|
||||||
- ``device-mapper: reload ioctl on osprober-linux-sda2 (253:0) failed: Device or resource busy``
|
|
||||||
This is caused by os-prober probing OS on the partitions used by ZFS,
|
|
||||||
harmless but os-prober can be disabled by::
|
|
||||||
|
|
||||||
echo GRUB_DISABLE_OS_PROBER=true >> /etc/default/grub
|
|
||||||
|
|
||||||
- ``/usr/sbin/grub2-probe: error: ../grub-core/kern/fs.c:120:unknown filesystem.``
|
|
||||||
This is fixed by /etc/grub.d/09_fix_root_on_zfs
|
|
||||||
|
|
||||||
#. For both legacy and EFI booting: mirror ESP content::
|
|
||||||
|
|
||||||
ESP_MIRROR=$(mktemp -d)
|
|
||||||
unalias -a
|
|
||||||
cp -r /boot/efi/EFI $ESP_MIRROR
|
|
||||||
for i in /boot/efis/*; do
|
|
||||||
cp -r $ESP_MIRROR/EFI $i
|
|
||||||
done
|
|
||||||
|
|
||||||
#. Automatically regenerate GRUB menu on kernel update::
|
|
||||||
|
|
||||||
tee /etc/dnf/plugins/post-transaction-actions.d/00-update-grub-menu-for-kernel.action <<EOF >/dev/null
|
|
||||||
# kernel-core package contains vmlinuz and initramfs
|
|
||||||
# change package name if non-standard kernel is used
|
|
||||||
kernel-core:in:/usr/local/sbin/update-grub-menu.sh
|
|
||||||
kernel-core:out:/usr/local/sbin/update-grub-menu.sh
|
|
||||||
EOF
|
|
||||||
|
|
||||||
tee /usr/local/sbin/update-grub-menu.sh <<-'EOF' >/dev/null
|
|
||||||
#!/bin/sh
|
|
||||||
export PATH=$PATH:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
|
|
||||||
export ZPOOL_VDEV_NAME_PATH=YES
|
|
||||||
source /etc/os-release
|
|
||||||
grub2-mkconfig -o /boot/efi/EFI/${ID}/grub.cfg
|
|
||||||
cp /boot/efi/EFI/${ID}/grub.cfg /boot/efi/EFI/${ID}/grub2/grub.cfg
|
|
||||||
cp /boot/efi/EFI/${ID}/grub.cfg /boot/grub2/grub.cfg
|
|
||||||
ESP_MIRROR=$(mktemp -d)
|
|
||||||
cp -r /boot/efi/EFI $ESP_MIRROR
|
|
||||||
for i in /boot/efis/*; do
|
|
||||||
cp -r $ESP_MIRROR/EFI $i
|
|
||||||
done
|
|
||||||
rm -rf $ESP_MIRROR
|
|
||||||
EOF
|
|
||||||
|
|
||||||
chmod +x /usr/local/sbin/update-grub-menu.sh
|
|
||||||
|
|
||||||
#. Notes for GRUB on RHEL
|
|
||||||
|
|
||||||
To support Secure Boot, GRUB has been heavily modified by Fedora,
|
|
||||||
namely:
|
|
||||||
|
|
||||||
- ``grub2-install`` is `disabled for UEFI <https://bugzilla.redhat.com/show_bug.cgi?id=1917213>`__
|
|
||||||
- Only a static, signed version of bootloader is copied to EFI system partition
|
|
||||||
- This signed bootloader does not have built-in support for either ZFS or LUKS containers
|
|
||||||
- This signed bootloader only loads configuration from ``/boot/efi/EFI/fedora/grub.cfg``
|
|
||||||
|
|
||||||
Unrelated to Secure Boot, GRUB has also been modified to provide optional
|
|
||||||
support for `systemd bootloader specification (bls) <https://systemd.io/BOOT_LOADER_SPECIFICATION/>`__.
|
|
||||||
Currently ``blscfg.mod`` is incompatible with root on ZFS.
|
|
||||||
|
|
||||||
As bls is disabled, you will need to regenerate GRUB menu after each kernel upgrade.
|
|
||||||
Or else the new kernel will not be recognized and system will boot the old kernel
|
|
||||||
on reboot.
|
|
||||||
|
|
||||||
Also see `Fedora docs for GRUB
|
|
||||||
<https://docs.fedoraproject.org/en-US/fedora/rawhide/system-administrators-guide/kernel-module-driver-configuration/Working_with_the_GRUB_2_Boot_Loader/>`__.
|
|
||||||
|
|
||||||
Finish Installation
|
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
#. Exit chroot::
|
|
||||||
|
|
||||||
exit
|
|
||||||
|
|
||||||
#. Take a snapshot of the clean installation for future use::
|
|
||||||
|
|
||||||
zfs snapshot -r rpool_$INST_UUID/$INST_ID@install
|
|
||||||
zfs snapshot -r bpool_$INST_UUID/$INST_ID@install
|
|
||||||
|
|
||||||
#. Unmount EFI system partition::
|
|
||||||
|
|
||||||
umount /mnt/boot/efi
|
|
||||||
umount /mnt/boot/efis/*
|
|
||||||
|
|
||||||
#. Export pools::
|
|
||||||
|
|
||||||
zpool export bpool_$INST_UUID
|
|
||||||
zpool export rpool_$INST_UUID
|
|
||||||
|
|
||||||
#. Reboot::
|
|
||||||
|
|
||||||
reboot
|
|
||||||
|
|
||||||
Post installaion
|
|
||||||
~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
#. If you have other data pools, generate list of datasets for `zfs-mount-generator
|
|
||||||
<https://manpages.ubuntu.com/manpages/focal/man8/zfs-mount-generator.8.html>`__ to mount them at boot::
|
|
||||||
|
|
||||||
DATA_POOL='tank0 tank1'
|
|
||||||
|
|
||||||
# tab-separated zfs properties
|
|
||||||
# see /etc/zfs/zed.d/history_event-zfs-list-cacher.sh
|
|
||||||
export \
|
|
||||||
PROPS="name,mountpoint,canmount,atime,relatime,devices,exec\
|
|
||||||
,readonly,setuid,nbmand,encroot,keylocation"
|
|
||||||
|
|
||||||
for i in $DATA_POOL; do
|
|
||||||
zfs list -H -t filesystem -o $PROPS -r $i > /etc/zfs/zfs-list.cache/$i
|
|
||||||
done
|
|
||||||
|
|
||||||
#. After reboot, consider adding a normal user::
|
|
||||||
|
|
||||||
# with root permissions
|
|
||||||
sudo -i
|
|
||||||
|
|
||||||
# store user name in a variable
|
|
||||||
myUser=UserName
|
|
||||||
|
|
||||||
# rename default `User` to new user name
|
|
||||||
zfs rename $(df --output=source /home | tail -n +2)/User $(df --output=source /home | tail -n +2)/${myUser}
|
|
||||||
|
|
||||||
# update entry in fstab
|
|
||||||
sed -i "s|/home/User|/home/${myUser}|g" /etc/fstab
|
|
||||||
|
|
||||||
# add user
|
|
||||||
useradd --no-create-home --user-group --home-dir /home/${myUser} --comment 'My Name' ${myUser}
|
|
||||||
|
|
||||||
# delegate snapshot and destroy permissions of the home dataset to
|
|
||||||
# new user
|
|
||||||
zfs allow -u ${myUser} mount,snapshot,destroy $(df --output=source /home | tail -n +2)/${myUser}
|
|
||||||
|
|
||||||
# fix permissions
|
|
||||||
chown --recursive ${myUser}:${myUser} /home/${myUser}
|
|
||||||
chmod 700 /home/${myUser}
|
|
||||||
|
|
||||||
# fix selinux context
|
|
||||||
restorecon /home/${myUser}
|
|
||||||
|
|
||||||
# set new password for user
|
|
||||||
passwd ${myUser}
|
|
||||||
|
|
||||||
Set up cron job to snapshot user home everyday::
|
|
||||||
|
|
||||||
dnf install cronie
|
|
||||||
systemctl enable --now crond
|
|
||||||
crontab -eu ${myUser}
|
|
||||||
#@daily zfs snap $(df --output=source /home/${myUser} | tail -n +2)@$(dd if=/dev/urandom of=/dev/stdout bs=1 count=100 2>/dev/null |tr -dc 'a-z0-9' | cut -c-6)
|
|
||||||
zfs list -t snapshot -S creation $(df --output=source /home/${myUser} | tail -n +2)
|
|
||||||
|
|
||||||
Install package groups::
|
|
||||||
|
|
||||||
dnf group list --hidden -v # query package groups
|
|
||||||
dnf group install 'Virtualization Host'
|
|
||||||
@@ -1,211 +0,0 @@
|
|||||||
.. highlight:: sh
|
|
||||||
|
|
||||||
Recovery
|
|
||||||
======================
|
|
||||||
|
|
||||||
.. contents:: Table of Contents
|
|
||||||
:local:
|
|
||||||
|
|
||||||
GRUB Tips
|
|
||||||
-------------
|
|
||||||
|
|
||||||
Boot from GRUB rescue
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
If bootloader file is damaged, it's still possible
|
|
||||||
to boot computer with GRUB rescue image.
|
|
||||||
|
|
||||||
This section is also applicable if you are in
|
|
||||||
``grub rescue>``.
|
|
||||||
|
|
||||||
#. On another computer, generate rescue image with::
|
|
||||||
|
|
||||||
pacman -S --needed mtools libisoburn grub
|
|
||||||
grub-install
|
|
||||||
grub-mkrescue -o grub-rescue.img
|
|
||||||
dd if=grub-rescue.img of=/dev/your-usb-stick
|
|
||||||
|
|
||||||
Boot computer from the rescue media.
|
|
||||||
Both legacy and EFI mode are supported.
|
|
||||||
|
|
||||||
Or `download generated GRUB rescue image <https://gitlab.com/m_zhou/bieaz/uploads/e0847a8675cda4317ea7f48abb1d9f10/grub-rescue-2.06.img.7z>`__.
|
|
||||||
|
|
||||||
#. List available disks with ``ls`` command::
|
|
||||||
|
|
||||||
grub> ls (hd # press tab
|
|
||||||
Possible devices are:
|
|
||||||
|
|
||||||
hd0 hd1 hd2 hd3
|
|
||||||
|
|
||||||
#. List partitions by pressing tab key:
|
|
||||||
|
|
||||||
.. code-block:: text
|
|
||||||
|
|
||||||
grub> ls (hd0 # press tab
|
|
||||||
Possible partitions are:
|
|
||||||
|
|
||||||
Device hd0: No known filesystem detected - Sector size 512B - Total size 20971520KiB
|
|
||||||
Partition hd0,gpt1: Filesystem type fat - Label `EFI', UUID 0DF5-3A76 - Partition start at 1024KiB - Total size 1048576KiB
|
|
||||||
Partition hd0,gpt2: No known filesystem detected - Partition start at 1049600KiB - Total size 4194304KiB
|
|
||||||
|
|
||||||
- If boot pool is encrypted:
|
|
||||||
|
|
||||||
Unlock it with ``cryptomount``::
|
|
||||||
|
|
||||||
grub> insmod luks
|
|
||||||
grub> cryptomount hd0,gpt2
|
|
||||||
Attempting to decrypt master key...
|
|
||||||
Enter passphrase for hd0,gpt2 (af5a240e13e24483acf02600d61e0f36):
|
|
||||||
Slot 1 opened
|
|
||||||
|
|
||||||
Unlocked LUKS container is ``(crypto0)``:
|
|
||||||
|
|
||||||
.. code-block:: text
|
|
||||||
|
|
||||||
grub> ls (crypto0)
|
|
||||||
Device crypto0: Filesystem type zfs - Label `bpool_ip3tdb' - Last modification
|
|
||||||
time 2021-05-03 12:14:08 Monday, UUID f14d7bdf89fe21fb - Sector size 512B -
|
|
||||||
Total size 4192256KiB
|
|
||||||
|
|
||||||
- If boot pool is not encrypted:
|
|
||||||
|
|
||||||
.. code-block:: text
|
|
||||||
|
|
||||||
grub> ls (hd0,gpt2)
|
|
||||||
Device hd0,gpt2: Filesystem type zfs - Label `bpool_ip3tdb' - Last modification
|
|
||||||
time 2021-05-03 12:14:08 Monday, UUID f14d7bdf89fe21fb - Sector size 512B -
|
|
||||||
Total size 4192256KiB
|
|
||||||
|
|
||||||
#. List boot environments nested inside ``bpool/$INST_ID/BOOT``::
|
|
||||||
|
|
||||||
grub> ls (crypto0)/sys/BOOT
|
|
||||||
@/ default/ be0/
|
|
||||||
|
|
||||||
#. Instruct GRUB to load configuration from ``be0`` boot environment::
|
|
||||||
|
|
||||||
grub> prefix=(crypto0)/sys/BOOT/be0/@/grub
|
|
||||||
grub> configfile $prefix/grub.cfg
|
|
||||||
|
|
||||||
#. GRUB menu should now appear.
|
|
||||||
|
|
||||||
#. After entering system, `reinstall GRUB <#grub-installation>`__.
|
|
||||||
|
|
||||||
Switch GRUB prefix when disk fails
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
If you are using LUKS encrypted boot pool with multiple disks,
|
|
||||||
the primary disk failed, GRUB will fail to load configuration.
|
|
||||||
|
|
||||||
If there's still enough redundancy for the boot pool, try fix
|
|
||||||
GRUB with the following method:
|
|
||||||
|
|
||||||
#. Ensure ``Slot 1 opened`` message
|
|
||||||
is shown
|
|
||||||
|
|
||||||
.. code-block:: text
|
|
||||||
|
|
||||||
Welcome to GRUB!
|
|
||||||
|
|
||||||
error: no such cryptodisk found.
|
|
||||||
Attempting to decrypt master key...
|
|
||||||
Enter passphrase for hd0,gpt2 (c0987ea1a51049e9b3056622804de62a):
|
|
||||||
Slot 1 opened
|
|
||||||
error: disk `cryptouuid/47ed1b7eb0014bc9a70aede3d8714faf' not found.
|
|
||||||
Entering rescue mode...
|
|
||||||
grub rescue>
|
|
||||||
|
|
||||||
If ``error: access denied.`` is shown,
|
|
||||||
try re-enter password with::
|
|
||||||
|
|
||||||
grub rescue> cryptomount hd0,gpt2
|
|
||||||
|
|
||||||
#. Check prefix::
|
|
||||||
|
|
||||||
grub rescue > set
|
|
||||||
# prefix=(cryptouuid/47ed1b7eb0014bc9a70aede3d8714faf)/sys/BOOT/be0@/grub
|
|
||||||
# root=cryptouuid/47ed1b7eb0014bc9a70aede3d8714faf
|
|
||||||
|
|
||||||
#. Set correct ``prefix`` and ``root`` by replacing
|
|
||||||
``cryptouuid/UUID`` with ``crypto0``::
|
|
||||||
|
|
||||||
grub rescue> prefix=(crypto0)/sys/BOOT/default@/grub
|
|
||||||
grub rescue> root=crypto0
|
|
||||||
|
|
||||||
#. Boot GRUB::
|
|
||||||
|
|
||||||
grub rescue> insmod normal
|
|
||||||
grub rescue> normal
|
|
||||||
|
|
||||||
GRUB should then boot normally.
|
|
||||||
|
|
||||||
#. After entering system, edit ``/etc/fstab`` to promote
|
|
||||||
one backup to ``/boot/efi``.
|
|
||||||
|
|
||||||
#. Make the change to ``prefix`` and ``root``
|
|
||||||
permanent by `reinstalling GRUB <#grub-installation>`__.
|
|
||||||
|
|
||||||
Access system in chroot
|
|
||||||
-----------------------
|
|
||||||
|
|
||||||
#. Go through `preparation <1-preparation.html>`__.
|
|
||||||
|
|
||||||
#. Import and unlock root and boot pool::
|
|
||||||
|
|
||||||
zpool import -NR /mnt rpool_$INST_UUID
|
|
||||||
zpool import -NR /mnt bpool_$INST_UUID
|
|
||||||
|
|
||||||
If using password::
|
|
||||||
|
|
||||||
zfs load-key rpool_$INST_UUID/$INST_ID
|
|
||||||
|
|
||||||
If using keyfile::
|
|
||||||
|
|
||||||
zfs load-key -L file:///path/to/keyfile rpool_$INST_UUID/$INST_ID
|
|
||||||
|
|
||||||
#. Find the current boot environment::
|
|
||||||
|
|
||||||
zfs list
|
|
||||||
BE=default
|
|
||||||
|
|
||||||
#. Mount root filesystem::
|
|
||||||
|
|
||||||
zfs mount rpool_$INST_UUID/$INST_ID/ROOT/$BE
|
|
||||||
|
|
||||||
#. chroot into the system::
|
|
||||||
|
|
||||||
arch-chroot /mnt /bin/bash --login
|
|
||||||
zfs mount -a
|
|
||||||
mount -a
|
|
||||||
|
|
||||||
#. Finish rescue. See `finish installation <#finish-installation>`__.
|
|
||||||
|
|
||||||
Backup and migrate existing installation
|
|
||||||
----------------------------------------
|
|
||||||
With the help of `zfs send
|
|
||||||
<https://openzfs.github.io/openzfs-docs/man/8/zfs-send.8.html>`__
|
|
||||||
it is relatively easy to perform a system backup and migration.
|
|
||||||
|
|
||||||
#. Create a snapshot of root file system::
|
|
||||||
|
|
||||||
zfs snapshot -r rpool/$INST_ID@backup
|
|
||||||
zfs snapshot -r bpool/$INST_ID@backup
|
|
||||||
|
|
||||||
#. Save snapshot to a file or pipe to SSH::
|
|
||||||
|
|
||||||
zfs send --options rpool/$INST_ID@backup > /backup/$INST_ID-rpool
|
|
||||||
zfs send --options bpool/$INST_ID@backup > /backup/$INST_ID-bpool
|
|
||||||
|
|
||||||
#. Re-create partitions and root/boot
|
|
||||||
pool on target system.
|
|
||||||
|
|
||||||
#. Restore backup::
|
|
||||||
|
|
||||||
zfs recv rpool_new/$INST_ID < /backup/$INST_ID-rpool
|
|
||||||
zfs recv bpool_new/$INST_ID < /backup/$INST_ID-bpool
|
|
||||||
|
|
||||||
#. Chroot and reinstall bootloader.
|
|
||||||
|
|
||||||
#. Update pool name in ``/etc/fstab``, ``/boot/grub/grub.cfg``
|
|
||||||
and ``/etc/zfs/zfs-list.cache/*``.
|
|
||||||
|
|
||||||
#. Update device name, etc, in ``/etc/fstab`` and ``/etc/crypttab``.
|
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
RHEL Root on ZFS
|
||||||
|
=======================================
|
||||||
|
`Start here <RHEL%20-based%20distro%20Root%20on%20ZFS/1-preparation.html>`__.
|
||||||
|
|
||||||
|
Contents
|
||||||
|
--------
|
||||||
|
.. toctree::
|
||||||
|
:maxdepth: 2
|
||||||
|
:glob:
|
||||||
|
|
||||||
|
RHEL-based distro Root on ZFS/*
|
||||||
@@ -0,0 +1,78 @@
|
|||||||
|
.. highlight:: sh
|
||||||
|
|
||||||
|
Preparation
|
||||||
|
======================
|
||||||
|
|
||||||
|
.. contents:: Table of Contents
|
||||||
|
:local:
|
||||||
|
|
||||||
|
#. Disable Secure Boot. ZFS modules can not be loaded if Secure Boot is enabled.
|
||||||
|
#. Download a variant of `AlmaLinux Minimal Live ISO
|
||||||
|
<https://repo.almalinux.org/almalinux/9/live/x86_64/>`__ and boot from it.
|
||||||
|
#. Connect to the Internet.
|
||||||
|
#. Set root password or ``/root/.ssh/authorized_keys``.
|
||||||
|
#. Start SSH server::
|
||||||
|
|
||||||
|
echo PermitRootLogin yes >> /etc/ssh/sshd_config
|
||||||
|
systemctl restart sshd
|
||||||
|
|
||||||
|
#. Connect from another computer::
|
||||||
|
|
||||||
|
ssh root@192.168.1.19
|
||||||
|
|
||||||
|
#. Target disk
|
||||||
|
|
||||||
|
List available disks with::
|
||||||
|
|
||||||
|
ls /dev/disk/by-id/*
|
||||||
|
|
||||||
|
If using virtio as disk bus, use ``/dev/disk/by-path/*``.
|
||||||
|
|
||||||
|
Declare disk array::
|
||||||
|
|
||||||
|
DISK='/dev/disk/by-id/ata-FOO /dev/disk/by-id/nvme-BAR'
|
||||||
|
|
||||||
|
For single disk installation, use::
|
||||||
|
|
||||||
|
DISK='/dev/disk/by-id/disk1'
|
||||||
|
|
||||||
|
#. Set partition size:
|
||||||
|
|
||||||
|
Set swap size. It's `recommended <https://chrisdown.name/2018/01/02/in-defence-of-swap.html>`__
|
||||||
|
to setup a swap partition. If you intend to use hibernation,
|
||||||
|
the minimum should be no less than RAM size. Skip if swap is not needed::
|
||||||
|
|
||||||
|
INST_PARTSIZE_SWAP=8
|
||||||
|
|
||||||
|
Root pool size, use all remaining disk space if not set::
|
||||||
|
|
||||||
|
INST_PARTSIZE_RPOOL=
|
||||||
|
|
||||||
|
#. Temporarily set SELinux to permissive in live environment::
|
||||||
|
|
||||||
|
setenforce 0
|
||||||
|
|
||||||
|
SELinux will be enabled on the installed system.
|
||||||
|
|
||||||
|
#. Add ZFS repo::
|
||||||
|
|
||||||
|
dnf install -y https://zfsonlinux.org/epel/zfs-release-el-2-1.noarch.rpm
|
||||||
|
|
||||||
|
#. Check available repos::
|
||||||
|
|
||||||
|
dnf repolist --all
|
||||||
|
|
||||||
|
#. Install ZFS packages::
|
||||||
|
|
||||||
|
dnf config-manager --disable zfs
|
||||||
|
dnf config-manager --enable zfs-kmod
|
||||||
|
dnf install -y zfs
|
||||||
|
# if gpg import fails, add --nogpgcheck
|
||||||
|
|
||||||
|
#. Load kernel modules::
|
||||||
|
|
||||||
|
modprobe zfs
|
||||||
|
|
||||||
|
#. Install partition tool::
|
||||||
|
|
||||||
|
dnf install -y gdisk dosfstools
|
||||||
@@ -0,0 +1,146 @@
|
|||||||
|
.. highlight:: sh
|
||||||
|
|
||||||
|
System Installation
|
||||||
|
======================
|
||||||
|
|
||||||
|
.. contents:: Table of Contents
|
||||||
|
:local:
|
||||||
|
|
||||||
|
#. Partition the disks::
|
||||||
|
|
||||||
|
for i in ${DISK}; do
|
||||||
|
|
||||||
|
sgdisk --zap-all $i
|
||||||
|
|
||||||
|
sgdisk -n1:1M:+1G -t1:EF00 $i
|
||||||
|
|
||||||
|
sgdisk -n2:0:+4G -t2:BE00 $i
|
||||||
|
|
||||||
|
test -z $INST_PARTSIZE_SWAP || sgdisk -n4:0:+${INST_PARTSIZE_SWAP}G -t4:8200 $i
|
||||||
|
|
||||||
|
if test -z $INST_PARTSIZE_RPOOL; then
|
||||||
|
sgdisk -n3:0:0 -t3:BF00 $i
|
||||||
|
else
|
||||||
|
sgdisk -n3:0:+${INST_PARTSIZE_RPOOL}G -t3:BF00 $i
|
||||||
|
fi
|
||||||
|
|
||||||
|
sgdisk -a1 -n5:24K:+1000K -t5:EF02 $i
|
||||||
|
done
|
||||||
|
|
||||||
|
#. Create boot pool::
|
||||||
|
|
||||||
|
zpool create \
|
||||||
|
-o compatibility=grub2 \
|
||||||
|
-o ashift=12 \
|
||||||
|
-o autotrim=on \
|
||||||
|
-O acltype=posixacl \
|
||||||
|
-O canmount=off \
|
||||||
|
-O compression=lz4 \
|
||||||
|
-O devices=off \
|
||||||
|
-O normalization=formD \
|
||||||
|
-O relatime=on \
|
||||||
|
-O xattr=sa \
|
||||||
|
-O mountpoint=/boot \
|
||||||
|
-R /mnt \
|
||||||
|
bpool \
|
||||||
|
mirror \
|
||||||
|
$(for i in ${DISK}; do
|
||||||
|
printf "$i-part2 ";
|
||||||
|
done)
|
||||||
|
|
||||||
|
If not using a multi-disk setup, remove ``mirror``.
|
||||||
|
|
||||||
|
You should not need to customize any of the options for the boot pool.
|
||||||
|
|
||||||
|
GRUB does not support all of the zpool features. See ``spa_feature_names``
|
||||||
|
in `grub-core/fs/zfs/zfs.c
|
||||||
|
<http://git.savannah.gnu.org/cgit/grub.git/tree/grub-core/fs/zfs/zfs.c#n276>`__.
|
||||||
|
This step creates a separate boot pool for ``/boot`` with the features
|
||||||
|
limited to only those that GRUB supports, allowing the root pool to use
|
||||||
|
any/all features.
|
||||||
|
|
||||||
|
Features enabled with ``-o compatibility=grub2`` can be seen
|
||||||
|
`here <https://github.com/openzfs/zfs/blob/master/cmd/zpool/compatibility.d/grub2>`__.
|
||||||
|
|
||||||
|
#. Create root pool::
|
||||||
|
|
||||||
|
zpool create \
|
||||||
|
-o ashift=12 \
|
||||||
|
-o autotrim=on \
|
||||||
|
-R /mnt \
|
||||||
|
-O acltype=posixacl \
|
||||||
|
-O canmount=off \
|
||||||
|
-O compression=zstd \
|
||||||
|
-O dnodesize=auto \
|
||||||
|
-O normalization=formD \
|
||||||
|
-O relatime=on \
|
||||||
|
-O xattr=sa \
|
||||||
|
-O mountpoint=/ \
|
||||||
|
rpool \
|
||||||
|
mirror \
|
||||||
|
$(for i in ${DISK}; do
|
||||||
|
printf "$i-part3 ";
|
||||||
|
done)
|
||||||
|
|
||||||
|
If not using a multi-disk setup, remove ``mirror``.
|
||||||
|
|
||||||
|
#. This section implements dataset layout as described in `overview <1-preparation.html>`__.
|
||||||
|
|
||||||
|
Create root system container:
|
||||||
|
|
||||||
|
- Unencrypted::
|
||||||
|
|
||||||
|
zfs create \
|
||||||
|
-o canmount=off \
|
||||||
|
-o mountpoint=none \
|
||||||
|
rpool/redhat
|
||||||
|
|
||||||
|
- Encrypted:
|
||||||
|
|
||||||
|
Pick a strong password. Once compromised, changing password will not keep your
|
||||||
|
data safe. See ``zfs-change-key(8)`` for more info::
|
||||||
|
|
||||||
|
zfs create \
|
||||||
|
-o canmount=off \
|
||||||
|
-o mountpoint=none \
|
||||||
|
-o encryption=on \
|
||||||
|
-o keylocation=prompt \
|
||||||
|
-o keyformat=passphrase \
|
||||||
|
rpool/redhat
|
||||||
|
|
||||||
|
Create system datasets::
|
||||||
|
|
||||||
|
zfs create -o canmount=on -o mountpoint=/ rpool/redhat/root
|
||||||
|
zfs create -o canmount=on -o mountpoint=/home rpool/redhat/home
|
||||||
|
zfs create -o canmount=off -o mountpoint=/var rpool/redhat/var
|
||||||
|
zfs create -o canmount=on rpool/redhat/var/lib
|
||||||
|
zfs create -o canmount=on rpool/redhat/var/log
|
||||||
|
|
||||||
|
Create boot dataset::
|
||||||
|
|
||||||
|
zfs create -o canmount=on -o mountpoint=/boot bpool/redhat
|
||||||
|
|
||||||
|
#. Format and mount ESP::
|
||||||
|
|
||||||
|
for i in ${DISK}; do
|
||||||
|
mkfs.vfat -n EFI ${i}-part1
|
||||||
|
mkdir -p /mnt/boot/efis/${i##*/}-part1
|
||||||
|
mount -t vfat ${i}-part1 /mnt/boot/efis/${i##*/}-part1
|
||||||
|
done
|
||||||
|
|
||||||
|
mkdir -p /mnt/boot/efi
|
||||||
|
mount -t vfat $(echo $DISK | cut -f1 -d\ )-part1 /mnt/boot/efi
|
||||||
|
|
||||||
|
#. Install packages::
|
||||||
|
|
||||||
|
dnf --installroot=/mnt --releasever=$(source /etc/os-release ; echo $VERSION_ID) -y install \
|
||||||
|
@core grub2-efi-x64 grub2-pc-modules grub2-efi-x64-modules shim-x64 efibootmgr kernel
|
||||||
|
|
||||||
|
dnf --installroot=/mnt --releasever=$(source /etc/os-release ; echo $VERSION_ID) -y install \
|
||||||
|
https://zfsonlinux.org/epel/zfs-release-el-2-1.noarch.rpm
|
||||||
|
|
||||||
|
dnf config-manager --installroot=/mnt --disable zfs
|
||||||
|
dnf config-manager --installroot=/mnt --enable zfs-kmod
|
||||||
|
|
||||||
|
dnf --installroot=/mnt --releasever=$(source /etc/os-release ; echo $VERSION_ID) \
|
||||||
|
--nogpgcheck -y install zfs zfs-dracut
|
||||||
@@ -0,0 +1,66 @@
|
|||||||
|
.. highlight:: sh
|
||||||
|
|
||||||
|
System Configuration
|
||||||
|
======================
|
||||||
|
|
||||||
|
.. contents:: Table of Contents
|
||||||
|
:local:
|
||||||
|
|
||||||
|
#. Generate fstab::
|
||||||
|
|
||||||
|
mkdir -p /mnt/etc/
|
||||||
|
for i in ${DISK}; do
|
||||||
|
echo UUID=$(blkid -s UUID -o value ${i}-part1) /boot/efis/${i##*/}-part1 vfat \
|
||||||
|
umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab
|
||||||
|
done
|
||||||
|
echo $(echo $DISK | cut -f1 -d\ )-part1 /boot/efi vfat \
|
||||||
|
noauto,umask=0022,fmask=0022,dmask=0022 0 1 >> /mnt/etc/fstab
|
||||||
|
|
||||||
|
#. Configure dracut::
|
||||||
|
|
||||||
|
echo 'add_dracutmodules+=" zfs "' > /mnt/etc/dracut.conf.d/zfs.conf
|
||||||
|
|
||||||
|
#. Force load mpt3sas module if used::
|
||||||
|
|
||||||
|
if grep mpt3sas /proc/modules; then
|
||||||
|
echo 'forced_drivers+=" mpt3sas "' >> /mnt/etc/dracut.conf.d/zfs.conf
|
||||||
|
fi
|
||||||
|
|
||||||
|
#. Set locale, keymap, timezone, hostname and root password::
|
||||||
|
|
||||||
|
rm -f /mnt/etc/localtime
|
||||||
|
systemd-firstboot --root=/mnt --prompt --root-password=PASSWORD --force
|
||||||
|
|
||||||
|
#. Generate host id::
|
||||||
|
|
||||||
|
zgenhostid -f -o /mnt/etc/hostid
|
||||||
|
|
||||||
|
#. Install locale package, example for English locale::
|
||||||
|
|
||||||
|
dnf --installroot=/mnt install -y glibc-minimal-langpack glibc-langpack-en
|
||||||
|
|
||||||
|
#. Enable ZFS services::
|
||||||
|
|
||||||
|
systemctl enable zfs-import-scan.service zfs-mount zfs-import.target zfs-zed zfs.target --root=/mnt
|
||||||
|
|
||||||
|
#. By default SSH server is enabled, allowing root login by password,
|
||||||
|
disable SSH server::
|
||||||
|
|
||||||
|
systemctl disable sshd --root=/mnt
|
||||||
|
systemctl enable firewalld --root=/mnt
|
||||||
|
|
||||||
|
#. Chroot::
|
||||||
|
|
||||||
|
m='/dev /proc /sys'
|
||||||
|
for i in $m; do mount --rbind $i /mnt/$i; done
|
||||||
|
|
||||||
|
history -w /mnt/home/sys-install-pre-chroot.txt
|
||||||
|
chroot /mnt /usr/bin/env DISK=$DISK bash --login
|
||||||
|
|
||||||
|
#. For SELinux, relabel filesystem on reboot::
|
||||||
|
|
||||||
|
fixfiles -F onboot
|
||||||
|
|
||||||
|
#. Set root password::
|
||||||
|
|
||||||
|
passwd
|
||||||
@@ -0,0 +1,129 @@
|
|||||||
|
.. highlight:: sh
|
||||||
|
|
||||||
|
Bootloader
|
||||||
|
======================
|
||||||
|
|
||||||
|
.. contents:: Table of Contents
|
||||||
|
:local:
|
||||||
|
|
||||||
|
Apply workarounds
|
||||||
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
|
Currently GRUB has multiple compatibility problems with ZFS,
|
||||||
|
especially with regards to newer ZFS features.
|
||||||
|
Workarounds have to be applied.
|
||||||
|
|
||||||
|
#. grub2-probe fails to get canonical path
|
||||||
|
|
||||||
|
When persistent device names ``/dev/disk/by-id/*`` are used
|
||||||
|
with ZFS, GRUB will fail to resolve the path of the boot pool
|
||||||
|
device. Error::
|
||||||
|
|
||||||
|
# /usr/bin/grub2-probe: error: failed to get canonical path of `/dev/virtio-pci-0000:06:00.0-part3'.
|
||||||
|
|
||||||
|
Solution::
|
||||||
|
|
||||||
|
echo 'export ZPOOL_VDEV_NAME_PATH=YES' >> /etc/profile.d/zpool_vdev_name_path.sh
|
||||||
|
source /etc/profile.d/zpool_vdev_name_path.sh
|
||||||
|
|
||||||
|
#. Pool name missing
|
||||||
|
|
||||||
|
See `this bug report <https://savannah.gnu.org/bugs/?59614>`__.
|
||||||
|
Root pool name is missing from ``root=ZFS=rpool_$INST_UUID/ROOT/default``
|
||||||
|
kernel cmdline in generated ``grub.cfg`` file.
|
||||||
|
|
||||||
|
A workaround is to replace the pool name detection with ``zdb``
|
||||||
|
command::
|
||||||
|
|
||||||
|
sed -i "s|rpool=.*|rpool=\`zdb -l \${GRUB_DEVICE} \| grep -E '[[:blank:]]name' \| cut -d\\\' -f 2\`|" /etc/grub.d/10_linux
|
||||||
|
|
||||||
|
Caution: this fix must be applied after every GRUB update and before generating the menu.
|
||||||
|
|
||||||
|
Install GRUB
|
||||||
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
#. If using virtio disk, add driver to initrd::
|
||||||
|
|
||||||
|
echo 'filesystems+=" virtio_blk "' >> /etc/dracut.conf.d/fs.conf
|
||||||
|
|
||||||
|
#. Create empty cache file and generate initrd::
|
||||||
|
|
||||||
|
rm -f /etc/zfs/zpool.cache
|
||||||
|
touch /etc/zfs/zpool.cache
|
||||||
|
chmod a-w /etc/zfs/zpool.cache
|
||||||
|
chattr +i /etc/zfs/zpool.cache
|
||||||
|
|
||||||
|
for directory in /lib/modules/*; do
|
||||||
|
kernel_version=$(basename $directory)
|
||||||
|
dracut --force --kver $kernel_version
|
||||||
|
done
|
||||||
|
|
||||||
|
#. Load ZFS modules and disable BLS::
|
||||||
|
|
||||||
|
echo 'GRUB_ENABLE_BLSCFG=false' >> /etc/default/grub
|
||||||
|
|
||||||
|
#. If using legacy booting, install GRUB to every disk::
|
||||||
|
|
||||||
|
for i in ${DISK}; do
|
||||||
|
grub2-install --target=i386-pc $i
|
||||||
|
done
|
||||||
|
|
||||||
|
#. If using EFI::
|
||||||
|
|
||||||
|
for i in ${DISK}; do
|
||||||
|
efibootmgr -cgp 1 -l "\EFI\almalinux\shimx64.efi" \
|
||||||
|
-L "almalinux-${i##*/}" -d ${i}
|
||||||
|
done
|
||||||
|
cp -r /usr/lib/grub/x86_64-efi/ /boot/efi/EFI/almalinux/
|
||||||
|
|
||||||
|
#. Generate GRUB Menu:
|
||||||
|
|
||||||
|
Generate menu::
|
||||||
|
|
||||||
|
grub2-mkconfig -o /boot/grub2/grub.cfg
|
||||||
|
cp /boot/grub2/grub.cfg /boot/efi/EFI/almalinux/
|
||||||
|
|
||||||
|
#. For both legacy and EFI booting: mirror ESP content::
|
||||||
|
|
||||||
|
ESP_MIRROR=$(mktemp -d)
|
||||||
|
unalias -a
|
||||||
|
cp -r /boot/efi/EFI $ESP_MIRROR
|
||||||
|
for i in /boot/efis/*; do
|
||||||
|
cp -r $ESP_MIRROR/EFI $i
|
||||||
|
done
|
||||||
|
rm -rf $ESP_MIRROR
|
||||||
|
|
||||||
|
#. Notes for GRUB on RHEL
|
||||||
|
|
||||||
|
As bls is disabled, you will need to regenerate GRUB menu after each kernel upgrade.
|
||||||
|
Or else the new kernel will not be recognized and system will boot the old kernel
|
||||||
|
on reboot.
|
||||||
|
|
||||||
|
Finish Installation
|
||||||
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
#. Exit chroot::
|
||||||
|
|
||||||
|
exit
|
||||||
|
|
||||||
|
#. Export pools::
|
||||||
|
|
||||||
|
umount -Rl /mnt
|
||||||
|
zpool export -a
|
||||||
|
|
||||||
|
#. Reboot::
|
||||||
|
|
||||||
|
reboot
|
||||||
|
|
||||||
|
#. On first reboot, the boot process will fail, with failure messages such
|
||||||
|
as "You are in Emergency Mode...Press Ctrl-D to continue".
|
||||||
|
|
||||||
|
Wait for the computer to automatically reboot and the problem will be resolved.
|
||||||
|
|
||||||
|
Post installaion
|
||||||
|
~~~~~~~~~~~~~~~~
|
||||||
|
#. Install package groups::
|
||||||
|
|
||||||
|
dnf group list --hidden -v # query package groups
|
||||||
|
dnf group install @gnome-desktop
|
||||||
|
|
||||||
|
#. Add new user, configure swap.
|
||||||
@@ -146,15 +146,15 @@ And for RHEL/CentOS 8 and newer::
|
|||||||
Use *zfs-testing* for DKMS packages and *zfs-testing-kmod*
|
Use *zfs-testing* for DKMS packages and *zfs-testing-kmod*
|
||||||
kABI-tracking kmod packages.
|
kABI-tracking kmod packages.
|
||||||
|
|
||||||
RHEL 8-based distro Root on ZFS
|
RHEL-based distro Root on ZFS
|
||||||
-------------------------------
|
-------------------------------
|
||||||
`Start here <RHEL%208-based%20distro%20Root%20on%20ZFS/0-overview.html>`__.
|
`Start here <RHEL%20-based%20distro%20Root%20on%20ZFS/1-preparation.html>`__.
|
||||||
|
|
||||||
.. toctree::
|
.. toctree::
|
||||||
:maxdepth: 1
|
:maxdepth: 1
|
||||||
:glob:
|
:glob:
|
||||||
|
|
||||||
RHEL 8-based distro Root on ZFS/*
|
RHEL-based distro Root on ZFS/*
|
||||||
|
|
||||||
.. _kABI-tracking kmod: https://elrepoproject.blogspot.com/2016/02/kabi-tracking-kmod-packages.html
|
.. _kABI-tracking kmod: https://elrepoproject.blogspot.com/2016/02/kabi-tracking-kmod-packages.html
|
||||||
.. _DKMS: https://en.wikipedia.org/wiki/Dynamic_Kernel_Module_Support
|
.. _DKMS: https://en.wikipedia.org/wiki/Dynamic_Kernel_Module_Support
|
||||||
|
|||||||
Reference in New Issue
Block a user