Add support for NixOS + Fedora, Arch, RH minor changes (#195)
* Add support for NixOS Signed-off-by: Maurice Zhou <jasper@apvc.uk> * updated files for immutable root fs Signed-off-by: Maurice Zhou <jasper@apvc.uk> * update ssh key location Signed-off-by: Maurice Zhou <jasper@apvc.uk> * change initrd host key name Signed-off-by: Maurice Zhou <jasper@apvc.uk> * add missing -p switch Signed-off-by: Maurice Zhou <jasper@apvc.uk> * comments about SSH and encrypted bpool Signed-off-by: Maurice Zhou <jasper@apvc.uk> * backup hardware-configuration.nix file Signed-off-by: Maurice Zhou <jasper@apvc.uk> * clearer wording for backup Signed-off-by: Maurice Zhou <jasper@apvc.uk> * global useDHCP flag is deprecated Signed-off-by: Maurice Zhou <jasper@apvc.uk> * point user to configuration.nix Signed-off-by: Maurice Zhou <jasper@apvc.uk> * fixes for systemd-autofs; network; typo Signed-off-by: Maurice Zhou <jasper@apvc.uk> * force command to return 0 Signed-off-by: Maurice Zhou <jasper@apvc.uk> * fix command to postMount Signed-off-by: Maurice Zhou <jasper@apvc.uk> * fix mount command Signed-off-by: Maurice Zhou <jasper@apvc.uk> * escape $i Signed-off-by: Maurice Zhou <jasper@apvc.uk> * create mountpoints before booting Signed-off-by: Maurice Zhou <jasper@apvc.uk> * replace postDevice,preMount command Signed-off-by: Maurice Zhou <jasper@apvc.uk> * disable zfs-mount Signed-off-by: Maurice Zhou <jasper@apvc.uk> * generate machine-id Signed-off-by: Maurice Zhou <jasper@apvc.uk>
This commit is contained in:
ne9z
11
docs/Getting Started/NixOS/Root on ZFS.rst
Normal file
11
docs/Getting Started/NixOS/Root on ZFS.rst
Normal file
@@ -0,0 +1,11 @@
|
||||
NixOS Root on ZFS
|
||||
=======================================
|
||||
`Start here <Root%20on%20ZFS/0-overview.html>`__.
|
||||
|
||||
Contents
|
||||
--------
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:glob:
|
||||
|
||||
Root on ZFS/*
|
||||
142
docs/Getting Started/NixOS/Root on ZFS/0-overview.rst
Normal file
142
docs/Getting Started/NixOS/Root on ZFS/0-overview.rst
Normal file
@@ -0,0 +1,142 @@
|
||||
.. highlight:: sh
|
||||
|
||||
Overview
|
||||
======================
|
||||
This document describes how to install NixOS with ZFS as root
|
||||
file system.
|
||||
|
||||
Caution
|
||||
~~~~~~~
|
||||
- This guide wipes entire physical disks. Back up existing data.
|
||||
- `GRUB does not and
|
||||
will not work on 4Kn drive with legacy (BIOS) booting.
|
||||
<http://savannah.gnu.org/bugs/?46700>`__
|
||||
|
||||
Partition layout
|
||||
~~~~~~~~~~~~~~~~
|
||||
|
||||
GUID partition table (GPT) is used.
|
||||
EFI system partition will be referred to as **ESP** in this document.
|
||||
|
||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
||||
| Name | legacy boot | ESP | Boot pool | swap | root pool | remaining space |
|
||||
+======================+======================+=======================+======================+=====================+=======================+=================+
|
||||
| File system | | vfat | ZFS | swap | ZFS | |
|
||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
||||
| Size | 1M | 2G | 4G | depends on RAM size | | |
|
||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
||||
| Optional encryption | | *Secure Boot* | | plain dm-crypt | ZFS native encryption | |
|
||||
| | | | | | | |
|
||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
||||
| Partition no. | 5 | 1 | 2 | 4 | 3 | |
|
||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
||||
| Mount point | | | /boot | | / | |
|
||||
| | | /boot/efis/disk-part1 | | | | |
|
||||
+----------------------+----------------------+-----------------------+----------------------+---------------------+-----------------------+-----------------+
|
||||
|
||||
Dataset layout
|
||||
~~~~~~~~~~~~~~
|
||||
|
||||
The dataset layout used in this guide follows stardard
|
||||
mutable file positions (``/var``, ``/etc``, ...), but can
|
||||
still be modified to `a immutable root <https://grahamc.com/blog/erase-your-darlings>`__
|
||||
after installation.
|
||||
|
||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
||||
| Dataset | canmount | mountpoint | container | notes |
|
||||
+===========================+======================+======================+=====================================+===========================================+
|
||||
| bpool | off | /boot | contains sys | |
|
||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
||||
| rpool | off | / | contains sys | |
|
||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
||||
| bpool/sys | off | none | contains BOOT | |
|
||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
||||
| rpool/sys | off | none | contains ROOT | sys is encryptionroot |
|
||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
||||
| bpool/sys/BOOT | off | none | contains boot environments | |
|
||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
||||
| rpool/sys/ROOT | off | none | contains boot environments | |
|
||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
||||
| rpool/sys/DATA | off | none | contains placeholder "default" | |
|
||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
||||
| rpool/sys/DATA/default | off | / | contains user datasets | child datsets inherits mountpoint |
|
||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
||||
| rpool/sys/DATA/local | off | / | contains /nix datasets | child datsets inherits mountpoint |
|
||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
||||
| rpool/sys/DATA/default/ | on | /home (inherited) | no | |
|
||||
| home | | | | user datasets, also called "shared |
|
||||
| | | | | datasets", "persistent datasets"; also |
|
||||
| | | | | include /var/lib, /srv, ... |
|
||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
||||
| bpool/sys/BOOT/default | noauto | /boot | no | noauto is used to switch BE. because of |
|
||||
| | | | | noauto, must use fstab to mount |
|
||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
||||
| rpool/sys/ROOT/default | noauto | / | no | mounted by initrd zfs hook |
|
||||
+---------------------------+----------------------+----------------------+-------------------------------------+-------------------------------------------+
|
||||
|
||||
Encryption
|
||||
~~~~~~~~~~
|
||||
|
||||
- Swap
|
||||
|
||||
Swap is always encrypted. By default, swap is encrypted
|
||||
with plain dm-crypt with key generated from ``/dev/urandom``
|
||||
at every boot. Swap content does not persist between reboots.
|
||||
|
||||
- Root pool
|
||||
|
||||
ZFS native encryption can be optionally enabled for ``rpool/sys``
|
||||
and child datasets.
|
||||
|
||||
User should be aware that, ZFS native encryption does not
|
||||
encrypt some metadata of the datasets.
|
||||
ZFS native encryption also does not change master key when ``zfs change-key`` is invoked.
|
||||
Therefore, you should wipe the disk when password is compromised to protect confidentiality.
|
||||
See `zfs-load-key.8 <https://openzfs.github.io/openzfs-docs/man/8/zfs-load-key.8.html>`__
|
||||
and `zfs-change-key.8 <https://openzfs.github.io/openzfs-docs/man/8/zfs-change-key.8.html>`__
|
||||
for more information regarding ZFS native encryption.
|
||||
|
||||
Encryption is enabled at dataset creation and can not be disabled later.
|
||||
|
||||
- Boot pool
|
||||
|
||||
Boot pool can not be encrypted.
|
||||
|
||||
- Bootloader
|
||||
|
||||
Bootloader can not be encrypted.
|
||||
|
||||
However, with Secure Boot, bootloader
|
||||
can be verified by motherboard firmware to be untempered,
|
||||
which should be sufficient for most purposes.
|
||||
|
||||
Secure Boot is not supported out-of-the-box due to ZFS module.
|
||||
|
||||
Booting with disk failure
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
This guide is written with disk failure in mind.
|
||||
|
||||
If disks used in Root on ZFS pool failed, but
|
||||
sufficient redundancy for both root pool and boot pool
|
||||
still exists, the system will still boot normally.
|
||||
|
||||
Swap partition on the failed disk will fail to mount,
|
||||
after an 1m30s timeout.
|
||||
|
||||
This feature is useful for use cases such
|
||||
as an unattended remote server.
|
||||
|
||||
Example:
|
||||
|
||||
- System has disks ``n>1``
|
||||
|
||||
- Installed with mirrored setup
|
||||
|
||||
- Mirrored setup can tolerate up to ``n-1`` disk failures
|
||||
|
||||
- Disconnect one or more disks, keep at least
|
||||
one disk connected
|
||||
|
||||
- System still boots, but fails to mount swap and
|
||||
EFI partition
|
||||
109
docs/Getting Started/NixOS/Root on ZFS/1-preparation.rst
Normal file
109
docs/Getting Started/NixOS/Root on ZFS/1-preparation.rst
Normal file
@@ -0,0 +1,109 @@
|
||||
.. highlight:: sh
|
||||
|
||||
Preparation
|
||||
======================
|
||||
|
||||
.. contents:: Table of Contents
|
||||
:local:
|
||||
|
||||
#. Download `Minimal ISO image
|
||||
<https://channels.nixos.org/nixos-21.05/latest-nixos-minimal-x86_64-linux.iso>`__ and boot from it.
|
||||
|
||||
#. Connect to network. See `NixOS manual <https://nixos.org/manual/nixos/stable/index.html#sec-installation-booting>`__.
|
||||
|
||||
#. SSH server is enabled by default. To connect, set root password with::
|
||||
|
||||
sudo passwd
|
||||
|
||||
#. Connect from another computer::
|
||||
|
||||
ssh root@192.168.1.19
|
||||
|
||||
#. Unique pool suffix. ZFS expects pool names to be
|
||||
unique, therefore it's recommended to create
|
||||
pools with a unique suffix::
|
||||
|
||||
INST_UUID=$(dd if=/dev/urandom bs=1 count=100 2>/dev/null | tr -dc 'a-z0-9' | cut -c-6)
|
||||
|
||||
#. Identify this installation in ZFS filesystem path::
|
||||
|
||||
INST_ID=nixos
|
||||
|
||||
#. Root on ZFS configuration file name::
|
||||
|
||||
INST_CONFIG_FILE='zfs.nix'
|
||||
|
||||
#. Target disk
|
||||
|
||||
List available disks with::
|
||||
|
||||
ls /dev/disk/by-id/*
|
||||
|
||||
If using virtio as disk bus, use
|
||||
``/dev/disk/by-path/*``.
|
||||
|
||||
Declare disk array::
|
||||
|
||||
DISK='/dev/disk/by-id/ata-FOO /dev/disk/by-id/nvme-BAR'
|
||||
|
||||
For single disk installation, use::
|
||||
|
||||
DISK='/dev/disk/by-id/disk1'
|
||||
|
||||
#. Choose a primary disk. This disk will be used
|
||||
for primary EFI partition, default to
|
||||
first disk in the array::
|
||||
|
||||
INST_PRIMARY_DISK=$(echo $DISK | cut -f1 -d\ )
|
||||
|
||||
#. Set vdev topology, possible values are:
|
||||
|
||||
- (not set, single disk or striped; no redundancy)
|
||||
- mirror
|
||||
- raidz1
|
||||
- raidz2
|
||||
- raidz3
|
||||
|
||||
::
|
||||
|
||||
INST_VDEV=
|
||||
|
||||
This will create a single vdev with the topology of your choice.
|
||||
It is also possible to manually create a pool with multiple vdevs, such as::
|
||||
|
||||
zpool create --options \
|
||||
poolName \
|
||||
mirror sda sdb \
|
||||
raidz2 sdc ... \
|
||||
raidz3 sde ... \
|
||||
spare sdf ...
|
||||
|
||||
Notice the cost of parity when using RAID-Z. See
|
||||
`here <https://www.delphix.com/blog/delphix-engineering/zfs-raidz-stripe-width-or-how-i-learned-stop-worrying-and-love-raidz>`__
|
||||
and `here <https://docs.google.com/spreadsheets/d/1tf4qx1aMJp8Lo_R6gpT689wTjHv6CGVElrPqTA0w_ZY/>`__.
|
||||
|
||||
Refer to `zpoolconcepts <https://openzfs.github.io/openzfs-docs/man/7/zpoolconcepts.7.html>`__
|
||||
and `zpool-create <https://openzfs.github.io/openzfs-docs/man/8/zpool-create.8.html>`__
|
||||
man pages for details.
|
||||
|
||||
#. Set partition size:
|
||||
|
||||
Set ESP size::
|
||||
|
||||
INST_PARTSIZE_ESP=2 # in GB
|
||||
|
||||
Set boot pool size. To avoid running out of space while using
|
||||
boot environments, the minimum is 4GB. Adjust the size if you
|
||||
intend to use multiple kernel/distros::
|
||||
|
||||
INST_PARTSIZE_BPOOL=4
|
||||
|
||||
Set swap size. It's `recommended <https://chrisdown.name/2018/01/02/in-defence-of-swap.html>`__
|
||||
to setup a swap partition. If you intend to use hibernation,
|
||||
the minimum should be no less than RAM size. Skip if swap is not needed::
|
||||
|
||||
INST_PARTSIZE_SWAP=8
|
||||
|
||||
Root pool size, use all remaining disk space if not set::
|
||||
|
||||
INST_PARTSIZE_RPOOL=
|
||||
@@ -0,0 +1,356 @@
|
||||
.. highlight:: sh
|
||||
|
||||
System Configuration
|
||||
======================
|
||||
|
||||
.. contents:: Table of Contents
|
||||
:local:
|
||||
|
||||
#. Optional: wipe solid-state drives with the generic tool
|
||||
`blkdiscard <https://utcc.utoronto.ca/~cks/space/blog/linux/ErasingSSDsWithBlkdiscard>`__,
|
||||
to clean previous partition tables and improve performance.
|
||||
|
||||
All content will be irrevocably destroyed::
|
||||
|
||||
for i in ${DISK}; do
|
||||
blkdiscard -f $i &
|
||||
done
|
||||
wait
|
||||
|
||||
This is a quick operation and should be completed under one
|
||||
minute.
|
||||
|
||||
For other device specific methods, see
|
||||
`Memory cell clearing <https://wiki.archlinux.org/title/Solid_state_drive/Memory_cell_clearing>`__
|
||||
|
||||
#. Partition the disks.
|
||||
See `Overview <0-overview.html>`__ for details::
|
||||
|
||||
for i in ${DISK}; do
|
||||
sgdisk --zap-all $i
|
||||
sgdisk -n1:1M:+${INST_PARTSIZE_ESP}G -t1:EF00 $i
|
||||
sgdisk -n2:0:+${INST_PARTSIZE_BPOOL}G -t2:BE00 $i
|
||||
if [ "${INST_PARTSIZE_SWAP}" != "" ]; then
|
||||
sgdisk -n4:0:+${INST_PARTSIZE_SWAP}G -t4:8200 $i
|
||||
fi
|
||||
if [ "${INST_PARTSIZE_RPOOL}" = "" ]; then
|
||||
sgdisk -n3:0:0 -t3:BF00 $i
|
||||
else
|
||||
sgdisk -n3:0:+${INST_PARTSIZE_RPOOL}G -t3:BF00 $i
|
||||
fi
|
||||
sgdisk -a1 -n5:24K:+1000K -t5:EF02 $i
|
||||
done
|
||||
|
||||
#. Create boot pool::
|
||||
|
||||
zpool create \
|
||||
-d -o feature@async_destroy=enabled \
|
||||
-o feature@bookmarks=enabled \
|
||||
-o feature@embedded_data=enabled \
|
||||
-o feature@empty_bpobj=enabled \
|
||||
-o feature@enabled_txg=enabled \
|
||||
-o feature@extensible_dataset=enabled \
|
||||
-o feature@filesystem_limits=enabled \
|
||||
-o feature@hole_birth=enabled \
|
||||
-o feature@large_blocks=enabled \
|
||||
-o feature@lz4_compress=enabled \
|
||||
-o feature@spacemap_histogram=enabled \
|
||||
-o ashift=12 \
|
||||
-o autotrim=on \
|
||||
-O acltype=posixacl \
|
||||
-O canmount=off \
|
||||
-O compression=lz4 \
|
||||
-O devices=off \
|
||||
-O normalization=formD \
|
||||
-O relatime=on \
|
||||
-O xattr=sa \
|
||||
-O mountpoint=/boot \
|
||||
-R /mnt \
|
||||
bpool_$INST_UUID \
|
||||
$INST_VDEV \
|
||||
$(for i in ${DISK}; do
|
||||
printf "$i-part2 ";
|
||||
done)
|
||||
|
||||
You should not need to customize any of the options for the boot pool.
|
||||
|
||||
GRUB does not support all of the zpool features. See ``spa_feature_names``
|
||||
in `grub-core/fs/zfs/zfs.c
|
||||
<http://git.savannah.gnu.org/cgit/grub.git/tree/grub-core/fs/zfs/zfs.c#n276>`__.
|
||||
This step creates a separate boot pool for ``/boot`` with the features
|
||||
limited to only those that GRUB supports, allowing the root pool to use
|
||||
any/all features.
|
||||
|
||||
Features enabled with ``-o compatibility=grub2`` can be seen
|
||||
`here <https://github.com/openzfs/zfs/blob/master/cmd/zpool/compatibility.d/grub2>`__.
|
||||
|
||||
#. Create root pool::
|
||||
|
||||
zpool create \
|
||||
-o ashift=12 \
|
||||
-o autotrim=on \
|
||||
-R /mnt \
|
||||
-O acltype=posixacl \
|
||||
-O canmount=off \
|
||||
-O compression=zstd \
|
||||
-O dnodesize=auto \
|
||||
-O normalization=formD \
|
||||
-O relatime=on \
|
||||
-O xattr=sa \
|
||||
-O mountpoint=/ \
|
||||
rpool_$INST_UUID \
|
||||
$INST_VDEV \
|
||||
$(for i in ${DISK}; do
|
||||
printf "$i-part3 ";
|
||||
done)
|
||||
|
||||
**Notes:**
|
||||
|
||||
- The use of ``ashift=12`` is recommended here because many drives
|
||||
today have 4 KiB (or larger) physical sectors, even though they
|
||||
present 512 B logical sectors. Also, a future replacement drive may
|
||||
have 4 KiB physical sectors (in which case ``ashift=12`` is desirable)
|
||||
or 4 KiB logical sectors (in which case ``ashift=12`` is required).
|
||||
- Setting ``-O acltype=posixacl`` enables POSIX ACLs globally. If you
|
||||
do not want this, remove that option, but later add
|
||||
``-o acltype=posixacl`` (note: lowercase “o”) to the ``zfs create``
|
||||
for ``/var/log``, as `journald requires ACLs
|
||||
<https://askubuntu.com/questions/970886/journalctl-says-failed-to-search-journal-acl-operation-not-supported>`__
|
||||
- Setting ``normalization=formD`` eliminates some corner cases relating
|
||||
to UTF-8 filename normalization. It also implies ``utf8only=on``,
|
||||
which means that only UTF-8 filenames are allowed. If you care to
|
||||
support non-UTF-8 filenames, do not use this option. For a discussion
|
||||
of why requiring UTF-8 filenames may be a bad idea, see `The problems
|
||||
with enforced UTF-8 only filenames
|
||||
<http://utcc.utoronto.ca/~cks/space/blog/linux/ForcedUTF8Filenames>`__.
|
||||
- ``recordsize`` is unset (leaving it at the default of 128 KiB). If you
|
||||
want to tune it (e.g. ``-o recordsize=1M``), see `these
|
||||
<https://jrs-s.net/2019/04/03/on-zfs-recordsize/>`__ `various
|
||||
<http://blog.programster.org/zfs-record-size>`__ `blog
|
||||
<https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSFileRecordsizeGrowth>`__
|
||||
`posts
|
||||
<https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSRecordsizeAndCompression>`__.
|
||||
- Setting ``relatime=on`` is a middle ground between classic POSIX
|
||||
``atime`` behavior (with its significant performance impact) and
|
||||
``atime=off`` (which provides the best performance by completely
|
||||
disabling atime updates). Since Linux 2.6.30, ``relatime`` has been
|
||||
the default for other filesystems. See `RedHat’s documentation
|
||||
<https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/power_management_guide/relatime>`__
|
||||
for further information.
|
||||
- Setting ``xattr=sa`` `vastly improves the performance of extended
|
||||
attributes
|
||||
<https://github.com/zfsonlinux/zfs/commit/82a37189aac955c81a59a5ecc3400475adb56355>`__.
|
||||
Inside ZFS, extended attributes are used to implement POSIX ACLs.
|
||||
Extended attributes can also be used by user-space applications.
|
||||
`They are used by some desktop GUI applications.
|
||||
<https://en.wikipedia.org/wiki/Extended_file_attributes#Linux>`__
|
||||
`They can be used by Samba to store Windows ACLs and DOS attributes;
|
||||
they are required for a Samba Active Directory domain controller.
|
||||
<https://wiki.samba.org/index.php/Setting_up_a_Share_Using_Windows_ACLs>`__
|
||||
Note that ``xattr=sa`` is `Linux-specific
|
||||
<https://openzfs.org/wiki/Platform_code_differences>`__. If you move your
|
||||
``xattr=sa`` pool to another OpenZFS implementation besides ZFS-on-Linux,
|
||||
extended attributes will not be readable (though your data will be). If
|
||||
portability of extended attributes is important to you, omit the
|
||||
``-O xattr=sa`` above. Even if you do not want ``xattr=sa`` for the whole
|
||||
pool, it is probably fine to use it for ``/var/log``.
|
||||
- Make sure to include the ``-part3`` portion of the drive path. If you
|
||||
forget that, you are specifying the whole disk, which ZFS will then
|
||||
re-partition, and you will lose the bootloader partition(s).
|
||||
|
||||
#. This section implements dataset layout as described in `overview <0-overview.html>`__.
|
||||
|
||||
Create root system container:
|
||||
|
||||
- Unencrypted::
|
||||
|
||||
zfs create \
|
||||
-o canmount=off \
|
||||
-o mountpoint=none \
|
||||
rpool_$INST_UUID/$INST_ID
|
||||
|
||||
- Encrypted:
|
||||
|
||||
Pick a strong password. Once compromised, changing password will not keep your
|
||||
data safe. See ``zfs-change-key(8)`` for more info::
|
||||
|
||||
zfs create \
|
||||
-o canmount=off \
|
||||
-o mountpoint=none \
|
||||
-o encryption=aes-256-gcm \
|
||||
-o keylocation=prompt \
|
||||
-o keyformat=passphrase \
|
||||
rpool_$INST_UUID/$INST_ID
|
||||
|
||||
Create other system datasets::
|
||||
|
||||
zfs create -o canmount=off -o mountpoint=none bpool_$INST_UUID/$INST_ID
|
||||
zfs create -o canmount=off -o mountpoint=none bpool_$INST_UUID/$INST_ID/BOOT
|
||||
zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID/ROOT
|
||||
zfs create -o canmount=off -o mountpoint=none rpool_$INST_UUID/$INST_ID/DATA
|
||||
zfs create -o mountpoint=/boot -o canmount=noauto bpool_$INST_UUID/$INST_ID/BOOT/default
|
||||
zfs create -o mountpoint=/ -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/default
|
||||
zfs create -o mountpoint=/ -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/local
|
||||
zfs create -o mountpoint=/ -o canmount=noauto rpool_$INST_UUID/$INST_ID/ROOT/default
|
||||
zfs mount rpool_$INST_UUID/$INST_ID/ROOT/default
|
||||
zfs mount bpool_$INST_UUID/$INST_ID/BOOT/default
|
||||
for i in {usr,var,var/lib};
|
||||
do
|
||||
zfs create -o canmount=off rpool_$INST_UUID/$INST_ID/DATA/default/$i
|
||||
done
|
||||
for i in {home,root,srv,usr/local,var/log,var/spool};
|
||||
do
|
||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/$i
|
||||
done
|
||||
chmod 750 /mnt/root
|
||||
for i in {nix,}; do
|
||||
zfs create -o canmount=on -o mountpoint=/$i rpool_$INST_UUID/$INST_ID/DATA/local/$i
|
||||
done
|
||||
|
||||
Datasets for immutable root filesystem::
|
||||
|
||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/state
|
||||
for i in {/etc/nixos,/etc/cryptkey.d}; do
|
||||
mkdir -p /mnt/state/$i /mnt/$i
|
||||
mount -o bind /mnt/state/$i /mnt/$i
|
||||
done
|
||||
zfs create -o mountpoint=/ -o canmount=noauto rpool_$INST_UUID/$INST_ID/ROOT/empty
|
||||
zfs snapshot rpool_$INST_UUID/$INST_ID/ROOT/empty@start
|
||||
|
||||
#. Format and mount ESP::
|
||||
|
||||
for i in ${DISK}; do
|
||||
mkfs.vfat -n EFI ${i}-part1
|
||||
mkdir -p /mnt/boot/efis/${i##*/}-part1
|
||||
mount -t vfat ${i}-part1 /mnt/boot/efis/${i##*/}-part1
|
||||
done
|
||||
|
||||
#. Create optional user data datasets to omit data from rollback::
|
||||
|
||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/games
|
||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/www
|
||||
# for GNOME
|
||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/AccountsService
|
||||
# for Docker
|
||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/docker
|
||||
# for NFS
|
||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/nfs
|
||||
# for LXC
|
||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/lxc
|
||||
# for LibVirt
|
||||
zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/libvirt
|
||||
##other application
|
||||
# zfs create -o canmount=on rpool_$INST_UUID/$INST_ID/DATA/default/var/lib/$name
|
||||
|
||||
Add other datasets when needed, such as PostgreSQL.
|
||||
|
||||
#. Generate initial NixOS system configuration::
|
||||
|
||||
nixos-generate-config --root /mnt
|
||||
|
||||
This command will generate two files, ``configuration.nix``
|
||||
and ``hardware-configuration-zfs.nix``, which will be the starting point
|
||||
of configuring the system.
|
||||
|
||||
#. Edit config file to import ZFS options::
|
||||
|
||||
sed -i "s|./hardware-configuration.nix|./hardware-configuration-zfs.nix ./${INST_CONFIG_FILE}|g" /mnt/etc/nixos/configuration.nix
|
||||
# backup, prevent being overwritten by nixos-generate-config
|
||||
mv /mnt/etc/nixos/hardware-configuration.nix /mnt/etc/nixos/hardware-configuration-zfs.nix
|
||||
|
||||
#. ZFS options::
|
||||
|
||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
||||
{ config, pkgs, ... }:
|
||||
|
||||
{ boot.supportedFilesystems = [ "zfs" ];
|
||||
networking.hostId = "$(head -c 8 /etc/machine-id)";
|
||||
boot.zfs.devNodes = "${INST_PRIMARY_DISK%/*}";
|
||||
EOF
|
||||
|
||||
ZFS datasets should be mounted with ``-o zfsutil`` option::
|
||||
|
||||
sed -i 's|fsType = "zfs";|fsType = "zfs"; options = [ "zfsutil" ];|g' \
|
||||
/mnt/etc/nixos/hardware-configuration-zfs.nix
|
||||
|
||||
Allow EFI system partition mounting to fail at boot::
|
||||
|
||||
sed -i 's|fsType = "vfat";|fsType = "vfat"; options = [ "x-systemd.idle-timeout=1min" "x-systemd.automount" "noauto" ];|g' \
|
||||
/mnt/etc/nixos/hardware-configuration-zfs.nix
|
||||
|
||||
Disable cache::
|
||||
|
||||
mkdir -p /mnt/state/etc/zfs/
|
||||
rm -f /mnt/state/etc/zfs/zpool.cache
|
||||
touch /mnt/state/etc/zfs/zpool.cache
|
||||
chmod a-w /mnt/state/etc/zfs/zpool.cache
|
||||
chattr +i /mnt/state/etc/zfs/zpool.cache
|
||||
|
||||
#. If swap is enabled::
|
||||
|
||||
if [ "${INST_PARTSIZE_SWAP}" != "" ]; then
|
||||
sed -i '/swapDevices/d' /mnt/etc/nixos/hardware-configuration-zfs.nix
|
||||
|
||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
||||
swapDevices = [
|
||||
EOF
|
||||
for i in $DISK; do
|
||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
||||
{ device = "$i-part4"; randomEncryption.enable = true; }
|
||||
EOF
|
||||
done
|
||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
||||
];
|
||||
EOF
|
||||
fi
|
||||
|
||||
#. For immutable root file system, save machine-id and other files::
|
||||
|
||||
mkdir -p /mnt/state/etc/{ssh,zfs}
|
||||
systemd-machine-id-setup --print > /mnt/state/etc/machine-id
|
||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
||||
systemd.services.zfs-mount.enable = false;
|
||||
environment.etc."machine-id".source = "/state/etc/machine-id";
|
||||
environment.etc."zfs/zpool.cache".source
|
||||
= "/state/etc/zfs/zpool.cache";
|
||||
EOF
|
||||
|
||||
#. Configure GRUB boot loader for both legacy boot and UEFI::
|
||||
|
||||
sed -i '/boot.loader/d' /mnt/etc/nixos/configuration.nix
|
||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
||||
boot.loader = {
|
||||
generationsDir.copyKernels = true;
|
||||
##for problematic UEFI firmware
|
||||
grub.efiInstallAsRemovable = true;
|
||||
efi.canTouchEfiVariables = false;
|
||||
##if UEFI firmware can detect entries
|
||||
#efi.canTouchEfiVariables = true;
|
||||
efi.efiSysMountPoint = "/boot/efis/${INST_PRIMARY_DISK##*/}-part1";
|
||||
grub.enable = true;
|
||||
grub.version = 2;
|
||||
grub.copyKernels = true;
|
||||
grub.efiSupport = true;
|
||||
grub.zfsSupport = true;
|
||||
# for systemd-autofs
|
||||
grub.extraPrepareConfig = ''
|
||||
mkdir -p /boot/efis
|
||||
for i in /boot/efis/*; do mount \$i ; done
|
||||
'';
|
||||
grub.devices = [
|
||||
EOF
|
||||
for i in $DISK; do
|
||||
printf " \"$i\"\n" >>/mnt/etc/nixos/${INST_CONFIG_FILE}
|
||||
done
|
||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
||||
];
|
||||
grub.mirroredBoots = [
|
||||
EOF
|
||||
for i in $DISK; do
|
||||
printf " { devices = [ \"$i\" ] ; efiSysMountPoint = \"/boot/efis/${i##*/}-part1\"; path = \"/boot\"; }\n" \
|
||||
>>/mnt/etc/nixos/${INST_CONFIG_FILE}
|
||||
done
|
||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
||||
];
|
||||
};
|
||||
EOF
|
||||
@@ -0,0 +1,254 @@
|
||||
.. highlight:: sh
|
||||
|
||||
Optional Configuration
|
||||
======================
|
||||
|
||||
.. contents:: Table of Contents
|
||||
:local:
|
||||
|
||||
Skip to `System Installation <./4-system-installation.html>`__ section if
|
||||
no optional configuration is needed.
|
||||
|
||||
Supply password with SSH
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Note: if you choose to encrypt boot pool, where decryption is handled
|
||||
by GRUB, as described in the next section, configuration performed
|
||||
in this section will have no effect.
|
||||
|
||||
This example uses DHCP::
|
||||
|
||||
mkdir -p /mnt/etc/state/ssh/
|
||||
ssh-keygen -t ed25519 -N "" -f /mnt/state/etc/ssh/ssh_host_initrd_ed25519_key
|
||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
||||
#networking.interfaces.enp1s0.useDHCP = true;
|
||||
boot = {
|
||||
initrd.network = {
|
||||
enable = true;
|
||||
ssh = {
|
||||
enable = true;
|
||||
hostKeys = [ /state/etc/ssh/ssh_host_initrd_ed25519_key ];
|
||||
authorizedKeys = [ "$YOUR_PUBLIC_KEY" ];
|
||||
};
|
||||
postCommands = ''
|
||||
echo "zfs load-key -a; killall zfs" >> /root/.profile
|
||||
'';
|
||||
};
|
||||
};
|
||||
EOF
|
||||
|
||||
Encrypt boot pool
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
Note: This will disable password with SSH. The password previously set for
|
||||
root pool will be replaced by keyfile, embedded in initrd.
|
||||
|
||||
#. Add package::
|
||||
|
||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
||||
environment.systemPackages = [ pkgs.cryptsetup ];
|
||||
EOF
|
||||
|
||||
#. LUKS password::
|
||||
|
||||
LUKS_PWD=secure-passwd
|
||||
|
||||
You will need to enter the same password for
|
||||
each disk at boot. As root pool key is
|
||||
protected by this password, the previous warning
|
||||
about password strength still apply.
|
||||
|
||||
Double-check password here. Complete reinstallation is
|
||||
needed if entered wrong.
|
||||
|
||||
#. Create encryption keys::
|
||||
|
||||
mkdir -p /mnt/etc/cryptkey.d/
|
||||
chmod 700 /mnt/etc/cryptkey.d/
|
||||
dd bs=32 count=1 if=/dev/urandom of=/mnt/etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs
|
||||
dd bs=32 count=1 if=/dev/urandom of=/mnt/etc/cryptkey.d/bpool_$INST_UUID-key-luks
|
||||
chmod u=r,go= /mnt/etc/cryptkey.d/*
|
||||
|
||||
#. Backup boot pool::
|
||||
|
||||
zfs snapshot -r bpool_$INST_UUID/$INST_ID@pre-luks
|
||||
zfs send -Rv bpool_$INST_UUID/$INST_ID@pre-luks > /mnt/root/bpool_$INST_UUID-${INST_ID}-pre-luks
|
||||
|
||||
#. Unmount EFI partition::
|
||||
|
||||
for i in ${DISK}; do
|
||||
umount /mnt/boot/efis/${i##*/}-part1
|
||||
done
|
||||
|
||||
#. Destroy boot pool::
|
||||
|
||||
zpool destroy bpool_$INST_UUID
|
||||
|
||||
#. Create LUKS containers::
|
||||
|
||||
for i in ${DISK}; do
|
||||
cryptsetup luksFormat -q --type luks1 --key-file /mnt/etc/cryptkey.d/bpool_$INST_UUID-key-luks $i-part2
|
||||
echo $LUKS_PWD | cryptsetup luksAddKey --key-file /mnt/etc/cryptkey.d/bpool_$INST_UUID-key-luks $i-part2
|
||||
cryptsetup open ${i}-part2 ${i##*/}-part2-luks-bpool_$INST_UUID --key-file /mnt/etc/cryptkey.d/bpool_$INST_UUID-key-luks
|
||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
||||
boot.initrd.luks.devices = {
|
||||
"${i##*/}-part2-luks-bpool_$INST_UUID" = {
|
||||
device = "${i}-part2";
|
||||
allowDiscards = true;
|
||||
keyFile = "/etc/cryptkey.d/bpool_$INST_UUID-key-luks";
|
||||
};
|
||||
};
|
||||
EOF
|
||||
done
|
||||
|
||||
GRUB 2.06 still does not have complete support for LUKS2, LUKS1
|
||||
is used instead.
|
||||
|
||||
#. Embed key file in initrd::
|
||||
|
||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
||||
boot.initrd.secrets = {
|
||||
"/etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs" = "/etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs";
|
||||
"/etc/cryptkey.d/bpool_$INST_UUID-key-luks" = "/etc/cryptkey.d/bpool_$INST_UUID-key-luks";
|
||||
};
|
||||
EOF
|
||||
|
||||
#. Recreate boot pool with mappers as vdev::
|
||||
|
||||
zpool create \
|
||||
-d -o feature@async_destroy=enabled \
|
||||
-o feature@bookmarks=enabled \
|
||||
-o feature@embedded_data=enabled \
|
||||
-o feature@empty_bpobj=enabled \
|
||||
-o feature@enabled_txg=enabled \
|
||||
-o feature@extensible_dataset=enabled \
|
||||
-o feature@filesystem_limits=enabled \
|
||||
-o feature@hole_birth=enabled \
|
||||
-o feature@large_blocks=enabled \
|
||||
-o feature@lz4_compress=enabled \
|
||||
-o feature@spacemap_histogram=enabled \
|
||||
-o ashift=12 \
|
||||
-o autotrim=on \
|
||||
-O acltype=posixacl \
|
||||
-O canmount=off \
|
||||
-O compression=lz4 \
|
||||
-O devices=off \
|
||||
-O normalization=formD \
|
||||
-O relatime=on \
|
||||
-O xattr=sa \
|
||||
-O mountpoint=/boot \
|
||||
-R /mnt \
|
||||
bpool_$INST_UUID \
|
||||
$INST_VDEV \
|
||||
$(for i in ${DISK}; do
|
||||
printf "/dev/mapper/${i##*/}-part2-luks-bpool_$INST_UUID ";
|
||||
done)
|
||||
|
||||
#. Restore boot pool backup::
|
||||
|
||||
zfs recv bpool_${INST_UUID}/${INST_ID} < /mnt/root/bpool_$INST_UUID-${INST_ID}-pre-luks
|
||||
rm /mnt/root/bpool_$INST_UUID-${INST_ID}-pre-luks
|
||||
|
||||
#. Mount boot dataset and EFI partitions::
|
||||
|
||||
zfs mount bpool_$INST_UUID/$INST_ID/BOOT/default
|
||||
|
||||
for i in ${DISK}; do
|
||||
mount ${i}-part1 /mnt/boot/efis/${i##*/}-part1
|
||||
done
|
||||
|
||||
#. As keys are stored in initrd,
|
||||
set secure permissions for ``/boot``::
|
||||
|
||||
chmod 700 /mnt/boot
|
||||
|
||||
#. Change root pool password to key file::
|
||||
|
||||
mkdir -p /etc/cryptkey.d/
|
||||
cp /mnt/etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs /etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs
|
||||
zfs change-key -l \
|
||||
-o keylocation=file:///etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs \
|
||||
-o keyformat=raw \
|
||||
rpool_$INST_UUID/$INST_ID
|
||||
|
||||
#. Import encrypted boot pool from ``/dev/mapper``::
|
||||
|
||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<-'EOF'
|
||||
systemd.services.zfs-import-bpool-mapper = {
|
||||
wantedBy = [ "zfs-import.target" ];
|
||||
description = "Import encrypted boot pool";
|
||||
after = [ "cryptsetup.target" ];
|
||||
before = [ "boot.mount" ];
|
||||
serviceConfig = {
|
||||
Type = "oneshot";
|
||||
ExecStart = ''${pkgs.zfs}/bin/zpool import -aNd /dev/mapper'';
|
||||
};
|
||||
};
|
||||
EOF
|
||||
|
||||
#. Enable GRUB cryptodisk::
|
||||
|
||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
||||
boot.loader.grub.enableCryptodisk = true;
|
||||
EOF
|
||||
|
||||
#. **Important**: Back up root dataset key ``/etc/cryptkey.d/rpool_$INST_UUID-${INST_ID}-key-zfs``
|
||||
to a secure location.
|
||||
|
||||
In the possible event of LUKS container corruption,
|
||||
data on root set will only be available
|
||||
with this key.
|
||||
|
||||
Persistent swap and hibernation
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
#. Optional: enable persistent swap partition. By default
|
||||
encryption key of swap partition is discarded on reboot::
|
||||
|
||||
INST_SWAPKEY=/mnt/etc/cryptkey.d/${INST_PRIMARY_DISK##*/}-part4-key-luks-swap
|
||||
INST_SWAPMAPPER=${INST_PRIMARY_DISK##*/}-part4-luks-swap
|
||||
|
||||
# fstab
|
||||
# remove existing swap entries
|
||||
for i in $DISK; do echo $i; done | grep -v ${INST_PRIMARY_DISK##*/} \
|
||||
| while read j; do sed -i "\,$j-part4\"; randomEncryption.enable,d" /mnt/etc/nixos/${INST_CONFIG_FILE} ; done
|
||||
sed -i "s|${INST_PRIMARY_DISK}-part4\"; randomEncryption.enable = true|/dev/mapper/${INST_SWAPMAPPER}\"|g" /mnt/etc/nixos/${INST_CONFIG_FILE}
|
||||
|
||||
# create key and format partition as LUKS container
|
||||
dd bs=32 count=1 if=/dev/urandom of=${INST_SWAPKEY};
|
||||
chmod u=r,go= /mnt/etc/cryptkey.d/*
|
||||
cryptsetup luksFormat -q --type luks2 --key-file ${INST_SWAPKEY} ${INST_PRIMARY_DISK}-part4
|
||||
cryptsetup luksOpen ${INST_PRIMARY_DISK}-part4 ${INST_SWAPMAPPER} --key-file ${INST_SWAPKEY}
|
||||
|
||||
# initialize swap space
|
||||
mkswap /dev/mapper/${INST_SWAPMAPPER}
|
||||
|
||||
# add initrd key
|
||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
||||
boot.initrd.secrets = {
|
||||
"/etc/cryptkey.d/${INST_PRIMARY_DISK##*/}-part4-key-luks-swap" = "/etc/cryptkey.d/${INST_PRIMARY_DISK##*/}-part4-key-luks-swap";
|
||||
};
|
||||
boot.initrd.luks.devices = {
|
||||
"${INST_SWAPMAPPER}" = {
|
||||
device = "${INST_PRIMARY_DISK}-part4";
|
||||
allowDiscards = true;
|
||||
keyFile = "/etc/cryptkey.d/${INST_PRIMARY_DISK##*/}-part4-key-luks-swap";
|
||||
};
|
||||
};
|
||||
EOF
|
||||
|
||||
#. Optional: after enabling persistent swap partition,
|
||||
enable hibernation::
|
||||
|
||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
||||
boot.resumeDevice = "/dev/mapper/${INST_SWAPMAPPER}";
|
||||
EOF
|
||||
|
||||
|
||||
Note that hibernation might not work with discrete graphics, virtio graphics or
|
||||
AMD APU integrated graphics. This is not specific to this guide.
|
||||
|
||||
Computer must resume from a continuous swap space, resume
|
||||
from multiple swap partitions is not supported.
|
||||
|
||||
Do not touch anything on disk while the computer is
|
||||
in hibernation, see `kernel documentation
|
||||
<https://www.kernel.org/doc/html/latest/power/swsusp.html>`__.
|
||||
176
docs/Getting Started/NixOS/Root on ZFS/4-system-installation.rst
Normal file
176
docs/Getting Started/NixOS/Root on ZFS/4-system-installation.rst
Normal file
@@ -0,0 +1,176 @@
|
||||
.. highlight:: sh
|
||||
|
||||
System Installation
|
||||
======================
|
||||
|
||||
.. contents:: Table of Contents
|
||||
:local:
|
||||
|
||||
|
||||
Additional configuration
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
As NixOS configuration is declarative, post-installation tasks,
|
||||
such as user accounts and package selection, can all be done by
|
||||
specifing them in configuration. See `NixOS manual <https://nixos.org/nixos/manual/>`__
|
||||
for details.
|
||||
|
||||
For timezone, hostname, networking, keyboard layout, etc,
|
||||
see ``/mnt/etc/nixos/configuration.nix``.
|
||||
|
||||
Set root password
|
||||
-----------------
|
||||
|
||||
This optional step is an example
|
||||
of declaratively configuring the system.
|
||||
|
||||
#. Generate password hash::
|
||||
|
||||
mkpasswd -m SHA-512 -s
|
||||
#output: $6$DeHnzc
|
||||
|
||||
#. Declare `initialHashedPassword
|
||||
<https://nixos.org/manual/nixos/stable/options.html#opt-users.users._name_.initialHashedPassword>`__
|
||||
for root user::
|
||||
|
||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<-'EOF'
|
||||
users.users.root.initialHashedPassword = "$6$DeHnzc";
|
||||
EOF
|
||||
|
||||
System installation
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
#. Finalize the config file::
|
||||
|
||||
tee -a /mnt/etc/nixos/${INST_CONFIG_FILE} <<EOF
|
||||
}
|
||||
EOF
|
||||
|
||||
#. Take a snapshot of the clean installation, without state
|
||||
for future use::
|
||||
|
||||
zfs snapshot -r rpool_$INST_UUID/$INST_ID@install_start
|
||||
zfs snapshot -r bpool_$INST_UUID/$INST_ID@install_start
|
||||
|
||||
#. Apply configuration
|
||||
|
||||
If root password hash is not set::
|
||||
|
||||
nixos-install -v --show-trace --root /mnt
|
||||
|
||||
You will be prompted for a new root password.
|
||||
|
||||
If password hash has been set::
|
||||
|
||||
nixos-install -v --show-trace --no-root-passwd --root /mnt
|
||||
|
||||
|
||||
Finish installation
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
#. Take a snapshot of the clean installation for future use::
|
||||
|
||||
zfs snapshot -r rpool_$INST_UUID/$INST_ID@install
|
||||
zfs snapshot -r bpool_$INST_UUID/$INST_ID@install
|
||||
|
||||
#. Unmount EFI system partition::
|
||||
|
||||
umount /mnt/boot/efis/*
|
||||
|
||||
#. Export pools::
|
||||
|
||||
zpool export bpool_$INST_UUID
|
||||
zpool export rpool_$INST_UUID
|
||||
|
||||
#. Reboot::
|
||||
|
||||
reboot
|
||||
|
||||
Immutable root file system
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
This section is optional.
|
||||
|
||||
Often, programs generate mutable files in paths such as
|
||||
``/etc`` and ``/var/lib``. The generated files can be considered a
|
||||
part of the system state.
|
||||
|
||||
This generated state is not declaratively managed
|
||||
by NixOS and can not be reproduced from NixOS configuration.
|
||||
|
||||
To ensure that the system state is fully managed by NixOS and reproducible,
|
||||
we need to periodically purge the system state and let NixOS
|
||||
regenerate root file system from scratch.
|
||||
|
||||
Also see: `Erase your darlings:
|
||||
immutable infrastructure for mutable systems <https://grahamc.com/blog/erase-your-darlings>`__.
|
||||
|
||||
Save mutable data to alternative path
|
||||
-------------------------------------
|
||||
|
||||
Before enabling purging on root dataset, we need to back up
|
||||
essential mutable data first, such as host SSH key and network connections.
|
||||
Below are some tips.
|
||||
|
||||
- Some programs support specifying another
|
||||
location for mutable data, such as
|
||||
Wireguard::
|
||||
|
||||
networking.wireguard.interfaces.wg0.privateKeyFile = "/state/etc/wireguard/wg0";
|
||||
|
||||
- For programs without a configurable data path,
|
||||
`environment.etc <https://nixos.org/manual/nixos/stable/options.html#opt-environment.etc>`__
|
||||
may be used::
|
||||
|
||||
environment.etc = {
|
||||
"ssh/ssh_host_rsa_key".source = "/state/etc/ssh/ssh_host_rsa_key";
|
||||
}
|
||||
|
||||
- systemd’s tmpfiles.d rules are also an option::
|
||||
|
||||
systemd.tmpfiles.rules = [
|
||||
"L /var/lib/bluetooth - - - - /state/var/lib/bluetooth"
|
||||
];
|
||||
|
||||
- Bind mount::
|
||||
|
||||
for i in {/etc/nixos,/etc/cryptkey.d}; do
|
||||
mkdir -p /state/$i /$i
|
||||
mount -o bind /state/$i /$i
|
||||
done
|
||||
nixos-generate-config --show-hardware-config
|
||||
|
||||
Boot from empty root file system
|
||||
--------------------------------
|
||||
|
||||
After backing up mutable data, you can try switching to
|
||||
an empty dataset as root file system.
|
||||
|
||||
#. Check current root file system::
|
||||
|
||||
ROOT_FS=$(df --output=source /|tail -n1)
|
||||
# rpool/ROOT/default
|
||||
|
||||
#. Set empty file system as root::
|
||||
|
||||
sed -i "s,${ROOT_FS},${ROOT_FS%/*}/empty,g" /etc/nixos/hardware-configuration-zfs.nix
|
||||
|
||||
#. Apply changes and reboot::
|
||||
|
||||
nixos-rebuild boot
|
||||
reboot
|
||||
|
||||
#. If everything went fine, add the output of the following command to configuration::
|
||||
|
||||
ROOT_FS=$(df --output=source /|tail -n1)
|
||||
cat <<EOF
|
||||
boot.initrd.postDeviceCommands = ''
|
||||
zpool import -Nf ${ROOT_FS%%/*}
|
||||
zfs rollback -r ${ROOT_FS%/*}/empty@start
|
||||
'';
|
||||
EOF
|
||||
|
||||
#. Apply and reboot::
|
||||
|
||||
nixos-rebuild boot
|
||||
reboot
|
||||
145
docs/Getting Started/NixOS/Root on ZFS/6-recovery.rst
Normal file
145
docs/Getting Started/NixOS/Root on ZFS/6-recovery.rst
Normal file
@@ -0,0 +1,145 @@
|
||||
.. highlight:: sh
|
||||
|
||||
Recovery
|
||||
======================
|
||||
|
||||
.. contents:: Table of Contents
|
||||
:local:
|
||||
|
||||
GRUB Tips
|
||||
-------------
|
||||
|
||||
Boot from GRUB rescue
|
||||
~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
If bootloader file is damaged, it's still possible
|
||||
to boot computer with GRUB rescue image.
|
||||
|
||||
This section is also applicable if you are in
|
||||
``grub rescue>``.
|
||||
|
||||
#. On another computer, generate rescue image with::
|
||||
|
||||
pacman -S --needed mtools libisoburn grub
|
||||
grub-install
|
||||
grub-mkrescue -o grub-rescue.img
|
||||
dd if=grub-rescue.img of=/dev/your-usb-stick
|
||||
|
||||
Boot computer from the rescue media.
|
||||
Both legacy and EFI mode are supported.
|
||||
|
||||
Or `download generated GRUB rescue image <https://gitlab.com/m_zhou/bieaz/uploads/4a1b7cefb42723de6eb04f9dc485be3b/grub-rescue.img.7z>`__.
|
||||
|
||||
#. List available disks with ``ls`` command::
|
||||
|
||||
grub> ls (hd # press tab
|
||||
Possible devices are:
|
||||
|
||||
hd0 hd1 hd2 hd3
|
||||
|
||||
#. List partitions by pressing tab key:
|
||||
|
||||
.. code-block:: text
|
||||
|
||||
grub> ls (hd0 # press tab
|
||||
Possible partitions are:
|
||||
|
||||
Device hd0: No known filesystem detected - Sector size 512B - Total size 20971520KiB
|
||||
Partition hd0,gpt1: Filesystem type fat - Label `EFI', UUID 0DF5-3A76 - Partition start at 1024KiB - Total size 1048576KiB
|
||||
Partition hd0,gpt2: No known filesystem detected - Partition start at 1049600KiB - Total size 4194304KiB
|
||||
|
||||
- If boot pool is encrypted:
|
||||
|
||||
Unlock it with ``cryptomount``::
|
||||
|
||||
grub> insmod luks
|
||||
grub> cryptomount hd0,gpt2
|
||||
Attempting to decrypt master key...
|
||||
Enter passphrase for hd0,gpt2 (af5a240e13e24483acf02600d61e0f36):
|
||||
Slot 1 opened
|
||||
|
||||
Unlocked LUKS container is ``(crypto0)``:
|
||||
|
||||
.. code-block:: text
|
||||
|
||||
grub> ls (crypto0)
|
||||
Device crypto0: Filesystem type zfs - Label `bpool_ip3tdb' - Last modification
|
||||
time 2021-05-03 12:14:08 Monday, UUID f14d7bdf89fe21fb - Sector size 512B -
|
||||
Total size 4192256KiB
|
||||
|
||||
- If boot pool is not encrypted:
|
||||
|
||||
.. code-block:: text
|
||||
|
||||
grub> ls (hd0,gpt2)
|
||||
Device hd0,gpt2: Filesystem type zfs - Label `bpool_ip3tdb' - Last modification
|
||||
time 2021-05-03 12:14:08 Monday, UUID f14d7bdf89fe21fb - Sector size 512B -
|
||||
Total size 4192256KiB
|
||||
|
||||
#. List boot environments nested inside ``bpool/$INST_ID/BOOT``::
|
||||
|
||||
grub> ls (crypto0)/sys/BOOT
|
||||
@/ default/ be0/
|
||||
|
||||
#. Instruct GRUB to load configuration from ``be0`` boot environment::
|
||||
|
||||
grub> prefix=(crypto0)/sys/BOOT/be0/@/grub
|
||||
grub> configfile $prefix/grub.cfg
|
||||
|
||||
#. GRUB menu should now appear.
|
||||
|
||||
#. After entering system, reinstall GRUB.
|
||||
|
||||
Switch GRUB prefix when disk fails
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
If you are using LUKS encrypted boot pool with multiple disks,
|
||||
the primary disk failed, GRUB will fail to load configuration.
|
||||
|
||||
If there's still enough redundancy for the boot pool, try fix
|
||||
GRUB with the following method:
|
||||
|
||||
#. Ensure ``Slot 1 opened`` message
|
||||
is shown
|
||||
|
||||
.. code-block:: text
|
||||
|
||||
Welcome to GRUB!
|
||||
|
||||
error: no such cryptodisk found.
|
||||
Attempting to decrypt master key...
|
||||
Enter passphrase for hd0,gpt2 (c0987ea1a51049e9b3056622804de62a):
|
||||
Slot 1 opened
|
||||
error: disk `cryptouuid/47ed1b7eb0014bc9a70aede3d8714faf' not found.
|
||||
Entering rescue mode...
|
||||
grub rescue>
|
||||
|
||||
If ``error: access denied.`` is shown,
|
||||
try re-enter password with::
|
||||
|
||||
grub rescue> cryptomount hd0,gpt2
|
||||
|
||||
#. Check prefix::
|
||||
|
||||
grub rescue > set
|
||||
# prefix=(cryptouuid/47ed1b7eb0014bc9a70aede3d8714faf)/sys/BOOT/be0@/grub
|
||||
# root=cryptouuid/47ed1b7eb0014bc9a70aede3d8714faf
|
||||
|
||||
#. Set correct ``prefix`` and ``root`` by replacing
|
||||
``cryptouuid/UUID`` with ``crypto0``::
|
||||
|
||||
grub rescue> prefix=(crypto0)/sys/BOOT/default@/grub
|
||||
grub rescue> root=crypto0
|
||||
|
||||
#. Boot GRUB::
|
||||
|
||||
grub rescue> insmod normal
|
||||
grub rescue> normal
|
||||
|
||||
GRUB should then boot normally.
|
||||
|
||||
#. After entering system, edit ``/etc/fstab`` to promote
|
||||
one backup to ``/boot/efi``.
|
||||
|
||||
#. Make the change to ``prefix`` and ``root``
|
||||
permanent by `reinstalling GRUB <#grub-installation>`__.
|
||||
64
docs/Getting Started/NixOS/index.rst
Normal file
64
docs/Getting Started/NixOS/index.rst
Normal file
@@ -0,0 +1,64 @@
|
||||
.. highlight:: sh
|
||||
|
||||
NixOS
|
||||
=====
|
||||
|
||||
Contents
|
||||
--------
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:glob:
|
||||
|
||||
*
|
||||
|
||||
Support
|
||||
-------
|
||||
Reach out to the community using the :ref:`mailing_lists` or IRC at
|
||||
`#zfsonlinux <ircs://irc.libera.chat/#zfsonlinux>`__ on `Libera Chat
|
||||
<https://libera.chat/>`__.
|
||||
|
||||
If you have a bug report or feature request
|
||||
related to this HOWTO, please `file a new issue and mention @ne9z
|
||||
<https://github.com/openzfs/openzfs-docs/issues/new?body=@ne9z,%20I%20have%20the%20following%20issue%20with%20the%20Nix%20ZFS%20HOWTO:>`__.
|
||||
|
||||
Installation
|
||||
------------
|
||||
|
||||
Note: this is for installing ZFS on an existing
|
||||
NixOS installation. To use ZFS as root file system,
|
||||
see below.
|
||||
|
||||
Live image ships with ZFS support by default.
|
||||
|
||||
#. Import separate configration file for ZFS options::
|
||||
|
||||
vim /etc/nixos/configuration.nix
|
||||
##add './zfs.nix' to 'imports'
|
||||
# imports = [ ./zfs.nix ];
|
||||
|
||||
#. Configure ZFS options::
|
||||
|
||||
tee -a /etc/nixos/zfs.nix <<EOF
|
||||
{ config, pkgs, ... }:
|
||||
|
||||
{ boot.supportedFilesystems = [ "zfs" ];
|
||||
networking.hostId = "$(head -c 8 /etc/machine-id)";
|
||||
}
|
||||
EOF
|
||||
|
||||
#. Apply configuation changes::
|
||||
|
||||
nixos-rebuild switch
|
||||
|
||||
Root on ZFS
|
||||
-----------
|
||||
ZFS can be used as root file system for NixOS.
|
||||
An installation guide is available.
|
||||
|
||||
`Start here <Root%20on%20ZFS/0-overview.html>`__.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:glob:
|
||||
|
||||
Root on ZFS/*
|
||||
Reference in New Issue
Block a user