jacobpascual/openzfs-docs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Adopt Developer Resources section

Browse Source 
Signed-off-by: George Melikov <mail@gmelikov.ru>
This commit is contained in:
George Melikov
2020-05-16 20:50:01 +03:00
parent 15f717f3f2
commit 8ce59fb624
9 changed files with 44 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files

245
docs/Developer Resources/Buildbot-Options.rst Normal file
View File

@@ -0,0 +1,245 @@
There are a number of ways to control the ZFS Buildbot at a commit
level. This page provides a summary of various options that the ZFS
Buildbot supports and how it impacts testing. More detailed information
regarding its implementation can be found at the `ZFS Buildbot Github
page <https://github.com/zfsonlinux/zfs-buildbot>`__.
Choosing Builders
-----------------
By default, all commits in your ZFS pull request are compiled by the
BUILD builders. Additionally, the top commit of your ZFS pull request is
tested by TEST builders. However, there is the option to override which
types of builder should be used on a per commit basis. In this case, you
can add
``Requires-builders: <none|all|style|build|arch|distro|test|perf|coverage|unstable>``
to your commit message. A comma separated list of options can be
provided. Supported options are:
- ``all``: This commit should be built by all available builders
- ``none``: This commit should not be built by any builders
- ``style``: This commit should be built by STYLE builders
- ``build``: This commit should be built by all BUILD builders
- ``arch``: This commit should be built by BUILD builders tagged as
'Architectures'
- ``distro``: This commit should be built by BUILD builders tagged as
'Distributions'
- ``test``: This commit should be built and tested by the TEST builders
(excluding the Coverage TEST builders)
- ``perf``: This commit should be built and tested by the PERF builders
- ``coverage`` : This commit should be built and tested by the Coverage
TEST builders
- ``unstable`` : This commit should be built and tested by the Unstable
TEST builders (currently only the Fedora Rawhide TEST builder)
A couple of examples on how to use ``Requires-builders:`` in commit
messages can be found below.
.. _preventing-a-commit-from-being-built-and-tested:
Preventing a commit from being built and tested.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
::
This is a commit message
This text is part of the commit message body.
Signed-off-by: Contributor <contributor@email.com>
Requires-builders: none
.. _submitting-a-commit-to-style-and-test-builders-only:
Submitting a commit to STYLE and TEST builders only.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
::
This is a commit message
This text is part of the commit message body.
Signed-off-by: Contributor <contributor@email.com>
Requires-builders: style test
Requiring SPL Versions
----------------------
Currently, the ZFS Buildbot attempts to choose the correct SPL branch to
build based on a pull request's base branch. In the cases where a
specific SPL version needs to be built, the ZFS buildbot supports
specifying an SPL version for pull request testing. By opening a pull
request against ZFS and adding ``Requires-spl:`` in a commit message,
you can instruct the buildbot to use a specific SPL version. Below are
examples of a commit messages that specify the SPL version.
Build SPL from a specific pull request
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
::
This is a commit message
This text is part of the commit message body.
Signed-off-by: Contributor <contributor@email.com>
Requires-spl: refs/pull/123/head
Build SPL branch ``spl-branch-name`` from ``zfsonlinux/spl`` repository
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
::
This is a commit message
This text is part of the commit message body.
Signed-off-by: Contributor <contributor@email.com>
Requires-spl: spl-branch-name
Requiring Kernel Version
------------------------
Currently, Kernel.org builders will clone and build the master branch of
Linux. In cases where a specific version of the Linux kernel needs to be
built, the ZFS buildbot supports specifying the Linux kernel to be built
via commit message. By opening a pull request against ZFS and adding
``Requires-kernel:`` in a commit message, you can instruct the buildbot
to use a specific Linux kernel. Below is an example commit message that
specifies a specific Linux kernel tag.
.. _build-linux-kernel-version-414:
Build Linux Kernel Version 4.14
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
::
This is a commit message
This text is part of the commit message body.
Signed-off-by: Contributor <contributor@email.com>
Requires-kernel: v4.14
Build Steps Overrides
---------------------
Each builder will execute or skip build steps based on its default
preferences. In some scenarios, it might be possible to skip various
build steps. The ZFS buildbot supports overriding the defaults of all
builders in a commit message. The list of available overrides are:
- ``Build-linux: <Yes|No>``: All builders should build Linux for this
commit
- ``Build-lustre: <Yes|No>``: All builders should build Lustre for this
commit
- ``Build-spl: <Yes|No>``: All builders should build the SPL for this
commit
- ``Build-zfs: <Yes|No>``: All builders should build ZFS for this
commit
- ``Built-in: <Yes|No>``: All Linux builds should build in SPL and ZFS
- ``Check-lint: <Yes|No>``: All builders should perform lint checks for
this commit
- ``Configure-lustre: <options>``: Provide ``<options>`` as configure
flags when building Lustre
- ``Configure-spl: <options>``: Provide ``<options>`` as configure
flags when building the SPL
- ``Configure-zfs: <options>``: Provide ``<options>`` as configure
flags when building ZFS
A couple of examples on how to use overrides in commit messages can be
found below.
Skip building the SPL and build Lustre without ldiskfs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
::
This is a commit message
This text is part of the commit message body.
Signed-off-by: Contributor <contributor@email.com>
Build-lustre: Yes
Configure-lustre: --disable-ldiskfs
Build-spl: No
Build ZFS Only
~~~~~~~~~~~~~~
::
This is a commit message
This text is part of the commit message body.
Signed-off-by: Contributor <contributor@email.com>
Build-lustre: No
Build-spl: No
Configuring Tests with the TEST File
------------------------------------
At the top level of the ZFS source tree, there is the ```TEST``
file <https://github.com/zfsonlinux/zfs/blob/master/TEST>`__ which
contains variables that control if and how a specific test should run.
Below is a list of each variable and a brief description of what each
variable controls.
- ``TEST_PREPARE_WATCHDOG`` - Enables the Linux kernel watchdog
- ``TEST_PREPARE_SHARES`` - Start NFS and Samba servers
- ``TEST_SPLAT_SKIP`` - Determines if ``splat`` testing is skipped
- ``TEST_SPLAT_OPTIONS`` - Command line options to provide to ``splat``
- ``TEST_ZTEST_SKIP`` - Determines if ``ztest`` testing is skipped
- ``TEST_ZTEST_TIMEOUT`` - The length of time ``ztest`` should run
- ``TEST_ZTEST_DIR`` - Directory where ``ztest`` will create vdevs
- ``TEST_ZTEST_OPTIONS`` - Options to pass to ``ztest``
- ``TEST_ZTEST_CORE_DIR`` - Directory for ``ztest`` to store core dumps
- ``TEST_ZIMPORT_SKIP`` - Determines if ``zimport`` testing is skipped
- ``TEST_ZIMPORT_DIR`` - Directory used during ``zimport``
- ``TEST_ZIMPORT_VERSIONS`` - Source versions to test
- ``TEST_ZIMPORT_POOLS`` - Names of the pools for ``zimport`` to use
for testing
- ``TEST_ZIMPORT_OPTIONS`` - Command line options to provide to
``zimport``
- ``TEST_XFSTESTS_SKIP`` - Determines if ``xfstest`` testing is skipped
- ``TEST_XFSTESTS_URL`` - URL to download ``xfstest`` from
- ``TEST_XFSTESTS_VER`` - Name of the tarball to download from
``TEST_XFSTESTS_URL``
- ``TEST_XFSTESTS_POOL`` - Name of pool to create and used by
``xfstest``
- ``TEST_XFSTESTS_FS`` - Name of dataset for use by ``xfstest``
- ``TEST_XFSTESTS_VDEV`` - Name of the vdev used by ``xfstest``
- ``TEST_XFSTESTS_OPTIONS`` - Command line options to provide to
``xfstest``
- ``TEST_ZFSTESTS_SKIP`` - Determines if ``zfs-tests`` testing is
skipped
- ``TEST_ZFSTESTS_DIR`` - Directory to store files and loopback devices
- ``TEST_ZFSTESTS_DISKS`` - Space delimited list of disks that
``zfs-tests`` is allowed to use
- ``TEST_ZFSTESTS_DISKSIZE`` - File size of file based vdevs used by
``zfs-tests``
- ``TEST_ZFSTESTS_ITERS`` - Number of times ``test-runner`` should
execute its set of tests
- ``TEST_ZFSTESTS_OPTIONS`` - Options to provide ``zfs-tests``
- ``TEST_ZFSTESTS_RUNFILE`` - The runfile to use when running
``zfs-tests``
- ``TEST_ZFSTESTS_TAGS`` - List of tags to provide to ``test-runner``
- ``TEST_ZFSSTRESS_SKIP`` - Determines if ``zfsstress`` testing is
skipped
- ``TEST_ZFSSTRESS_URL`` - URL to download ``zfsstress`` from
- ``TEST_ZFSSTRESS_VER`` - Name of the tarball to download from
``TEST_ZFSSTRESS_URL``
- ``TEST_ZFSSTRESS_RUNTIME`` - Duration to run ``runstress.sh``
- ``TEST_ZFSSTRESS_POOL`` - Name of pool to create and use for
``zfsstress`` testing
- ``TEST_ZFSSTRESS_FS`` - Name of dataset for use during ``zfsstress``
tests
- ``TEST_ZFSSTRESS_FSOPT`` - File system options to provide to
``zfsstress``
- ``TEST_ZFSSTRESS_VDEV`` - Directory to store vdevs for use during
``zfsstress`` tests
- ``TEST_ZFSSTRESS_OPTIONS`` - Command line options to provide to
``runstress.sh``

246
docs/Developer Resources/Building ZFS.rst Normal file
View File

@@ -0,0 +1,246 @@
Building ZFS
============
GitHub Repositories
~~~~~~~~~~~~~~~~~~~
The official source for ZFS on Linux is maintained at GitHub by the
`zfsonlinux <https://github.com/zfsonlinux/>`__ organization. The
project consists of two primary git repositories named
`spl <https://github.com/zfsonlinux/spl>`__ and
`zfs <https://github.com/zfsonlinux/zfs>`__, both are required to build
ZFS on Linux.
**NOTE:** The SPL was merged in to the
`zfs <https://github.com/zfsonlinux/zfs>`__ repository, the last major
release with a separate SPL is ``0.7``.
- **SPL**: The SPL is thin shim layer which is responsible for
implementing the fundamental interfaces required by OpenZFS. It's
this layer which allows OpenZFS to be used across multiple platforms.
- **ZFS**: The ZFS repository contains a copy of the upstream OpenZFS
code which has been adapted and extended for Linux. The vast majority
of the core OpenZFS code is self-contained and can be used without
modification.
Installing Dependencies
~~~~~~~~~~~~~~~~~~~~~~~
The first thing you'll need to do is prepare your environment by
installing a full development tool chain. In addition, development
headers for both the kernel and the following libraries must be
available. It is important to note that if the development kernel
headers for the currently running kernel aren't installed, the modules
won't compile properly.
The following dependencies should be installed to build the latest ZFS
0.8 release.
- **RHEL/CentOS 7**:
.. code:: sh
sudo yum install epel-release gcc make autoconf automake libtool rpm-build dkms libtirpc-devel libblkid-devel libuuid-devel libudev-devel openssl-devel zlib-devel libaio-devel libattr-devel elfutils-libelf-devel kernel-devel-$(uname -r) python python2-devel python-setuptools python-cffi libffi-devel
- **RHEL/CentOS 8, Fedora**:
.. code:: sh
sudo dnf install gcc make autoconf automake libtool rpm-build dkms libtirpc-devel libblkid-devel libuuid-devel libudev-devel openssl-devel zlib-devel libaio-devel libattr-devel elfutils-libelf-devel kernel-devel-$(uname -r) python3 python3-devel python3-setuptools python3-cffi libffi-devel
- **Debian, Ubuntu**:
.. code:: sh
sudo apt install build-essential autoconf automake libtool gawk alien fakeroot dkms libblkid-dev uuid-dev libudev-dev libssl-dev zlib1g-dev libaio-dev libattr1-dev libelf-dev linux-headers-$(uname -r) python3 python3-dev python3-setuptools python3-cffi libffi-dev
Build Options
~~~~~~~~~~~~~
There are two options for building ZFS on Linux, the correct one largely
depends on your requirements.
- **Packages**: Often it can be useful to build custom packages from
git which can be installed on a system. This is the best way to
perform integration testing with systemd, dracut, and udev. The
downside to using packages it is greatly increases the time required
to build, install, and test a change.
- **In-tree**: Development can be done entirely in the SPL and ZFS
source trees. This speeds up development by allowing developers to
rapidly iterate on a patch. When working in-tree developers can
leverage incremental builds, load/unload kernel modules, execute
utilities, and verify all their changes with the ZFS Test Suite.
The remainder of this page focuses on the **in-tree** option which is
the recommended method of development for the majority of changes. See
the :doc:`custom packages <./Custom Packages>` page for additional information on building
custom packages.
Developing In-Tree
~~~~~~~~~~~~~~~~~~
Clone from GitHub
^^^^^^^^^^^^^^^^^
Start by cloning the SPL and ZFS repositories from GitHub. The
repositories have a **master** branch for development and a series of
**\*-release** branches for tagged releases. After checking out the
repository your clone will default to the master branch. Tagged releases
may be built by checking out spl/zfs-x.y.z tags with matching version
numbers or matching release branches. Avoid using mismatched versions,
this can result build failures due to interface changes.
**NOTE:** SPL was merged in to the
`zfs <https://github.com/zfsonlinux/zfs>`__ repository, last release
with separate SPL is ``0.7``.
::
git clone https://github.com/zfsonlinux/zfs
If you need 0.7 release or older:
::
git clone https://github.com/zfsonlinux/spl
Configure and Build
^^^^^^^^^^^^^^^^^^^
For developers working on a change always create a new topic branch
based off of master. This will make it easy to open a pull request with
your change latter. The master branch is kept stable with extensive
`regression testing <http://build.zfsonlinux.org/>`__ of every pull
request before and after it's merged. Every effort is made to catch
defects as early as possible and to keep them out of the tree.
Developers should be comfortable frequently rebasing their work against
the latest master branch.
If you want to build 0.7 release or older, you should compile SPL first:
::
cd ./spl
git checkout master
sh autogen.sh
./configure
make -s -j$(nproc)
In this example we'll use the master branch and walk through a stock
**in-tree** build, so we don't need to build SPL separately. Start by
checking out the desired branch then build the ZFS and SPL source in the
tradition autotools fashion.
::
cd ./zfs
git checkout master
sh autogen.sh
./configure
make -s -j$(nproc)
| **tip:** ``--with-linux=PATH`` and ``--with-linux-obj=PATH`` can be
passed to configure to specify a kernel installed in a non-default
location. This option is also supported when building ZFS.
| **tip:** ``--enable-debug`` can be set to enable all ASSERTs and
additional correctness tests. This option is also supported when
building ZFS.
| **tip:** for version ``<=0.7`` ``--with-spl=PATH`` and
``--with-spl-obj=PATH``, where ``PATH`` is a full path, can be passed
to configure if it is unable to locate the SPL.
**Optional** Build packages
::
make deb #example for Debian/Ubuntu
Install
^^^^^^^
You can run ``zfs-tests.sh`` without installing ZFS, see below. If you
have reason to install ZFS after building it, pay attention to how your
distribution handles kernel modules. On Ubuntu, for example, the modules
from this repository install in the ``extra`` kernel module path, which
is not in the standard ``depmod`` search path. Therefore, for the
duration of your testing, edit ``/etc/depmod.d/ubuntu.conf`` and add
``extra`` to the beginning of the search path.
You may then install using
``sudo make install; sudo ldconfig; sudo depmod``. You'd uninstall with
``sudo make uninstall; sudo ldconfig; sudo depmod``.
.. _running-zloopsh-and-zfs-testssh:
Running zloop.sh and zfs-tests.sh
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If you wish to run the ZFS Test Suite (ZTS), then ``ksh`` and a few
additional utilities must be installed.
- **RHEL/CentOS 7:**
.. code:: sh
sudo yum install ksh bc fio acl sysstat mdadm lsscsi parted attr dbench nfs-utils samba rng-tools pax perf
- **RHEL/CentOS 8, Fedora:**
.. code:: sh
sudo dnf install ksh bc fio acl sysstat mdadm lsscsi parted attr dbench nfs-utils samba rng-tools pax perf
- **Debian, Ubuntu:**
.. code:: sh
sudo apt install ksh bc fio acl sysstat mdadm lsscsi parted attr dbench nfs-kernel-server samba rng-tools pax linux-tools-common selinux-utils quota
There are a few helper scripts provided in the top-level scripts
directory designed to aid developers working with in-tree builds.
- **zfs-helper.sh:** Certain functionality (i.e. /dev/zvol/) depends on
the ZFS provided udev helper scripts being installed on the system.
This script can be used to create symlinks on the system from the
installation location to the in-tree helper. These links must be in
place to successfully run the ZFS Test Suite. The **-i** and **-r**
options can be used to install and remove the symlinks.
::
sudo ./scripts/zfs-helpers.sh -i
- **zfs.sh:** The freshly built kernel modules can be loaded using
``zfs.sh``. This script can latter be used to unload the kernel
modules with the **-u** option.
::
sudo ./scripts/zfs.sh
- **zloop.sh:** A wrapper to run ztest repeatedly with randomized
arguments. The ztest command is a user space stress test designed to
detect correctness issues by concurrently running a random set of
test cases. If a crash is encountered, the ztest logs, any associated
vdev files, and core file (if one exists) are collected and moved to
the output directory for analysis.
::
sudo ./scripts/zloop.sh
- **zfs-tests.sh:** A wrapper which can be used to launch the ZFS Test
Suite. Three loopback devices are created on top of sparse files
located in ``/var/tmp/`` and used for the regression test. Detailed
directions for the ZFS Test Suite can be found in the
`README <https://github.com/zfsonlinux/zfs/tree/master/tests>`__
located in the top-level tests directory.
::
./scripts/zfs-tests.sh -vx
**tip:** The **delegate** tests will be skipped unless group read
permission is set on the zfs directory and its parents.

207
docs/Developer Resources/Custom Packages.rst Normal file
View File

@@ -0,0 +1,207 @@
Custom Packages
===============
The following instructions assume you are building from an official
`release tarball <https://github.com/zfsonlinux/zfs/releases/latest>`__
(version 0.8.0 or newer) or directly from the `git
repository <https://github.com/zfsonlinux/zfs>`__. Most users should not
need to do this and should preferentially use the distribution packages.
As a general rule the distribution packages will be more tightly
integrated, widely tested, and better supported. However, if your
distribution of choice doesn't provide packages, or you're a developer
and want to roll your own, here's how to do it.
The first thing to be aware of is that the build system is capable of
generating several different types of packages. Which type of package
you choose depends on what's supported on your platform and exactly what
your needs are.
- **DKMS** packages contain only the source code and scripts for
rebuilding the kernel modules. When the DKMS package is installed
kernel modules will be built for all available kernels. Additionally,
when the kernel is upgraded new kernel modules will be automatically
built for that kernel. This is particularly convenient for desktop
systems which receive frequent kernel updates. The downside is that
because the DKMS packages build the kernel modules from source a full
development environment is required which may not be appropriate for
large deployments.
- **kmods** packages are binary kernel modules which are compiled
against a specific version of the kernel. This means that if you
update the kernel you must compile and install a new kmod package. If
you don't frequently update your kernel, or if you're managing a
large number of systems, then kmod packages are a good choice.
- **kABI-tracking kmod** Packages are similar to standard binary kmods
and may be used with Enterprise Linux distributions like Red Hat and
CentOS. These distributions provide a stable kABI (Kernel Application
Binary Interface) which allows the same binary modules to be used
with new versions of the distribution provided kernel.
By default the build system will generate user packages and both DKMS
and kmod style kernel packages if possible. The user packages can be
used with either set of kernel packages and do not need to be rebuilt
when the kernel is updated. You can also streamline the build process by
building only the DKMS or kmod packages as shown below.
Be aware that when building directly from a git repository you must
first run the *autogen.sh* script to create the *configure* script. This
will require installing the GNU autotools packages for your
distribution. To perform any of the builds, you must install all the
necessary development tools and headers for your distribution.
It is important to note that if the development kernel headers for the
currently running kernel aren't installed, the modules won't compile
properly.
- `Red Hat, CentOS and Fedora <#red-hat-centos-and-fedora>`__
- `Debian and Ubuntu <#debian-and-ubuntu>`__
RHEL, CentOS and Fedora
-----------------------
Make sure that the required packages are installed to build the latest
ZFS 0.8 release:
- **RHEL/CentOS 7**:
.. code:: sh
sudo yum install epel-release gcc make autoconf automake libtool rpm-build dkms libtirpc-devel libblkid-devel libuuid-devel libudev-devel openssl-devel zlib-devel libaio-devel libattr-devel elfutils-libelf-devel kernel-devel-$(uname -r) python python2-devel python-setuptools python-cffi libffi-devel
- **RHEL/CentOS 8, Fedora**:
.. code:: sh
sudo dnf install gcc make autoconf automake libtool rpm-build kernel-rpm-macros dkms libtirpc-devel libblkid-devel libuuid-devel libudev-devel openssl-devel zlib-devel libaio-devel libattr-devel elfutils-libelf-devel kernel-devel-$(uname -r) python3 python3-devel python3-setuptools python3-cffi libffi-devel
`Get the source code <#get-the-source-code>`__.
DKMS
~~~~
Building rpm-based DKMS and user packages can be done as follows:
.. code:: sh
$ cd zfs
$ ./configure
$ make -j1 rpm-utils rpm-dkms
$ sudo yum localinstall *.$(uname -p).rpm *.noarch.rpm
kmod
~~~~
The key thing to know when building a kmod package is that a specific
Linux kernel must be specified. At configure time the build system will
make an educated guess as to which kernel you want to build against.
However, if configure is unable to locate your kernel development
headers, or you want to build against a different kernel, you must
specify the exact path with the *--with-linux* and *--with-linux-obj*
options.
.. code:: sh
$ cd zfs
$ ./configure
$ make -j1 rpm-utils rpm-kmod
$ sudo yum localinstall *.$(uname -p).rpm
kABI-tracking kmod
~~~~~~~~~~~~~~~~~~
The process for building kABI-tracking kmods is almost identical to for
building normal kmods. However, it will only produce binaries which can
be used by multiple kernels if the distribution supports a stable kABI.
In order to request kABI-tracking package the *--with-spec=redhat*
option must be passed to configure.
**NOTE:** This type of package is not available for Fedora.
.. code:: sh
$ cd zfs
$ ./configure --with-spec=redhat
$ make -j1 rpm-utils rpm-kmod
$ sudo yum localinstall *.$(uname -p).rpm
Debian and Ubuntu
-----------------
Make sure that the required packages are installed:
.. code:: sh
sudo apt install build-essential autoconf automake libtool gawk alien fakeroot dkms libblkid-dev uuid-dev libudev-dev libssl-dev zlib1g-dev libaio-dev libattr1-dev libelf-dev linux-headers-$(uname -r) python3 python3-dev python3-setuptools python3-cffi libffi-dev
`Get the source code <#get-the-source-code>`__.
.. _kmod-1:
kmod
~~~~
The key thing to know when building a kmod package is that a specific
Linux kernel must be specified. At configure time the build system will
make an educated guess as to which kernel you want to build against.
However, if configure is unable to locate your kernel development
headers, or you want to build against a different kernel, you must
specify the exact path with the *--with-linux* and *--with-linux-obj*
options.
.. code:: sh
$ cd zfs
$ ./configure
$ make -j1 deb-utils deb-kmod
$ for file in *.deb; do sudo gdebi -q --non-interactive $file; done
.. _dkms-1:
DKMS
~~~~
Building deb-based DKMS and user packages can be done as follows:
.. code:: sh
$ sudo apt-get install dkms
$ cd zfs
$ ./configure
$ make -j1 deb-utils deb-dkms
$ for file in *.deb; do sudo gdebi -q --non-interactive $file; done
Get the Source Code
-------------------
Released Tarball
~~~~~~~~~~~~~~~~
The released tarball contains the latest fully tested and released
version of ZFS. This is the preferred source code location for use in
production systems. If you want to use the official released tarballs,
then use the following commands to fetch and prepare the source.
.. code:: sh
$ wget http://archive.zfsonlinux.org/downloads/zfsonlinux/zfs/zfs-x.y.z.tar.gz
$ tar -xzf zfs-x.y.z.tar.gz
Git Master Branch
~~~~~~~~~~~~~~~~~
The Git *master* branch contains the latest version of the software, and
will probably contain fixes that, for some reason, weren't included in
the released tarball. This is the preferred source code location for
developers who intend to modify ZFS. If you would like to use the git
version, you can clone it from Github and prepare the source like this.
.. code:: sh
$ git clone https://github.com/zfsonlinux/zfs.git
$ cd zfs
$ ./autogen.sh
Once the source has been prepared you'll need to decide what kind of
packages you're building and jump the to appropriate section above. Note
that not all package types are supported for all platforms.

210
docs/Developer Resources/Git and GitHub for beginners.rst Normal file
View File

@@ -0,0 +1,210 @@
Git and GitHub for beginners (ZoL edition)
==========================================
This is a very basic rundown of how to use Git and GitHub to make
changes.
Recommended reading: `ZFS on Linux
CONTRIBUTING.md <https://github.com/zfsonlinux/zfs/blob/master/.github/CONTRIBUTING.md>`__
First time setup
----------------
If you've never used Git before, you'll need a little setup to start
things off.
::
git config --global user.name "My Name"
git config --global user.email myemail@noreply.non
Cloning the initial repository
------------------------------
The easiest way to get started is to click the fork icon at the top of
the main repository page. From there you need to download a copy of the
forked repository to your computer:
::
git clone https://github.com/<your-account-name>/zfs.git
This sets the "origin" repository to your fork. This will come in handy
when creating pull requests. To make pulling from the "upstream"
repository as changes are made, it is very useful to establish the
upstream repository as another remote (man git-remote):
::
cd zfs
git remote add upstream https://github.com/zfsonlinux/zfs.git
Preparing and making changes
----------------------------
In order to make changes it is recommended to make a branch, this lets
you work on several unrelated changes at once. It is also not
recommended to make changes to the master branch unless you own the
repository.
::
git checkout -b my-new-branch
From here you can make your changes and move on to the next step.
Recommended reading: `C Style and Coding Standards for
SunOS <https://www.cis.upenn.edu/~lee/06cse480/data/cstyle.ms.pdf>`__,
`ZFS on Linux Developer
Resources <https://github.com/zfsonlinux/zfs/wiki/Developer-Resources>`__,
`OpenZFS Developer
Resources <http://open-zfs.org/wiki/Developer_resources>`__
Testing your patches before pushing
-----------------------------------
Before committing and pushing, you may want to test your patches. There
are several tests you can run against your branch such as style
checking, and functional tests. All pull requests go through these tests
before being pushed to the main repository, however testing locally
takes the load off the build/test servers. This step is optional but
highly recommended, however the test suite should be run on a virtual
machine or a host that currently does not use ZFS. You may need to
install ``shellcheck`` and ``flake8`` to run the ``checkstyle``
correctly.
::
sh autogen.sh
./configure
make checkstyle
Recommended reading: `Building
ZFS <https://github.com/zfsonlinux/zfs/wiki/Building-ZFS>`__, `ZFS Test
Suite
README <https://github.com/zfsonlinux/zfs/blob/master/tests/README.md>`__
Committing your changes to be pushed
------------------------------------
When you are done making changes to your branch there are a few more
steps before you can make a pull request.
::
git commit --all --signoff
This command opens an editor and adds all unstaged files from your
branch. Here you need to describe your change and add a few things:
::
# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
# On branch my-new-branch
# Changes to be committed:
# (use "git reset HEAD <file>..." to unstage)
#
# modified: hello.c
#
The first thing we need to add is the commit message. This is what is
displayed on the git log, and should be a short description of the
change. By style guidelines, this has to be less than 72 characters in
length.
Underneath the commit message you can add a more descriptive text to
your commit. The lines in this section have to be less than 72
characters.
When you are done, the commit should look like this:
::
Add hello command
This is a test commit with a descriptive commit message.
This message can be more than one line as shown here.
Signed-off-by: My Name <myemail@noreply.non>
Closes #9998
Issue #9999
# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
# On branch my-new-branch
# Changes to be committed:
# (use "git reset HEAD <file>..." to unstage)
#
# modified: hello.c
#
You can also reference issues and pull requests if you are filing a pull
request for an existing issue as shown above. Save and exit the editor
when you are done.
Pushing and creating the pull request
-------------------------------------
Home stretch. You've made your change and made the commit. Now it's time
to push it.
::
git push --set-upstream origin my-new-branch
This should ask you for your github credentials and upload your changes
to your repository.
The last step is to either go to your repository or the upstream
repository on GitHub and you should see a button for making a new pull
request for your recently committed branch.
Correcting issues with your pull request
----------------------------------------
Sometimes things don't always go as planned and you may need to update
your pull request with a correction to either your commit message, or
your changes. This can be accomplished by re-pushing your branch. If you
need to make code changes or ``git add`` a file, you can do those now,
along with the following:
::
git commit --amend
git push --force
This will return you to the commit editor screen, and push your changes
over top of the old ones. Do note that this will restart the process of
any build/test servers currently running and excessively pushing can
cause delays in processing of all pull requests.
Maintaining your repository
---------------------------
When you wish to make changes in the future you will want to have an
up-to-date copy of the upstream repository to make your changes on. Here
is how you keep updated:
::
git checkout master
git pull upstream master
git push origin master
This will make sure you are on the master branch of the repository, grab
the changes from upstream, then push them back to your repository.
Final words
-----------
This is a very basic introduction to Git and GitHub, but should get you
on your way to contributing to many open source projects. Not all
projects have style requirements and some may have different processes
to getting changes committed so please refer to their documentation to
see if you need to do anything different. One topic we have not touched
on is the ``git rebase`` command which is a little more advanced for
this wiki article.
Additional resources: `Github Help <https://help.github.com/>`__,
`Atlassian Git Tutorials <https://www.atlassian.com/git/tutorials>`__

573
docs/Developer Resources/OpenZFS Exceptions.rst Normal file
View File

@@ -0,0 +1,573 @@
OpenZFS Exceptions
==================
Commit exceptions used to explicitly reference a given Linux commit.
These exceptions are useful for a variety of reasons.
**This page is used to generate**
`OpenZFS Tracking <http://build.zfsonlinux.org/openzfs-tracking.html>`__
**page.**
Format:
^^^^^^^
- ``<openzfs issue>|-|<comment>`` - The OpenZFS commit isn't applicable
to Linux, or the OpenZFS -> ZFS on Linux commit matching is unable to
associate the related commits due to lack of information (denoted by
a -).
- ``<openzfs issue>|<commit>|<comment>`` - The fix was merged to Linux
prior to their being an OpenZFS issue.
- ``<openzfs issue>|!|<comment>`` - The commit is applicable but not
applied for the reason described in the comment.
+------------------+-------------------+-----------------------------+
| OpenZFS issue id | status/ZFS commit | comment |
+==================+===================+=============================+
| 10500 | 03916905 | |
+------------------+-------------------+-----------------------------+
| 10154 | - | Not applicable to Linux |
+------------------+-------------------+-----------------------------+
| 10067 | - | The only ZFS change was to |
| | | zfs remap, which was |
| | | removed on Linux. |
+------------------+-------------------+-----------------------------+
| 9884 | - | Not applicable to Linux |
+------------------+-------------------+-----------------------------+
| 9851 | - | Not applicable to Linux |
+------------------+-------------------+-----------------------------+
| 9683 | - | Not applicable to Linux due |
| | | to devids not being used |
+------------------+-------------------+-----------------------------+
| 9680 | - | Applied and rolled back in |
| | | OpenZFS, additional changes |
| | | needed. |
+------------------+-------------------+-----------------------------+
| 9672 | 29445fe3 | |
+------------------+-------------------+-----------------------------+
| 9626 | 59e6e7ca | |
+------------------+-------------------+-----------------------------+
| 9635 | - | Not applicable to Linux |
+------------------+-------------------+-----------------------------+
| 9623 | 22448f08 | |
+------------------+-------------------+-----------------------------+
| 9621 | 305bc4b3 | |
+------------------+-------------------+-----------------------------+
| 9539 | 5228cf01 | |
+------------------+-------------------+-----------------------------+
| 9512 | b4555c77 | |
+------------------+-------------------+-----------------------------+
| 9487 | 48fbb9dd | |
+------------------+-------------------+-----------------------------+
| 9466 | 272b5d73 | |
+------------------+-------------------+-----------------------------+
| 9433 | 0873bb63 | |
+------------------+-------------------+-----------------------------+
| 9421 | 64c1dcef | |
+------------------+-------------------+-----------------------------+
| 9237 | - | Introduced by 8567 which |
| | | was never applied to Linux |
+------------------+-------------------+-----------------------------+
| 9194 | - | Not applicable the '-o |
| | | ashift=value' option is |
| | | provided on Linux |
+------------------+-------------------+-----------------------------+
| 9077 | - | Not applicable to Linux |
+------------------+-------------------+-----------------------------+
| 9027 | 4a5d7f82 | |
+------------------+-------------------+-----------------------------+
| 9018 | 3ec34e55 | |
+------------------+-------------------+-----------------------------+
| 8984 | ! | WIP to support NFSv4 ACLs |
+------------------+-------------------+-----------------------------+
| 8969 | - | Not applicable to Linux |
+------------------+-------------------+-----------------------------+
| 8942 | 650258d7 | |
+------------------+-------------------+-----------------------------+
| 8941 | 390d679a | |
+------------------+-------------------+-----------------------------+
| 8858 | - | Not applicable to Linux |
+------------------+-------------------+-----------------------------+
| 8856 | - | Not applicable to Linux due |
| | | to Encryption (b525630) |
+------------------+-------------------+-----------------------------+
| 8809 | ! | Adding libfakekernel needs |
| | | to be done by refactoring |
| | | existing code. |
+------------------+-------------------+-----------------------------+
| 8713 | 871e0732 | |
+------------------+-------------------+-----------------------------+
| 8661 | 1ce23dca | |
+------------------+-------------------+-----------------------------+
| 8648 | f763c3d1 | |
+------------------+-------------------+-----------------------------+
| 8602 | a032ac4 | |
+------------------+-------------------+-----------------------------+
| 8601 | d99a015 | Equivalent fix included in |
| | | initial commit |
+------------------+-------------------+-----------------------------+
| 8590 | 935e2c2 | |
+------------------+-------------------+-----------------------------+
| 8569 | - | This change isn't relevant |
| | | for Linux. |
+------------------+-------------------+-----------------------------+
| 8567 | - | An alternate fix was |
| | | applied for Linux. |
+------------------+-------------------+-----------------------------+
| 8552 | 935e2c2 | |
+------------------+-------------------+-----------------------------+
| 8521 | ee6370a7 | |
+------------------+-------------------+-----------------------------+
| 8502 | ! | Apply when porting OpenZFS |
| | | 7955 |
+------------------+-------------------+-----------------------------+
| 8477 | 92e43c1 | |
+------------------+-------------------+-----------------------------+
| 8454 | - | An alternate fix was |
| | | applied for Linux. |
+------------------+-------------------+-----------------------------+
| 8408 | 5f1346c | |
+------------------+-------------------+-----------------------------+
| 8379 | - | This change isn't relevant |
| | | for Linux. |
+------------------+-------------------+-----------------------------+
| 8376 | - | This change isn't relevant |
| | | for Linux. |
+------------------+-------------------+-----------------------------+
| 8311 | ! | Need to assess |
| | | applicability to Linux. |
+------------------+-------------------+-----------------------------+
| 8304 | - | This change isn't relevant |
| | | for Linux. |
+------------------+-------------------+-----------------------------+
| 8300 | 44f09cd | |
+------------------+-------------------+-----------------------------+
| 8265 | - | The large_dnode feature has |
| | | been implemented for Linux. |
+------------------+-------------------+-----------------------------+
| 8168 | 78d95ea | |
+------------------+-------------------+-----------------------------+
| 8138 | 44f09cd | The spelling fix to the zfs |
| | | man page came in with the |
| | | mdoc conversion. |
+------------------+-------------------+-----------------------------+
| 8108 | - | An equivalent Linux |
| | | specific fix was made. |
+------------------+-------------------+-----------------------------+
| 8064 | - | This change isn't relevant |
| | | for Linux. |
+------------------+-------------------+-----------------------------+
| 8021 | 7657def | |
+------------------+-------------------+-----------------------------+
| 8022 | e55ebf6 | |
+------------------+-------------------+-----------------------------+
| 8013 | - | The change is illumos |
| | | specific and not applicable |
| | | for Linux. |
+------------------+-------------------+-----------------------------+
| 7982 | - | The change is illumos |
| | | specific and not applicable |
| | | for Linux. |
+------------------+-------------------+-----------------------------+
| 7970 | c30e58c | |
+------------------+-------------------+-----------------------------+
| 7956 | cda0317 | |
+------------------+-------------------+-----------------------------+
| 7955 | ! | Need to assess |
| | | applicability to Linux. If |
| | | porting, apply 8502. |
+------------------+-------------------+-----------------------------+
| 7869 | df7eecc | |
+------------------+-------------------+-----------------------------+
| 7816 | - | The change is illumos |
| | | specific and not applicable |
| | | for Linux. |
+------------------+-------------------+-----------------------------+
| 7803 | - | This functionality is |
| | | provided by |
| | | ``upda |
| | | te_vdev_config_dev_strs()`` |
| | | on Linux. |
+------------------+-------------------+-----------------------------+
| 7801 | 0eef1bd | Commit f25efb3 in |
| | | openzfs/master has a small |
| | | change for linting which is |
| | | being ported. |
+------------------+-------------------+-----------------------------+
| 7779 | - | The change isn't relevant, |
| | | ``zfs_ctldir.c`` was |
| | | rewritten for Linux. |
+------------------+-------------------+-----------------------------+
| 7740 | 32d41fb | |
+------------------+-------------------+-----------------------------+
| 7739 | 582cc014 | |
+------------------+-------------------+-----------------------------+
| 7730 | e24e62a | |
+------------------+-------------------+-----------------------------+
| 7710 | - | None of the illumos build |
| | | system is used under Linux. |
+------------------+-------------------+-----------------------------+
| 7602 | 44f09cd | |
+------------------+-------------------+-----------------------------+
| 7591 | 541a090 | |
+------------------+-------------------+-----------------------------+
| 7586 | c443487 | |
+------------------+-------------------+-----------------------------+
| 7570 | - | Due to differences in the |
| | | block layer all discards |
| | | are handled asynchronously |
| | | under Linux. This |
| | | functionality could be |
| | | ported but it's unclear to |
| | | what purpose. |
+------------------+-------------------+-----------------------------+
| 7542 | - | The Linux libshare code |
| | | differs significantly from |
| | | the upstream OpenZFS code. |
| | | Since this change doesn't |
| | | address a Linux specific |
| | | issue it doesn't need to be |
| | | ported. The eventual plan |
| | | is to retire all of the |
| | | existing libshare code and |
| | | use the ZED to more |
| | | flexibly control filesystem |
| | | sharing. |
+------------------+-------------------+-----------------------------+
| 7512 | - | None of the illumos build |
| | | system is used under Linux. |
+------------------+-------------------+-----------------------------+
| 7497 | - | DTrace is isn't readily |
| | | available under Linux. |
+------------------+-------------------+-----------------------------+
| 7446 | ! | Need to assess |
| | | applicability to Linux. |
+------------------+-------------------+-----------------------------+
| 7430 | 68cbd56 | |
+------------------+-------------------+-----------------------------+
| 7402 | 690fe64 | |
+------------------+-------------------+-----------------------------+
| 7345 | 058ac9b | |
+------------------+-------------------+-----------------------------+
| 7278 | - | Dynamic ARC tuning is |
| | | handled slightly |
| | | differently under Linux and |
| | | this case is covered by |
| | | arc_tuning_update() |
+------------------+-------------------+-----------------------------+
| 7238 | - | zvol_swap test already |
| | | disabled in ZoL |
+------------------+-------------------+-----------------------------+
| 7194 | d7958b4 | |
+------------------+-------------------+-----------------------------+
| 7164 | b1b85c87 | |
+------------------+-------------------+-----------------------------+
| 7041 | 33c0819 | |
+------------------+-------------------+-----------------------------+
| 7016 | d3c2ae1 | |
+------------------+-------------------+-----------------------------+
| 6914 | - | Under Linux the |
| | | arc_meta_limit can be tuned |
| | | with the |
| | | zfs_arc_meta_limit_percent |
| | | module option. |
+------------------+-------------------+-----------------------------+
| 6875 | ! | WIP to support NFSv4 ACLs |
+------------------+-------------------+-----------------------------+
| 6843 | f5f087e | |
+------------------+-------------------+-----------------------------+
| 6841 | 4254acb | |
+------------------+-------------------+-----------------------------+
| 6781 | 15313c5 | |
+------------------+-------------------+-----------------------------+
| 6765 | ! | WIP to support NFSv4 ACLs |
+------------------+-------------------+-----------------------------+
| 6764 | ! | WIP to support NFSv4 ACLs |
+------------------+-------------------+-----------------------------+
| 6763 | ! | WIP to support NFSv4 ACLs |
+------------------+-------------------+-----------------------------+
| 6762 | ! | WIP to support NFSv4 ACLs |
+------------------+-------------------+-----------------------------+
| 6648 | 6bb24f4 | |
+------------------+-------------------+-----------------------------+
| 6578 | 6bb24f4 | |
+------------------+-------------------+-----------------------------+
| 6577 | 6bb24f4 | |
+------------------+-------------------+-----------------------------+
| 6575 | 6bb24f4 | |
+------------------+-------------------+-----------------------------+
| 6568 | 6bb24f4 | |
+------------------+-------------------+-----------------------------+
| 6528 | 6bb24f4 | |
+------------------+-------------------+-----------------------------+
| 6494 | - | The ``vdev_disk.c`` and |
| | | ``vdev_file.c`` files have |
| | | been reworked extensively |
| | | for Linux. The proposed |
| | | changes are not needed. |
+------------------+-------------------+-----------------------------+
| 6468 | 6bb24f4 | |
+------------------+-------------------+-----------------------------+
| 6465 | 6bb24f4 | |
+------------------+-------------------+-----------------------------+
| 6434 | 472e7c6 | |
+------------------+-------------------+-----------------------------+
| 6421 | ca0bf58 | |
+------------------+-------------------+-----------------------------+
| 6418 | 131cc95 | |
+------------------+-------------------+-----------------------------+
| 6391 | ee06391 | |
+------------------+-------------------+-----------------------------+
| 6390 | 85802aa | |
+------------------+-------------------+-----------------------------+
| 6388 | 0de7c55 | |
+------------------+-------------------+-----------------------------+
| 6386 | 485c581 | |
+------------------+-------------------+-----------------------------+
| 6385 | f3ad9cd | |
+------------------+-------------------+-----------------------------+
| 6369 | 6bb24f4 | |
+------------------+-------------------+-----------------------------+
| 6368 | 2024041 | |
+------------------+-------------------+-----------------------------+
| 6346 | 058ac9b | |
+------------------+-------------------+-----------------------------+
| 6334 | 1a04bab | |
+------------------+-------------------+-----------------------------+
| 6290 | 017da6 | |
+------------------+-------------------+-----------------------------+
| 6250 | - | Linux handles crash dumps |
| | | in a fundamentally |
| | | different way than Illumos. |
| | | The proposed changes are |
| | | not needed. |
+------------------+-------------------+-----------------------------+
| 6249 | 6bb24f4 | |
+------------------+-------------------+-----------------------------+
| 6248 | 6bb24f4 | |
+------------------+-------------------+-----------------------------+
| 6220 | - | The b_thawed debug code was |
| | | unused under Linux and |
| | | removed. |
+------------------+-------------------+-----------------------------+
| 6209 | - | The Linux user space mutex |
| | | implementation is based on |
| | | phtread primitives. |
+------------------+-------------------+-----------------------------+
| 6095 | f866a4ea | |
+------------------+-------------------+-----------------------------+
| 6091 | c11f100 | |
+------------------+-------------------+-----------------------------+
| 5984 | 480f626 | |
+------------------+-------------------+-----------------------------+
| 5966 | 6bb24f4 | |
+------------------+-------------------+-----------------------------+
| 5961 | 22872ff | |
+------------------+-------------------+-----------------------------+
| 5882 | 83e9986 | |
+------------------+-------------------+-----------------------------+
| 5815 | - | This patch could be adapted |
| | | if needed use equivalent |
| | | Linux functionality. |
+------------------+-------------------+-----------------------------+
| 5770 | c3275b5 | |
+------------------+-------------------+-----------------------------+
| 5769 | dd26aa5 | |
+------------------+-------------------+-----------------------------+
| 5768 | - | The change isn't relevant, |
| | | ``zfs_ctldir.c`` was |
| | | rewritten for Linux. |
+------------------+-------------------+-----------------------------+
| 5766 | 4dd1893 | |
+------------------+-------------------+-----------------------------+
| 5693 | 0f7d2a4 | |
+------------------+-------------------+-----------------------------+
| 5692 | ! | This functionality should |
| | | be ported in such a way |
| | | that it can be integrated |
| | | with ``filefrag(8)``. |
+------------------+-------------------+-----------------------------+
| 5684 | 6bb24f4 | |
+------------------+-------------------+-----------------------------+
| 5410 | 0bf8501 | |
+------------------+-------------------+-----------------------------+
| 5409 | b23d543 | |
+------------------+-------------------+-----------------------------+
| 5379 | - | This particular issue never |
| | | impacted Linux due to the |
| | | need for a modified |
| | | zfs_putpage() |
| | | implementation. |
+------------------+-------------------+-----------------------------+
| 5316 | - | The illumos idmap facility |
| | | isn't available under |
| | | Linux. This patch could |
| | | still be applied to |
| | | minimize code delta or all |
| | | HAVE_IDMAP chunks could be |
| | | removed on Linux for better |
| | | readability. |
+------------------+-------------------+-----------------------------+
| 5313 | ec8501e | |
+------------------+-------------------+-----------------------------+
| 5312 | ! | This change should be made |
| | | but the ideal time to do it |
| | | is when the spl repository |
| | | is folded in to the zfs |
| | | repository (planned for |
| | | 0.8). At this time we'll |
| | | want to cleanup many of the |
| | | includes. |
+------------------+-------------------+-----------------------------+
| 5219 | ef56b07 | |
+------------------+-------------------+-----------------------------+
| 5179 | 3f4058c | |
+------------------+-------------------+-----------------------------+
| 5149 | - | Equivalent Linux |
| | | functionality is provided |
| | | by the |
| | | ``zvol_max_discard_blocks`` |
| | | module option. |
+------------------+-------------------+-----------------------------+
| 5148 | - | Discards are handled |
| | | differently under Linux, |
| | | there is no DKIOCFREE |
| | | ioctl. |
+------------------+-------------------+-----------------------------+
| 5136 | e8b96c6 | |
+------------------+-------------------+-----------------------------+
| 4752 | aa9af22 | |
+------------------+-------------------+-----------------------------+
| 4745 | 411bf20 | |
+------------------+-------------------+-----------------------------+
| 4698 | 4fcc437 | |
+------------------+-------------------+-----------------------------+
| 4620 | 6bb24f4 | |
+------------------+-------------------+-----------------------------+
| 4573 | 10b7549 | |
+------------------+-------------------+-----------------------------+
| 4571 | 6e1b9d0 | |
+------------------+-------------------+-----------------------------+
| 4570 | b1d13a6 | |
+------------------+-------------------+-----------------------------+
| 4391 | 78e2739 | |
+------------------+-------------------+-----------------------------+
| 4465 | cda0317 | |
+------------------+-------------------+-----------------------------+
| 4263 | 6bb24f4 | |
+------------------+-------------------+-----------------------------+
| 4242 | - | Neither vnodes or their |
| | | associated events exist |
| | | under Linux. |
+------------------+-------------------+-----------------------------+
| 4206 | 2820bc4 | |
+------------------+-------------------+-----------------------------+
| 4188 | 2e7b765 | |
+------------------+-------------------+-----------------------------+
| 4181 | 44f09cd | |
+------------------+-------------------+-----------------------------+
| 4161 | - | The Linux user space |
| | | reader/writer |
| | | implementation is based on |
| | | phtread primitives. |
+------------------+-------------------+-----------------------------+
| 4128 | ! | The |
| | | ldi_ev_register_callbacks() |
| | | interface doesn't exist |
| | | under Linux. It may be |
| | | possible to receive similar |
| | | notifications via the scsi |
| | | error handlers or possibly |
| | | a different interface. |
+------------------+-------------------+-----------------------------+
| 4072 | - | None of the illumos build |
| | | system is used under Linux. |
+------------------+-------------------+-----------------------------+
| 3947 | 7f9d994 | |
+------------------+-------------------+-----------------------------+
| 3928 | - | Neither vnodes or their |
| | | associated events exist |
| | | under Linux. |
+------------------+-------------------+-----------------------------+
| 3871 | d1d7e268 | |
+------------------+-------------------+-----------------------------+
| 3747 | 090ff09 | |
+------------------+-------------------+-----------------------------+
| 3705 | - | The Linux implementation |
| | | uses the lz4 workspace kmem |
| | | cache to resolve the stack |
| | | issue. |
+------------------+-------------------+-----------------------------+
| 3606 | c5b247f | |
+------------------+-------------------+-----------------------------+
| 3580 | - | Linux provides generic |
| | | ioctl handlers get/set |
| | | block device information. |
+------------------+-------------------+-----------------------------+
| 3543 | 8dca0a9 | |
+------------------+-------------------+-----------------------------+
| 3512 | 67629d0 | |
+------------------+-------------------+-----------------------------+
| 3507 | 43a696e | |
+------------------+-------------------+-----------------------------+
| 3444 | 6bb24f4 | |
+------------------+-------------------+-----------------------------+
| 3371 | 44f09cd | |
+------------------+-------------------+-----------------------------+
| 3311 | 6bb24f4 | |
+------------------+-------------------+-----------------------------+
| 3301 | - | The Linux implementation of |
| | | ``vdev_disk.c`` does not |
| | | include this comment. |
+------------------+-------------------+-----------------------------+
| 3258 | 9d81146 | |
+------------------+-------------------+-----------------------------+
| 3254 | ! | WIP to support NFSv4 ACLs |
+------------------+-------------------+-----------------------------+
| 3246 | cc92e9d | |
+------------------+-------------------+-----------------------------+
| 2933 | - | None of the illumos build |
| | | system is used under Linux. |
+------------------+-------------------+-----------------------------+
| 2897 | fb82700 | |
+------------------+-------------------+-----------------------------+
| 2665 | 32a9872 | |
+------------------+-------------------+-----------------------------+
| 2130 | 460a021 | |
+------------------+-------------------+-----------------------------+
| 1974 | - | This change was entirely |
| | | replaced in the ARC |
| | | restructuring. |
+------------------+-------------------+-----------------------------+
| 1898 | - | The zfs_putpage() function |
| | | was rewritten to properly |
| | | integrate with the Linux |
| | | VM. |
+------------------+-------------------+-----------------------------+
| 1700 | - | Not applicable to Linux, |
| | | the discard implementation |
| | | is entirely different. |
+------------------+-------------------+-----------------------------+
| 1618 | ca67b33 | |
+------------------+-------------------+-----------------------------+
| 1337 | 2402458 | |
+------------------+-------------------+-----------------------------+
| 1126 | e43b290 | |
+------------------+-------------------+-----------------------------+
| 763 | 3cee226 | |
+------------------+-------------------+-----------------------------+
| 742 | ! | WIP to support NFSv4 ACLs |
+------------------+-------------------+-----------------------------+
| 701 | 460a021 | |
+------------------+-------------------+-----------------------------+
| 348 | - | The Linux implementation of |
| | | ``vdev_disk.c`` must have |
| | | this differently. |
+------------------+-------------------+-----------------------------+
| 243 | - | Manual updates have been |
| | | made separately for Linux. |
+------------------+-------------------+-----------------------------+
| 184 | - | The zfs_putpage() function |
| | | was rewritten to properly |
| | | integrate with the Linux |
| | | VM. |
+------------------+-------------------+-----------------------------+

318
docs/Developer Resources/OpenZFS Patches.rst Normal file
View File

@@ -0,0 +1,318 @@
OpenZFS Patches
===============
The ZFS on Linux project is an adaptation of the upstream `OpenZFS
repository <https://github.com/openzfs/openzfs/>`__ designed to work in
a Linux environment. This upstream repository acts as a location where
new features, bug fixes, and performance improvements from all the
OpenZFS platforms can be integrated. Each platform is responsible for
tracking the OpenZFS repository and merging the relevant improvements
back in to their release.
For the ZFS on Linux project this tracking is managed through an
`OpenZFS tracking <http://build.zfsonlinux.org/openzfs-tracking.html>`__
page. The page is updated regularly and shows a list of OpenZFS commits
and their status in regard to the ZFS on Linux master branch.
This page describes the process of applying outstanding OpenZFS commits
to ZFS on Linux and submitting those changes for inclusion. As a
developer this is a great way to familiarize yourself with ZFS on Linux
and to begin quickly making a valuable contribution to the project. The
following guide assumes you have a `github
account <https://help.github.com/articles/signing-up-for-a-new-github-account/>`__,
are familiar with git, and are used to developing in a Linux
environment.
Porting OpenZFS changes to ZFS on Linux
---------------------------------------
Setup the Environment
~~~~~~~~~~~~~~~~~~~~~
**Clone the source.** Start by making a local clone of the
`spl <https://github.com/zfsonlinux/spl>`__ and
`zfs <https://github.com/zfsonlinux/zfs>`__ repositories.
::
$ git clone -o zfsonlinux https://github.com/zfsonlinux/spl.git
$ git clone -o zfsonlinux https://github.com/zfsonlinux/zfs.git
**Add remote repositories.** Using the GitHub web interface
`fork <https://help.github.com/articles/fork-a-repo/>`__ the
`zfs <https://github.com/zfsonlinux/zfs>`__ repository in to your
personal GitHub account. Add your new zfs fork and the
`openzfs <https://github.com/openzfs/openzfs/>`__ repository as remotes
and then fetch both repositories. The OpenZFS repository is large and
the initial fetch may take some time over a slow connection.
::
$ cd zfs
$ git remote add <your-github-account> git@github.com:<your-github-account>/zfs.git
$ git remote add openzfs https://github.com/openzfs/openzfs.git
$ git fetch --all
**Build the source.** Compile the spl and zfs master branches. These
branches are always kept stable and this is a useful verification that
you have a full build environment installed and all the required
dependencies are available. This may also speed up the compile time
latter for small patches where incremental builds are an option.
::
$ cd ../spl
$ sh autogen.sh && ./configure --enable-debug && make -s -j$(nproc)
$
$ cd ../zfs
$ sh autogen.sh && ./configure --enable-debug && make -s -j$(nproc)
Pick a patch
~~~~~~~~~~~~
Consult the `OpenZFS
tracking <http://build.zfsonlinux.org/openzfs-tracking.html>`__ page and
select a patch which has not yet been applied. For your first patch you
will want to select a small patch to familiarize yourself with the
process.
Porting a Patch
~~~~~~~~~~~~~~~
There are 2 methods:
- `cherry-pick (easier) <#cherry-pick>`__
- `manual merge <#manual-merge>`__
Please read about `manual merge <#manual-merge>`__ first to learn the
whole process.
Cherry-pick
^^^^^^^^^^^
You can start to
`cherry-pick <https://git-scm.com/docs/git-cherry-pick>`__ by your own,
but we have made a special
`script <https://github.com/zfsonlinux/zfs-buildbot/blob/master/scripts/openzfs-merge.sh>`__,
which tries to
`cherry-pick <https://git-scm.com/docs/git-cherry-pick>`__ the patch
automatically and generates the description.
0) Prepare environment:
Mandatory git settings (add to ``~/.gitconfig``):
::
[merge]
renameLimit = 999999
[user]
email = mail@yourmail.com
name = Your Name
Download the script:
::
wget https://raw.githubusercontent.com/zfsonlinux/zfs-buildbot/master/scripts/openzfs-merge.sh
1) Run:
::
./openzfs-merge.sh -d path_to_zfs_folder -c openzfs_commit_hash
This command will fetch all repositories, create a new branch
``autoport-ozXXXX`` (XXXX - OpenZFS issue number), try to cherry-pick,
compile and check cstyle on success.
If it succeeds without any merge conflicts - go to ``autoport-ozXXXX``
branch, it will have ready to pull commit. Congratulations, you can go
to step 7!
Otherwise you should go to step 2.
2) Resolve all merge conflicts manually. Easy method - install
`Meld <http://meldmerge.org/>`__ or any other diff tool and run
``git mergetool``.
3) Check all compile and cstyle errors (See `Testing a
patch <#testing-a-patch>`__).
4) Commit your changes with any description.
5) Update commit description (last commit will be changed):
::
./openzfs-merge.sh -d path_to_zfs_folder -g openzfs_commit_hash
6) Add any porting notes (if you have modified something):
``git commit --amend``
7) Push your commit to github:
``git push <your-github-account> autoport-ozXXXX``
8) Create a pull request to ZoL master branch.
9) Go to `Testing a patch <#testing-a-patch>`__ section.
Manual merge
^^^^^^^^^^^^
**Create a new branch.** It is important to create a new branch for
every commit you port to ZFS on Linux. This will allow you to easily
submit your work as a GitHub pull request and it makes it possible to
work on multiple OpenZFS changes concurrently. All development branches
need to be based off of the ZFS master branch and it's helpful to name
the branches after the issue number you're working on.
::
$ git checkout -b openzfs-<issue-nr> master
**Generate a patch.** One of the first things you'll notice about the
ZFS on Linux repository is that it is laid out differently than the
OpenZFS repository. Organizationally it is much flatter, this is
possible because it only contains the code for OpenZFS not an entire OS.
That means that in order to apply a patch from OpenZFS the path names in
the patch must be changed. A script called zfs2zol-patch.sed has been
provided to perform this translation. Use the ``git format-patch``
command and this script to generate a patch.
::
$ git format-patch --stdout <commit-hash>^..<commit-hash> | \
./scripts/zfs2zol-patch.sed >openzfs-<issue-nr>.diff
**Apply the patch.** In many cases the generated patch will apply
cleanly to the repository. However, it's important to keep in mind the
zfs2zol-patch.sed script only translates the paths. There are often
additional reasons why a patch might not apply. In some cases hunks of
the patch may not be applicable to Linux and should be dropped. In other
cases a patch may depend on other changes which must be applied first.
The changes may also conflict with Linux specific modifications. In all
of these cases the patch will need to be manually modified to apply
cleanly while preserving the its original intent.
::
$ git am ./openzfs-<commit-nr>.diff
**Update the commit message.** By using ``git format-patch`` to generate
the patch and then ``git am`` to apply it the original comment and
authorship will be preserved. However, due to the formatting of the
OpenZFS commit you will likely find that the entire commit comment has
been squashed in to the subject line. Use ``git commit --amend`` to
cleanup the comment and be careful to follow `these standard
guidelines <http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html>`__.
The summary line of an OpenZFS commit is often very long and you should
truncate it to 50 characters. This is useful because it preserves the
correct formatting of ``git log --pretty=oneline`` command. Make sure to
leave a blank line between the summary and body of the commit. Then
include the full OpenZFS commit message wrapping any lines which exceed
72 characters. Finally, add a ``Ported-by`` tag with your contact
information and both a ``OpenZFS-issue`` and ``OpenZFS-commit`` tag with
appropriate links. You'll want to verify your commit contains all of the
following information:
- The subject line from the original OpenZFS patch in the form:
"OpenZFS <issue-nr> - short description".
- The original patch authorship should be preserved.
- The OpenZFS commit message.
- The following tags:
- **Authored by:** Original patch author
- **Reviewed by:** All OpenZFS reviewers from the original patch.
- **Approved by:** All OpenZFS reviewers from the original patch.
- **Ported-by:** Your name and email address.
- **OpenZFS-issue:** https ://www.illumos.org/issues/issue
- **OpenZFS-commit:** https
://github.com/openzfs/openzfs/commit/hash
- **Porting Notes:** An optional section describing any changes
required when porting.
For example, OpenZFS issue 6873 was `applied to
Linux <https://github.com/zfsonlinux/zfs/commit/b3744ae>`__ from this
upstream `OpenZFS
commit <https://github.com/openzfs/openzfs/commit/ee06391>`__.
::
OpenZFS 6873 - zfs_destroy_snaps_nvl leaks errlist
Authored by: Chris Williamson <chris.williamson@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Ported-by: Denys Rtveliashvili <denys@rtveliashvili.name>
lzc_destroy_snaps() returns an nvlist in errlist.
zfs_destroy_snaps_nvl() should nvlist_free() it before returning.
OpenZFS-issue: https://www.illumos.org/issues/6873
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/ee06391
Testing a Patch
~~~~~~~~~~~~~~~
**Build the source.** Verify the patched source compiles without errors
and all warnings are resolved.
::
$ make -s -j$(nproc)
**Run the style checker.** Verify the patched source passes the style
checker, the command should return without printing any output.
::
$ make cstyle
**Open a Pull Request.** When your patch builds cleanly and passes the
style checks `open a new pull
request <https://help.github.com/articles/creating-a-pull-request/>`__.
The pull request will be queued for `automated
testing <https://github.com/zfsonlinux/zfs-buildbot/>`__. As part of the
testing the change is built for a wide range of Linux distributions and
a battery of functional and stress tests are run to detect regressions.
::
$ git push <your-github-account> openzfs-<issue-nr>
**Fix any issues.** Testing takes approximately 2 hours to fully
complete and the results are posted in the GitHub `pull
request <https://github.com/zfsonlinux/zfs/pull/4594>`__. All the tests
are expected to pass and you should investigate and resolve any test
failures. The `test
scripts <https://github.com/zfsonlinux/zfs-buildbot/tree/master/scripts>`__
are all available and designed to run locally in order reproduce an
issue. Once you've resolved the issue force update the pull request to
trigger a new round of testing. Iterate until all the tests are passing.
::
# Fix issue, amend commit, force update branch.
$ git commit --amend
$ git push --force <your-github-account> openzfs-<issue-nr>
Merging the Patch
~~~~~~~~~~~~~~~~~
**Review.** Lastly one of the ZFS on Linux maintainers will make a final
review of the patch and may request additional changes. Once the
maintainer is happy with the final version of the patch they will add
their signed-off-by, merge it to the master branch, mark it complete on
the tracking page, and thank you for your contribution to the project!
Porting ZFS on Linux changes to OpenZFS
---------------------------------------
Often an issue will be first fixed in ZFS on Linux or a new feature
developed. Changes which are not Linux specific should be submitted
upstream to the OpenZFS GitHub repository for review. The process for
this is described in the `OpenZFS
README <https://github.com/openzfs/openzfs/>`__.

18
docs/Developer Resources/index.rst Normal file
View File

@@ -0,0 +1,18 @@
Developer Resources
===================
.. toctree::
:maxdepth: 2
:caption: Contents:
:glob:
Custom Packages
Building ZFS
Buildbot Status <http://build.zfsonlinux.org/tgrid?length=100&branch=master&category=Platforms&rev_order=desc>
Buildbot Issue Tracking <http://build.zfsonlinux.org/known-issues.html>
Buildbot Options <https://github.com/zfsonlinux/zfs/wiki/Buildbot-Options>
OpenZFS Tracking <http://build.zfsonlinux.org/openzfs-tracking.html>
OpenZFS Patches
OpenZFS Exceptions
OpenZFS Documentation <http://open-zfs.org/wiki/Developer_resources>
Git and GitHub for beginners