Introduction

PJDFSTest is a file system test suite. It was originally written to validate the ZFS port to FreeBSD, but it now supports multiple operating systems and file systems. This is a complete rewrite of the original test suite in Rust.

NOTE: The documentation is still a work-in-progress.

Build

cd rust
cargo run

Run as root

cd rust
cargo build && sudo ./target/debug/pjdfstest

Getting started

The test suite is as file system agnostic as possible and tries to comply with the POSIX specification. Typically, tests which make use of non-POSIX features are opt-in and only tests for syscalls which must be available on every POSIX system are ran. It can be configured with the configuration file, by specifying additional features supported by the file system/operating system.

Command-line interface

pjdfstest [OPTIONS] [--] TEST_PATTERNS

• -h, --help - Print help message
• -c, --configuration-file CONFIGURATION-FILE - Path of the configuration file
• -l, --list-features - List opt-in features
• -e, --exact - Match names exactly
• -v, --verbose - Verbose mode
• -p, --path PATH - Path where the test suite will be executed
• [--] TEST_PATTERNS - Filter tests which match against the provided patterns

Example: pjdfstest -c pjdfstest.toml chmod

Filter tests

It is possible to filter which tests should be ran, by specifying which parts should match. Tests are usually identified by syscall and optionally the file type on which it operates.

Rootless running

The test suite can be ran without privileges. However, not all tests can be completed without privileges, therefore the coverage will be incomplete. For example, tests which need to switch users will not be run.

Dummy users/groups

The test suite needs dummy users and groups to be set up. This should be handled automatically when installing it via a package, but they need to be created otherwise. By default, the users (with the same name for the group associated to each of them) to create are:

• nobody
• tests
• pjdfstest

It is also possible to specify other users with the configuration file.

Create users

FreeBSD

cat <<EOF | adduser -w none -S -f -
pjdfstest::::::Dummy User for pjdfstest:/nonexistent:/sbin/nologin:
EOF

Linux

cat <<EOF | newusers
tests:x:::Dummy User for pjdfstest:/:/usr/bin/nologin
pjdfstest:x:::Dummy User for pjdfstest:/:/usr/bin/nologin
EOF

Configuration file

The test runner can read a configuration file. For now, only the TOML format is supported. Its path can be specified by using the -c PATH flag.

Sections

[features]

Some features are not available for every file system. For tests requiring such features, the execution becomes opt-in. The user can enable their execution, by adding the corresponding feature as a key in this section. A list of these opt-in features is provided when executing the runner with -l argument.

For example, with posix_fallocate:

[features]
posix_fallocate = {}

# Can also be specified by using key notation
[features.posix_fallocate]

Feature configuration

TODO

file_flags

Some tests are related to file flags. However, not all file systems and operating systems support all flags. To give a sufficient level of granularity, each supported flag can be specified in the configuration with the file_flags array.

[features]
posix_fallocate = {}
file_flags = ["UF_IMMUTABLE"]

secondary_fs

Some tests require a secondary file system. This can be specified in the configuration with the secondary_fs key, but also with the secondary_fs argument. The argument takes precedence over the configuration.

[features]
secondary_fs = "/mnt/ISO"

[dummy_auth]

This section allows to modify the mecanism for switching users, which is required by some tests.

[dummy_auth]
entries = [
  ["nobody", "nobody"],
  # nogroup instead for some Linux distros
  # ["nobody", "nogroup"],
  ["tests", "tests"],
  ["pjdfstest", "pjdfstest"],
]

• entries - An entry is composed of a username and its associated group. Exactly 3 entries need to be specified if the runner default ones cannot be used.

[settings]

[settings]
naptime = 0.001
allow_remount = false

• naptime - The duration for a "short" sleep. It should be greater than the timestamp granularity of the file system under test. The default value is 1 second.
• allow_remount - If set to true, the runner will run the EROFS tests, which require to remount the file system on which pjdsfstest is run as read-only.

Structure

The package is made of the tests, and a test runner to launch them.

Tests (tests/)

To present how tests are organized, we take the chmod syscall as example.

There is a separate module for each syscall being tested. Within each of those modules, there may be either a single file, or a separate file for each aspect of the syscall.

The hierarchy is like this:

graph TD
TG[Syscall module<br /><i>chmod</i>] --> TC1[Aspect<br /><i>errno</i>]

TC1 --> TC1F1[Test case]
TC1 --> TC1F2[Test case]
TC1 --> TC1F3[Test case]
TC1 --> TC1F4[Test case]

TG --> TC2[Aspect<br /><i>permission</i>]

TC2 --> TC2F1[Test case]
TC2 --> TC2F2[Test case]

Layout

src/tests
├── chmod (syscall)
│   ├── errno.rs (aspect)
│   ├── mod.rs (syscall declaration)
│   └── permission.rs (aspect)
└── mod.rs (glues syscalls together)

tests/mod.rs

All the modules for the test groups should be declared in this file.

pub mod chmod;

Syscall module

A syscall module contains test cases related to a specific syscall. Its declaration should be in the mod.rs file of the relevant folder (chmod/ in our case). Common syscall-specific helpers can go here.

Aspect

An optional aspect module contains test cases that all relate to a common aspect of the syscall. Here "aspect" is a subjective area of related functionality. The aspect module may be either:

• in a single file, which contains all the test functions,
• in a folder, which contains multiple modules for the test functions and a mod.rs file, in which the case is declared.

Except in the case of a very large set of test functions, the first style should be preferred.

Test case

Each test case exercises a minimal piece of the syscall's functionality. Each must be registered with the test_case! macro.

crate::test_case! {ctime => [Regular, Fifo, Block, Char, Socket]}
fn ctime(ctx: &mut TestContext, f_type: FileType) {
    let path = ctx.create(f_type).unwrap();
    let ctime_before = stat(&path).unwrap().st_ctime;

    sleep(Duration::from_secs(1));

    chmod(&path, Mode::from_bits_truncate(0o111)).unwrap();

    let ctime_after = stat(&path).unwrap().st_ctime;
    assert!(ctime_after > ctime_before);
}

Test runner (main.rs)

The test runner has to run the tests, and provide a command-line interface to allow the user to modify how the tests should be run. It takes the tests from the specified test groups.

Test declaration

Test cases have the same structure than usual Rust tests, that is unwraping Results and using assertion macros (assert and assert_eq), the exception being that it should take a &mut TestContext parameter. It might also take a FileType argument if required. It also needs an additional declaration with the test_case! macro alongside the function, with the function name being the only mandatory argument.

For example:

// chmod/00.t:L58
crate::test_case! {ctime => [Regular, Fifo, Block, Char, Socket]}
fn ctime(ctx: &mut TestContext, f_type: FileType) {
    let path = ctx.create(f_type).unwrap();
    let ctime_before = stat(&path).unwrap().st_ctime;

    sleep(Duration::from_secs(1));

    chmod(&path, Mode::from_bits_truncate(0o111)).unwrap();

    let ctime_after = stat(&path).unwrap().st_ctime;
    assert!(ctime_after > ctime_before);
}

Description

It is possible to provide doc comments which will be used as documentation for developers but also be displayed to users when they run the test. The doc comments should be written in the test_case! declaration, before anything. For example:

crate::test_case! {
    /// The file mode of a newly created file should not affect whether
    /// posix_fallocate will work, only the create args
    /// https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=154873
    affected_only_create_flags, serialized, root, FileSystemFeature::PosixFallocate
}

Parameterization

It is possible to give additional parameters to the test case macro, to modify the execution of the tests or add requirements.

File-system exclusive features

Some features are not available for every file system. For tests requiring such features, the execution becomes opt-in. A variant of the FileSystemFeature enum corresponding to this feature should be specified after potential root requirement and before file flags. Multiple features can be specified, separated by a comma ,.

For example:

#[cfg(target_os = "freebsd")]
crate::test_case! {eperm_immutable_flag, FileSystemFeature::Chflags, FileSystemFeature::PosixFallocate ...}

Adding features

New features can be added to the FileSystemFeature enum. A description of the feature should be provided as documentation for both developers and users.

Guards

It is possible to specify "guards", which are functions which checks if a requirement is met and return an error if not, so the test is skipped. They can be specified by appending function names after a ; separator, after potential root requirement and features.

The function has to take a &Config argument which contains the current configuration and a &Path which represents the parent folder of the potential test context which would be created.

Guard signature

/// Function which indicates if the test should be skipped by returning an error.
pub type Guard = fn(&Config, &Path) -> anyhow::Result<()>;

Example

fn has_reasonable_link_max(_: &Config, base_path: &Path) -> anyhow::Result<()> {
    let link_max = pathconf(base_path, nix::unistd::PathconfVar::LINK_MAX)?
        .ok_or_else(|| anyhow::anyhow!("Failed to get LINK_MAX value"))?;
    
    if link_max >= LINK_MAX_LIMIT {
        anyhow::bail!("LINK_MAX value is too high ({link_max}, expected smaller than {LINK_MAX_LIMIT}");
    }

    Ok(())
}

crate::test_case! {
    /// link returns EMLINK if the link count of the file named by name1 would exceed {LINK_MAX}
    link_count_max; has_reasonable_link_max
}
...

Root privileges

Some tests may need root privileges to run. To declare that a test function require such privileges, root should be added to its declaration. For example:

crate::test_case!{change_perm, root}

The root requirement is automatically added for privileged file types, namely block and char.

File types

Some test cases need to test over different file types. The file types should be added at the end of the test case declaration, within brackets and with a fat arrow before (=> [Regular]). The test function should also accept a FileType parameter to operate on.

For example:

crate::test_case! {change_perm, root, FileSystemFeature::Chflags; FileFlags::SF_IMMUTABLE, FileFlags::UF_IMMUTABLE 
=> [Regular, Fifo, Block, Char, Socket]}
fn change_perm(ctx: &mut TestContext, f_type: FileType) {

Platform-specific functions

Some functions (like lchmod) are not supported on every operating system. When a test make use of such function, it is possible to restrain its compilation to the supported operating systems, with the attribute #[cfg(target_os = ...)]. It is also possible to apply this attribute on an aspect, or even on a syscall module. For example:

#[cfg(target_os = "freebsd")]
mod lchmod;

Serialized test cases

Some test cases need functions only available when they are run serialized, especially when they affect the whole process. An example is changing user (SerializedTestContext::as_user). To have access to these functions, the test should be declared with a SerializedTestContext parameter in place of TestContext and the serialized keyword should be prepended before features.

For example:

crate::test_case! {
    /// The file mode of a newly created file should not affect whether
    /// posix_fallocate will work, only the create args
    /// https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=154873
    affected_only_create_flags, serialized, root, FileSystemFeature::PosixFallocate
}
fn affected_only_create_flags(ctx: &mut SerializedTestContext) {
    ctx.as_user(Some(Uid::from_raw(65534)), None, || {
        let path = subdir.join("test1");
        let file = open(&path, OFlag::O_CREAT | OFlag::O_RDWR, Mode::empty()).unwrap();
        assert!(posix_fallocate(file, 0, 1).is_ok());
    });
}