Skip to content
Snippets Groups Projects
Select Git revision
  • main default protected
  • cocci-1.3
  • test-hans
3 results
Blame Permalink
README.md 11.90 KiB

Linux-media CI

This is a set of scripts to test and validate the media subsystem of the Linux Kernel. It is meant to help contributors, committers and maintainers of our beloved subsystem.

Preparation

The scripts can be used in four different ways:

  • Gitlab
  • Public containers
  • Self generated container
  • Locally

Gitlab

This is the simplest way to run your tests.

You need a Gitlab project that forks the Linux kernel, configured the following way:

  • Settings > CI/CD > General pipelines > CI/CD configuration file >

    • gitlab/media-committers.yml@linux-media/media-ci: if your project lives in fdo.org
    • https://gitlab.freedesktop.org/linux-media/media-ci/-/raw/main/gitlab/media-committers.yml: if your project lives outside fdo.org

  • Settings > CI/CD > Runners

    • Make sure you have runners available here. media-ci do not have any hardware requirement on the runners

  • Settings > CI/CD > Variables > CI/CD Variables

    • Create a variable named FULL_CI with value 1 if you want to run the full test (discouraged, and slow).
    • Create a variable named RUN_CI_DEFAULT_BRANCH with value 1 if you want to run CI also on the default branch.
    • Create a variable named TEST_MEDIA_TARGET if you want to test your code against the test-media regression test. The value of the variable will be passed to test-media (typically mc or all). If the variable is set to bypass the test-media regression test will not run. Please note that right now we are running the test without kvm, so they will take longer than in your machine.
    • Create the variable V4L_UTILS_REPO if you want to test against a custom v4l-utils repository. Eg: git://linuxtv.org/v4l-utils.git@master

Once you have your project set up, just push a new branch with your changes to trigger a pipeline:

user@host:~/linux$ git remote add gitlab git@gitlab.freedesktop.org:linux-media/users/$user
user@host:~/linux$ git push gitlab HEAD:refs/heads/user/newfeature

Linux-media developers are encouraged to fork https://gitlab.freedesktop.org/linux-media/media-committers.git under https://gitlab.freedesktop.org/linux-media/users/$username. Please avoid creating a new project without forking. It will use much more fdo.org resources.

We provide a browser extension to improve the readability of the artifacts:

Public container

The repository generates a container{:.external} with all the tools required to test your kernel tree. You just need to have a container manager on your system like podman or docker to use it. This can be installed using your favourite distro{:.external}. Eg for Debian:

user@host:~/linux$ apt-get install podman

The container has the media-ci scripts located at /media-ci, and all the dependencies are already pre-installed.

The container can be instanced like this:

user@host:~/linux$ podman run --rm -it -v .:/workdir registry.freedesktop.org/linux-media/media-ci/abi sh /media-ci/testsuites/abi.sh --kernel_dir /workdir --junit_dir /workdir/junit

Where:

-v .:/workdir Mounts the current directory (your kernel tree) in the folder /workdir on the container

/media-ci/testsuites/abi.sh --kernel_dir /workdir --junit_dir /workdir/junit is the actual test

Self generated container

The containers can also be built locally. After cloning the media-ci (this) repository to your computer run:

user@host:~/linux$ ~/work/media-ci$ podman build -f docker/abi/Dockerfile -t abi .

This will generate a container tagged abi on your local container repository.

Call it like:

user@host:~/linux$ podman run --rm -it -v .:/workdir abi sh /media-ci/testsuites/abi.sh --kernel_dir /workdir --junit_dir /workdir/junit

Check the file gitlab/media-committers.yml for the different container names and how they are called.

Locally

Clone the media-ci repository and execute the script third_party/setup.sh inside it.

The list of dependencies can be inferred by looking at docker/media-ci/Dockerfile.

Call the different test*.sh scripts from a clean linux repository. The scripts will run the different tests and clean-up after them if they succeed. The exit code of the script determines if the test has passed or failed. E.g:

user@host:~/linux$ sh ~/media-ci/test-pahole.sh
"/home/media-ci/test-pahole.sh " Completed!

Tests

The tests are grouped in testsuites that can be considered the top level unit testing.

They can be found in the repository with the name testsuites/*.sh they call one or more smaller tests that are named test-*.sh, that can be also called directly.

When a test fails, it will have a non-zero retcode and will print the error.

When a test succeeds it will have a 0 retcode and will revert the repository to a clean state.

Test behaviour can be modified with environment variables and arguments. Just run the commands with the --help argument to get the list of valid arguments and env variables. E.g.

Usage: test-pahole.sh [--help] [--kernel_dir value] [--cross_compile value] [--cross_compile32 value] [--cc value] [--arch value]

Arguments:
 --help: show this help
 --kernel_dir /home/user/linux: Path of the kernel directory
 --cross_compile x86_64-linux-gnu-: Cross-compile prefix
 --cross_compile32 i686-linux-gnu-: Cross-compile prefix for 32-bit ABI-checks comparison
 --cc gcc: Compiler
 --arch x86_64: Kernel architecture

Environment Variables:
 --kernel_dir: KERNEL_DIR
 --cross_compile: CROSS_COMPILE
 --cross_compile32: CROSS_COMPILE32
 --cc: CC
 --arch: ARCH

ABI testsuite

It validates that the ABI has not changed. It contains three tests:

  • test-compile-all.sh:
    • Makes sure that all the media code is compiled.
  • test-pahole.sh:
    • Compiles pahole/pahole.c for all the supported arches.
    • It checks that there are not holes or padding.
    • It checks that arm32 and x86 are identical to arm64 and x86_64 respectively.
  • test-abi-dumper.sh:
    • Autogenerates a C file that contains all the media structures.
    • Use abi-compliance-checker (from abi-dumper) to validate that there is no ABI breakage.
Usage: testsuites/abi.sh [--help] [--kernel_dir value]

Arguments:
 --help: show this help
 --kernel_dir /home/user/linux: Path of the kernel directory

Environment Variables:
 --kernel_dir: KERNEL_DIR

Build testsuite

It validates that the code from the media subsystem can be built. It contains one test:

  • test-build.sh:
    • Builds the code using the provided options

The build testsuite builds for x86 and x86_64 using the allyesconfig. It also builds using manually created config files under the testdata/configs/ folder.

Usage: testsuites/build.sh [--help] [--kernel_dir value]

Arguments:
 --help: show this help
 --kernel_dir /home/user/linux: Path of the kernel directory

Environment Variables:
 --kernel_dir: KERNEL_DIR

LLVM testsuite

Similar to the build testsuite but using the clang compiler instead of gcc. It builds for arm, arm64, x86, x86_64, and powerpc with the allyesconfig option

Usage: testsuites/llvm.sh [--help] [--kernel_dir value]

Arguments:
 --help: show this help
 --kernel_dir /home/user/linux: Path of the kernel directory

Environment Variables:
 --kernel_dir: KERNEL_DIR

Build ancient testsuite

Similar to the build testsuite but using the oldest toolchain supported by the kernel, as per Documentation/process/changes.rst.

In this case we only build for x86_64 with the allyesconfig option.

Usage: testsuites/build-ancient.sh [--help] [--kernel_dir value]

Arguments:
 --help: show this help
 --kernel_dir /home/user/linux: Path of the kernel directory

Environment Variables:
 --kernel_dir: KERNEL_DIR

Bisect testsuite

It validates that a patchset can be bisectable. For every patch the following tests are executed to evaluate its bisectability:

  • test-build.sh:
    • Building allyesconfig ignoring warnings
  • test-misc.sh:
    • Checks for unsafe strcpy and family
Usage: testsuites/bisect.sh [--help] [--kernel_dir value] [--git_origin value]

Arguments:
 --help: show this help
 --kernel_dir /home/user/linux: Path of the kernel directory
 --git_origin origin/main: GIT ref used for reference

Environment Variables:
 --kernel_dir: KERNEL_DIR
 --git_origin: GIT_ORIGIN

Checkpatch testsuite

It validates that the code passes the checkpatch.pl check in strict mode, as well as some custom media rules like:

  • Device tree code is not landed via our tree.
  • The top maintainers are not directly in cc.
  • Subject starts with "media:".
Usage: testsuites/checkpatch.sh [--help] [--kernel_dir value] [--git_origin value] [--patches_max value]

Arguments:
 --help: show this help
 --kernel_dir /home/user/linux: Path of the kernel directory
 --git_origin origin/main: GIT ref used for reference
 --patches_max 50: Max number of patches to be analyzed

Environment Variables:
 --kernel_dir: KERNEL_DIR
 --git_origin: GIT_ORIGIN
 --patches_max: PATCHES_MAX

Doc testsuite

The documentation is formatted properly. It contains two tests:

  • test-kernel-doc.sh:
    • Validates kernel-doc for all the media header files.
  • test-spec.sh:
    • Creates a spec book with the media documentation.
Usage: testsuites/doc.sh [--help] [--kernel_dir value]

Arguments:
 --help: show this help
 --kernel_dir /home/user/linux: Path of the kernel directory

Environment Variables:
 --kernel_dir: KERNEL_DIR

Static testsuite

Kernel static analyzers do not introduce new warnings.

  • test-compile-all.sh:
    • Checks that all the media code can be compile tested.
  • test-sparse.sh:
    • Use the sparse static analyzer building x86_64 in allyesconfig.
  • test-smatch.sh:
    • Use the smatch static analyzer building x86_64 in allyesconfig.
  • test-coccinelle.sh:
    • Use the coccinelle static analyzer.
Usage: testsuites/static.sh [--help] [--kernel_dir value]

Arguments:
 --help: show this help
 --kernel_dir /home/user/linux: Path of the kernel directory

Environment Variables:
 --kernel_dir: KERNEL_DIR

Virtme testsuite

Runs the test-media script in a virtual machine using virtme. It runs the test twice, in 64 bit and 32 bit compatibility mode.

Usage: virtme.sh [--help] [--kernel_dir value] [--junit_dir value] [--do_setup value]

Arguments:
 --help: show this help
 --kernel_dir /home/user/linux: Path of the kernel directory
 --junit_dir /home/user/linux/junit: Output directory for junit logs and reports
 --do_setup 0: Setup the required dependencies
 --virtme_args /home/user/media-ci/third_party/v4l-utils "mc -kmemleak": Arguments for the virme script

Environment Variables:
 --kernel_dir: KERNEL_DIR
 --junit_dir: JUNIT_DIR
 --do_setup: DO_SETUP
 --virtme_args: VIRTME_ARGS

Trust testsuite

Validates that patches have been properly reviewed. The list of trusted reviewers is maintained in a separated repository. It needs to be imported using the script third_party/setup-committers.txt.

The files are named: committers.txt, core_committers.txt and maintainers.txt.

Usage: trust.sh [--help] [--kernel_dir value] [--git_origin value] [--patches_max value] [--junit_dir value] [--do_setup value]

Arguments:
 --help: show this help
 --kernel_dir /home/user/linux: Path of the kernel directory
 --git_origin origin/master: GIT ref used for reference
 --patches_max 100: Max number of patches to be analyzed
 --junit_dir /home/user/linux/junit: Output directory for junit logs and reports
 --do_setup 0: Setup the required dependencies

Environment Variables:
 --kernel_dir: KERNEL_DIR
 --git_origin: GIT_ORIGIN
 --patches_max: PATCHES_MAX
 --junit_dir: JUNIT_DIR
 --do_setup: DO_SETUP