index.md 9.79 KB
Newer Older
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
# Intel GFX CI


## What Is It About?

We are continously testing [i915][i915-desc] (a Linux kernel device driver
for Intel Graphics) in an automated fashion, with a goal of having production
ready upstream.

## Hardware List

Our CI system is composed of multiple Intel machines spanning many
generations and display types. You can find our full hardware list here:

 * [list of machines](/hardware.html)
 * [raw data dump](/hardware/) (dmidecode, various debugfses, etc.)


## Pre-Merge Testing

**This is the crux of this CI system.** We test each patch that goes in our
driver ([i915][i915-desc]) or in the DRM drivers test suite we use ([IGT GPU
Tools][igt-desc]) before it lands in the repository and compare the results
with the [post-merge baseline](#git-trees-tested-post-merge) ([we filter out
known bugs](#results-filtering-and-bug-tracking)). This shifts the cost of
integration on the people making the change and helps us avoid time-consuming
bisection and reverts.

[i915-desc]:      https://01.org/linuxgraphics/gfx-docs/drm/gpu/i915.html
[igt-desc]:       https://gitlab.freedesktop.org/drm/igt-gpu-tools#igt-gpu-tools

Since we accept patches through mailing lists, this is where you can find the
33 34 35 36 37 38 39
results - they are sent out as a replies to the original mail. We run
[BAT](#basic-acceptance-tests-aka-bat) and [FULL
IGT](#full-igt-aka-sharded-runs) (if BAT is successful). On a usual day you
can expect result emails for a BAT within 1 hour and for FULL IGT within 6
hours. You will receive the emails even if the run is successful.

Here are the mailing lists we currently support:
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

| Mailing List                           | List Info              | Archive                   | [Patchwork][patchwork-desc] |
| --                                     | --                     | --                        | --                          |
| intel-gfx@lists.freedesktop.org        | [info][intel-gfx-info] | [archive][intel-gfx-arch] | [patchwork][intel-gfx-pw]   |
| igt-dev@lists.freedesktop.org          | [info][igt-dev-info]   | [archive][igt-dev-arch]   | [patchwork][igt-dev-pw]     |
| intel-gfx-trybot@lists.freedesktop.org | [info][trybot-info]    | [archive][trybot-arch]    | [patchwork][trybot-pw]      |

[intel-gfx-info]: https://lists.freedesktop.org/mailman/listinfo/intel-gfx
[intel-gfx-arch]: https://lists.freedesktop.org/archives/intel-gfx/
[intel-gfx-pw]:   https://patchwork.freedesktop.org/project/intel-gfx/series/

[igt-dev-info]:   https://lists.freedesktop.org/mailman/listinfo/igt-dev
[igt-dev-arch]:   https://lists.freedesktop.org/archives/igt-dev/
[igt-dev-pw]:     https://patchwork.freedesktop.org/project/igt/series/

[trybot-info]:    https://lists.freedesktop.org/mailman/listinfo/intel-gfx-trybot
[trybot-arch]:    https://lists.freedesktop.org/archives/intel-gfx-trybot/
[trybot-pw]:      https://patchwork.freedesktop.org/project/intel-gfx-trybot/series/

[patchwork-desc]: https://gitlab.freedesktop.org/patchwork-fdo/patchwork-fdo#patchwork-fdo

Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
61
### Queues
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
62

Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
63
You can check our pre-merge queues for [BAT](#basic-acceptance-tests-aka-bat)
64 65
and [Full/Sharded](#full-igt-aka-sharded-runs). They are helpful for
assessing how busy the CI is. See [Queues](/queue/) for
66
details.
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
67

68 69
### Testing Combined IGT And Kernel Changes
See [Testing Combined IGT And Kernel Changes](/test-with.md) for details.
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
70 71 72 73 74 75 76 77 78 79 80 81

## Git Trees Tested Post-Merge

We test many trees post-merge. Some of them are our baseline for pre-merge
testing (drm-tip and IGT GPU Tools), while the others helps us to make sure
that what we submit upstream works and catch any potential issues cased by
changes in other drivers/trees before we actually integrate with them.

The testing is sparse, i.e. we poll the branch for changes periodically, and
if something has changed we run it through our CI, even if that means
multiple commits/merges.

82
| CI Results                                 | Documentation          | Repository                       |
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
83 84
| --                                         | --                     | --                               |
| [drm-tip][]                                | [docs][drm-tip-doc]    | [repo][drm-tip-repo]             |
85
| [IGT GPU Tools][drm-tip]<sup>\*</sup>      | [docs][igt-doc]        | [repo][igt-repo]                 |
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
86 87 88 89 90 91 92
| [Linus' tree][linus]                       | ???                    | [repo][linus-repo]               |
| [linux-next][]                             | [docs][linux-next-doc] | [repo][linux-next-repo]          |
| [drm-intel-fixes][drm-intel-fixes]         | [docs][drm-intel-doc]  | [repo][drm-intel-fixes-repo]     |
| [drm-intel-next-fixes][dinf]               | [docs][drm-intel-doc]  | [repo][dinf-repo]                |
| [drm-intel-next-queued][dinq]              | [docs][drm-intel-doc]  | [repo][dinq-repo]                |
| [drm-misc-fixes][drm-misc-fixes]           | [docs][drm-misc-doc]   | [repo][drm-misc-fixes-repo]      |
| [drm-misc-next-fixes][drm-misc-next-fixes] | [docs][drm-misc-doc]   | [repo][drm-misc-next-fixes-repo] |
93
| [drm-intel-media][drm-intel-media]         | ???                    | [repo][drm-intel-media-repo]     |
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125
| [Dave Airlie's branch][airlied]            | ???                    | [repo][airlied-repo]             |

*\*: IGT results are part of the drm-tip visualisation*

[drm-tip]:                   /tree/drm-tip/
[drm-tip-repo]:              https://cgit.freedesktop.org/drm-tip
[drm-tip-doc]:               https://drm.pages.freedesktop.org/maintainer-tools/drm-tip.html

[igt-repo]:                  https://gitlab.freedesktop.org/drm/igt-gpu-tools
[igt-doc]:                   https://gitlab.freedesktop.org/drm/igt-gpu-tools#igt-gpu-tools

[linus]:                     /tree/linus/
[linus-repo]:                https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

[linux-next]:                /tree/linux-next/
[linux-next-repo]:           https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/
[linux-next-doc]:            https://www.kernel.org/doc/man-pages/linux-next.html

[drm-intel-fixes]:           /tree/drm-intel-fixes/
[drm-intel-fixes-repo]:      /tree/drm-intel-fixes/
[dinf]:                      /tree/drm-intel-next-fixes/
[dinf-repo]:                 https://cgit.freedesktop.org/drm-intel/log/?h=drm-intel-next-fixes
[dinq]:                      /tree/drm-intel-next-queued/
[dinq-repo]:                 https://cgit.freedesktop.org/drm-intel/log/?h=drm-intel-next-queued
[drm-intel-doc]:             https://drm.pages.freedesktop.org/maintainer-tools/repositories.html#the-upstream-i915-driver-repository

[drm-misc-fixes]:            /tree/drm-misc-fixes/
[drm-misc-fixes-repo]:       https://cgit.freedesktop.org/drm-misc/log/?h=drm-misc-fixes
[drm-misc-next-fixes]:       /tree/drm-misc-next-fixes/
[drm-misc-next-fixes-repo]:  https://cgit.freedesktop.org/drm-misc/log/?h=drm-misc-next-fixes
[drm-misc-doc]:              https://drm.pages.freedesktop.org/maintainer-tools/drm-misc.html

126 127 128
[drm-intel-media]:           /tree/drm-intel-media/
[drm-intel-media-repo]:      https://cgit.freedesktop.org/~tursulin/drm-intel/?h=media

Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186
[airlied]:                   /tree/airlied/
[airlied-repo]:              https://cgit.freedesktop.org/~airlied/linux/log/?h=for-intel-ci


## Results Filtering And Bug Tracking

Bugs caught by our system are filled to the following bug trackers:

 * [freedesktop's bugzilla][fdo-bugzilla] - all things IGT and DRM
 * [kernel.org's bugzilla][korg-bugzilla] - all other upstream bugs

[fdo-bugzilla]:  https://bugs.freedesktop.org/
[korg-bugzilla]: https://bugzilla.kernel.org/

We also maintain a set of filters that tie those bugs to failures we see in
our CI using machine names/types and patterns in dmesg/test output. This way
we are able to filter known issues out of pre-merge results to decrease noise
for developers, keep track of bug's life cycle, and reproduction rate. The
tool makes us able to confirm that a supposed fix is indeed fixing things.

You can **browse the CI issues** with the related data using our tool called
[cibuglog][] (updated hourly).

[cibuglog]: https://intel-gfx-ci.01.org/cibuglog/


## The Kinds Of Runs

### Basic Acceptance Tests (aka BAT)

BAT is the most basic run in our portfolio - it consists of tests that help
us ensure that the testing configuration is in a working condition. It gates
all the sharded runs - if it breaks, then the build is deemed to broken to
use more of the CI time on it. It utilizes [fast-feedback.testlist][], which
you can find in the IGT repository.

[fast-feedback.testlist]: https://gitlab.freedesktop.org/drm/igt-gpu-tools/blob/master/tests/intel-ci/fast-feedback.testlist

### Full IGT (aka sharded runs)

If the BAT run is succesful, then we continue with the sharded run. Much
broader set of tests (everything you can find in IGT filtered through our
[blacklist.txt][]) is executed.

[blacklist.txt]: https://gitlab.freedesktop.org/drm/igt-gpu-tools/blob/master/tests/intel-ci/blacklist.txt


### Idle Runs

Those runs are complementary to the ones above, used mainly to gather extra
data and increase coverage. As the name suggests, they are run when CI would
be idle otherwise (i.e. there nothing to test for the regular
pre-merge/post-merge).

- [drmtip][drmtip-results] - Full IGT but on the same hosts that BAT uses,
  thus increasing coverege.
- [KASAN][kasan-results] - same as above but runs with
  [The Kernel Address Sanitizer][kasan] enabled.
187
- 100 re-runs - re-run BAT 100 times with fixed IGT and kernel for extra data
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
188 189 190 191 192 193 194 195 196
  on flip-floppers.

[drmtip-results]: /tree/drm-tip/drmtip.html
[kasan-results]:  /tree/drm-tip/kasan.html

[kasan]: https://www.kernel.org/doc/html/latest/dev-tools/kasan.html

## Contacts

197
 * **IRC:** #intel-gfx-ci @ freenode
198
 * **Mailing list:** [i915-ci-infra @ lists.freedesktop.org](https://lists.freedesktop.org/mailman/listinfo/i915-ci-infra)
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
199 200
 * **hardware/CI maintainer:** Tomi Sarvela - tomi.p.sarvela @ intel.com
 * **cibuglog/metrics maintainer:** Martin Peres - martin.peres @ intel.com