index.md 10.5 KB
Newer Older
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
# Intel GFX CI


## What Is It About?

We are continously testing [i915][i915-desc] (a Linux kernel device driver
for Intel Graphics) in an automated fashion, with a goal of having production
ready upstream.

## Hardware List

Our CI system is composed of multiple Intel machines spanning many
generations and display types. You can find our full hardware list here:

 * [list of machines](/hardware.html)
 * [raw data dump](/hardware/) (dmidecode, various debugfses, etc.)


## Pre-Merge Testing

**This is the crux of this CI system.** We test each patch that goes in our
driver ([i915][i915-desc]) or in the DRM drivers test suite we use ([IGT GPU
Tools][igt-desc]) before it lands in the repository and compare the results
with the [post-merge baseline](#git-trees-tested-post-merge) ([we filter out
known bugs](#results-filtering-and-bug-tracking)). This shifts the cost of
integration on the people making the change and helps us avoid time-consuming
bisection and reverts.

[i915-desc]:      https://01.org/linuxgraphics/gfx-docs/drm/gpu/i915.html
[igt-desc]:       https://gitlab.freedesktop.org/drm/igt-gpu-tools#igt-gpu-tools

Since we accept patches through mailing lists, this is where you can find the
33 34 35 36 37 38 39
results - they are sent out as a replies to the original mail. We run
[BAT](#basic-acceptance-tests-aka-bat) and [FULL
IGT](#full-igt-aka-sharded-runs) (if BAT is successful). On a usual day you
can expect result emails for a BAT within 1 hour and for FULL IGT within 6
hours. You will receive the emails even if the run is successful.

Here are the mailing lists we currently support:
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
40

41 42 43 44 45
| Mailing List                           | List Info              | Archive                   | [Patchwork][patchwork-desc]                                      |
| --                                     | --                     | --                        | --                                                               |
| intel-gfx@lists.freedesktop.org        | [info][intel-gfx-info] | [archive][intel-gfx-arch] | [patchwork][intel-gfx-pw]                                        |
| igt-dev@lists.freedesktop.org          | [info][igt-dev-info]   | [archive][igt-dev-arch]   | [patchwork][igt-dev-pw]                                          |
| intel-gfx-trybot@lists.freedesktop.org | [info][trybot-info]    | [archive][trybot-arch]    | [kernel patchwork][trybot-pw]<br/>[IGT patchwork][trybot-igt-pw] |
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
46

47 48 49
And [here is a guide for reading pre-merge results emails](/pre-merge-results.md).


Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
50 51 52 53 54 55 56 57 58 59 60
[intel-gfx-info]: https://lists.freedesktop.org/mailman/listinfo/intel-gfx
[intel-gfx-arch]: https://lists.freedesktop.org/archives/intel-gfx/
[intel-gfx-pw]:   https://patchwork.freedesktop.org/project/intel-gfx/series/

[igt-dev-info]:   https://lists.freedesktop.org/mailman/listinfo/igt-dev
[igt-dev-arch]:   https://lists.freedesktop.org/archives/igt-dev/
[igt-dev-pw]:     https://patchwork.freedesktop.org/project/igt/series/

[trybot-info]:    https://lists.freedesktop.org/mailman/listinfo/intel-gfx-trybot
[trybot-arch]:    https://lists.freedesktop.org/archives/intel-gfx-trybot/
[trybot-pw]:      https://patchwork.freedesktop.org/project/intel-gfx-trybot/series/
61
[trybot-igt-pw]:  https://patchwork.freedesktop.org/project/igt-trybot/series/
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
62 63 64

[patchwork-desc]: https://gitlab.freedesktop.org/patchwork-fdo/patchwork-fdo#patchwork-fdo

65

66 67 68 69 70 71 72 73 74 75
### Trybot

Trybot mailing list is for trying out changes on our infrastracture before
they are ready for a review. It has the lowest priority of all the mailing
lists. It supports both IGT and kernel patches.

IGT patches sould have `i-g-t` tag in the subject, e.g. `[PATCH i-g-t] My
patch`. This can be automated with: <br/>
`git config --local format.subjectPrefix "PATCH i-g-t"`

Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
76
### Queues
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
77

Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
78
You can check our pre-merge queues for [BAT](#basic-acceptance-tests-aka-bat)
79 80
and [Full/Sharded](#full-igt-aka-sharded-runs). They are helpful for
assessing how busy the CI is. See [Queues](/queue/) for
81
details.
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
82

83 84
### Testing Combined IGT And Kernel Changes
See [Testing Combined IGT And Kernel Changes](/test-with.md) for details.
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
85 86 87 88 89 90 91 92 93 94 95 96

## Git Trees Tested Post-Merge

We test many trees post-merge. Some of them are our baseline for pre-merge
testing (drm-tip and IGT GPU Tools), while the others helps us to make sure
that what we submit upstream works and catch any potential issues cased by
changes in other drivers/trees before we actually integrate with them.

The testing is sparse, i.e. we poll the branch for changes periodically, and
if something has changed we run it through our CI, even if that means
multiple commits/merges.

97
| CI Results                                 | Documentation          | Repository                       |
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
98 99
| --                                         | --                     | --                               |
| [drm-tip][]                                | [docs][drm-tip-doc]    | [repo][drm-tip-repo]             |
100
| [IGT GPU Tools][drm-tip]<sup>\*</sup>      | [docs][igt-doc]        | [repo][igt-repo]                 |
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
101 102 103 104 105 106 107
| [Linus' tree][linus]                       | ???                    | [repo][linus-repo]               |
| [linux-next][]                             | [docs][linux-next-doc] | [repo][linux-next-repo]          |
| [drm-intel-fixes][drm-intel-fixes]         | [docs][drm-intel-doc]  | [repo][drm-intel-fixes-repo]     |
| [drm-intel-next-fixes][dinf]               | [docs][drm-intel-doc]  | [repo][dinf-repo]                |
| [drm-intel-next-queued][dinq]              | [docs][drm-intel-doc]  | [repo][dinq-repo]                |
| [drm-misc-fixes][drm-misc-fixes]           | [docs][drm-misc-doc]   | [repo][drm-misc-fixes-repo]      |
| [drm-misc-next-fixes][drm-misc-next-fixes] | [docs][drm-misc-doc]   | [repo][drm-misc-next-fixes-repo] |
108
| [drm-intel-media][drm-intel-media]         | ???                    | [repo][drm-intel-media-repo]     |
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140
| [Dave Airlie's branch][airlied]            | ???                    | [repo][airlied-repo]             |

*\*: IGT results are part of the drm-tip visualisation*

[drm-tip]:                   /tree/drm-tip/
[drm-tip-repo]:              https://cgit.freedesktop.org/drm-tip
[drm-tip-doc]:               https://drm.pages.freedesktop.org/maintainer-tools/drm-tip.html

[igt-repo]:                  https://gitlab.freedesktop.org/drm/igt-gpu-tools
[igt-doc]:                   https://gitlab.freedesktop.org/drm/igt-gpu-tools#igt-gpu-tools

[linus]:                     /tree/linus/
[linus-repo]:                https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

[linux-next]:                /tree/linux-next/
[linux-next-repo]:           https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/
[linux-next-doc]:            https://www.kernel.org/doc/man-pages/linux-next.html

[drm-intel-fixes]:           /tree/drm-intel-fixes/
[drm-intel-fixes-repo]:      /tree/drm-intel-fixes/
[dinf]:                      /tree/drm-intel-next-fixes/
[dinf-repo]:                 https://cgit.freedesktop.org/drm-intel/log/?h=drm-intel-next-fixes
[dinq]:                      /tree/drm-intel-next-queued/
[dinq-repo]:                 https://cgit.freedesktop.org/drm-intel/log/?h=drm-intel-next-queued
[drm-intel-doc]:             https://drm.pages.freedesktop.org/maintainer-tools/repositories.html#the-upstream-i915-driver-repository

[drm-misc-fixes]:            /tree/drm-misc-fixes/
[drm-misc-fixes-repo]:       https://cgit.freedesktop.org/drm-misc/log/?h=drm-misc-fixes
[drm-misc-next-fixes]:       /tree/drm-misc-next-fixes/
[drm-misc-next-fixes-repo]:  https://cgit.freedesktop.org/drm-misc/log/?h=drm-misc-next-fixes
[drm-misc-doc]:              https://drm.pages.freedesktop.org/maintainer-tools/drm-misc.html

141 142 143
[drm-intel-media]:           /tree/drm-intel-media/
[drm-intel-media-repo]:      https://cgit.freedesktop.org/~tursulin/drm-intel/?h=media

Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201
[airlied]:                   /tree/airlied/
[airlied-repo]:              https://cgit.freedesktop.org/~airlied/linux/log/?h=for-intel-ci


## Results Filtering And Bug Tracking

Bugs caught by our system are filled to the following bug trackers:

 * [freedesktop's bugzilla][fdo-bugzilla] - all things IGT and DRM
 * [kernel.org's bugzilla][korg-bugzilla] - all other upstream bugs

[fdo-bugzilla]:  https://bugs.freedesktop.org/
[korg-bugzilla]: https://bugzilla.kernel.org/

We also maintain a set of filters that tie those bugs to failures we see in
our CI using machine names/types and patterns in dmesg/test output. This way
we are able to filter known issues out of pre-merge results to decrease noise
for developers, keep track of bug's life cycle, and reproduction rate. The
tool makes us able to confirm that a supposed fix is indeed fixing things.

You can **browse the CI issues** with the related data using our tool called
[cibuglog][] (updated hourly).

[cibuglog]: https://intel-gfx-ci.01.org/cibuglog/


## The Kinds Of Runs

### Basic Acceptance Tests (aka BAT)

BAT is the most basic run in our portfolio - it consists of tests that help
us ensure that the testing configuration is in a working condition. It gates
all the sharded runs - if it breaks, then the build is deemed to broken to
use more of the CI time on it. It utilizes [fast-feedback.testlist][], which
you can find in the IGT repository.

[fast-feedback.testlist]: https://gitlab.freedesktop.org/drm/igt-gpu-tools/blob/master/tests/intel-ci/fast-feedback.testlist

### Full IGT (aka sharded runs)

If the BAT run is succesful, then we continue with the sharded run. Much
broader set of tests (everything you can find in IGT filtered through our
[blacklist.txt][]) is executed.

[blacklist.txt]: https://gitlab.freedesktop.org/drm/igt-gpu-tools/blob/master/tests/intel-ci/blacklist.txt


### Idle Runs

Those runs are complementary to the ones above, used mainly to gather extra
data and increase coverage. As the name suggests, they are run when CI would
be idle otherwise (i.e. there nothing to test for the regular
pre-merge/post-merge).

- [drmtip][drmtip-results] - Full IGT but on the same hosts that BAT uses,
  thus increasing coverege.
- [KASAN][kasan-results] - same as above but runs with
  [The Kernel Address Sanitizer][kasan] enabled.
202
- 100 re-runs - re-run BAT 100 times with fixed IGT and kernel for extra data
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
203 204 205 206 207 208 209 210 211
  on flip-floppers.

[drmtip-results]: /tree/drm-tip/drmtip.html
[kasan-results]:  /tree/drm-tip/kasan.html

[kasan]: https://www.kernel.org/doc/html/latest/dev-tools/kasan.html

## Contacts

212
 * **IRC:** #intel-gfx-ci @ freenode
213
 * **Mailing list:** [i915-ci-infra @ lists.freedesktop.org](https://lists.freedesktop.org/mailman/listinfo/i915-ci-infra)
Arkadiusz Hiler's avatar
Arkadiusz Hiler committed
214 215
 * **hardware/CI maintainer:** Tomi Sarvela - tomi.p.sarvela @ intel.com
 * **cibuglog/metrics maintainer:** Martin Peres - martin.peres @ intel.com