On a slow network, most of the time for a setup is taken by the initial
clone of the kernel repository. It's common for kernel developers to
have a kernel clone lying around, so allow to borrow the git objects
from there.

Testing on a relatively fast connection, the initial checkout goes down
from ~7m45s to 2m21s.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>

It's tedious to run setup and having to be around for each prompt. Just
add a -y|--yes switch like we have in package managers to assume
yes/default.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>

The amount of repos/branches is not restricted to the ones in drm-intel
and drm-misc anymore. Now there are topic branches on drm repo and also
the drm-xe repo. Use wildcard matches to list anything under drm-* repos
and any topic branch, even those on drm repo.

One main difference is that now `dim create-workdir all` will really
checkout all topic branches. Users may be advised to checkout only what
it's interesting for them.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>

Add initial documentation describing the xe rules and flow. Also add
the xe maintainers to the dim pull request cc list.

v2:
- Updates suggested by Lucas

v3:
- More updates suggested by Lucas

v4:
- Updates suggested by Oded and Jani.

Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>

I occasionally bounce patches to intel-gfx@ to get CI for patches that
were only sent to dri-devel@ or were caught in the moderation
queue. This results in a Resent-From: header being added.

Applying such a patch leads to the From: match in apply_patch() to match
Resent-From: rather than the actual From:. If whoever bounced the patch
is also applying the patch, their Signed-off-by won't be automatically
added.

Match From: from the beginning of the line to fix this.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>

Since commit e700ea2f ("dim: Introduce a new configuration file in
.config/dim") it's not possible to run `dim checkpatch` anymore without
a config file. Check for the existence of both $XDG_CONFIG_HOME/dim/dimrc
and $HOME/.dimrc before sourcing them.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>

The dim configuration has been stored in ~/.dimrc so far, but using
XDG_CONFIG_HOME allows for a tidier home directory.

Introduce a new XDG_CONFIG_HOME/dim/dimrc path.

Signed-off-by: Maxime Ripard <mripard@kernel.org>

XDG_CACHE_HOME is meant to hold "user-specific, non-essential data"
which seems like a good fit and avoids bloating ~.

Signed-off-by: Maxime Ripard <mripard@kernel.org>

XDG_CACHE_HOME is meant to hold "user-specific, non-essential data"
which seems like a good fit and avoids bloating ~.

Signed-off-by: Maxime Ripard <mripard@kernel.org>

The code has multiple places where the .dim-last-path file is
referenced, so let's move it to a variable.

Signed-off-by: Maxime Ripard <mripard@kernel.org>

As we checkpatch the applied commits rather than the patches, avoid
mailmap conversions, if any. Otherwise we might get complaints about
author Signed-off-by missing.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

GNU grep 3.8 started to complain about backslashes where
they're not needed:
 grep: warning: stray \ before /
Get rid of them.

Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Acked-by: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Apparently my git (v2.40.0) does more warnings and we're not silencing
all the possible ones in the commit_rerere_cache function

Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>

In case a fix was not propagated yet to upstream, the
original commit sha won't be found hence the fix of this
fix will be missed during the cherry-pick.

Since dim cherry-picks reliably adds the -x "(cherry picked
from" message, let's also grep in the log to see if that
was the case, before we give up and forget.

Also, if that's the case, let's then fix the 'Fixes:' tag.

v2: Actually replaces the 'Fixes:' tag, otherwise dim doesn't
allow us to push giving the following message:
"Fixes: SHA1 in not pointing at an ancestor:"
"dim: ERROR: issues in commits detected, aborting"

v3: - Also consider fix of fixes in drm-intel-fixes besides
      the drm-intel-next-fixes.
    - Use intel_remote branches instead of local.
    - And run the check unconditionally as suggested by Daniel.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>

After setting up drm-rerere, the integration config must be read again
to setup the environment variables with list of remotes/branches.
Otherwise the setup will be left incomplete, with drm-tip and drm-rerere
being the only branches configured.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>

Commit ea3d1418 ("dim: Use git apply to apply patch") changed dim
behaviour in that it now accepts fixup patches that can create new
files, and those need to be applied using 'git apply' instead of plain
'patch'. Bump dim version to 2 to be able to require this feature.

We'll also need to bump the dim version requirement (DIM_MIN_VERSION) to
2 in nightly.conf to account for the fixups that need this change.

Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Sam Ravnborg <sam@ravnborg.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Acked-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

This allows us to add new files in a fixup, required for unbreaking
drm-tip.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: Sam Ravnborg <sam@ravnborg.org>

We call assert_repo_clean() from a few places and for some of them, it's
not clear PWD would matter, example: dim ub

Clarify what's the working tree dim is complaining about.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>

This is useful for generating drm-tip without pushing, so one can try a
build from drm-tip branch before pushing.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>

If we already have a fixup file for a certain branch, we would override
any previous fixups. Just appending to the previous diff (even if
already containing the same path) seems to work better.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>

If the email comes from a source that encodes the mbox body as base64
for whatever reason, the grep would fail. If it fails, fallback to using
git-mailinfo which will re-encode it.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>

Javier Martinez Canillas authored 3 years ago and Simona Vetter committed 3 years ago

By default the git-format-patch command generates patches with prefixes
for the source and destination (-p1), and is also what git-am uses as a
default. The command strips the first leading path component when patch
is applied (unless a different -p<num> argument is used).

But the patch generating behaviour can be changed with git-format-patch
--no-prefix argument (or setting 'diff.noprefix = true' in .gitconfig).

Patches with no source and destination prefixes can confuse the git-am
3-way merge logic, since stripping the first path component will lead
to wrong paths for newly added files.

To avoid this, check that patches to apply are using git-format-patch's
defaults prefixes to make sure that git-am defaults are safe to use too.

Since there are developers that generate patches without using git, make
this an error but allow to still continue when using the dim -f option.

Signed-off-by: Javier Martinez Canillas <javierm@redhat.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>

Way out of date.

Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Lyude Paul <lyude@redhat.com>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Maxime Ripard <mripard@kernel.org>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>

We should never run the pull with the rebase strategy. Ever.

That messes up with our commits. Specially with the top
one where dim expects a merge commit and it amends from and
link info.

So, let's forbid the rebase strategy overriding any user
config.

Cc: Jani Nikula <jani.nikula@intel.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>

There's nothing we can do, but also I was baffled about what's going on.
The reason was that a topic branch was deleted on the server, but not
removed from nightly.conf. Which means git merge fails, but not in a way
we expect, so we keep falling over until this git commit (which should
always succeed) also falls over and the scrip just stops.

Make the failure a notch more verbose for next time around.

Reviewed-by: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>

Another thing to do having obtained a drm-intel-gt-next co-maintainer hat.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Suggested-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>

This will allow integrating drm-intel-gt-next with linux-next

Signed-off-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>

We have all these reminders to actually type a commit message, but
--no-commit doesn't imply --no-ff, so let's fix that.

Noticed because Maarten managed to do a fast-forward backmerge into
drm-misc-next somehow.

Reviewed-by: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>

It's not exactly trivial to add the smarts to properly check for pushing
backmerges, rebases, topic branches and subtree branches (such as
gvt). For a start, prompt the user with hints about what's going on.

Cc: Daniel Vetter <daniel@ffwll.ch>
Acked-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>

Also try the mail address, just in case that matches and all the name
mangling tests we do don't help.

Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>

Ville Syrjälä authored 4 years ago and Jani Nikula committed 3 years ago

We actually want to use git filter-branch, so make it shut
up and do what it's told.

Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>

Ville Syrjälä authored 4 years ago and Jani Nikula committed 3 years ago

Trying to suck in a Fixes tag doesn't work currently due to the
double quotes contained within. Escape those so the shell stuff
doesn't get confused and we get the expected results.

Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>

Simon Ser authored 4 years ago and Simona Vetter committed 4 years ago

Python 2 has been end-of-life'd, and d79c5784 ("dim: fix handling of 8-bit
non-UTF-8 messages") relies on functions introduced in Python 3.2.

Signed-off-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>

Simon Ser authored 4 years ago and Simona Vetter committed 4 years ago

Python's open() function will return a file object that decodes input
bytes to an UTF-8 string. Python assumes all files are UTF-8 by default
(unless an explicit encoding param is passed).

This works fine with 7-bit and UTF-8 messages. However, when a message
uses a 8-bit Content-Transfer-Encoding and a non-UTF-8 charset (such as
iso-8859-1), Python will error out.

To prevent this, open the file in binary mode to prevent Python from
doing any charset conversion under-the-hood.

Signed-off-by: Simon Ser <contact@emersion.fr>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>

Switching from using drm-intel-next-queued to drm-intel-next requires a
flag-day change. This is the dim change to switch to drm-intel-next. For
now, keep the sub-command names such as "conq" and "push-queued", and
change everything under the hood. Prevent pushes to
drm-intel-next-queued.

The flag-day change should be rolled with:

1) Bump minimum dim version with DIM_MIN_VERSION=1 in nightly.conf

2) Sync drm-intel-next-queued to drm-intel-next

3) Push this change to maintainer-tools

Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Acked-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>

The character set decoding added in commit b66d07db ("dim: decode
email message content charset to unicode") started failing with unicode
decoding failures under certain conditions. (Specifically python 3 and
mboxes downloaded from patchwork.)

Instead of raising UnicodeDecodeErrors, replace values that can't be
converted with U+FFFD (REPLACEMENT CHARACTER, �).

Reported-by: Dave Airlie <airlied@gmail.com>
Cc: Dave Airlie <airlied@gmail.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>

Simon Ser authored 4 years ago and Lucas De Marchi committed 4 years ago

Patchwork adds trailing newlines to long Message-Id header fields.
message_get_id returns a string containing the newline, and
dim_commit_add_tag blows up on that:

    sed: -e expression #1, char 57: unknown command: `<'

Signed-off-by: Simon Ser <contact@emersion.fr>
Closes: #17


Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>

Remove the special casing of drm-intel-next/drm-intel-next-queued
tagging and pull requests. Instead of having a two-step process for
dinq, follow the same pull request process for all branches. As the
first step, use drm-intel-next-queued for pull requests, and ignore
drm-intel-next for now. The change is only relevant for maintainers.

In the future, we'll want to switch from drm-intel-next-queued to
drm-intel-next completely. This will need a flag day switch for all
committers and maintainers.

We also decouple driver date update from dinq pull requests. We may want
to apply the driver date update to drm-intel-gt-next branch instead.

With the change, we also lose the ability to add multiple drm-intel-next
tags, and create a single pull request for them. In practice, we haven't
really used this possibility, as it lacks the merging of the annotated
tag changelogs together, missing information. We may want to resurrect
the feature in an improved form in the future, generalized for all
branches.

Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Acked-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>

Add simple single-digit dim version scheme, and add 'dim version'
subcommand to print it. Start off with version 0.

If $DIM_MIN_VERSION is set, bail out if the requirement is not met. The
idea is to set this in nightly.conf to be able to soft-enforce dim
updates before rolling out changes. (It's obviously possible to bypass
the requirement.)

We do have the dim uptodate nag, but it never bails out.

Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>