Draft: RFC: Convert to SPDX license headers (!20266) · Merge requests · Mesa / mesa

Alyssa Rosenzweig requested to merge alyssa/mesa:treewide/spdx-headers into main Dec 11, 2022

SPDX license headers are concise and machine-readable. This has a number of benefits:

Avoid license proliferation. Within NIR, panfrost, and asahi I found 5 different distinct variants of the MIT license. Running my script on other parts of the tree revealed even more variants but I need to go to bed so not handling here yet without discussion.
Reduces friction to create new files. Ostensibly this is a nice cultural benefit, make it less work to create new files -- pare the copy/paste boilerplate, have something short enough you could type out if you wanted to. And aesthetically it makes new files seem less daunting -- 20 lines of header for 30 lines of code is discouraging, but 2 lines of header for 30 lines of code isn't fundamentally unreasonable for a simple NIR pass. In practice this can have technical effects too -- lowering the barrier to making new files likely means people will split code into more files, which has (hopefully positive) effects on project compile time in the long run
Consistency between files. Across the tree we have at least a half dozen variants of the MIT license text (probably more), plus code that uses SPDX headers instead (Google's Vulkan drivers).. Asahi I've begun switching over manually so you can tell old vs new code based on the headers...
Less for reviewers to scroll through adding files. Minimal actual cognitive burden for reviewers thanks to banner blindness, but the big headers still bloat diffs that add/delete files.
The immense difficulty involved trying to parse the various MIT license texts for SPDX conversion in itself justified the need for SPDX...

So, let's follow the lead of projects like Linux and switch to SPDX license headers. So far I've converted only the parts of the tree that are under my supervision (NIR, panfrost, and asahi), @jekstrand seemed receptive for NIR as well. If people are on board with this, it'd be nice to convert the whole tree. While there is nominally churn, I don't forsee negative effects on backports (unlike e.g. clang-formatting the codebase) so I'm not too worried.

The actual conversion is invoked as

find . \( -name '*.h' -o -name '*.c' -o -name 'meson.build' -o -name '*.py' -o -name '*.cpp' -o -name '*.xml' \) -print0 | parallel -0 python3 ~/m2s/mit-to-spdx.py {} {}

with a custom script to do the conversion (below). Originally I tried to use KDE's licensedigger https://github.com/KDE/licensedigger but I couldn't get it to work for Mesa. As it turns out, the fact that NIR/panfrost/asahi hit 5 different variants of the MIT text makes things hard to do accurately.

The major outstanding issue (a question for the lawyers, I imagine) is what the legal ramifications are for collapsing these 5 variants into a single SPDX line. The variants differ in typography, whether or not they include the "(including the next paragraph)" bit, and who is specified as disclaiming liability ("AUTHORS OR COPYRIGHT HOLDERS" usually, but sometimes other text is used, e.g. "VMWARE AND/OR ITS SUPPLIERS"). I've included all 5 variants in a new "licenses/" directory at the root for this RFC, but ideally I'd get acks from the relevant copyright holders (/ and their lawyers) before we merge any of the common code changes. It's not clear to me which of the 5 the SPDX "MIT" license refers to, seemingly all of them? If even 'trivial' relicensing is needed for this change, this might get sticky ... I own copyright on most of the code in asahi but that's about it, for everything else there are piles of other copyright owners.

At any rate -- I think this is the right direction for Mesa to move in, and I'd like to get the conversation started about how to get there.

mit-to-spdx.py

import sys
import re

fn = sys.argv[1]

def strip_c(line):
    return line.replace(' (C) ', ' ').replace(' © ', ' ')

lines = [strip_c(c) for c in open(fn).read().split('\n')]

def strip_leading_star(line, leading):
    line = line.strip()
    if len(line) > 0 and line[0] == leading.strip():
        line = line[1:].strip()

    return line

# sublicense
# no (including the next paragraph)
# without limitation the rights
# AUTHORS OR COPYRIGHT HOLDERS
# NONINFRINGEMENT
MIT = """
Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the "Software"),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
sell copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
USE OR OTHER DEALINGS IN THE SOFTWARE.
"""

# sub license
# without limitation on the rights
# no (including the next paragraph)
# AUTHOR(S) AND/OR THEIR SUPPLIERS
# NON-INFRINGEMENT
MIT2 = """
Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the "Software"),
to deal in the Software without restriction, including without limitation
on the rights to use, copy, modify, merge, publish, distribute, sub
license, and/or sell copies of the Software, and to permit persons to whom
the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice (including the next
paragraph) shall be included in all copies or substantial portions of the
Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL
THE AUTHOR(S) AND/OR THEIR SUPPLIERS BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
USE OR OTHER DEALINGS IN THE SOFTWARE.
"""

# sublicense
# without limitation the rights
# (including the next paragraph)
# AUTHORS OR COPYRIGHT HOLDERS
# NONINFRINGEMENT
MIT3 = """
Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the "Software"),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice (including the next
paragraph) shall be included in all copies or substantial portions of the
Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
USE OR OTHER DEALINGS IN THE SOFTWARE.
"""

# sub license
# without limitation the rights
# (including the next paragraph)
# VMWARE AND/OR ITS SUPPLIERS
# NON-INFRINGEMENT
#
# nir_vla.h uses this..
MIT4 = """
Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the "Software"),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sub license,
and/or sell copies of the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice (including the next
paragraph) shall be included in all copies or substantial portions of the
Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL
VMWARE AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
USE OR OTHER DEALINGS IN THE SOFTWARE.
"""

# sub license
# without limitation the rights
# (including the next paragraph)
# AUTHORS OR COPYRIGHT HOLDERS
# NON-INFRINGEMENT
#
# From lima
MIT5 = """
Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the "Software"),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sub license,
and/or sell copies of the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice (including the next
paragraph) shall be included in all copies or substantial portions of the
Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
USE OR OTHER DEALINGS IN THE SOFTWARE.
"""

# MIT on oneline
MITs = [MIT, MIT2, MIT3, MIT4, MIT5]

MIT_words = [m.replace('\n', ' ').replace('  ', ' ').strip() for m in MITs]

# Try to match MIT header from start of stripped
def match_mit(candidate):
    tmp = re.sub('\s+', ' ', ' '.join(candidate)).strip()
    for m in MIT_words:
        if tmp[0:len(m)] == m:
            return True
    return False

SPDX = 'SPDX-License-Identifier: MIT'

# Detect MIT lines

def doit(leading):
    stripped = [strip_leading_star(line, leading).replace('  ', ' ').replace('  ', ' ').strip() for line in lines]
    for i, line in enumerate(stripped[0:40]):
        if match_mit(stripped[i:]):
            if i > 1 and len(stripped[i - 1]) == 0:
                i = i - 1

            endj = None
            for j, line2 in enumerate(stripped[i:]):
                if line2.endswith('SOFTWARE.'):
                    endj = j
            assert(endj != None)
            rebuilt_lines = lines[0:i] + [leading + SPDX] + lines[(i + endj + 1):]
            open(fn, "w").write('\n'.join(rebuilt_lines))
            sys.exit(0)

if fn.endswith('xml'):
    if lines[1].startswith('  '):
        # bifrost XML
        doit('  ')
    else:
        # perf XML
        doit('')
else:
    # normal files
    doit('# ' if fn.endswith('build') or fn.endswith('py') else ' * ')
    doit('# ' if not (fn.endswith('build') or fn.endswith('py')) else ' * ')

# No MIT found
sys.exit(1)

Edited Dec 11, 2022 by Yonggang Luo

Draft: RFC: Convert to SPDX license headers

Merge request reports