Commit 189b72f2 authored by Arkadiusz Hiler's avatar Arkadiusz Hiler

Don't mangle \n and \r combinations in the generated mboxes

Generated mboxes weren't representing newline (aka LF or \n) and
carriage return (aka CR or \r) found in the original patch faithfully.

This is mostly due to how email.Message is implemented - it tries to
normalize newlines in plain text emails. This doesn't work for us as we
want to provide the patches in an unchanged form.

To do that we have to overwrite as_string() and as_bytes() methods on
our Message class that will use modified Generator that doesn't mangle
newlines.

The difference in the checksums is due to slightly different header
wrapping caused by the changes in generator.

Additionally tests are provided to make sure that we ingest emails
correctly (no mangling there) and that the generated mbox is also
a-okay.

Fixes: #33Signed-off-by: default avatarArkadiusz Hiler <arkadiusz.hiler@intel.com>
parent fdb77357
Pipeline #172701 passed with stage
in 1 minute and 24 seconds
From intel-gfx-trybot-bounces@lists.freedesktop.org Tue Dec 10 15:39:33 2019
X-Original-To: intel-gfx-trybot@lists.freedesktop.org
Delivered-To: intel-gfx-trybot@lists.freedesktop.org
X-Greylist: delayed 425 seconds by postgrey-1.36 at gabe;
Tue, 10 Dec 2019 13:38:59 UTC
From: Petri Latvala <petri.latvala@intel.com>
To: <intel-gfx-trybot@lists.freedesktop.org>
Subject: [PATCH i-g-t 2/2] runner/json_tests: Test handling of unprintable
output from tests
Date: Tue, 10 Dec 2019 15:31:43 +0200
Message-ID: <20191210133143.15645-1-petri.latvala@intel.com>
X-Mailer: git-send-email 2.19.1
X-BeenThere: intel-gfx-trybot@lists.freedesktop.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Intel graphics driver testing & development using CI system
<intel-gfx-trybot.lists.freedesktop.org>
List-Unsubscribe: <https://lists.freedesktop.org/mailman/options/intel-gfx-trybot>,
<mailto:intel-gfx-trybot-request@lists.freedesktop.org?subject=unsubscribe>
List-Archive: <https://lists.freedesktop.org/archives/intel-gfx-trybot>
List-Post: <mailto:intel-gfx-trybot@lists.freedesktop.org>
List-Help: <mailto:intel-gfx-trybot-request@lists.freedesktop.org?subject=help>
List-Subscribe: <https://lists.freedesktop.org/mailman/listinfo/intel-gfx-trybot>,
<mailto:intel-gfx-trybot-request@lists.freedesktop.org?subject=subscribe>
CC: Petri Latvala <petri.latvala@intel.com>
Content-Type: multipart/mixed; boundary="===============0078474997=="
Errors-To: intel-gfx-trybot-bounces@lists.freedesktop.org
Sender: Intel-gfx-trybot <intel-gfx-trybot-bounces@lists.freedesktop.org>
Return-Path: intel-gfx-trybot-bounces@lists.freedesktop.org
MIME-Version: 1.0
X-TUID: pNCZR+SGfuBc
Status: RO
Content-Length: 19731
Tags: inbox
--===============0078474997==
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
A simple test output with numbers from 1 to 255, both in plain text
form and as a single byte with that particular value.
Note that the json spec doesn't require \u-encoding for characters
other than '"', '\' and the range U+0000 to U+001F, the raw
non-\u-encoded UTF-8 in the reference.json file for bytes 128 and up
is what libjson-c outputs for those codepoints and is valid.
The validity of the json file can be verified with iconv, i.e.
$ iconv -f UTF-8 reference.json -o /dev/null && echo it is utf-8
Signed-off-by: Petri Latvala <petri.latvala@intel.com>
---
.../unprintable-characters/0/dmesg.txt | 5 +
.../unprintable-characters/0/err.txt | 258 +++++++++++++++++
.../unprintable-characters/0/journal.txt | 2 +
.../unprintable-characters/0/out.txt | 259 ++++++++++++++++++
.../unprintable-characters/README.txt | 2 +
.../unprintable-characters/endtime.txt | 1 +
.../unprintable-characters/joblist.txt | 1 +
.../unprintable-characters/metadata.txt | 12 +
.../unprintable-characters/reference.json | 72 +++++
.../unprintable-characters/starttime.txt | 1 +
.../unprintable-characters/uname.txt | 1 +
runner/runner_json_tests.c | 1 +
12 files changed, 615 insertions(+)
create mode 100644 runner/json_tests_data/unprintable-characters/0/dmesg.t=
xt
create mode 100644 runner/json_tests_data/unprintable-characters/0/err.txt
create mode 100644 runner/json_tests_data/unprintable-characters/0/journal=
.txt
create mode 100644 runner/json_tests_data/unprintable-characters/0/out.txt
create mode 100644 runner/json_tests_data/unprintable-characters/README.tx=
t
create mode 100644 runner/json_tests_data/unprintable-characters/endtime.t=
xt
create mode 100644 runner/json_tests_data/unprintable-characters/joblist.t=
xt
create mode 100644 runner/json_tests_data/unprintable-characters/metadata.=
txt
create mode 100644 runner/json_tests_data/unprintable-characters/reference=
.json
create mode 100644 runner/json_tests_data/unprintable-characters/starttime=
.txt
create mode 100644 runner/json_tests_data/unprintable-characters/uname.txt
diff --git a/runner/json_tests_data/unprintable-characters/0/dmesg.txt b/ru=
nner/json_tests_data/unprintable-characters/0/dmesg.txt
new file mode 100644
index 00000000..6aac2735
--- /dev/null
+++ b/runner/json_tests_data/unprintable-characters/0/dmesg.txt
@@ -0,0 +1,5 @@
+6,951,3216186095083,-;Console: switching to colour dummy device 80x25
+14,952,3216186095097,-;[IGT] successtest: executing
+14,953,3216186101115,-;[IGT] successtest: starting subtest first-subtest
+14,955,3216186101160,-;[IGT] successtest: exiting, ret=3D0
+6,956,3216186101299,-;Console: switching to colour frame buffer device 240=
x75
diff --git a/runner/json_tests_data/unprintable-characters/0/err.txt b/runn=
er/json_tests_data/unprintable-characters/0/err.txt
new file mode 100644
index 00000000..34d19985
--- /dev/null
+++ b/runner/json_tests_data/unprintable-characters/0/err.txt
@@ -0,0 +1,258 @@
+Starting subtest: first-subtest
+1:=01
+2:=02
+3:=03
+4:=04
+5:=05
+6:=06
+7:=07
+8:=08
+9:=09
+10:
+
+11:=0B
+12:=0C
+13:=0D
+14:=0E
+15:=0F
+16:=10
+17:=11
+18:=12
+19:=13
+20:=14
+21:=15
+22:=16
+23:=17
+24:=18
+25:=19
+26:=1A
+27:=1B
+28:=1C
+29:=1D
+30:=1E
+31:=1F
+32:=20
+33:!
+34:"
+35:#
+36:$
+37:%
+38:&
+39:'
+40:(
+41:)
+42:*
+43:+
+44:,
+45:-
+46:.
+47:/
+48:0
+49:1
+50:2
+51:3
+52:4
+53:5
+54:6
+55:7
+56:8
+57:9
+58::
+59:;
+60:<
+61:=3D
+62:>
+63:?
+64:@
+65:A
+66:B
+67:C
+68:D
+69:E
+70:F
+71:G
+72:H
+73:I
+74:J
+75:K
+76:L
+77:M
+78:N
+79:O
+80:P
+81:Q
+82:R
+83:S
+84:T
+85:U
+86:V
+87:W
+88:X
+89:Y
+90:Z
+91:[
+92:\
+93:]
+94:^
+95:_
+96:`
+97:a
+98:b
+99:c
+100:d
+101:e
+102:f
+103:g
+104:h
+105:i
+106:j
+107:k
+108:l
+109:m
+110:n
+111:o
+112:p
+113:q
+114:r
+115:s
+116:t
+117:u
+118:v
+119:w
+120:x
+121:y
+122:z
+123:{
+124:|
+125:}
+126:~
+127:=7F
+128:=EF=BF=BD
+129:=EF=BF=BD
+130:=EF=BF=BD
+131:=EF=BF=BD
+132:=EF=BF=BD
+133:=EF=BF=BD
+134:=EF=BF=BD
+135:=EF=BF=BD
+136:=EF=BF=BD
+137:=EF=BF=BD
+138:=EF=BF=BD
+139:=EF=BF=BD
+140:=EF=BF=BD
+141:=EF=BF=BD
+142:=EF=BF=BD
+143:=EF=BF=BD
+144:=EF=BF=BD
+145:=EF=BF=BD
+146:=EF=BF=BD
+147:=EF=BF=BD
+148:=EF=BF=BD
+149:=EF=BF=BD
+150:=EF=BF=BD
+151:=EF=BF=BD
+152:=EF=BF=BD
+153:=EF=BF=BD
+154:=EF=BF=BD
+155:=EF=BF=BD
+156:=EF=BF=BD
+157:=EF=BF=BD
+158:=EF=BF=BD
+159:=EF=BF=BD
+160:=EF=BF=BD
+161:=EF=BF=BD
+162:=EF=BF=BD
+163:=EF=BF=BD
+164:=EF=BF=BD
+165:=EF=BF=BD
+166:=EF=BF=BD
+167:=EF=BF=BD
+168:=EF=BF=BD
+169:=EF=BF=BD
+170:=EF=BF=BD
+171:=EF=BF=BD
+172:=EF=BF=BD
+173:=EF=BF=BD
+174:=EF=BF=BD
+175:=EF=BF=BD
+176:=EF=BF=BD
+177:=EF=BF=BD
+178:=EF=BF=BD
+179:=EF=BF=BD
+180:=EF=BF=BD
+181:=EF=BF=BD
+182:=EF=BF=BD
+183:=EF=BF=BD
+184:=EF=BF=BD
+185:=EF=BF=BD
+186:=EF=BF=BD
+187:=EF=BF=BD
+188:=EF=BF=BD
+189:=EF=BF=BD
+190:=EF=BF=BD
+191:=EF=BF=BD
+192:=EF=BF=BD
+193:=EF=BF=BD
+194:=EF=BF=BD
+195:=EF=BF=BD
+196:=EF=BF=BD
+197:=EF=BF=BD
+198:=EF=BF=BD
+199:=EF=BF=BD
+200:=EF=BF=BD
+201:=EF=BF=BD
+202:=EF=BF=BD
+203:=EF=BF=BD
+204:=EF=BF=BD
+205:=EF=BF=BD
+206:=EF=BF=BD
+207:=EF=BF=BD
+208:=EF=BF=BD
+209:=EF=BF=BD
+210:=EF=BF=BD
+211:=EF=BF=BD
+212:=EF=BF=BD
+213:=EF=BF=BD
+214:=EF=BF=BD
+215:=EF=BF=BD
+216:=EF=BF=BD
+217:=EF=BF=BD
+218:=EF=BF=BD
+219:=EF=BF=BD
+220:=EF=BF=BD
+221:=EF=BF=BD
+222:=EF=BF=BD
+223:=EF=BF=BD
+224:=EF=BF=BD
+225:=EF=BF=BD
+226:=EF=BF=BD
+227:=EF=BF=BD
+228:=EF=BF=BD
+229:=EF=BF=BD
+230:=EF=BF=BD
+231:=EF=BF=BD
+232:=EF=BF=BD
+233:=EF=BF=BD
+234:=EF=BF=BD
+235:=EF=BF=BD
+236:=EF=BF=BD
+237:=EF=BF=BD
+238:=EF=BF=BD
+239:=EF=BF=BD
+240:=EF=BF=BD
+241:=EF=BF=BD
+242:=EF=BF=BD
+243:=EF=BF=BD
+244:=EF=BF=BD
+245:=EF=BF=BD
+246:=EF=BF=BD
+247:=EF=BF=BD
+248:=EF=BF=BD
+249:=EF=BF=BD
+250:=EF=BF=BD
+251:=EF=BF=BD
+252:=EF=BF=BD
+253:=EF=BF=BD
+254:=EF=BF=BD
+255:=EF=BF=BD
+Subtest first-subtest: SUCCESS (0.000s)
diff --git a/runner/json_tests_data/unprintable-characters/0/journal.txt b/=
runner/json_tests_data/unprintable-characters/0/journal.txt
new file mode 100644
index 00000000..86a30e07
--- /dev/null
+++ b/runner/json_tests_data/unprintable-characters/0/journal.txt
@@ -0,0 +1,2 @@
+first-subtest
+exit:0 (0.014s)
diff --git a/runner/json_tests_data/unprintable-characters/0/out.txt b/runn=
er/json_tests_data/unprintable-characters/0/out.txt
new file mode 100644
index 00000000..c597674a
--- /dev/null
+++ b/runner/json_tests_data/unprintable-characters/0/out.txt
@@ -0,0 +1,259 @@
+IGT-Version: 1.23-g0c763bfd (x86_64) (Linux: 4.18.0-1-amd64 x86_64)
+Starting subtest: first-subtest
+1:=01
+2:=02
+3:=03
+4:=04
+5:=05
+6:=06
+7:=07
+8:=08
+9:=09
+10:
+
+11:=0B
+12:=0C
+13:=0D
+14:=0E
+15:=0F
+16:=10
+17:=11
+18:=12
+19:=13
+20:=14
+21:=15
+22:=16
+23:=17
+24:=18
+25:=19
+26:=1A
+27:=1B
+28:=1C
+29:=1D
+30:=1E
+31:=1F
+32:=20
+33:!
+34:"
+35:#
+36:$
+37:%
+38:&
+39:'
+40:(
+41:)
+42:*
+43:+
+44:,
+45:-
+46:.
+47:/
+48:0
+49:1
+50:2
+51:3
+52:4
+53:5
+54:6
+55:7
+56:8
+57:9
+58::
+59:;
+60:<
+61:=3D
+62:>
+63:?
+64:@
+65:A
+66:B
+67:C
+68:D
+69:E
+70:F
+71:G
+72:H
+73:I
+74:J
+75:K
+76:L
+77:M
+78:N
+79:O
+80:P
+81:Q
+82:R
+83:S
+84:T
+85:U
+86:V
+87:W
+88:X
+89:Y
+90:Z
+91:[
+92:\
+93:]
+94:^
+95:_
+96:`
+97:a
+98:b
+99:c
+100:d
+101:e
+102:f
+103:g
+104:h
+105:i
+106:j
+107:k
+108:l
+109:m
+110:n
+111:o
+112:p
+113:q
+114:r
+115:s
+116:t
+117:u
+118:v
+119:w
+120:x
+121:y
+122:z
+123:{
+124:|
+125:}
+126:~
+127:=7F
+128:=EF=BF=BD
+129:=EF=BF=BD
+130:=EF=BF=BD
+131:=EF=BF=BD
+132:=EF=BF=BD
+133:=EF=BF=BD
+134:=EF=BF=BD
+135:=EF=BF=BD
+136:=EF=BF=BD
+137:=EF=BF=BD
+138:=EF=BF=BD
+139:=EF=BF=BD
+140:=EF=BF=BD
+141:=EF=BF=BD
+142:=EF=BF=BD
+143:=EF=BF=BD
+144:=EF=BF=BD
+145:=EF=BF=BD
+146:=EF=BF=BD
+147:=EF=BF=BD
+148:=EF=BF=BD
+149:=EF=BF=BD
+150:=EF=BF=BD
+151:=EF=BF=BD
+152:=EF=BF=BD
+153:=EF=BF=BD
+154:=EF=BF=BD
+155:=EF=BF=BD
+156:=EF=BF=BD
+157:=EF=BF=BD
+158:=EF=BF=BD
+159:=EF=BF=BD
+160:=EF=BF=BD
+161:=EF=BF=BD
+162:=EF=BF=BD
+163:=EF=BF=BD
+164:=EF=BF=BD
+165:=EF=BF=BD
+166:=EF=BF=BD
+167:=EF=BF=BD
+168:=EF=BF=BD
+169:=EF=BF=BD
+170:=EF=BF=BD
+171:=EF=BF=BD
+172:=EF=BF=BD
+173:=EF=BF=BD
+174:=EF=BF=BD
+175:=EF=BF=BD
+176:=EF=BF=BD
+177:=EF=BF=BD
+178:=EF=BF=BD
+179:=EF=BF=BD
+180:=EF=BF=BD
+181:=EF=BF=BD
+182:=EF=BF=BD
+183:=EF=BF=BD
+184:=EF=BF=BD
+185:=EF=BF=BD
+186:=EF=BF=BD
+187:=EF=BF=BD
+188:=EF=BF=BD
+189:=EF=BF=BD
+190:=EF=BF=BD
+191:=EF=BF=BD
+192:=EF=BF=BD
+193:=EF=BF=BD
+194:=EF=BF=BD
+195:=EF=BF=BD
+196:=EF=BF=BD
+197:=EF=BF=BD
+198:=EF=BF=BD
+199:=EF=BF=BD
+200:=EF=BF=BD
+201:=EF=BF=BD
+202:=EF=BF=BD
+203:=EF=BF=BD
+204:=EF=BF=BD
+205:=EF=BF=BD
+206:=EF=BF=BD
+207:=EF=BF=BD
+208:=EF=BF=BD
+209:=EF=BF=BD
+210:=EF=BF=BD
+211:=EF=BF=BD
+212:=EF=BF=BD
+213:=EF=BF=BD
+214:=EF=BF=BD
+215:=EF=BF=BD
+216:=EF=BF=BD
+217:=EF=BF=BD
+218:=EF=BF=BD
+219:=EF=BF=BD
+220:=EF=BF=BD
+221:=EF=BF=BD
+222:=EF=BF=BD
+223:=EF=BF=BD
+224:=EF=BF=BD
+225:=EF=BF=BD
+226:=EF=BF=BD
+227:=EF=BF=BD
+228:=EF=BF=BD
+229:=EF=BF=BD
+230:=EF=BF=BD
+231:=EF=BF=BD
+232:=EF=BF=BD
+233:=EF=BF=BD
+234:=EF=BF=BD
+235:=EF=BF=BD
+236:=EF=BF=BD
+237:=EF=BF=BD
+238:=EF=BF=BD
+239:=EF=BF=BD
+240:=EF=BF=BD
+241:=EF=BF=BD
+242:=EF=BF=BD
+243:=EF=BF=BD
+244:=EF=BF=BD
+245:=EF=BF=BD
+246:=EF=BF=BD
+247:=EF=BF=BD
+248:=EF=BF=BD
+249:=EF=BF=BD
+250:=EF=BF=BD
+251:=EF=BF=BD
+252:=EF=BF=BD
+253:=EF=BF=BD
+254:=EF=BF=BD
+255:=EF=BF=BD
+Subtest first-subtest: SUCCESS (0.000s)
diff --git a/runner/json_tests_data/u