Skip to content

x86: Fix unchecked misaligned array access

amyspark requested to merge amyspark/orc:fix-avx2-alignment-crash into main

Hi all,

This PR is to address a possible Orc crash when accessing a misaligned array parameter. Currently, Orc hard assumes that all supplied arrays are naturally aligned. This may not be so, e.g. because one has supplied .dest 4 x align 2 int16_t (intentionally misaligning the parameter) or because the supplied array is itself misaligned. At least on Linux, it is possible to make Orc crash by supplying an array parameter that matches any of the two above.

The fix for this is twofold:

  • if an array can be supplied misaligned (e.g. .dest 4 x align 2 int16_t, or .dest x int16_t) convert all accesses to unaligned
  • if the array parameter has fixed its alignment (.dest 4 x align 4 int16_t), but the region split detects otherwise, bail out to scalar iteration

Additionally, while testing #72 (comment 2515202), I found an Orc crash with MSYS in the preamble:

Thread 1 "orctest2" received signal SIGSEGV, Segmentation fault.
0x00006ffffffd0005 in ?? ()
(gdb) bt
#0  0x00006ffffffd0005 in ?? ()
#1  0x0000000a0001f150 in ?? ()
#2  0x00000001004011fa in pa_volume_s16ne_orc_2ch (d1=d1@entry=0x7ffffbbf0, p1=p1@entry=0,
    n=n@entry=1024) at orctest2.exe.p/svolume.orc.c:356
#3  0x000000010047a7c1 in main () at ../orctest.c:17
(gdb) disas 0x00006ffffffd0005,+100
Dump of assembler code from 0x6ffffffd0005 to 0x6ffffffd0069:
=> 0x00006ffffffd0005:  vmovd  0x278(%rdi),%xmm0
   0x00006ffffffd000d:  vpinsrd $0x1,0x298(%rdi),%xmm0,%xmm0
   0x00006ffffffd0017:  vpbroadcastq %xmm0,%ymm0
   0x00006ffffffd001c:  vpxor  %ymm1,%ymm1,%ymm1

That preamble is way too short (it doesn't store any registers on the stack?) so I investigated, and realised that Orc doesn't consider at any step that Cygwin/MSYS2 is as a Win32-like platform for ABI purposes. This PR fixes that too.

Fixes #72 (closed)

Edited by amyspark

Merge request reports