While we lack value range tracking, this patch tries to 'manually' propogate
the division by 4 to calculate SSBO element-offset, into a possible previous
shift operation (shift left or right); checking that it is safe to do so.

This should help in cases like ie. when accessing a field in an array of
structs, where the offset is likely defined as base plus a multiplication
by a struct or array element size.

See dEQP test 'dEQP-GLES31.functional.ssbo.atomic.xor.highp_uint'
for an example of a shader that benefits from this.

Reviewed-by: Rob Clark <robdclark@gmail.com>

These intrinsics have the offset in dwords already computed in the last
source, so the change here is basically using that instead of emitting
the ir3_SHR to divide the byte-offset by 4.

The improvement in shader stats is significant, of up to ~15% in
instruction count in some cases. Tested only on a5xx.

shader-db is unfortunately not very useful here because shaders that use
SSBO require GLSL versions that are not supported by freedreno yet.

For examples, most Khronos CTS tests under 'dEQP-GLES31.functional.ssbo.*'
are helped.

A random case:

dEQP-GLES31.functional.ssbo.layout.2_level_array.packed.row_major_mat3x2

with current master:

; CL prog 14/1: 1252 instructions, 0 half, 48 full
; 8 const, 8 constlen
; 61 (ss), 43 (sy)

with the SSBO dword-offset moved to NIR:

; CL prog 14/1: 1053 instructions, 0 half, 45 full
; 7 const, 7 constlen
; 34 (ss), 73 (sy)

The SHR previously emitted for every single SSBO instruction disappears
in most cases, and the dword-offset ends up embedded in the STGB
instruction as immediate in many cases as well.

There are also a few of those tests that are currently failing on register
allocation, that start to pass as a result of reducing the pressure. At least
these, probably more:

dEQP-GLES31.functional.ssbo.layout.random.unsized_arrays.24
dEQP-GLES31.functional.ssbo.layout.random.arrays_of_arrays.6
dEQP-GLES31.functional.ssbo.layout.random.arrays_of_arrays.17
dEQP-GLES31.functional.ssbo.layout.random.nested_structs_arrays.14
dEQP-GLES31.functional.ssbo.layout.random.nested_structs_arrays_instance_arrays.5
dEQP-GLES31.functional.ssbo.layout.random.nested_structs_arrays_instance_arrays.7

No regressions observed with relevant CTS and piglit tests.

Reviewed-by: Rob Clark <robdclark@gmail.com>

This NIR->NIR pass implements offset computations that are currently
done on the IR3 backend compiler, to give NIR a better chance of
optimizing them.

For now, it supports lowering the dword-offset computation for SSBO
instructions. It will take an SSBO intrinsic and replace it with the
new ir3-specific version that adds an extra source. That source will
hold the SSA value resulting from inserting a division by 4 (an SHR op)
of the original byte-offset source already provided by NIR in one of
the intrinsic sources.

Note that on a6xx the original byte-offset is not needed, so we could
potentially replace that source instead of adding a new one. But to
keep things simple and consistent we always add the new source and
a6xx will just ignore the original one.

Reviewed-by: Rob Clark <robdclark@gmail.com>

These are ir3 specific versions of SSBO intrinsics that add an
extra source to hold the element offset (dword), which is what the
backend instructions need.

The original byte-offset source provided by NIR is not replaced
because on a4xx and a5xx the backend still needs it.

Reviewed-by: Rob Clark <robdclark@gmail.com>

Kevin Strasser authored Jan 28, 2019 and Emil Velikov committed Mar 13, 2019

indexConfigAttrib iterates over every index in the dri driver, possibly
exceeding __DRI_ATTRIB_MAX. In other words, if the dri driver has newer
attributes libEGL will end up reading from uninitialized memory through
dri2_to_egl_attribute_map[].

Signed-off-by: Kevin Strasser <kevin.strasser@intel.com>
Cc: mesa-stable@lists.freedesktop.org
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>

Chris Wilson authored Feb 22, 2019 and Kenneth Graunke committed Mar 13, 2019

Always use the streaming load (since we know we have Broadwell+, all of
our target CPU support sse41) for reading back form the tiled surface
for mapping the resource. This means we hit the fast WC handling paths
on Atoms (without LLC), and for big Core (with LLC) using the streaming
load is no less efficient as we do not require the tiled buffer to be
pulled into the CPU cache.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

Chris Wilson authored Feb 22, 2019 and Kenneth Graunke committed Mar 13, 2019

On !llc machines (Atoms), reading from a linear buffers is slow and so
copying from one resource into the linear staging buffer is still slow.
However, we can tell the GPU to snoop the CPU cache when reading from and
writing to the staging buffer eliminating the slow uncached reads.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

Chris Wilson authored Feb 25, 2019 and Kenneth Graunke committed Mar 13, 2019

We prefer fast CPU access to read back the query results.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

Due to lack of write mask in SPIR-V store, generators may produce
multiple stores to the same vector but using different array derefs.
Use the combining store pass to clean this up.  For example,

    layout(binding = 3) buffer block {
        vec4 v;
    };

    void main() {
        v.x = 11;
        v.y = 22;
    }

after going to SPIR-V and NIR, ends up with in two store_derefs to
v[0] and v[1]

    vec2 32 ssa_4 = deref_struct &ssa_3->field0 (ssbo vec4) /* &((block *)ssa_2)->field0 */
    vec2 32 ssa_6 = deref_array &(*ssa_4)[0] (ssbo float) /* &((block *)ssa_2)->field0[0] */
    intrinsic store_deref (ssa_6, ssa_7) (1, 0) /* wrmask=x */ /* access=0 */
    vec1 32 ssa_13 = load_const (0x00000001 /* 0.000000 */)
    vec2 32 ssa_14 = deref_array &(*ssa_4)[1] (ssbo float) /* &((block *)ssa_2)->field0[1] */
    intrinsic store_deref (ssa_14, ssa_15) (1, 0) /* wrmask=x */ /* access=0 */

producing two different sends instructions in skl.  The combining pass
transform the snippet above into

    vec2 32 ssa_4 = deref_struct &ssa_3->field0 (ssbo vec4) /* &((block *)ssa_2)->field0 */
    vec4 32 ssa_18 = vec4 ssa_7, ssa_15, ssa_16, ssa_17
    intrinsic store_deref (ssa_4, ssa_18) (3, 0) /* wrmask=xy */ /* access=0 */

producing a single sends instruction.

v2: Move this from spirv_to_nir into the general optimization pass for
    intel compiler.  (Jason)

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

Shader-db results for skl:

    total instructions in shared programs: 15232903 -> 15224781 (-0.05%)
    instructions in affected programs: 61246 -> 53124 (-13.26%)
    helped: 221
    HURT: 0

    total cycles in shared programs: 371440470 -> 371398018 (-0.01%)
    cycles in affected programs: 281363 -> 238911 (-15.09%)
    helped: 221
    HURT: 0

Results for bdw are very similar.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

v2: (all from Jason)
    Reuse existing function for the end of the block combinations.
    Check the SSA values are coming from the right place in tests.
    Document the case when the store to array_deref is reused.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

vindex is always 0.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>

The struct version sets IDXEN=1, while the raw version sets IDXEN=0.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>

This drastically reduces the number of SGPRs because the driver
now uses descriptors per vertex binding, instead of per vertex
attribute format.

29077 shaders in 15096 tests
Totals:
SGPRS: 1354285 -> 1282109 (-5.33 %)
VGPRS: 909896 -> 908800 (-0.12 %)
Spilled SGPRs: 24840 -> 24811 (-0.12 %)
Code Size: 49221144 -> 48986628 (-0.48 %) bytes
Max Waves: 243930 -> 244229 (0.12 %)

Totals from affected shaders:
SGPRS: 390648 -> 318472 (-18.48 %)
VGPRS: 288432 -> 287336 (-0.38 %)
Spilled SGPRs: 94 -> 65 (-30.85 %)
Code Size: 11548412 -> 11313896 (-2.03 %) bytes
Max Waves: 86460 -> 86759 (0.35 %)

This gives a really tiny boost.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>

They are required for using typed buffer loads.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>

Be more generic, this will be used by an upcoming series.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>

So we can free it later.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

So we can unmap it later.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

We were not copying the saturate bit from the original instruction
to the new replacement instruction.  This caused major misrendering
in DiRT Rally on iris, where comparisons leading to discards failed
due to the missing saturate, causing lots of extra garbage pixels to
be drawn in text rendering, trees, and so on.

This did not show up on i965 because st/nir performs a more aggressive
version of nir_opt_peephole_select, yielding more b32csel operations.

Fixes: 52c7df16 i965/fs: Merge CMP and SEL into CSEL on Gen8+

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>

Tessellation control shader outputs act as if they have memory backing
them and you can have multiple writes to different components of the
same vector in-flight at the same time.  When this happens, the load vec
store pattern that gets used by ir_triop_vector_insert doesn't yield the
correct results.  Instead, just emit a sequence of conditional
assignments.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Cc: mesa-stable@lists.freedesktop.org

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>

The previous code was completely broken when it came to constructing the
undef values.  I'm not sure how it ever worked.  For the case of a copy
that reads an undefined value, we can just delete the copy because the
destination is a valid undefined value.  This saves us the effort of
trying to construct a value for an arbitrary copy_deref intrinsic.

Fixes: e8a8937a "nir: add partial loop unrolling support"
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>

We don't care about the information but there's no sense in throwing a
debug warning about it.  It's harmless but annoying to users.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=109984


Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>

This makes v3d match vc4's destroy path.

Fixes: e113b21c ("v3d: Add renderonly support.")

Noticed while trying to get a CTS run again.

Fixes: 33886474 ("v3d: Use the DAG datastructure for QPU instruction scheduling.")

Grigori Goronzy authored Aug 03, 2017 and Adam Jackson committed Mar 12, 2019

v2: Only reject no-error contexts for too-old GL if we're actually
trying to create a no-error context (Adam Jackson)
v3: Fix share contexts (Adam Jackson)

Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Eric Anholt <eric@anholt.net>

This fixes random SteamVR corruption, see
https://github.com/ValveSoftware/SteamVR-for-Linux/issues/181



Fixes: 4d30f2c6 ("radv/winsys: remove the max IBs per submit limit for the fallback path")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>

Danylo Piliaiev authored Mar 12, 2019 and Lionel Landwerlin committed Mar 12, 2019

pool->next and pool->free_list were reset before their usage in
anv_descriptor_pool_free_set

Fixes: 775aabdd "anv: destroy descriptor sets when pool gets reset"
Signed-off-by: Danylo Piliaiev <danylo.piliaiev@globallogic.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>

This reduces the runtime of dEQP-GLES3.functional.shaders.precision.* from
11.5s to 3.3s.  This brings CTS runs down to 4 hours on one of my target
devices.

The IO scalarization pass that we run to help with linking end up
turning some shader I/O such as that for tessellation and geometry
shaders into many scalar URB operations rather than one vector one.  To
alleviate this, we now vectorize the I/O once again.  This fixes a 10%
performance regression in the GfxBench tessellation test that was caused
by scalarizing.

Shader-db results on Kaby Lake:

    total instructions in shared programs: 15224023 -> 15220871 (-0.02%)
    instructions in affected programs: 342009 -> 338857 (-0.92%)
    helped: 1236
    HURT: 443

    total spills in shared programs: 23471 -> 23465 (-0.03%)
    spills in affected programs: 6 -> 0
    helped: 1
    HURT: 0

    total fills in shared programs: 31770 -> 31766 (-0.01%)
    fills in affected programs: 4 -> 0
    helped: 1
    HURT: 0

Cycles was just a lot of churn do to moves being different places.  Most
of the pure churn in instructions was +/- one or two instructions in
fragment shaders.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=107510


Fixes: 4434591b "intel/nir: Call nir_lower_io_to_scalar_early"
Fixes: 8d822246 "intel/nir: Enable nir_opt_find_array_copies"
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>

This pass tries to turn scalar and array-of-scalar IO variables into
vector IO variables whenever possible.

Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Cc: "19.0" <mesa-stable@lists.freedesktop.org>

Rhys Perry authored Dec 06, 2018 and Samuel Pitoiset committed Mar 12, 2019

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>

Prodea Alexandru-Liviu authored Mar 07, 2019 and Jose Fonseca committed Mar 12, 2019

This ensures Mesa3D build doesn't fail in this case as encountered when
bisecting Scons source code while regression testing
https://bugs.freedesktop.org/show_bug.cgi?id=109443


and when testing 3.0.5.a.2

Technical details:
Scons version string has consistently been in this format:
MajorVersion.MinorVersion.Patch[.alpha/beta.yyyymmdd]
so these formulas should strip alpha/beta flags and return Scons version:

- as string - `'.'.join(SCons.__version__.split('.')[:3])`
- as tuple of integers - `tuple(map(int, SCons.__version__.split('.')[:3]))`

- v2: Fixed Scons version retrieval formulas as string and tuple of integers.
- v3: Fixed Scons version string format description.

Cc: "19.0" <mesa-stable@lists.freedesktop.org>
Reviewed-by: Jose Fonseca <jfonseca@vmware.com>

This reverts commit 47fc3598.

Reason is that patch did not take in to account situation where we might
have both OpenGL and Vulkan using glsl_types at the same time.

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>

This reduces compilation time for my shader-db collection from around 40
seconds to 30, vs. 19 seconds for TGSI. There are still some shaders
that TGSI caches but NIR doesn't, partly because of more aggressive
cross-stage optimizations with NIR.

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>

Oftentimes various nir shaders after lowering will be the same, or
almost the same. For example, this can happen when the same shader is
linked with different shaders to form different pipelines and
cross-stage optimizations don't kick in to change it. We want to avoid
running the backend twice on these shaders. We were already doing this
with radeonsi, but we were storing a few extra pieces of information
that made this much less effective compared to TGSI. The worse offender
by far was the program name, which caused most of the cache misses. This
pass strips out these pieces of information, controlled by the NIR_STRIP
debug env variable.

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

The values should match the ones that are emitted.

This fixes new CTS dEQP-VK.rasterization.primitive_size.points.*.

Fixes: f4e499ec ("radv: add initial non-conformant radv vulkan driver")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>