It doesn't matter too much where we do the lowering so long as we get
some opportunity to optimize the resulting bitcasting operations before
we go into the back-end compiler.  Pushing it late gives us more options
for where we put SSBO deref lowering.

For now, it's hidden behind a cap.  Hopefully, we can eventually drop
that along with all the manual offset code in spirv_to_nir.

The choice of whether or not we should use block_load/store isn't a
choice between external and not so much as a choice between deref
instructions and manually calculated offsets.  In vtn_pointer_from_ssa,
we guard the index+offset case behind vtn_pointer_uses_ssa_offset and
then branch out from there.

Instead of baking in uvec2 for UBO and SSBO pointers and uint for push
constant and shared memory pointers, make it configurable.

Previously, we hard-coded the rule about workgroup variables and the
builder lower_workgroup_access_to_offsets flag.  Instead base it on the
handy helper we have for exactly this sort of thing.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

Variable pointers being well-defined across the block boundary requires
a couple of very specific SPIR-V validation rules.  Normally, we'd trust
the validator to catch these but since CTS tests have been found in the
wild which violate them, we'll carry our own checks.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

This new pass is for lowering explicitly laid out memory coming in from
SPIR-V or a similar source.  It's quite a bit more complicated than the
normal lower_io because we have to be able to handle matrices.  The
way the stride information is stored for matrices is awkward and dealing
with row-major matrices is especially painful.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

This commit adds a new num_components value for intrinsic sources of -1
which means that it consumes everything and the number of components
effectively isn't validated.  This is useful for deref sources which
just take the result of the deref and we leave it up to the driver to
decide what that size should be.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

We added this assert when first moving derefs over to instructions to
ensure that deref chains could go all the way back to the variables.
Now that we're going to start using derefs for things that we can do
variable pointers on such as UBOs and SSBOs, we need to be able to run
derefs through phi nodes, selects, and basically anything else.

We already detect any incomplete deref chains (where the deref is used
for something other than another deref or a load/store) and flag the
variable as used thanks to deref_used_for_not_store.  All that's left to
do is to properly skip casts when cleaning up.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

This pass is used when, for instance, we lazily change the mode of
variables rather than rewriting this.  Since we only do this in cases
where we know we have full deref chains, it's ok to just skip them in
fixup_deref_modes.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

The code which constructs deref paths already gives you the path
starting at the nearest deref_cast or deref_var.  All we need to do for
casts is handle the case where the start of the path isn't a deref_var.
For ptr_as_array derefs, we just bail if we have any after the
divergence point between the two derefs.  We may be able to do better in
the future but this works for now.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

When handling casts, we can't blindly propagate the parent of a cast
into a ptr_as_array deref because doing so might loose the stride
information from the cast.  Instead, before we can propagate into
ptr_as_array derefs, we need to check that the cast is a cast of an
array deref and that the stride matches.  For other types of derefs, we
can continue to propagate casts as normal because they don't need the
stride.  We also add an optimization which can combine a ptr_as_array
deref with it parent if it is also an array deref of some form.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

We also add a ptr_stride field to cast derefs to provide a stride for
ptr_as_array derefs.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

We're going to want to do more deref optimizations going forward and
this gives us a central place to do them.  Also, cast propagation will
get a bit more complicated with the addition of ptr_as_array derefs.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

Instead of just storing the decorations in the vtn_type, propagate them
all the way through to the glsl_type.  For array strides, this means we
need to handle them earlier so we break array stride handling into it's
own function and explicitly call it for both pointer and array types.

Due to type deduplication in the SPIR-V, we may have explicit layout
decorations on all sorts of types that don't actually want them.  In
order to prevent these leaking into unfortunate places in NIR, we
explicitly strip them off before creating NIR variables and when casting
pointers to non-external memory.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

SPIR-V allows for matrix and array types to be decorated with explicit
byte stride decorations and matrix types to be decorated row- or
column-major.  This commit adds support to glsl_type to encode this
information.  Because this doesn't work nicely with std430 and std140
alignments, we add asserts to ensure that we don't use any of the std430
or std140 layout functions with explicitly laid out types.

In SPIR-V, the layout information for matrices is applied to the parent
struct member instead of to the matrix type itself.  However, this is
gets rather clumsy when you're walking derefs trying to compute offsets
because, the moment you hit a matrix, you have to crawl back the deref
chain and find the struct.  Instead, we take the same path here as we've
taken in spirv_to_nir and put the decorations on the matrix type itself.

This also subtly adds support for strided vector types.  These don't
come up in SPIR-V directly but you can get one as the result of taking a
column from a row-major matrix or a row from a column-major matrix.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

This is C++ so we can just poke at the fields of glsl_type if we wish
and calling get_instance is way easier and more reliable than handling
each instance separately.  While we're at it, we re-arrange the base
type labels to match the enum order and add 8-bit type support.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

It was added in bce6f998 even though it's completely redundant with
glsl_array_type().

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

Previously, NIR had a single nir_var_uniform mode used for atomic
counters, UBOs, samplers, images, and normal uniforms.  This commit
splits this into nir_var_uniform and nir_var_ubo where nir_var_uniform
is still a bit of a catch-all but the nir_var_ubo is specific to UBOs.
While we're at it, we also rename shader_storage to ssbo to follow the
convention.

We need this so that we can distinguish between normal uniforms and UBO
access at the deref level without going all the way back variable and
seeing if it has an interface type.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

I have no idea how shader_storage made it into the list of banned
variable modes for stores but it clearly should be allowed.  This only
doesn't cause us a problem today because we never actually use derefs on
shader_storage variables.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

This doesn't currently change anything because array indices are
required to be 32 bits and all derefs are also 32 bits.  However, we
will one day have 64-bit derefs for OpenCL.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

We already had code in link_as_ssa to handle bit sizes; we just need to
use it.  While we're at it we clean up link_as_ssa a bit and add an
explicit bit_size parameter in preparation for a day when we have derefs
that aren't 32 bit.

Cc: mesa-stable@lists.freedesktop.org
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

Cc: mesa-stable@lists.freedesktop.org
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

This simplifies our deref handling by emitting the actual NIR deref
instructions on-the-fly instead of of building up a deref chain and then
emitting them at the last moment.  In order for this to work with the
parts of the compiler that assume they can chase deref chains, we have
to run nir_rematerialize_derefs_in_use_blocks_impl to put the derefs
back in the right places in cases like loops where dominance may not
quite work out the way SPIR-V thinks it does.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

The SPIR-V spec was recently updated to clarify that array indices are
treated as signed integers.

Cc: mesa-stable@lists.freedesktop.org
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

The loop through instructions doesn't set the cursor for us so unless we
set it somewhere, we may end up emitting instructions in the wrong
place.  The only reason why we haven't been bitten by this in the past
is that it only happens in a few variable pointers cases and the CTS
tests for those don't use much control flow so things were getting
emitted in the correct order by accident.

Cc: mesa-stable@lists.freedesktop.org
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

This crops up both in the actual SPIR-V VectorInsert/Extract opcodes as
well as various places where we deal with vector derefs.

Cc: mesa-stable@lists.freedesktop.org
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

Instead of going all the way back to the variable, just look at the
deref.  The modes are guaranteed to be the same by nir_validate whenever
the variable can be found.  This fixes clear_unused_for_modes for
derefs that don't have an accessible variable.

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>

Instead of going all the way back to the variable, just look at the
deref.  The modes are guaranteed to be the same by nir_validate whenever
the variable can be found.  This fixes apply_barrier_for_modes for
derefs that don't have an accessible variable.

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>

This is instead of looking all the way back to the variable which may
not exist for all derefs.  This makes this code properly ignore casts
with modes other than the mode[s] we care about (where casts aren't
allowed).

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>

This is instead of looking all the way back to the variable which may
not exist for all derefs.  This makes this code properly ignore casts
with modes other than the mode[s] we care about (where casts aren't
allowed).

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>

This is instead of looking all the way back to the variable which may
not exist for all derefs.  This makes this code properly ignore casts
with modes other than the mode[s] we care about (where casts aren't
allowed).

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>