A previous method that I tried was treating the helped and hurt data
as samples from separate populations, and these were compared using a
T-test.  Since we're applying a common change to "both" sample sets, I
don't think this is a valid analysis.

Instead, I think it is more valid to look at the entire change set as a
sample of a single population and compare the mean of that sample to
zero.  Only the changed samples are examined because the vast majority
of the sample in unaffected.  If the mean of the entire sample was used,
the mean confidence interval would always include zero.  It would be
more valid, I believe, include shaders that were affected but had no
change in instruction or cycle count.  I don't know of a way to
determine this using the existing shader-db infrastructure.

These two different methods communicate two different things.  The first
tries to determine whether the shaders hurt are affected more or less
than the ...