glx: Be more paranoid about variable-length requests [CVE-2014-8093 1/6]

If the size computation routine returns -1 we should just reject the
request outright.  Clamping it to zero could give an attacker the
opportunity to also mangle cmdlen in such a way that the subsequent
length check passes, and the request would get executed, thus passing
data we wanted to reject to the renderer.
Reviewed-by: Keith Packard <keithp@keithp.com>
Reviewed-by: Julien Cristau <jcristau@debian.org>
Reviewed-by: Michal Srb <msrb@suse.com>
Reviewed-by: Andy Ritger <aritger@nvidia.com>
Signed-off-by: Adam Jackson <ajax@redhat.com>
Signed-off-by: Alan Coopersmith <alan.coopersmith@oracle.com>

glx/glxcmds.c

@@ -2062,7 +2062,7 @@ __glXDisp_Render(__GLXclientState * cl, GLbyte * pc)
             extra = (*entry.varsize) (pc + __GLX_RENDER_HDR_SIZE,
                                       client->swapped);
             if (extra < 0) {
-                extra = 0;
+                return BadLength;
             }
             if (cmdlen != __GLX_PAD(entry.bytes + extra)) {
                 return BadLength;
@@ -2179,7 +2179,7 @@ __glXDisp_RenderLarge(__GLXclientState * cl, GLbyte * pc)
             extra = (*entry.varsize) (pc + __GLX_RENDER_LARGE_HDR_SIZE,
                                       client->swapped);
             if (extra < 0) {
-                extra = 0;
+                return BadLength;
             }
             /* large command's header is 4 bytes longer, so add 4 */
             if (cmdlen != __GLX_PAD(entry.bytes + 4 + extra)) {