buffer overflow in nouveau driver on mesa 20.0.2
Firefox's probabilistic heap checker (see https://bugzilla.mozilla.org/show_bug.cgi?id=1523268) is reporting crashes in the nouveau driver in the wild, e.g. https://crash-stats.mozilla.com/report/index/f1db01f8-55fd-4487-ab73-2d29f0200330
The crash report points at a buffer overflow.
Top frames of crashing thread:
0 libgallium_dri.so nv50_validate_tic ../src/gallium/drivers/nouveau/nv50/nv50_tex.c:324
1 libgallium_dri.so nv50_validate_textures ../src/gallium/drivers/nouveau/nv50/nv50_tex.c:339
2 libgallium_dri.so nv50_state_validate ../src/gallium/drivers/nouveau/nv50/nv50_state_validate.c:549
3 libgallium_dri.so nv50_state_validate_3d ../src/gallium/drivers/nouveau/nv50/nv50_state_validate.c:572
4 libgallium_dri.so nv50_draw_vbo ../src/gallium/drivers/nouveau/nv50/nv50_vbo.c:799
5 libgallium_dri.so cso_draw_arrays ../src/gallium/auxiliary/cso_cache/cso_context.c:1756
6 libgallium_dri.so st_pbo_draw ../src/mesa/state_tracker/st_pbo.c:282
7 libgallium_dri.so try_pbo_upload_common ../src/mesa/state_tracker/st_cb_texture.c:1322
8 libgallium_dri.so st_TexSubImage ../src/mesa/state_tracker/st_cb_texture.c:1561
9 libgallium_dri.so <name omitted> ../src/mesa/main/teximage.c:3584
10 libgallium_dri.so <name omitted> ../src/mesa/main/teximage.c:3604
11 libgallium_dri.so <name omitted> ../src/mesa/main/teximage.c:3864
12 libxul.so <gleam::gl::GlesFns as gleam::gl::Gl>::tex_sub_image_2d_pbo hg:hg.mozilla.org/mozilla-central:third_party/rust/gleam/src/gles_fns.rs:a7625a1bcac7e2b793e89fa78b16638219d193a3:754
13 libxul.so webrender::device::gl::UploadTarget::update_impl hg:hg.mozilla.org/mozilla-central:gfx/wr/webrender/src/device/gl.rs:a7625a1bcac7e2b793e89fa78b16638219d193a3:4016
14 libxul.so webrender::device::gl::PixelBuffer::flush_chunks hg:hg.mozilla.org/mozilla-central:gfx/wr/webrender/src/device/gl.rs:a7625a1bcac7e2b793e89fa78b16638219d193a3:3795
15 libxul.so webrender::renderer::Renderer::update_gpu_cache hg:hg.mozilla.org/mozilla-central:gfx/wr/webrender/src/device/gl.rs:a7625a1bcac7e2b793e89fa78b16638219d193a3:3829
16 libxul.so webrender::renderer::Renderer::render_impl hg:hg.mozilla.org/mozilla-central:gfx/wr/webrender/src/renderer.rs:a7625a1bcac7e2b793e89fa78b16638219d193a3:3562
17 libxul.so webrender::renderer::Renderer::render hg:hg.mozilla.org/mozilla-central:gfx/wr/webrender/src/renderer.rs:a7625a1bcac7e2b793e89fa78b16638219d193a3:3048
18 libxul.so wr_renderer_render hg:hg.mozilla.org/mozilla-central:gfx/webrender_bindings/src/bindings.rs:a7625a1bcac7e2b793e89fa78b16638219d193a3:603
19 libxul.so mozilla::wr::RendererOGL::UpdateAndRender(mozilla::Maybe<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits> > const&, mozilla::Maybe<mozilla::wr::ImageFormat> const&, mozilla::Maybe<mozilla::Range<unsigned char> > const&, bool, mozilla::wr::RendererStats*) hg:hg.mozilla.org/mozilla-central:gfx/webrender_bindings/RendererOGL.cpp:a7625a1bcac7e2b793e89fa78b16638219d193a3:154
Allocation stack from PHC:
0 firefox-bin replace_calloc(unsigned long, unsigned long)+0xe4
1 libgallium_dri.so nouveau_buffer_create+0x31 ../src/gallium/drivers/nouveau/nouveau_buffer.c:644
2 libgallium_dri.so st_bufferobj_data+0x27a ../src/mesa/main/bufferobj.c:2087
3 libgallium_dri.so <name omitted>+0xbe ../src/mesa/main/bufferobj.c:2087
4 libgallium_dri.so <name omitted>+0xf3 ../src/mesa/main/bufferobj.c:2130
5 libxul.so webrender::device::gl::Device::create_upload_buffer+0x4f
6 libxul.so webrender::renderer::Renderer::update_gpu_cache+0x3bd7
7 libxul.so webrender::renderer::Renderer::render_impl+0x69d7
8 libxul.so webrender::renderer::Renderer::render+0x46
9 libxul.so wr_renderer_render+0x68
10 libxul.so mozilla::wr::RendererOGL::UpdateAndRender(mozilla::Maybe<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits> > const&, mozilla::Maybe<mozilla::wr::ImageFormat> const&, mozilla::Maybe<mozilla::Range<unsigned char> > const&, bool, mozilla::wr::RendererStats*)+0x163
11 libxul.so mozilla::wr::RenderThread::UpdateAndRender(mozilla::wr::WrWindowId, mozilla::layers::BaseTransactionId<mozilla::VsyncIdType> const&, mozilla::TimeStamp const&, bool, mozilla::Maybe<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits> > const&, mozilla::Maybe<mozilla::wr::ImageFormat> const&, mozilla::Maybe<mozilla::Range<unsigned char> > const&, bool)+0x1fa
12 libxul.so mozilla::wr::RenderThread::HandleFrameOneDoc(mozilla::wr::WrWindowId, bool)+0x227
13 libxul.so mozilla::detail::RunnableMethodImpl<mozilla::wr::RenderThread*, void (mozilla::wr::RenderThread::*)(mozilla::wr::WrWindowId, bool), true, (mozilla::RunnableKind)0, mozilla::wr::WrWindowId, bool>::Run()+0x2f
14 libxul.so base::MessagePumpDefault::Run(base::MessagePump::Delegate*)+0x674
15 libxul.so MessageLoop::Run()+0x4f