In gst-msdk, a mfx session may be shared between different gst elements, each element tries to set the frame allocator. However, per the MSDK documation1, the behavior is undefined if reset the frame allocator while the previous allocator is in use. Fortunately all elements use the same frame allocator, so we can avoid to call MFXVideoCORE_SetFrameAllocator again.