Skip to content
Snippets Groups Projects
Forked from drm / msm
Source project has a limited visibility.
  • Jiebin Sun's avatar
    72d1e611
    ipc/msg: mitigate the lock contention with percpu counter · 72d1e611
    Jiebin Sun authored
    The msg_bytes and msg_hdrs atomic counters are frequently updated when IPC
    msg queue is in heavy use, causing heavy cache bounce and overhead. 
    Change them to percpu_counter greatly improve the performance.  Since
    there is one percpu struct per namespace, additional memory cost is
    minimal.  Reading of the count done in msgctl call, which is infrequent. 
    So the need to sum up the counts in each CPU is infrequent.
    
    Apply the patch and test the pts/stress-ng-1.4.0
    -- system v message passing (160 threads).
    
    Score gain: 3.99x
    
    CPU: ICX 8380 x 2 sockets
    Core number: 40 x 2 physical cores
    Benchmark: pts/stress-ng-1.4.0
    -- system v message passing (160 threads)
    
    [akpm@linux-foundation.org: coding-style cleanups]
    [jiebin.sun@intel.com: avoid negative value by overflow in msginfo]
      Link: https://lkml.kernel.org/r/20220920150809.4014944-1-jiebin.sun@intel.com
    [akpm@linux-foundation.org: fix min() warnings]
    Link: https://lkml.kernel.org/r/20220913192538.3023708-3-jiebin.sun@intel.com
    
    
    Signed-off-by: default avatarJiebin Sun <jiebin.sun@intel.com>
    Reviewed-by: default avatarTim Chen <tim.c.chen@linux.intel.com>
    Cc: Alexander Mikhalitsyn <alexander.mikhalitsyn@virtuozzo.com>
    Cc: Alexey Gladkov <legion@kernel.org>
    Cc: Christoph Lameter <cl@linux.com>
    Cc: Davidlohr Bueso <dave@stgolabs.net>
    Cc: Dennis Zhou <dennis@kernel.org>
    Cc: "Eric W . Biederman" <ebiederm@xmission.com>
    Cc: Manfred Spraul <manfred@colorfullife.com>
    Cc: Shakeel Butt <shakeelb@google.com>
    Cc: Tejun Heo <tj@kernel.org>
    Cc: Vasily Averin <vasily.averin@linux.dev>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    72d1e611
    History
    ipc/msg: mitigate the lock contention with percpu counter
    Jiebin Sun authored
    The msg_bytes and msg_hdrs atomic counters are frequently updated when IPC
    msg queue is in heavy use, causing heavy cache bounce and overhead. 
    Change them to percpu_counter greatly improve the performance.  Since
    there is one percpu struct per namespace, additional memory cost is
    minimal.  Reading of the count done in msgctl call, which is infrequent. 
    So the need to sum up the counts in each CPU is infrequent.
    
    Apply the patch and test the pts/stress-ng-1.4.0
    -- system v message passing (160 threads).
    
    Score gain: 3.99x
    
    CPU: ICX 8380 x 2 sockets
    Core number: 40 x 2 physical cores
    Benchmark: pts/stress-ng-1.4.0
    -- system v message passing (160 threads)
    
    [akpm@linux-foundation.org: coding-style cleanups]
    [jiebin.sun@intel.com: avoid negative value by overflow in msginfo]
      Link: https://lkml.kernel.org/r/20220920150809.4014944-1-jiebin.sun@intel.com
    [akpm@linux-foundation.org: fix min() warnings]
    Link: https://lkml.kernel.org/r/20220913192538.3023708-3-jiebin.sun@intel.com
    
    
    Signed-off-by: default avatarJiebin Sun <jiebin.sun@intel.com>
    Reviewed-by: default avatarTim Chen <tim.c.chen@linux.intel.com>
    Cc: Alexander Mikhalitsyn <alexander.mikhalitsyn@virtuozzo.com>
    Cc: Alexey Gladkov <legion@kernel.org>
    Cc: Christoph Lameter <cl@linux.com>
    Cc: Davidlohr Bueso <dave@stgolabs.net>
    Cc: Dennis Zhou <dennis@kernel.org>
    Cc: "Eric W . Biederman" <ebiederm@xmission.com>
    Cc: Manfred Spraul <manfred@colorfullife.com>
    Cc: Shakeel Butt <shakeelb@google.com>
    Cc: Tejun Heo <tj@kernel.org>
    Cc: Vasily Averin <vasily.averin@linux.dev>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>