1. 09 Oct, 2008 13 commits
    • Tejun Heo's avatar
      block: move stats from disk to part0 · 074a7aca
      Tejun Heo authored
      Move stats related fields - stamp, in_flight, dkstats - from disk to
      part0 and unify stat handling such that...
      * part_stat_*() now updates part0 together if the specified partition
        is not part0.  ie. part_stat_*() are now essentially all_stat_*().
      * {disk|all}_stat_*() are gone.
      * part_round_stats() is updated similary.  It handles part0 stats
        automatically and disk_round_stats() is killed.
      * part_{inc|dec}_in_fligh() is implemented which automatically updates
        part0 stats for parts other than part0.
      * disk_map_sector_rcu() is updated to return part0 if no part matches.
        Combined with the above changes, this makes NULL special case
        handling in callers unnecessary.
      * Separate stats show code paths for disk are collapsed into part
        stats show code paths.
      * Rename disk_stat_lock/unlock() to part_stat_lock/unlock()
      While at it, reposition stat handling macros a bit and add missing
      parentheses around macro parameters.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Signed-off-by: default avatarJens Axboe <jens.axboe@oracle.com>
    • Tejun Heo's avatar
      block: kill GENHD_FL_FAIL and use part0->make_it_fail · eddb2e26
      Tejun Heo authored
      GENHD_FL_FAIL for disk is what make_it_fail is for parts.  Kill it and
      use part0->make_it_fail.  Sysfs node handling is unified too.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Signed-off-by: default avatarJens Axboe <jens.axboe@oracle.com>
    • Tejun Heo's avatar
      block: always set bdev->bd_part · 0762b8bd
      Tejun Heo authored
      Till now, bdev->bd_part is set only if the bdev was for parts other
      than part0.  This patch makes bdev->bd_part always set so that code
      paths don't have to differenciate common handling.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Signed-off-by: default avatarJens Axboe <jens.axboe@oracle.com>
    • Tejun Heo's avatar
      block: fix diskstats access · c9959059
      Tejun Heo authored
      There are two variants of stat functions - ones prefixed with double
      underbars which don't care about preemption and ones without which
      disable preemption before manipulating per-cpu counters.  It's unclear
      whether the underbarred ones assume that preemtion is disabled on
      entry as some callers don't do that.
      This patch unifies diskstats access by implementing disk_stat_lock()
      and disk_stat_unlock() which take care of both RCU (for partition
      access) and preemption (for per-cpu counter access).  diskstats access
      should always be enclosed between the two functions.  As such, there's
      no need for the versions which disables preemption.  They're removed
      and double underbars ones are renamed to drop the underbars.  As an
      extra argument is added, there's no danger of using the old version
      disk_stat_lock() uses get_cpu() and returns the cpu index and all
      diskstat functions which access per-cpu counters now has @cpu
      argument to help RT.
      This change adds RCU or preemption operations at some places but also
      collapses several preemption ops into one at others.  Overall, the
      performance difference should be negligible as all involved ops are
      very lightweight per-cpu ones.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Signed-off-by: default avatarJens Axboe <jens.axboe@oracle.com>
    • Tejun Heo's avatar
      block: fix disk->part[] dereferencing race · e71bf0d0
      Tejun Heo authored
      disk->part[] is protected by its matching bdev's lock.  However,
      non-critical accesses like collecting stats and printing out sysfs and
      proc information used to be performed without any locking.  As
      partitions can come and go dynamically, partitions can go away
      underneath those non-critical accesses.  As some of those accesses are
      writes, this theoretically can lead to silent corruption.
      This patch fixes the race by using RCU for the partition array and dev
      reference counter to hold partitions.
      * Rename disk->part[] to disk->__part[] to make sure no one outside
        genhd layer proper accesses it directly.
      * Use RCU for disk->__part[] dereferencing.
      * Implement disk_{get|put}_part() which can be used to get and put
        partitions from gendisk respectively.
      * Iterators are implemented to help iterate through all partitions
      * Functions which require RCU readlock are marked with _rcu suffix.
      * Use disk_put_part() in __blkdev_put() instead of directly putting
        the contained kobject.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Signed-off-by: default avatarJens Axboe <jens.axboe@oracle.com>
    • Tejun Heo's avatar
      block: misc updates · 310a2c10
      Tejun Heo authored
      This patch makes the following misc updates in preparation for
      disk->part dereference fix and extended block devt support.
      * implment part_to_disk()
      * fix comment about gendisk->part indexing
      * rename get_part() to disk_map_sector()
      * don't use n which is always zero while printing disk information in
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Signed-off-by: default avatarJens Axboe <jens.axboe@oracle.com>
    • Randy Dunlap's avatar
      Add some block/ source files to the kernel-api docbook. Fix kernel-doc... · 710027a4
      Randy Dunlap authored
      Add some block/ source files to the kernel-api docbook. Fix kernel-doc notation in them as needed. Fix changed function parameter names. Fix typos/spellos. In comments, change REQ_SPECIAL to REQ_TYPE_SPECIAL and REQ_BLOCK_PC to REQ_TYPE_BLOCK_PC.
      Signed-off-by: default avatarRandy Dunlap <randy.dunlap@oracle.com>
      Signed-off-by: default avatarJens Axboe <jens.axboe@oracle.com>
    • Mikulas Patocka's avatar
      drop vmerge accounting · 5df97b91
      Mikulas Patocka authored
      Remove hw_segments field from struct bio and struct request. Without virtual
      merge accounting they have no purpose.
      Signed-off-by: default avatarMikulas Patocka <mpatocka@redhat.com>
      Signed-off-by: default avatarJens Axboe <jens.axboe@oracle.com>
    • David Woodhouse's avatar
      Allow elevators to sort/merge discard requests · e17fc0a1
      David Woodhouse authored
      But blkdev_issue_discard() still emits requests which are interpreted as
      soft barriers, because naïve callers might otherwise issue subsequent
      writes to those same sectors, which might cross on the queue (if they're
      reallocated quickly enough).
      Callers still _can_ issue non-barrier discard requests, but they have to
      take care of queue ordering for themselves.
      Signed-off-by: default avatarDavid Woodhouse <David.Woodhouse@intel.com>
      Signed-off-by: default avatarJens Axboe <jens.axboe@oracle.com>
    • David Woodhouse's avatar
      Add 'discard' request handling · fb2dce86
      David Woodhouse authored
      Some block devices benefit from a hint that they can forget the contents
      of certain sectors. Add basic support for this to the block core, along
      with a 'blkdev_issue_discard()' helper function which issues such
      The caller doesn't get to provide an end_io functio, since
      blkdev_issue_discard() will automatically split the request up into
      multiple bios if appropriate. Neither does the function wait for
      completion -- it's expected that callers won't care about when, or even
      _if_, the request completes. It's only a hint to the device anyway. By
      definition, the file system doesn't _care_ about these sectors any more.
      [With feedback from OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> and
      Jens Axboe <jens.axboe@oracle.com]
      Signed-off-by: default avatarDavid Woodhouse <David.Woodhouse@intel.com>
      Signed-off-by: default avatarJens Axboe <jens.axboe@oracle.com>
    • David Woodhouse's avatar
    • Jens Axboe's avatar
    • Jens Axboe's avatar
  2. 27 Aug, 2008 1 commit
    • FUJITA Tomonori's avatar
      block: move cmdfilter from gendisk to request_queue · abf54393
      FUJITA Tomonori authored
      cmd_filter works only for the block layer SG_IO with SCSI block
      devices. It breaks scsi/sg.c, bsg, and the block layer SG_IO with SCSI
      character devices (such as st). We hit a kernel crash with them.
      The problem is that cmd_filter code accesses to gendisk (having struct
      blk_scsi_cmd_filter) via inode->i_bdev->bd_disk. It works for only
      SCSI block device files. With character device files, inode->i_bdev
      leads you to struct cdev. inode->i_bdev->bd_disk->blk_scsi_cmd_filter
      isn't safe.
      SCSI ULDs don't expose gendisk; they keep it private. bsg needs to be
      independent on any protocols. We shouldn't change ULDs to expose their
      This patch moves struct blk_scsi_cmd_filter from gendisk to
      request_queue, a common object, which eveyone can access to.
      The user interface doesn't change; users can change the filters via
      /sys/block/. gendisk has a pointer to request_queue so the cmd_filter
      code accesses to struct blk_scsi_cmd_filter.
      Signed-off-by: default avatarFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
      Signed-off-by: default avatarJens Axboe <jens.axboe@oracle.com>
  3. 01 Aug, 2008 1 commit
  4. 15 Jul, 2008 1 commit
  5. 03 Jul, 2008 2 commits
  6. 28 May, 2008 1 commit
  7. 25 May, 2008 1 commit
    • Carlos R. Mafra's avatar
      Remove argument from open_softirq which is always NULL · 962cf36c
      Carlos R. Mafra authored
      As git-grep shows, open_softirq() is always called with the last argument
      being NULL
      block/blk-core.c:       open_softirq(BLOCK_SOFTIRQ, blk_done_softirq, NULL);
      kernel/hrtimer.c:       open_softirq(HRTIMER_SOFTIRQ, run_hrtimer_softirq, NULL);
      kernel/rcuclassic.c:    open_softirq(RCU_SOFTIRQ, rcu_process_callbacks, NULL);
      kernel/rcupreempt.c:    open_softirq(RCU_SOFTIRQ, rcu_process_callbacks, NULL);
      kernel/sched.c: open_softirq(SCHED_SOFTIRQ, run_rebalance_domains, NULL);
      kernel/softirq.c:       open_softirq(TASKLET_SOFTIRQ, tasklet_action, NULL);
      kernel/softirq.c:       open_softirq(HI_SOFTIRQ, tasklet_hi_action, NULL);
      kernel/timer.c: open_softirq(TIMER_SOFTIRQ, run_timer_softirq, NULL);
      net/core/dev.c: open_softirq(NET_TX_SOFTIRQ, net_tx_action, NULL);
      net/core/dev.c: open_softirq(NET_RX_SOFTIRQ, net_rx_action, NULL);
      This observation has already been made by Matthew Wilcox in June 2002
      "I notice that none of the current softirq routines use the data element
      passed to them."
      and the situation hasn't changed since them. So it appears we can safely
      remove that extra argument to save 128 (54) bytes of kernel data (text).
      Signed-off-by: default avatarCarlos R. Mafra <crmafra@ift.unesp.br>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
  8. 15 May, 2008 1 commit
    • Neil Brown's avatar
      Remove blkdev warning triggered by using md · e7e72bf6
      Neil Brown authored
      As setting and clearing queue flags now requires that we hold a spinlock
      on the queue, and as blk_queue_stack_limits is called without that lock,
      get the lock inside blk_queue_stack_limits.
      For blk_queue_stack_limits to be able to find the right lock, each md
      personality needs to set q->queue_lock to point to the appropriate lock.
      Those personalities which didn't previously use a spin_lock, us
      q->__queue_lock.  So always initialise that lock when allocated.
      With this in place, setting/clearing of the QUEUE_FLAG_PLUGGED bit will no
      longer cause warnings as it will be clear that the proper lock is held.
      Thanks to Dan Williams for review and fixing the silly bugs.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      Cc: Dan Williams <dan.j.williams@intel.com>
      Cc: Jens Axboe <jens.axboe@oracle.com>
      Cc: Alistair John Strachan <alistair@devzero.co.uk>
      Cc: Nick Piggin <npiggin@suse.de>
      Cc: "Rafael J. Wysocki" <rjw@sisk.pl>
      Cc: Jacek Luczak <difrost.kernel@gmail.com>
      Cc: Prakash Punnoor <prakash@punnoor.de>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
  9. 07 May, 2008 2 commits
    • Jens Axboe's avatar
      block: avoid duplicate calls to get_part() in disk stat code · 28f13702
      Jens Axboe authored
      get_part() is fairly expensive, as it O(N) loops over partitions
      to find the right one. In lots of normal IO paths we end up looking
      up the partition twice, to make matters even worse. Change the
      stat add code to accept a passed in partition instead.
      Signed-off-by: default avatarJens Axboe <jens.axboe@oracle.com>
    • Jens Axboe's avatar
      block: optimize generic_unplug_device() · dbaf2c00
      Jens Axboe authored
      Original patch from Mikulas Patocka <mpatocka@redhat.com>
      Mike Anderson was doing an OLTP benchmark on a computer with 48 physical
      disks mapped to one logical device via device mapper.
      He found that there was a slowdown on request_queue->lock in function
      generic_unplug_device. The slowdown is caused by the fact that when some
      code calls unplug on the device mapper, device mapper calls unplug on all
      physical disks. These unplug calls take the lock, find that the queue is
      already unplugged, release the lock and exit.
      With the below patch, performance of the benchmark was increased by 18%
      (the whole OLTP application, not just block layer microbenchmarks).
      So I'm submitting this patch for upstream. I think the patch is correct,
      because when more threads call simultaneously plug and unplug, it is
      unspecified, if the queue is or isn't plugged (so the patch can't make
      this worse). And the caller that plugged the queue should unplug it
      anyway. (if it doesn't, there's 3ms timeout).
      Signed-off-by: default avatarMikulas Patocka <mpatocka@redhat.com>
      Signed-off-by: default avatarJens Axboe <jens.axboe@oracle.com>
  10. 01 May, 2008 1 commit
  11. 29 Apr, 2008 5 commits
  12. 04 Mar, 2008 3 commits
  13. 19 Feb, 2008 2 commits
    • Tejun Heo's avatar
      block: add request->raw_data_len · 6b00769f
      Tejun Heo authored
      With padding and draining moved into it, block layer now may extend
      requests as directed by queue parameters, so now a request has two
      sizes - the original request size and the extended size which matches
      the size of area pointed to by bios and later by sgs.  The latter size
      is what lower layers are primarily interested in when allocating,
      filling up DMA tables and setting up the controller.
      Both padding and draining extend the data area to accomodate
      controller characteristics.  As any controller which speaks SCSI can
      handle underflows, feeding larger data area is safe.
      So, this patch makes the primary data length field, request->data_len,
      indicate the size of full data area and add a separate length field,
      request->raw_data_len, for the unmodified request size.  The latter is
      used to report to higher layer (userland) and where the original
      request size should be fed to the controller or device.
      Signed-off-by: default avatarTejun Heo <htejun@gmail.com>
      Cc: James Bottomley <James.Bottomley@HansenPartnership.com>
      Signed-off-by: default avatarJens Axboe <jens.axboe@oracle.com>
    • Adrian Bunk's avatar
      make blk-core.c:request_cachep static again · 5ece6c52
      Adrian Bunk authored
      request_cachep needlessly became global.
      Signed-off-by: default avatarAdrian Bunk <bunk@kernel.org>
      Signed-off-by: default avatarJens Axboe <axboe@carl.home.kernel.dk>
  14. 08 Feb, 2008 3 commits
  15. 01 Feb, 2008 2 commits
  16. 29 Jan, 2008 1 commit