Commit c6b4fcba authored by Joe Thornber's avatar Joe Thornber Committed by Alasdair G Kergon

dm: add cache target

Add a target that allows a fast device such as an SSD to be used as a
cache for a slower device such as a disk.

A plug-in architecture was chosen so that the decisions about which data
to migrate and when are delegated to interchangeable tunable policy
modules.  The first general purpose module we have developed, called
"mq" (multiqueue), follows in the next patch.  Other modules are
under development.
Signed-off-by: default avatarJoe Thornber <>
Signed-off-by: default avatarHeinz Mauelshagen <>
Signed-off-by: default avatarMike Snitzer <>
Signed-off-by: default avatarAlasdair G Kergon <>
parent 7a87edfe
dm-cache is a device mapper target written by Joe Thornber, Heinz
Mauelshagen, and Mike Snitzer.
It aims to improve performance of a block device (eg, a spindle) by
dynamically migrating some of its data to a faster, smaller device
(eg, an SSD).
This device-mapper solution allows us to insert this caching at
different levels of the dm stack, for instance above the data device for
a thin-provisioning pool. Caching solutions that are integrated more
closely with the virtual memory system should give better performance.
The target reuses the metadata library used in the thin-provisioning
The decision as to what data to migrate and when is left to a plug-in
policy module. Several of these have been written as we experiment,
and we hope other people will contribute others for specific io
scenarios (eg. a vm image server).
Migration - Movement of the primary copy of a logical block from one
device to the other.
Promotion - Migration from slow device to fast device.
Demotion - Migration from fast device to slow device.
The origin device always contains a copy of the logical block, which
may be out of date or kept in sync with the copy on the cache device
(depending on policy).
The target is constructed by passing three devices to it (along with
other parameters detailed later):
1. An origin device - the big, slow one.
2. A cache device - the small, fast one.
3. A small metadata device - records which blocks are in the cache,
which are dirty, and extra hints for use by the policy object.
This information could be put on the cache device, but having it
separate allows the volume manager to configure it differently,
e.g. as a mirror for extra robustness.
Fixed block size
The origin is divided up into blocks of a fixed size. This block size
is configurable when you first create the cache. Typically we've been
using block sizes of 256k - 1024k.
Having a fixed block size simplifies the target a lot. But it is
something of a compromise. For instance, a small part of a block may be
getting hit a lot, yet the whole block will be promoted to the cache.
So large block sizes are bad because they waste cache space. And small
block sizes are bad because they increase the amount of metadata (both
in core and on disk).
The cache has two modes, writeback and writethrough.
If writeback, the default, is selected then a write to a block that is
cached will go only to the cache and the block will be marked dirty in
the metadata.
If writethrough is selected then a write to a cached block will not
complete until it has hit both the origin and cache devices. Clean
blocks should remain clean.
A simple cleaner policy is provided, which will clean (write back) all
dirty blocks in a cache. Useful for decommissioning a cache.
Migration throttling
Migrating data between the origin and cache device uses bandwidth.
The user can set a throttle to prevent more than a certain amount of
migration occuring at any one time. Currently we're not taking any
account of normal io traffic going to the devices. More work needs
doing here to avoid migrating during those peak io moments.
For the time being, a message "migration_threshold <#sectors>"
can be used to set the maximum number of sectors being migrated,
the default being 204800 sectors (or 100MB).
Updating on-disk metadata
On-disk metadata is committed every time a REQ_SYNC or REQ_FUA bio is
written. If no such requests are made then commits will occur every
second. This means the cache behaves like a physical disk that has a
write cache (the same is true of the thin-provisioning target). If
power is lost you may lose some recent writes. The metadata should
always be consistent in spite of any crash.
The 'dirty' state for a cache block changes far too frequently for us
to keep updating it on the fly. So we treat it as a hint. In normal
operation it will be written when the dm device is suspended. If the
system crashes all cache blocks will be assumed dirty when restarted.
Per-block policy hints
Policy plug-ins can store a chunk of data per cache block. It's up to
the policy how big this chunk is, but it should be kept small. Like the
dirty flags this data is lost if there's a crash so a safe fallback
value should always be possible.
For instance, the 'mq' policy, which is currently the default policy,
uses this facility to store the hit count of the cache blocks. If
there's a crash this information will be lost, which means the cache
may be less efficient until those hit counts are regenerated.
Policy hints affect performance, not correctness.
Policy messaging
Policies will have different tunables, specific to each one, so we
need a generic way of getting and setting these. Device-mapper
messages are used. Refer to cache-policies.txt.
Discard bitset resolution
We can avoid copying data during migration if we know the block has
been discarded. A prime example of this is when mkfs discards the
whole block device. We store a bitset tracking the discard state of
blocks. However, we allow this bitset to have a different block size
from the cache blocks. This is because we need to track the discard
state for all of the origin device (compare with the dirty bitset
which is just for the smaller cache device).
Target interface
cache <metadata dev> <cache dev> <origin dev> <block size>
<#feature args> [<feature arg>]*
<policy> <#policy args> [policy args]*
metadata dev : fast device holding the persistent metadata
cache dev : fast device holding cached data blocks
origin dev : slow device holding original data blocks
block size : cache unit size in sectors
#feature args : number of feature arguments passed
feature args : writethrough. (The default is writeback.)
policy : the replacement policy to use
#policy args : an even number of arguments corresponding to
key/value pairs passed to the policy
policy args : key/value pairs passed to the policy
E.g. 'sequential_threshold 1024'
See cache-policies.txt for details.
Optional feature arguments are:
writethrough : write through caching that prohibits cache block
content from being different from origin block content.
Without this argument, the default behaviour is to write
back cache block contents later for performance reasons,
so they may differ from the corresponding origin blocks.
A policy called 'default' is always registered. This is an alias for
the policy we currently think is giving best all round performance.
As the default policy could vary between kernels, if you are relying on
the characteristics of a specific policy, always request it by name.
<#used metadata blocks>/<#total metadata blocks> <#read hits> <#read misses>
<#write hits> <#write misses> <#demotions> <#promotions> <#blocks in cache>
<#dirty> <#features> <features>* <#core args> <core args>* <#policy args>
<policy args>*
#used metadata blocks : Number of metadata blocks used
#total metadata blocks : Total number of metadata blocks
#read hits : Number of times a READ bio has been mapped
to the cache
#read misses : Number of times a READ bio has been mapped
to the origin
#write hits : Number of times a WRITE bio has been mapped
to the cache
#write misses : Number of times a WRITE bio has been
mapped to the origin
#demotions : Number of times a block has been removed
from the cache
#promotions : Number of times a block has been moved to
the cache
#blocks in cache : Number of blocks resident in the cache
#dirty : Number of blocks in the cache that differ
from the origin
#feature args : Number of feature args to follow
feature args : 'writethrough' (optional)
#core args : Number of core arguments (must be even)
core args : Key/value pairs for tuning the core
e.g. migration_threshold
#policy args : Number of policy arguments to follow (must be even)
policy args : Key/value pairs
e.g. 'sequential_threshold 1024
Policies will have different tunables, specific to each one, so we
need a generic way of getting and setting these. Device-mapper
messages are used. (A sysfs interface would also be possible.)
The message format is:
<key> <value>
dmsetup message my_cache 0 sequential_threshold 1024
The test suite can be found here:
dmsetup create my_cache --table '0 41943040 cache /dev/mapper/metadata \
/dev/mapper/ssd /dev/mapper/origin 512 1 writeback default 0'
dmsetup create my_cache --table '0 41943040 cache /dev/mapper/metadata \
/dev/mapper/ssd /dev/mapper/origin 1024 1 writeback \
mq 4 sequential_threshold 1024 random_threshold 8'
......@@ -268,6 +268,19 @@ config DM_DEBUG_BLOCK_STACK_TRACING
If unsure, say N.
config DM_CACHE
tristate "Cache target (EXPERIMENTAL)"
depends on BLK_DEV_DM
default n
dm-cache attempts to improve performance of a block device by
moving frequently used data to a smaller, higher performance
device. Different 'policy' plugins can be used to change the
algorithms used to select which blocks are promoted, demoted,
cleaned etc. It supports writeback and writethrough modes.
config DM_MIRROR
tristate "Mirror target"
depends on BLK_DEV_DM
......@@ -11,6 +11,7 @@ dm-mirror-y += dm-raid1.o
dm-log-userspace-y \
+= dm-log-userspace-base.o dm-log-userspace-transfer.o
dm-thin-pool-y += dm-thin.o dm-thin-metadata.o
dm-cache-y += dm-cache-target.o dm-cache-metadata.o dm-cache-policy.o
md-mod-y += md.o bitmap.o
raid456-y += raid5.o
......@@ -44,6 +45,7 @@ obj-$(CONFIG_DM_ZERO) += dm-zero.o
obj-$(CONFIG_DM_RAID) += dm-raid.o
obj-$(CONFIG_DM_THIN_PROVISIONING) += dm-thin-pool.o
obj-$(CONFIG_DM_VERITY) += dm-verity.o
obj-$(CONFIG_DM_CACHE) += dm-cache.o
ifeq ($(CONFIG_DM_UEVENT),y)
dm-mod-objs += dm-uevent.o
......@@ -179,6 +179,15 @@ int dm_bio_detain(struct dm_bio_prison *prison,
int dm_get_cell(struct dm_bio_prison *prison,
struct dm_cell_key *key,
struct dm_bio_prison_cell *cell_prealloc,
struct dm_bio_prison_cell **cell_result)
return bio_detain(prison, key, NULL, cell_prealloc, cell_result);
* @inmates must have been initialised prior to this call
......@@ -56,6 +56,17 @@ struct dm_bio_prison_cell *dm_bio_prison_alloc_cell(struct dm_bio_prison *prison
void dm_bio_prison_free_cell(struct dm_bio_prison *prison,
struct dm_bio_prison_cell *cell);
* Creates, or retrieves a cell for the given key.
* Returns 1 if pre-existing cell returned, zero if new cell created using
* @cell_prealloc.
int dm_get_cell(struct dm_bio_prison *prison,
struct dm_cell_key *key,
struct dm_bio_prison_cell *cell_prealloc,
struct dm_bio_prison_cell **cell_result);
* An atomic op that combines retrieving a cell, and adding a bio to it.
* Copyright (C) 2012 Red Hat, Inc.
* This file is released under the GPL.
#include "persistent-data/dm-block-manager.h"
* It's helpful to get sparse to differentiate between indexes into the
* origin device, indexes into the cache device, and indexes into the
* discard bitset.
typedef dm_block_t __bitwise__ dm_oblock_t;
typedef uint32_t __bitwise__ dm_cblock_t;
typedef dm_block_t __bitwise__ dm_dblock_t;
static inline dm_oblock_t to_oblock(dm_block_t b)
return (__force dm_oblock_t) b;
static inline dm_block_t from_oblock(dm_oblock_t b)
return (__force dm_block_t) b;
static inline dm_cblock_t to_cblock(uint32_t b)
return (__force dm_cblock_t) b;
static inline uint32_t from_cblock(dm_cblock_t b)
return (__force uint32_t) b;
static inline dm_dblock_t to_dblock(dm_block_t b)
return (__force dm_dblock_t) b;
static inline dm_block_t from_dblock(dm_dblock_t b)
return (__force dm_block_t) b;
This diff is collapsed.
* Copyright (C) 2012 Red Hat, Inc.
* This file is released under the GPL.
#include "dm-cache-block-types.h"
#include "dm-cache-policy-internal.h"
/* FIXME: remove this restriction */
* The metadata device is currently limited in size.
* We have one block of index, which can hold 255 index entries. Each
* index entry contains allocation info about 16k metadata blocks.
* A metadata device larger than 16GB triggers a warning.
#define DM_CACHE_METADATA_MAX_SECTORS_WARNING (16 * (1024 * 1024 * 1024 >> SECTOR_SHIFT))
* Ext[234]-style compat feature flags.
* A new feature which old metadata will still be compatible with should
* define a DM_CACHE_FEATURE_COMPAT_* flag (rarely useful).
* A new feature that is not compatible with old code should define a
* DM_CACHE_FEATURE_INCOMPAT_* flag and guard the relevant code with
* that flag.
* A new feature that is not compatible with old code accessing the
* metadata RDWR should define a DM_CACHE_FEATURE_RO_COMPAT_* flag and
* guard the relevant code with that flag.
* As these various flags are defined they should be added to the
* following masks.
* Reopens or creates a new, empty metadata volume.
* Returns an ERR_PTR on failure.
struct dm_cache_metadata *dm_cache_metadata_open(struct block_device *bdev,
sector_t data_block_size,
bool may_format_device,
size_t policy_hint_size);
void dm_cache_metadata_close(struct dm_cache_metadata *cmd);
* The metadata needs to know how many cache blocks there are. We don't
* care about the origin, assuming the core target is giving us valid
* origin blocks to map to.
int dm_cache_resize(struct dm_cache_metadata *cmd, dm_cblock_t new_cache_size);
dm_cblock_t dm_cache_size(struct dm_cache_metadata *cmd);
int dm_cache_discard_bitset_resize(struct dm_cache_metadata *cmd,
sector_t discard_block_size,
dm_dblock_t new_nr_entries);
typedef int (*load_discard_fn)(void *context, sector_t discard_block_size,
dm_dblock_t dblock, bool discarded);
int dm_cache_load_discards(struct dm_cache_metadata *cmd,
load_discard_fn fn, void *context);
int dm_cache_set_discard(struct dm_cache_metadata *cmd, dm_dblock_t dblock, bool discard);
int dm_cache_remove_mapping(struct dm_cache_metadata *cmd, dm_cblock_t cblock);
int dm_cache_insert_mapping(struct dm_cache_metadata *cmd, dm_cblock_t cblock, dm_oblock_t oblock);
int dm_cache_changed_this_transaction(struct dm_cache_metadata *cmd);
typedef int (*load_mapping_fn)(void *context, dm_oblock_t oblock,
dm_cblock_t cblock, bool dirty,
uint32_t hint, bool hint_valid);
int dm_cache_load_mappings(struct dm_cache_metadata *cmd,
const char *policy_name,
load_mapping_fn fn,
void *context);
int dm_cache_set_dirty(struct dm_cache_metadata *cmd, dm_cblock_t cblock, bool dirty);
struct dm_cache_statistics {
uint32_t read_hits;
uint32_t read_misses;
uint32_t write_hits;
uint32_t write_misses;
void dm_cache_metadata_get_stats(struct dm_cache_metadata *cmd,
struct dm_cache_statistics *stats);
void dm_cache_metadata_set_stats(struct dm_cache_metadata *cmd,
struct dm_cache_statistics *stats);
int dm_cache_commit(struct dm_cache_metadata *cmd, bool clean_shutdown);
int dm_cache_get_free_metadata_block_count(struct dm_cache_metadata *cmd,
dm_block_t *result);
int dm_cache_get_metadata_dev_size(struct dm_cache_metadata *cmd,
dm_block_t *result);
void dm_cache_dump(struct dm_cache_metadata *cmd);
* The policy is invited to save a 32bit hint value for every cblock (eg,
* for a hit count). These are stored against the policy name. If
* policies are changed, then hints will be lost. If the machine crashes,
* hints will be lost.
* The hints are indexed by the cblock, but many policies will not
* neccessarily have a fast way of accessing efficiently via cblock. So
* rather than querying the policy for each cblock, we let it walk its data
* structures and fill in the hints in whatever order it wishes.
int dm_cache_begin_hints(struct dm_cache_metadata *cmd, struct dm_cache_policy *p);
* requests hints for every cblock and stores in the metadata device.
int dm_cache_save_hint(struct dm_cache_metadata *cmd,
dm_cblock_t cblock, uint32_t hint);
#endif /* DM_CACHE_METADATA_H */
* Copyright (C) 2012 Red Hat. All rights reserved.
* This file is released under the GPL.
#include "dm-cache-policy.h"
* Little inline functions that simplify calling the policy methods.
static inline int policy_map(struct dm_cache_policy *p, dm_oblock_t oblock,
bool can_block, bool can_migrate, bool discarded_oblock,
struct bio *bio, struct policy_result *result)
return p->map(p, oblock, can_block, can_migrate, discarded_oblock, bio, result);
static inline int policy_lookup(struct dm_cache_policy *p, dm_oblock_t oblock, dm_cblock_t *cblock)
return p->lookup(p, oblock, cblock);
static inline void policy_set_dirty(struct dm_cache_policy *p, dm_oblock_t oblock)
if (p->set_dirty)
p->set_dirty(p, oblock);
static inline void policy_clear_dirty(struct dm_cache_policy *p, dm_oblock_t oblock)
if (p->clear_dirty)
p->clear_dirty(p, oblock);
static inline int policy_load_mapping(struct dm_cache_policy *p,
dm_oblock_t oblock, dm_cblock_t cblock,
uint32_t hint, bool hint_valid)
return p->load_mapping(p, oblock, cblock, hint, hint_valid);
static inline int policy_walk_mappings(struct dm_cache_policy *p,
policy_walk_fn fn, void *context)
return p->walk_mappings ? p->walk_mappings(p, fn, context) : 0;
static inline int policy_writeback_work(struct dm_cache_policy *p,
dm_oblock_t *oblock,
dm_cblock_t *cblock)
return p->writeback_work ? p->writeback_work(p, oblock, cblock) : -ENOENT;
static inline void policy_remove_mapping(struct dm_cache_policy *p, dm_oblock_t oblock)
return p->remove_mapping(p, oblock);
static inline void policy_force_mapping(struct dm_cache_policy *p,
dm_oblock_t current_oblock, dm_oblock_t new_oblock)
return p->force_mapping(p, current_oblock, new_oblock);
static inline dm_cblock_t policy_residency(struct dm_cache_policy *p)
return p->residency(p);
static inline void policy_tick(struct dm_cache_policy *p)
if (p->tick)
return p->tick(p);
static inline int policy_emit_config_values(struct dm_cache_policy *p, char *result, unsigned maxlen)
ssize_t sz = 0;
if (p->emit_config_values)
return p->emit_config_values(p, result, maxlen);
return 0;
static inline int policy_set_config_value(struct dm_cache_policy *p,
const char *key, const char *value)
return p->set_config_value ? p->set_config_value(p, key, value) : -EINVAL;
* Creates a new cache policy given a policy name, a cache size, an origin size and the block size.
struct dm_cache_policy *dm_cache_policy_create(const char *name, dm_cblock_t cache_size,
sector_t origin_size, sector_t block_size);
* Destroys the policy. This drops references to the policy module as well
* as calling it's destroy method. So always use this rather than calling
* the policy->destroy method directly.
void dm_cache_policy_destroy(struct dm_cache_policy *p);
* In case we've forgotten.
const char *dm_cache_policy_get_name(struct dm_cache_policy *p);
size_t dm_cache_policy_get_hint_size(struct dm_cache_policy *p);
* Copyright (C) 2012 Red Hat. All rights reserved.
* This file is released under the GPL.
#include "dm-cache-policy-internal.h"
#include "dm.h"
#include <linux/module.h>
#include <linux/slab.h>
#define DM_MSG_PREFIX "cache-policy"
static DEFINE_SPINLOCK(register_lock);
static LIST_HEAD(register_list);
static struct dm_cache_policy_type *__find_policy(const char *name)
struct dm_cache_policy_type *t;
list_for_each_entry(t, &register_list, list)
if (!strcmp(t->name, name))
return t;
return NULL;
static struct dm_cache_policy_type *__get_policy_once(const char *name)
struct dm_cache_policy_type *t = __find_policy(name);
if (t && !try_module_get(t->owner)) {
DMWARN("couldn't get module %s", name);
return t;
static struct dm_cache_policy_type *get_policy_once(const char *name)
struct dm_cache_policy_type *t;
t = __get_policy_once(name);
return t;
static struct dm_cache_policy_type *get_policy(const char *name)
struct dm_cache_policy_type *t;
t = get_policy_once(name);
if (IS_ERR(t))
return NULL;
if (t)
return t;
request_module("dm-cache-%s", name);
t = get_policy_once(name);
if (IS_ERR(t))
return NULL;
return t;
static void put_policy(struct dm_cache_policy_type *t)
int dm_cache_policy_register(struct dm_cache_policy_type *type)
int r;
/* One size fits all for now */
if (type->hint_size != 0 && type->hint_size != 4) {
DMWARN("hint size must be 0 or 4 but %llu supplied.", (unsigned long long) type->hint_size);
return -EINVAL;
if (__find_policy(type->name)) {
DMWARN("attempt to register policy under duplicate name %s", type->name);
r = -EINVAL;
} else {
list_add(&type->list, &register_list);
r = 0;
return r;
void dm_cache_policy_unregister(struct dm_cache_policy_type *type)