simulating a slow drive

I want to simulate a slow drive, to better grasp the symptoms of a database suffering from a slow disk.

Until now I used to follow up on suspicions by testing with  IOzone or sysbench/fileio , but:

• This is sometimes difficult to do if the sysadmin is not me, and will not/can not cooperate
• This is imperfect because we would typically do such tests at night when the SAN which I suspect is less used ;-)

I could use a SD card or something, but most of my test setups are on VM, I would prefer a software solution.
I almost started to write my own slow FUSE filesystem, when I finally found a nice trick from this post:
http://askubuntu.com/questions/13590/can-i-simulate-a-slow-hard-drive

Use nbd, the Network Block Device, and then rate limit access to it using say trickle.
sudo apt-get install nbd-client nbd-server trickle

Knowing neither trickle nor nbd, I tested each  before doing the setup:

1) Trickle test

Trickle will limit the network speed of a command:

trickle -u <upload speed> -d <download speed> command

Example:$ wget http://192.168.0.6/videos2007/CIMG0500-small.avi

=> 11M/s

$ trickle -d 20 wget http://192.168.0.6/videos2007/CIMG0500-small.avi

=> 22 K/s




2) NBD test

NBD stands for Network Block Device. I understand it to be a simple way to use a block device from a remote machine (simpler than drbd or iscsi targets/initiators implementations)

Blocks may be files or partitions, logical volumes... Let's try to create a file :

dd if=/dev/zero of=/home/phil/apps/NBD/nbd-drive bs=1024 count=10240

Then run the nbd server on that file:

nbd-server 9500 /home/phil/apps/NBD/nbd-drive 

It seems to launch a daemon 'nbd-server', which do listen on 9500:


$ netstat -plnt
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
...          
tcp        0      0 0.0.0.0:9500            0.0.0.0:*               LISTEN      2195/nbd-server 



Let's try to connect the client:

sudo nbd-client localhost 9500 /dev/nbd0

Seems to launch a daemon as well:
root      2276     1  0 07:04 ?        00:00:00 nbd-client localhost 9500 /dev/nbd0
root      2277     2  0 07:04 ?        00:00:00 [nbd0]

We can now format this device and mount it:
sudo mkfs /dev/nbd0 
sudo mount /dev/nbd0 /mnt

$ df -h
Filesystem            Size  Used Avail Use% Mounted on
...
/dev/nbd0             9.7M   92K  9.1M   1% /mnt

The test is OK. Let's umount and release:

sudo umount /mnt

sudo nbd-client -d /dev/nbd0

killall nbd-server

3) Trickle + NBD to make a slow drive

I succeeded by running the server through trickle this way:

$ trickle -d 20 -u 20 nbd-server 9500 /home/phil/apps/NBD/nbd-drive bs=1024 count=10240

trickle: Could not reach trickled, working independently: No such file or directory

Then started the client again with:

sudo nbd-client localhost 9500 /dev/nbd0

And indeed this block device was really slow :

$ sudo dd if=/dev/nbd0 of=/dev/null bs=65536 skip=100 count=10

10+0 records in

10+0 records out

655360 bytes (655 kB) copied, 28.5783 s, 22.9 kB/s

Finally I wrote that script to make the drive ready when I need it:

$ cat mountslowdrive.sh 

#!/bin/bash

FILE_OR_PARTITION=/home/phil/apps/NBD/nbd-drive

MOUNTPOINT=/home/phil/apps/NBD/mnt

if [ $# -eq 0 ] ; then

  echo making/mounting slow drive

  trickle -d 20 -u 20 nbd-server 9500 $FILE_OR_PARTITION bs=1024 count=10240

  sudo nbd-client localhost 9500 /dev/nbd0

  sudo mount /dev/nbd0 $MOUNTPOINT

else

  if [ \( $# -eq 1 \) -a \(  $1 == "-u" \) ]; then

    echo unmountint slow drive

    sudo umount $MOUNTPOINT

    sudo nbd-client -d /dev/nbd0

    killall nbd-server

  else

    echo  USAGE: $0 [-u]

    echo         -u : umount the slow drive

  fi

fi

Todo:
Use the -l or /etc/nbd_server.allow option of the server to restrict the clients (Otherwise any client on the network can use the block device... )