atconsultancy logo
cowloop logo
atcomputing logo

spacerHome

spacerNews

spacerDownload

spacerFAQ

 

SourceForge.net Logo

Tux

Introduction

The cowloop driver allows block-devices or files to be used in a read-write fashion without modifying the block-device or file itself. Modified data-blocks are not written back to the original read-only device, instead they are written to a separate file called the copy-on-write file or cowfile.
The cowloop driver offers (default) 16 cowdevices. To every cowdevice a read-only device (file) can be assigned with a corresponding cowfile to store the modifications for that read-only device.

The cowloop driver offers various possibilities, such as:

  • A filesystem stored on a non-writable medium (on a write-protected floppy, in a file on a cdrom/dvd) can still be mounted read-write via a cowdevice.

  • One filesystem can be used as a basis for various independent sessions that can be active in parallel.
    For each session a separate cowdevice is prepared. Every cowdevice could use the same read-only device (or file) holding the filesystem, while every cowdevice uses a dedicated cowfile holding the specific modifications for that session. Each cowdevice is mounted separately via the cowloop driver and unmounted again after use.

  • A database stored in a file or on a block-device can be used read-write without modifying the original contents.

  • ......
The cowloop driver can be used for any kind of filesystem-type (even non-filesystem data can be accessed in a read-write fashion). Obviously the filesystem-module that lies on top of the cowloop driver should allow write-accesses to that particular type of filesystem.

Functional overview

For every cowdevice the combination of a read-only file and a cowfile should be assigned. This assignment can be made dynamically (while the cowloop module is loaded) with the command cowdev; this command can also be used to deactivate a cowdevice again.
For the first cowdevice an assigment can be made already during loading of the cowloop module by specifying one read-only file and one cowfile as parameters for the modprobe command.

Every combination of a read-only file and corresponding cowfile is made available as a new cowdevice (e.g. two independent cowdevices /dev/cow/0 and /dev/cow/1 in the figure below).

driver schematics

The cowloop driver accesses the read-only file as a sequence of 1 Kb blocks.

Concerning the example of cowdevice /dev/cow/0 in the figure above:
Initially every block read from this device is retrieved from the read-only file (e.g. block 3). When a block is written to the cowdevice, the cowloop driver writes the block to the copy-on-write file. On reading, the cowloop driver first examines the cowfile to see whether a modified version of the requested block is present. Hence modified blocks are retrieved from the cowfile (e.g. block 1), otherwise the original block is retrieved from the read-only file (e.g. block 2).

 

Suppose that the underlying read-only file contains a filesystem, it can be mounted read-write with the command

  mount /dev/cow/0 /mnt/fs0

All modifications on data-blocks and metadata-blocks of the filesystem are written to the copy-on-write file. After the filesystem has been unmounted and the cowdevice deactivated, the read-only file still contains the original (untouched) filesystem. However when it is used in combination with the cowfile via the cowloop driver, the modified filesystem appears again.

Installing cowloop

The following description is a summary of a more extensive explanation as you can find in file HOWTO.install, which is part of the distribution.

First go to the directory where the source-tree of cowloop has been unpacked. Now check whether the default major number for the cowloop driver is already in use on your system:

    ls -l /dev | grep '241,'
If it is in use by another driver already, change the major number of the cowloop driver in the file Makefile (COWMAJOR=241), in the source-file cowloop.c (#define COWMAJOR 241) and in the build-script makecows. All these three files are in the src/ subdirectory. Note: from version 3.3 on the driver will obtain its major number dynamically.

To build and install cowloop, start the command:

    make install
This compiles the cowloop driver and copies it to the /lib/modules/version/misc directory, where version is the actual kernel-version running on the system. The related commands are compiled and copied to the /usr/sbin directory.
Furthermore the device-files /dev/cow/0, /dev/cow/1, /dev/cow/2, /dev/cow/3, etc. are created. Note: from version 3.3 on the driver will dynamically create these /dev/cow/N names when the corresponding cowdevice is activated (see the cowdev -a utility). This has made the makecows installation script superfluous.

Loading and unloading cowloop

The command

  modprobe cowloop [maxcows=...] [rdofile=... cowfile=... [option=r]]
takes care that the cowloop driver is loaded into the kernel.

The parameter maxcows= can be used to specify how many cowdevices should be supported in parallel. The default value is 16, accessible via the cowdevices /dev/cow/0 till /dev/cow/15.
The current maximum value is 254. Version 3.3. will increase the maximum to 1022. (more info on how to modify this maximum is in the file HOWTO.install).

It is possible to activate the first cowdevice /dev/cow/0 already while loading the cowloop driver:

  • rdofile=
    The pathname of the read-only file or filesystem. The read-only file(system) may be of type regular or block-device.
  • cowfile=
    The pathname of the copy-on-write file used as storage-space for modified blocks of the read-only file(system). The copy-on-write file is always a regular file.

    If the given copy-on-write file does not yet exist, a new file with this name will be created by the cowloop driver.

    If an existing regular file is specified as cowfile, its contents will be used again for the current read-only file (the cowfile is supposed to contain modified blocks from an earlier session for the same read-only file).

  • option=
    Notice that an existing cowfile must be in a consistent state. This means that the previous session should have been finished properly by deactivating the cowdevice via the command cowdev or by unloading the cowloop driver (command rmmod).
    When the cowfile has not been properly closed, the cowloop driver refuses to assign it again unless the parameter option=r is given with the command modprobe. This option forces that the cowfile is repaired automatically if it appears to be inconsistent. In that case the command modprobe might take some time!

    Notice that this option should only be used for special purposes. Inconsistent cowfiles should preferably be repaired with the command cowrepair before activating the cowdevice.

When failures occur during driver-loading, more information about the reason of the failure can be found in the /var/log/messages syslog file.

 

The command

    rmmod cowloop
takes care that the cowloop driver is unloaded again (all active cowdevices will be deactivated automatically).

Cowfile structure

A cowfile is used by the cowloop driver to store the modified blocks that are written to a cowdevice. Apart from these modified blocks, a copy-on-write file contains some additional administration:

cowloop logo

  • header
    A block of 1 Kbytes containing information like the magic number (a special bit-pattern), the flag indicating if the cowfile has been properly closed, the size of the corresponding read-only file and a checksum of the first blocks from the read-only file. Using the information in the header, the cowloop driver can roughly check whether an existing cowloop file is
    • consistent (closed properly),
    • used with the same read-only file as before, and
    • used with a read-only file that has not been modified since the cowdevice has been previously deactivated.
  • bitmap
    The bitmap contains one bit per 1 Kbytes block in the read-only file. This bit indicates if the corresponding block has been modified (i.e. the block should be retrieved from the cowfile) or not (i.e. the block should be retrieved from the read-only file).
    The bitmap itself also consists of a number of 1 Kbytes blocks. The size of the bitmap obviously depends on the size of the read-only file.
  • modified blocks
    Area containing the modified blocks.
    A modified block is written in this area at the same offset as the corresponding block in the read-only file. Therefore the cowfile contains many "holes" of unwritten space (it is a so-called sparse file). Only the written blocks really consume disk-space.

The maximum size of the cowfile is evident from the figure above: the size of the orginal file + the sizes of the header and the bitmap. The command "ls -l cowfile" shows the logical size of the file. In most cases the cowfile will be sparse, i.e. contain holes. The actual amount of diskspace consumed is given in kilobytes by "du -k cowfile".

Be careful when making copies of cowfiles. Not all programs are aware of sparse files. Commands like cpio and tar in particular store cowfiles using their maximum size.

Commands

The following commands can be used to interact with the cowloop driver. Note that the distribution package contains extensive manual pages for each of these.

  • Command cowdev
    The command cowdev can be used to activate a new cowdevice by offering a read-only file and a corresponding cowfile. This command can also be used to deactivate a cowdevice again and to list the active cowdevices.

    SYNOPSIS:
          cowdev -l
          cowdev -a rdofile cowfile [cowdevice]
          cowdev -d cowdevice

    -l List cowdevices.
    Use the flag "-l" to generate a list with the cowdevices that are currently in use. For every cowdevice the assigned read-only file and cowfile is shown.
    -a Activate cowdevice.
    Use the flag "-a" to activate a new cowdevice, assigning the specified read-only file and corresponding cowfile.
    Optionally a specific cowdevice (/dev/cow/...) can be requested. When this device is already in use, the activation will fail. When no specific cowdevice is requested, the cowloop driver searches for the first free cowdevice.
    When the activation succeeds, the command cowdev returns the name of the activated cowdevice.
    -d Deactivate cowdevice.
    Use the flag "-d" to deactivate a cowdevice. This might fail when the cowdevice is still in use (e.g. the cowdevice is still mounted).

    EXAMPLE:

          # cowdev -a /dev/sda2 /tmp/myfs1.cow
          /dev/cow/5
      
          # cowdev -l
          cowdevice    read-only file   copy-on-write file              
          ---------------------------------------------------
          ...
          /dev/cow/5   /dev/sda2        /tmp/myfs1.cow                  
      
          # mount /dev/cow/5 /mnt/myfs
      
          # ......
      
          # umount /dev/cow/5
          # cowdev -d /dev/cow/5
        

  • Command cowsync
    The cowloop driver maintains an up-to-date copy of the header and bitmap in memory. When a cowdevice is deactivated (or when the driver is unloaded), these in-memory copies are flushed to the cowfile. As long as the cowdevice is active, the cowfile is in an inconsistent state (state 'dirty').
    The command cowsync can be used to request the cowloop driver to flush the in-memory header and bitmap for all active cowfiles. After that the cowfiles are temporarily consistent again (state 'clean') until the driver modifies the in-memory header or bitmap again.

    SYNOPSIS:      cowsync

     

The following commands can be used to manipulate cowfiles (no need for the cowloop driver):

  • Command cowlist
    The command cowlist can be used to show the information from the header of a cowfile.

    SYNOPSIS:      cowlist cowfile

    EXAMPLE:

          $ cowlist /tmp/myfs1.cow
      
          Info about cowfile /tmp/myfs1.cow:
               state cowfile:     clean
              header-version:         1
                 data offset:     36864
                     mapunit:      1024
               bitmap-blocks:        32 (of 1024 bytes)
            cowblocks in use:       762 (of 1024 bytes)
                size rdofile:    256000 (of 1024 bytes)
        

  • Command cowrepair
    When a cowdevice has not been properly deactivated (e.g. due to a system crash), the cowfile is in an inconsistent state (state 'dirty'). The cowloop driver refuses to activate such cowfile again, unless the inconsistencies are repaired. When the first cowdevice is already activated during loading of the cowloop driver (via parameters "rdofile=" and "cowfile=") the additional parameter "option=r" can be used. In that case the cowloop driver implicitly repairs the cowfile in kernel mode which is not preferable though in some cases inevitable.

    Preferably a cowfile should be repaired using the command cowrepair before it is offered to be activated for a new cowdevice.

    SYNOPSIS:      cowrepair [-fnv] cowfile

    -f Forced repair.
    The command cowrepair only repairs a cowfile which has the state 'dirty'. With the flag "-f" the cowfile will be made consistent even if the state is 'clean'.
    -n No modifications (fake mode).
    With the flag "-n" the consistency of the cowfile is verified and messages are shown about the state. However no corrections are issued.
    -v Verbose.
    With the flag "-v" additional messages are shown about the inconsistencies in the cowfile.

    EXAMPLE:

          $ cowrepair /tmp/myfs1.cow
          cowfile /tmp/myfs1.cow is not dirty
      
          $ cowrepair -f /tmp/myfs1.cow
          number of corrections in bitmap: 0
        

  • Command cowmerge
    The command cowmerge writes all modifications stored in the specified cowfile into the specified read-only file (that is used in a read-write fashion now).
    Beware that you lose the original contents of the read-only file which makes other cowfiles related to this read-only file useless!
    After the merge, the updated read-only file can be used in combination with new cowfiles to store modifications made from now on.

    SYNOPSIS:      cowmerge rdofile cowfile

    EXAMPLE:

          # cowdev -a /dev/sda2 /tmp/myfs.cow
          /dev/cow/0
      
          # mount /dev/cow/0 /mnt/myfs
      
          # ......                                 (work for a while...)
      
          # umount /dev/cow/0
    
          # cowdev -d /dev/cow/0
    
          # cowmerge /dev/sda2 /tmp/myfs.cow 
          # rm /tmp/myfs.cow                       (cowfile is useless now)
    
          # cowdev -a /dev/sda2 /tmp/myfs.cow      (new cowfile)
          /dev/cow/0
    
        

Status information

As soon as a cowdevice is activated, status information can be read from the file /proc/cow/n (n is the number of the cowdevice /dev/cow/n):

  $ cat /proc/cow/5
     cowloop version:      2.14

     number of opens:         1
       pid of thread:      2533

      read-only file: /dev/sda2
            rdoreads:      3129

  copy-on-write file: /tmp/myfs1.cow
       state cowfile:     dirty
       bitmap-blocks:        32 (of 1024 bytes)
    cowblocks in use:       762 (of 1024 bytes)
            cowreads:      1007
           cowwrites:      1770
Most relevant values are:
  • cowloop version
    Version of the current cowloop driver.
  • number of opens
    Number of times that the cowdevice is currently open. The cowdevice can only be deactivated when this value is 0.
  • read-only file
    Name of the file specified as read-only file for this cowdevice.
  • rdoreads
    Number of read-requests issued on the read-only file since it has been activated.
  • copy-on-write file
    Name of the file specified as cowfile for this cowdevice.
  • state cowfile
    Current state of the cowfile. The state 'dirty' means that the bitmap in memory has been modified but not yet flushed to the cowfile. The bitmap is automatically flushed when the cowdevice is deactivated.
    With the command cowsync a flush of the bitmap can be forced without deactivating the cowdevices. After that the state 'clean' is shown until the driver modifies the in-memory bitmap for that cowdevice again.
  • bitmap blocks
    Number of blocks (1 Kbytes) used for the bitmap.
  • cowblocks in use
    Number of modified blocks (1 Kbytes) in use in the cowfile. This value can be used to determine the filling-degree of the sparse copy-on-write file.
  • cowreads
    Number of read-requests issued on the copy-on-write file since it has been activated.

  • cowwrites
    Number of write-requests issued on the copy-on-write file since it has been activated.

Note: from version 3.3. on the use of the /proc file system has been discontinued, and the debugfs has taken its place. The debugfs is usually mounted under /sys/kernel/debug. This change was required by general kernel driver policy.

Example 1: filesystem on write-protected floppy

Obviously, you may read "USB stick" instead of "floppy" here....

Mount a filesystem on a write-protected floppy read-only to see the list of files:

  # mount -r /dev/fd0 /mnt/floppy
  # ls -l /mnt/floppy
  -rw-r--r--  1 root   root     44 Jun 30 13:14 adjtime
  -rw-r--r--  1 root   root   1343 Jun 30 13:14 aliases
  -rw-r-----  1 root   root  12288 Jun 30 13:14 aliases.db
  -rw-------  1 root   root      1 Jun 30 13:14 at.deny
  -rw-r--r--  1 root   root    872 Jun 30 13:14 atsar.conf
  -rw-r--r--  1 root   root    212 Jun 30 13:14 auto.master
  # umount /dev/fd0
The same filesystem can be mounted read-write via the cowloop driver (already loaded via the command modprobe), specifying the name of a new cowfile:
  # cowdev -a /dev/fd0 /tmp/ses1.cow
  /dev/cow/0
  # mount /dev/cow/0 /mnt/floppy
  # ls -l /mnt/floppy
  -rw-r--r--  1 root   root     44 Jun 30 13:14 adjtime
  -rw-r--r--  1 root   root   1343 Jun 30 13:14 aliases
  -rw-r-----  1 root   root  12288 Jun 30 13:14 aliases.db
  -rw-------  1 root   root      1 Jun 30 13:14 at.deny
  -rw-r--r--  1 root   root    872 Jun 30 13:14 atsar.conf
  -rw-r--r--  1 root   root    212 Jun 30 13:14 auto.master
After entering the command mount the situation is as shown in this figure:

cowfloppy schematics

Files can be modified and removed now:
  # vi /mnt/floppy/adjtime
  # ls -l /mnt/floppy/adjtime
  -rw-r--r--  1 root   root     47 Jun 30 13:24 /mnt/floppy/adjtime
  # rm /mnt/floppy/aliases
  # umount /dev/cow/0
  # cowdev -d /dev/cow/0
The contents of the floppy itself is unmodified. The modified blocks of the filesystem (superblock, inodes, bitmaps, data-blocks) are written to the cowfile.
Apparently the cowfile is almost half a Megabyte (according to command ls), but it only consumes 18 Kbytes of physical disk space (according to command du):
  # ls -l /tmp/ses1.cow
  -rw-------  1 root   root 475584 Jun 30 13:25 /tmp/ses1.cow
  # du -k /tmp/ses1.cow 
  18      /tmp/ses1.cow
Later on the filesystem can be mounted read-write again via the cowloop driver, using the same cowfile:
  # cowdev -a /dev/fd0 /tmp/ses1.cow
  /dev/cow/3
  # mount /dev/cow/3 /mnt/floppy
  # ls -l /mnt/floppy
  -rw-r--r--  1 root   root     47 Jun 30 13:24 adjtime
  -rw-r-----  1 root   root  12288 Jun 30 13:14 aliases.db
  -rw-------  1 root   root      1 Jun 30 13:14 at.deny
  -rw-r--r--  1 root   root    872 Jun 30 13:14 atsar.conf
  -rw-r--r--  1 root   root    212 Jun 30 13:14 auto.master
When the cowfile /tmp/ses1.cow is removed before activating it again via the command "cowdev -a", the cowloop driver creates a new empty cowfile. In that case the original floppy-contents appears again after the mount.

Example 2: compressed filesystem on cdrom

A compressed ext2-filesystem is stored in a file called chicken on a cdrom. It is not allowed to mount this filesystem read-write, since it is compressed (accessed via the cloop-driver that does not allow modifications). And even if it was not compressed, the filesystem is located in a file on cdrom for which write-access is impossible anyhow (if the higher kernel layers "know" that the lower driver has no "write"-support, as is the case with e.g. an ISO9660 file system, all write requests will be blocked long before they reach the driver (and thus cowloop); more on this in the FAQ).

The following sequence of commands is necessary to mount the compressed filesystem read-write (the cowloop driver is supposed to be loaded already with the modprobe command):

  # mount /dev/cdrom /mnt/cdrom           
  # modprobe cloop file=/mnt/cdrom/chicken
  # cowdev -a /dev/cloop /tmp/chicken.cow
  /dev/cow/0    # the command returns this name now
  # mount /dev/cow/0 /cow+chicken

  $ cp /etc/termcap /cow+chicken

  # umount /dev/cow/0
  # cowdev -d /dev/cow/0
  # rmmod cloop
  # umount /dev/cdrom
After the command mount has been issued for cowdevice /dev/cow/0, the situation is as shown in this figure:

cowdemocd schematics

Obviously the contents of the compressed ext2-filesystem on cdrom is unmodified; the modified filesystem-blocks are stored in the /tmp/chicken.cow file. Later on the first four commands can be issued again to continue with this modified filesystem and make additional changes.

Notice that a filesystem-type is preferred that does not use journaling (e.g. ext2 instead of ext3) because the additional write-requests for journaling would unnecessarily populate the cowfile). Also, mount-options such as -o noatime are useful for the same reason.


The described sequence can be used to generate Linux LiveCD's to run Linux stand-alone from a cdrom without modifying the hard disk. In that case the cowfile could be located in memory by allocating it on a RAM-disk or on a USB memory-stick.

Example 3: mount multiple instances of one filesystem

The device /dev/sda2 can be formatted as a filesystem and populated with lots of data:

  # mkfs   /dev/sda2
  # mount  /dev/sda2 /mnt/myfs

  # cp .... lotsofdata .... /mnt/myfs

  # umount /dev/sda2
This filesystem can now be mounted several times via the cowloop driver to obtain multiple writable instances of the same filesystem:
  # modprobe cowloop
  # cowdev -a /dev/sda2 /fsdir/inst0.cow
  /dev/cow/0     # the command returns this name now
  # cowdev -a /dev/sda2 /fsdir/inst1.cow
  /dev/cow/1     # the command returns this name now
  # cowdev -a /dev/sda2 /fsdir/inst2.cow
  /dev/cow/2     # the command returns this name now
  # cowdev -l
  cowdevice     read-only file     copy-on-write file              
  ---------------------------------------------------
  /dev/cow/0    /dev/sda2          /fsdir/inst0.cow                
  /dev/cow/1    /dev/sda2          /fsdir/inst1.cow                
  /dev/cow/2    /dev/sda2          /fsdir/inst2.cow                

  # mount /dev/cow/0 /mnt/fs0
  # mount /dev/cow/1 /mnt/fs1
  # mount /dev/cow/2 /mnt/fs2
  # df
  Filesystem   1K-blocks      Used Available Use% Mounted on
  ....
  /dev/cow/0     2016044   1057832    855800  56% /mnt/fs0
  /dev/cow/1     2016044   1057832    855800  56% /mnt/fs1
  /dev/cow/2     2016044   1057832    855800  56% /mnt/fs2
Every instance can be modified independent of the other instances, while the underlying filesystem on device /dev/sda2 will not be modified at all.

Cowloop versus device mapper

A standard facility in modern Linux kernels is the so called "device mapper". This paragraph briefly compares its pros and cons with respect to cowloop.

Manual page 'dmsetup(8)' describes its basic use, but a comparison with cowloop requires the so-called 'snapshot' target type, which is as of yet not documented on that page. The file 'Documentation/dm/snapshot.txt' has slightly more information. This is how to do it (the italic parts are fixed, the roman parts are choices to be filled in by yourself):

  # dmsetup create cowdevname 0 size snapshot \
	lowerdevice cowfiledevice p chunksize
The cowdevname is the name you choose for the newly to-be-created block device, comparable to our cow-blockdevice (in our examples the /dev/cow/0 etc.). Its /dev name will be created automatically, and it provides you with a linear range of blocks, starting with block number 0 and size blocks large.

The lowerdevice is the block device to be shielded from write accesses, comparable to our 'rdofile'. The cowfiledevice is the place where written block data will be diverted to, comparable to our 'cowfile'. The device mapper requires both to be actual block device names, where cowloop has the possibility to use regular files for either one.

The p parameter is for 'persistent': the content of the cowfiledevice will survive a reboot. The alternative for this parameter is n, which will keep more information in memory: less overhead, but the information is gone after a reboot.

Finally, the chunksize parameter determines the size (in 512-byte units) of the copied-on-write chunks. The documentation uses 64 as a sample value.

Apart from the fact that this 'device mapper' is a standard 2.6 kernel facility in most mainstream distributions, the big advantage is that device mapper is significantly more efficient than cowloop is. The reason for this, but also the price you pay for this, is that device mapper requires the cowfiledevice (where the diverted writes are stored) to be a block device instead of a regular file. Obviously, it is much easier to reroute a block driver request from one driver to another one, than it is to convert a block driver read/write request into a file read/write operation. But block devices are very coarse resources in a Linux system, whereas (large amounts of) regular files are cheap and easy. In our opinion, this point alone justifies the existence of cowloop besides the device mapper facility.

Cowloop allows the cowfile to be merged back directly into the original rdofile, with the help of the cowmerge-command. The device mapper requires an intermediate copy of the entire block device to be made when this facility is required.

For the sake of completeness: the problem with write calls being blocked by higher kernel layers, if the file system concerned does not have a write routine on board (as described in our FAQ) still exists with the device mapper as it does with cowloop.

spacerIntroduction

spacerOverview

spacerInstall

spacerLoad & unload

spacerCowfiles

spacerCommands

spacerStatus info

spacerExample 1

spacerExample 2

spacerExample 3

spacerCowloop vs.
   device mapper

small @ logoView as single page


© AT Consultancy bv, Nijmegen, The Netherlands  -  www.ATConsultancy.nl
© AT Computing bv, Nijmegen, The Netherlands  -  www.ATComputing.nl - www.ATComputing.net