Backing Up to Disk Volumes

Bacula 1.34 User's Guide |
Recycling Your Volumes | Index | Using Pools to Manage Disk Volumes
# Backing Up to Disk Volumes If you have a lot of hard disk storage or you absolutely must have your backups run within a small time window, you may want to direct Bacula to backup to disk Volumes rather than tape Volumes. This chapter is intended to give you some of the options that are available to you so that you can manage those disk volumes. ## Key Concepts and Resource Records Getting Bacula to write to disk rather than tape in the simplest case is rather easy. In the Storage daemon's configuration file, you simply define an Archive Device to be a directory. For example, if you want your disk backups to go into the directory /home/bacula/backups, you could use the following:
    Device {
      Name = FileBackup
      Media Type = File
      Archive Device = /home/bacula/backups
      Random Access = Yes;
      AutomaticMount = yes;
      RemovableMedia = no;
      AlwaysOpen = no;
    }
Assuming you have the appropriate Storage resource in your Director's configuration file that references the above Device resource,
    Storage {
      Name = FileStorage
      Address = ...
      Password = ...
      Device = FileBackup
      Media Type = File
    }
Bacula will then write the archive to the file /home/bacula/backups/<volume-name> where <volume-name> is the volume name of a Volume defined in the Pool. For example, if you have labeled a Volume named Vol001, Bacula will write to the file /home/bacula/backups/Vol001. Although you can later move the archive file to another directory, you should not rename it or it will become unreadable by Bacula. This is because each archive has the filename as part of the internal label, and the internal label must agree with the system filename before Bacula will use it. Although this is quite simple, there are a number of problems, the first is that unless you specify otherwise, Bacula will always write to the same volume until you run out of disk space. ### Pool Options to Limit the Volume Usage Some of the options you have, all of which are specified in the Pool record, are: To write each Volume only once (i.e. one Job per Volume or file in this case), use: UseVolumeOnce = yes. To write nnn Jobs to each Volume, use: Maximum Volume Jobs = nnn. To limit the maximum size of each Volume, use: Maximum Volume Bytes = mmmm. To limit the use time (i.e. write the Volume for a maximum of 5 days), use: Volume Use Duration = ttt. As mentioned above, each of those directives are specified in the Pool or Pools that you use for your Volumes. In the case of Maximum Job Volumes, Maximum Volume Bytes, and Volume Use Duration, you can actually specify the desired value on a Volume by Volume basis. The value specified in the Pool record becomes the default when labeling new Volumes. As an example of the use of one of the above, suppose your Pool resource contains:
    Pool {
      Name = File
      Pool Type = Backup
      Volume Use Duration = 23h
    }
then if you run a backup once a day (every 24 hours), Bacula will use a new Volume each backup because each Volume it writes can only be used for 23 hours after the first write. ### Automatic Volume Labeling Use of the above records brings up another problem -- that of labeling your Volumes. For automated disk backup, you can either manually label each of your Volumes, or you can have Bacula automatically label new Volumes when they are needed. While, the automatic Volume labeling in version 1.30 and prior is a bit simplistic, but it does allow for automation, the features added in version 1.31 permit automatic creation of a wide variety of labels including information from environment variables and special Bacula Counter variables. Please note that automatic Volume can also be used with tapes, but it is not nearly so practical since the tapes must be pre-mounted. This requires some user interaction. Automatic labeling from templates does NOT work with autochangers since Bacula will not access unknown slots. There are several methods of labeling all volumes in an autochanger magazine. For more information on this, please see the Autochanger chapter of this manual. Automatic Volume labeling is enabled by making a change to both the Pool resource (Director) and to the Device resource (Storage daemon) shown above. In the case of the Pool resource, you must provide Bacula with a label format that it will use to create new names. In the simplest form, the label format is simply the Volume name, to which Bacula will append a four digit number. This number starts at 0001 and is incremented for each Volume the pool contains. Thus if you modify your Pool resource to be:
    Pool {
      Name = File
      Pool Type = Backup
      Volume Use Duration = 23h
      LabelFormat = "Vol"
    }
Bacula will create Volume names Vol0001, Vol0002, and so on when new Volumes are needed. Much more complex and elaborate labels can be created using variable expansion defined in the Variable Expansion chapter of this manual. The second change that is necessary to make automatic labeling work is to give the Storage daemon permission to automatically label Volumes. Do so by adding LabelMedia = yes to the Device resource as follows:
    Device {
      Name = File
      Media Type = File
      Archive Device = /home/bacula/backups
      Random Access = Yes;
      AutomaticMount = yes;
      RemovableMedia = no;
      AlwaysOpen = no;
      LabelMedia = yes
    }
You can find more details of the Label Format Pool record in Label Format description of the Pool resource records. ### Restricting the Number of Volumes and Recycling Automatic labeling discussed above brings up the problem of Volume management. With the above scheme, a new Volume will be created every day. If you have not specified Retention periods, your Catalog will continue to fill keeping track of all the files Bacula has backed up, and this procedure will create one new archive file (Volume) every day. The tools Bacula gives you to help automatically manage these problems are the following: Catalog file record retention periods, the File Retention = ttt record in the Client resource. Catalog job record retention periods, the Job Retention = ttt record in the Client resource. The AutoPrune = yes record in the Client resource to permit application of the above two retention periods. The Volume Retention = ttt record in the Pool resource. The AutoPrune = yes record in the Pool resource to permit application of the Volume retention period. The Recycle = yes record in the Pool resource to permit automatic recycling of Volumes whose Volume retention period has expired. The Recycle Oldest Volume = yes record in the Pool resource tells Bacula to Prune the oldest volume in the Pool, and if all files were pruned to recyle this volume and use it. The Recycle Current Volume = yes record in the Pool resource tells Bacula to Prune the currently mounted volume in the Pool, and if all files were pruned to recyle this volume and use it. The Purge Oldest Volume = yes record in the Pool resource permits a forced recycling of the oldest Volume when a new one is needed. N.B. This record ignores retention periods! We highly recommend not to use this record, but instead use Recycle Oldest Volume The Maximum Volumes = nnn record in the Pool resource to limit the number of Volumes that can be created. The first three records (File Retention, Job Retention, and AutoPrune) determine the amount of time that Job and File records will remain in your Catalog, and they are discussed in detail in the Automatic Volume Recycling chapter of this manual. Volume Retention, AutoPrune, and Recycle determine how long Bacula will keep your Volumes before reusing them, and they are also discussed in detail in the Automatic Volume Recycling chapter of this manual. The Maximum Volumes record can also be used in conjunction with the Volume Retention period to limit the total number of archive Volumes (files) that Bacula will create. By setting an appropriate Volume Retention period, a Volume will be purged just before it is needed and thus Bacula can cycle through a fixed set of Volumes. Cycling through a fixed set of Volumes can also be done by setting Recycle Oldest Volume = yes or Recycle Current Volume = yes. In this case, when Bacula needs a new Volume, it will prune the specified volume. ## An Example The following example is not very practical, but can be used to demonstrate the proof of concept in a relatively short period of time. The example consists of a single client that is backed up to a set of 12 archive files (Volumes). Each Volume is used (written) only once, and there are four Full saves done every hour (so the whole thing cycles around after three hours). The Director's configuration file is as follows:
    Director {
      Name = my-dir
      QueryFile = "~/bacula/bin/query.sql"
      PidDirectory = "~/bacula/working"
      WorkingDirectory = "~/bacula/working"
      Password = dir_password
    }

    Schedule {
      Name = "FourPerHour"
      Run = Level=Full Pool=Recycle Storage=File hourly at 0:05
      Run = Level=Full Pool=Recycle Storage=File hourly at 0:20
      Run = Level=Full Pool=Recycle Storage=File hourly at 0:35
      Run = Level=Full Pool=Recycle Storage=File hourly at 0:50
    }

    Job {
      Name = "RecycleExample"
      Type = Backup
      Level = Full
      Client = Rufus
      FileSet= "Example FileSet"
      Messages = Standard
      Storage = FileStorage
      Pool = Recycle
      Schedule = FourPerHour
    }

    FileSet {
      Name = "Example FileSet"
      Include = compression=GZIP signature=SHA1 {
        /home/kern/bacula/bin
      }
    }

    Client {
      Name = Rufus
      Address = rufus
      Catalog = BackupDB
      Password = client_password
    }

    Storage {
      Name = FileStorage
      Address = rufus
      Password = local_storage_password
      Device = RecycleDir
      Media Type = File
    }

    Catalog {
      Name = BackupDB
      dbname = bacula; user = bacula; password = ""
    }

    Messages {
      Name = Standard
      ...
    }

    Pool {
      Name = Recycle
      Use Volume Once = yes
      Pool Type = Backup
      LabelFormat = "Vol"
      AutoPrune = yes
      VolumeRetention = 2h
      Maximum Volumes = 12
      Recycle = yes
    }
and the Storage daemon's configuration file is:
    Storage {
      Name = my-sd
      WorkingDirectory = "~/bacula/working"
      Pid Directory = "~/bacula/working"
      MaximumConcurrentJobs = 10
    }

    Director {
      Name = my-dir
      Password = local_storage_password
    }

    Device {
      Name = RecycleDir
      Media Type = File
      Archive Device = /home/bacula/backups
      LabelMedia = yes;
      Random Access = Yes;
      AutomaticMount = yes;
      RemovableMedia = no;
      AlwaysOpen = no;
    }

    Messages {
      Name = Standard
      director = my-dir = all
    }
In this example, the Jobs will be backed up to directory /home/bacula/backups with Volume names Vol0001, Vol0002, ... Vol0012. Every backup Job will write a new volume cycling through the volume numbers, and two hours after a job has started, the volume will be pruned Volume Retention = 2h. With a little bit of work, you can change the above example into a weekly or monthly cycle (take care about the amount of archive disk space used). ## Backing up to Multiple Disks Bacula can, of course, use multiple disks, but in general, each disk must be a separate Device specification in the Storage daemon's conf file, and you must then select what clients to backup to each disk. The situation is a bit more complicated if you want to treat two disk drives logically as a single drive, which Bacula does not directly support. However, it is possible to back up your data to multiple disks as if they were a single drive by linking the Volumes from the first disk to the second disk. For example, assume that you have two disks named /disk1 and /disk2>. If you then create a standard Storage daemon Device resource for backing up to the first disk, it will look like the following:
    Device {
      Name = client1
      Media Type = File
      Archive Device = /disk1
      LabelMedia = yes;
      Random Access = Yes;
      AutomaticMount = yes;
      RemovableMedia = no;
      AlwaysOpen = no;
    }
Since there is no way to get the above Device resource to reference both /disk1 and /disk2 we do it by pre-creating Volumes on /disk2 with the following:
    ln -s /disk2/Disk2-vol001 /disk1/Disk2-vol001
    ln -s /disk2/Disk2-vol002 /disk1/Disk2-vol002
    ln -s /disk2/Disk2-vol003 /disk1/Disk2-vol003
    ...
At this point, you can label the Volumes as Volume Disk2-vol001, Disk2-vol002, ... and Bacula will use them as if they were on /disk1 but actually write the data to /disk2. The only minor inconvenience with this method is that you must explicitly name the disks and cannot use automatic labeling unless you arrange to have the labels exactly match the links you have created. ## Considerations for Multiple Clients If we take the above example and add a second Client, here are a few considerations: Although the second client can write to the same set of Volumes, you will probably want to write to a different set. You can write to a different set of Volumes by defining a second Pool, which has a different name and a different LabelFormat. If you wish the Volumes for the second client to go into a different directory (perhaps even on a different filesystem to spread the load), you would do so by defining a second Device resource in the Storage daemon. The Name must be different, and the Archive Device could be different. To ensure that Volumes are never mixed from one pool to another, you might also define a different MediaType (e.g. File1). In this example, we have two clients, each with a different Pool and a different number of archive files retained. They also write to different directories with different Volume labeling. The Director's configuration file is as follows:
    Director {
      Name = my-dir
      QueryFile = "~/bacula/bin/query.sql"
      PidDirectory = "~/bacula/working"
      WorkingDirectory = "~/bacula/working"
      Password = dir_password
    }

    # Basic weekly schedule
    Schedule {
      Name = "WeeklySchedule"
      Run = Level=Full fri at 1:30
      Run = Level=Incremental sat-thu at 1:30
    }

    FileSet {
      Name = "Example FileSet"
      Include = compression=GZIP signature=SHA1 {
        /home/kern/bacula/bin
      }
    }

    Job {
      Name = "Backup-client1"
      Type = Backup
      Level = Full
      Client = client1
      FileSet= "Example FileSet"
      Messages = Standard
      Storage = File1
      Pool = client1
      Schedule = "WeeklySchedule"
    }

    Job {
      Name = "Backup-client2"
      Type = Backup
      Level = Full
      Client = client2
      FileSet= "Example FileSet"
      Messages = Standard
      Storage = File2
      Pool = client2
      Schedule = "WeeklySchedule"
    }

    Client {
      Name = client1
      Address = client1
      Catalog = BackupDB
      Password = client1_password
      File Retention = 7d
    }

    Client {
      Name = client2
      Address = client2
      Catalog = BackupDB
      Password = client2_password
    }

    # Two Storage definitions permits different directories
    Storage {
      Name = File1
      Address = rufus
      Password = local_storage_password
      Device = client1
      Media Type = File
    }

    Storage {
      Name = File2
      Address = rufus
      Password = local_storage_password
      Device = client2
      Media Type = File
    }

    Catalog {
      Name = BackupDB
      dbname = bacula; user = bacula; password = ""
    }

    Messages {
      Name = Standard
      ...
    }

    # Two pools permits different cycling periods and Volume names
    # Cycle through 15 Volumes (two weeks)
    Pool {
      Name = client1
      Use Volume Once = yes
      Pool Type = Backup
      LabelFormat = "Client1-"
      AutoPrune = yes
      VolumeRetention = 13d
      Maximum Volumes = 15
      Recycle = yes
    }

    # Cycle through 8 Volumes (1 week)
    Pool {
      Name = client2
      Use Volume Once = yes
      Pool Type = Backup
      LabelFormat = "Client2-"
      AutoPrune = yes
      VolumeRetention = 6d
      Maximum Volumes = 8
      Recycle = yes
    }
and the Storage daemon's configuration file is:
    Storage {
      Name = my-sd
      WorkingDirectory = "~/bacula/working"
      Pid Directory = "~/bacula/working"
      MaximumConcurrentJobs = 10
    }

    Director {
      Name = my-dir
      Password = local_storage_password
    }

    # Archive directory for Client1
    Device {
      Name = client1
      Media Type = File
      Archive Device = /home/bacula/client1
      LabelMedia = yes;
      Random Access = Yes;
      AutomaticMount = yes;
      RemovableMedia = no;
      AlwaysOpen = no;
    }

    # Archive directory for Client2
    Device {
      Name = client2
      Media Type = File
      Archive Device = /home/bacula/client2
      LabelMedia = yes;
      Random Access = Yes;
      AutomaticMount = yes;
      RemovableMedia = no;
      AlwaysOpen = no;
    }

    Messages {
      Name = Standard
      director = my-dir = all
    }
Recycling Your Volumes | Index | Using Pools to Manage Disk Volumes
Bacula 1.34 User's Guide The Network Backup Solution | Copyright © 2000-2004 Kern Sibbald and John Walker