
#include "bacula_template.inc" title="Backing Up to Disk Volumes" num=12

<h1>Backing Up to Disk Volumes</h1>
If you have a lot of hard disk storage or you absolutely must have
your backups run within a small time window, you may want to 
direct Bacula to backup to disk Volumes rather than tape Volumes.
This chapter is intended to give you some of the options that
are available to you so that you can manage those disk volumes.

<a name="Concepts"></a>
<h2>Key Concepts and Resource Records</h2>
Getting Bacula to write to disk rather than tape in the simplest
case is rather easy. 
In the Storage daemon's configuration file, you simply define an
<b>Archive Device</b> to be a directory. For example, if you want your
disk backups to go into the directory <b>/home/bacula/backups</b>, you
could use the following:
<pre>
Device {
  Name = FileBackup
  Media Type = File
  Archive Device = /home/bacula/backups
  Random Access = Yes;
  AutomaticMount = yes;
  RemovableMedia = no;
  AlwaysOpen = no;
}
</pre>
Assuming you have the appropriate <b>Storage</b> resource in your
Director's configuration file that references the above Device resource,
<pre>
Storage {
  Name = FileStorage
  Address = ...
  Password = ...
  Device = FileBackup
  Media Type = File
}
</pre>
Bacula will then write the archive to the file
<b>/home/bacula/backups/&lt;volume-name&gt;</b> where
&lt;volume-name&gt; is the volume name of a Volume defined in the Pool.
For example, if you have labeled a Volume named <b>Vol001</b>, Bacula will
write to the file <b>/home/bacula/backups/Vol001</b>.  Although
you can later move the archive file to another directory, you should
not rename it or it will become unreadable by Bacula. This  
is because each archive has the filename as part of the internal label, 
and the internal label must agree with the system filename before
Bacula will use it.
<p>Although this is quite simple, there are a number of
problems, the first is that unless you specify otherwise, 
Bacula will always write to the same volume until you 
run out of disk space.
<p>
<h3>Pool Options to Limit the Volume Usage</h3>
Some of the options you have, all of which are specified in
the Pool record, are:
<ul>
  <li>To write each Volume only once (i.e. one Job per Volume or file
      in this case), use:<p>
      <b>UseVolumeOnce = yes</b>. </li>
  <li>To write nnn Jobs to each Volume, use:<p>
      <b>Maximum Volume Jobs = nnn</b>.</li>
  <li>To limit the maximum size of each Volume, use:<p>
      <b>Maximum Volume Bytes = mmmm</b>.</li>
  <li>To limit the use time (i.e. write the Volume for
      a maximum of 5 days), use:<p>
      <b>Volume Use Duration = ttt</b>.</li>
</ul>

As mentioned above, each of those directives are specified in the
Pool or Pools that you use for your Volumes. In the case of
<b>Maximum Job Volumes</b>, <b>Maximum Volume Bytes</b>, and
<b>Volume Use Duration</b>, you can actually specify the desired
value on a Volume by Volume basis. The value specified in the
Pool record becomes the default when labeling new Volumes.
As an example of the use of one of the above, suppose your
Pool resource contains:
<pre>
Pool {
  Name = File
  Pool Type = Backup
  Volume Use Duration = 23h
}
</pre>
then if you run a backup once a day (every 24 hours), Bacula
will use a new Volume each backup because each Volume it writes
can only be used for 23 hours after the first write.

<a name="AutomaticLabeling"></a>
<h3>Automatic Volume Labeling</h3>
Use of the above records brings up another problem -- that of
labeling your Volumes. For automated disk backup, you can either
manually label each of your Volumes, or you can have Bacula automatically
label new Volumes when they are needed. While, the automatic
Volume labeling in version 1.30 and prior
is a bit simplistic, but it does allow for 
automation, the features added in version 1.31 permit
automatic creation of a wide variety of labels including
information from environment variables and special Bacula
Counter variables.
<p>
Please note that automatic Volume can also be used with 
tapes, but it is not nearly so practical since the tapes must 
be pre-mounted. This requires some user interaction. Automatic
labeling from templates
does NOT work with autochangers since Bacula will 
not access unknown slots. There are several methods of
labeling all volumes in an autochanger magazine. For more
information on this, please see the <a href="autochangers.html">
Autochanger</a> chapter of this manual.
<p>
Automatic Volume labeling is enabled by making a change to both the Pool
resource (Director) and to the Device resource (Storage daemon) shown
above.  In the case of the Pool resource, you must provide Bacula with a
label format that it will use to create new names.  In the simplest
form, the label format is simply the Volume name, to which Bacula will
append a four digit number.  This number starts at 0001 and is
incremented for each Volume the pool contains.  Thus if you modify your
Pool resource to be:

<pre>
Pool {
  Name = File
  Pool Type = Backup
  Volume Use Duration = 23h
  LabelFormat = &quot;Vol&quot;
}
</pre>
Bacula will create Volume names Vol0001, Vol0002, and so on when
new Volumes are needed.  Much more complex and elaborate labels
can be created using variable expansion defined in the
<a href="vars.html">Variable Expansion</a> chapter of this manual.
<p>
The second change that is necessary to make automatic labeling
work is to give the Storage daemon permission to automatically
label Volumes. Do so by adding <b>LabelMedia = yes</b> to the 
Device resource as follows:
<pre>
Device {
  Name = File
  Media Type = File
  Archive Device = /home/bacula/backups
  Random Access = Yes;
  AutomaticMount = yes;
  RemovableMedia = no;
  AlwaysOpen = no;
  LabelMedia = yes
}
</pre>
You can find more details of the <b>Label Format</b> Pool record
in <a href="dirdconf.html#Label">Label Format</a> description of
the Pool resource records.

<a name="Recycling"></a>
<h3>Restricting the Number of Volumes and Recycling</h3>
Automatic labeling discussed above brings up the problem of Volume
management.  With the above scheme, a new Volume will be created every
day.  If you have not specified Retention periods, your Catalog
will continue to fill keeping track of all the files Bacula has backed
up, and this procedure will create one new archive file (Volume) every day.

<p>The tools Bacula gives you to help automatically manage these
problems are the following:
<ol>
  <li>Catalog file record retention periods, the 
      <a href="dirdconf.html#FileRetention"><b>File Retention = ttt</b></a>
      record in the Client resource.</li>
  <li>Catalog job record retention periods, the 
      <a href="dirdconf.html#JobRetention"><b>Job Retention = ttt</b></a>
      record in the Client resource.</li>
  <li>The <a href="dirdconf.html#AutoPrune">
     <b>AutoPrune = yes</b></a> record in the Client resource to permit
      application of the above two retention periods.</li>
  <li>The <a href="dirdconf.html#VolRetention">
      <b>Volume Retention = ttt</b></a> record in the Pool resource.</li>
  <li>The <a href="dirdconf.html#PoolAutoPrune">
     <b>AutoPrune = yes</b></a> record in the Pool resource to permit
      application of the Volume retention period.</li>
  <li>The <a href="dirdconf.html#PoolRecycle">
     <b>Recycle = yes</b></a> record in the Pool resource to permit
      automatic recycling of Volumes whose Volume retention period has
      expired.</li>
  <li>The <a href="dirdconf.html#RecycleOldest"> <b>Recycle Oldest
      Volume = yes</b></a> record in the Pool resource tells Bacula
      to Prune the oldest volume in the Pool, and if all files
      were pruned to recyle this volume and use it.</li>
  <li>The <a href="dirdconf.html#RecycleCurrent"> <b>Recycle Current
      Volume = yes</b></a> record in the Pool resource tells Bacula
      to Prune the currently mounted volume in the Pool, and if all files
      were pruned to recyle this volume and use it.</li>
  <li>The <a href="dirdconf.html#PurgeOldest">
     <b>Purge Oldest Volume = yes</b></a> record in the Pool resource 
      permits a forced recycling of the oldest Volume when a new one is
      needed. 
      <b>N.B. This record ignores retention periods! We highly recommend
      not to use this record, but instead use Recycle Oldest Volume</b></li>
  <li>The <a href="dirdconf.html#MaxVolumes">
      <b>Maximum Volumes = nnn</b></a> record in the Pool resource to limit
      the number of Volumes that can be created.</li>
</ol>
The first three records (File Retention, Job Retention, and AutoPrune)
determine the amount of time that Job and File records will remain
in your Catalog, and they
are discussed in detail in the <a href="recycling.html">Automatic 
Volume Recycling</a> chapter of this manual.
<p>Volume Retention, AutoPrune, and Recycle determine how long
Bacula will keep your Volumes before reusing them, and they are
also discussed in detail in the <a href="recycling.html">Automatic 
Volume Recycling</a> chapter of this manual.
<p>The Maximum Volumes record can also be used in conjunction with the
Volume Retention period to limit the total number of archive Volumes
(files) that Bacula will create.  By setting an appropriate Volume
Retention period, a Volume will be purged just before it is needed and
thus Bacula can cycle through a fixed set of Volumes.  Cycling through a
fixed set of Volumes can also be done by setting
<b>Recycle Oldest Volume = yes</b> or <b>Recycle Current Volume = yes</b>.
In this case, when Bacula needs
a new Volume, it will prune the specified volume.

<a name="Example"></a>
<h2>An Example</h2>
The following example is not very practical, but can be used
to demonstrate the proof of concept in a relatively short 
period of time. The example consists of a single client that
is backed up to a set of 12 archive files (Volumes). Each
Volume is used (written) only once, and there are
four Full saves done every hour (so the whole thing cycles
around after three hours).

<p>The Director's configuration file is as follows:
<pre>
Director {
  Name = my-dir
  QueryFile = "~/bacula/bin/query.sql"
  PidDirectory = "~/bacula/working"
  WorkingDirectory = "~/bacula/working"
  Password = dir_password
}

Schedule {
  Name = "FourPerHour"
  Run = Level=Full Pool=Recycle Storage=File hourly at 0:05
  Run = Level=Full Pool=Recycle Storage=File hourly at 0:20
  Run = Level=Full Pool=Recycle Storage=File hourly at 0:35
  Run = Level=Full Pool=Recycle Storage=File hourly at 0:50
}

Job {
  Name = "RecycleExample"
  Type = Backup
  Level = Full
  Client = Rufus
  FileSet= "Example FileSet"
  Messages = Standard
  Storage = FileStorage 
  Pool = Recycle
  Schedule = FourPerHour
}

FileSet {
  Name = "Example FileSet"
  Include = compression=GZIP signature=SHA1 { 
    /home/kern/bacula/bin
  }
}

Client {
  Name = Rufus
  Address = rufus
  Catalog = BackupDB
  Password = client_password
}


Storage {
  Name = FileStorage
  Address = rufus
  Password = local_storage_password 
  Device = RecycleDir 
  Media Type = File   
}

Catalog {
  Name = BackupDB
  dbname = bacula; user = bacula; password = ""
}

Messages {
  Name = Standard
  ...
}

Pool {
  Name = Recycle
  Use Volume Once = yes
  Pool Type = Backup
  LabelFormat = &quot;Vol&quot;
  AutoPrune = yes
  VolumeRetention = 2h
  Maximum Volumes = 12
  Recycle = yes
}
</pre>

and the Storage daemon's configuration file is:

<pre>
Storage {
  Name = my-sd    
  WorkingDirectory = "~/bacula/working"
  Pid Directory = "~/bacula/working"
  MaximumConcurrentJobs = 10
}

Director {
  Name = my-dir
  Password = local_storage_password
}

Device {
  Name = RecycleDir
  Media Type = File
  Archive Device = /home/bacula/backups
  LabelMedia = yes;
  Random Access = Yes;
  AutomaticMount = yes; 
  RemovableMedia = no;
  AlwaysOpen = no;
}

Messages {
  Name = Standard
  director = my-dir = all
}
</pre>

In this example, the Jobs will be backed up to directory 
<b>/home/bacula/backups</b> with Volume names Vol0001, Vol0002, ...
Vol0012. Every backup Job will write a new volume cycling    
through the volume numbers, and two hours after a job has
started, the volume will be pruned <b>Volume Retention = 2h</b>.
<p>
With a little bit of work, you can change the above example
into a weekly or monthly cycle (take care about the amount of
archive disk space used).

<a name="MultipleDisks"></a>
<h2>Backing up to Multiple Disks</h2>
Bacula can, of course, use multiple disks, but in
general, each disk must be a separate Device specification
in the Storage daemon's conf file, and you must then
select what clients to backup to each disk.
<p>
The situation is a bit more complicated if you want
to treat two disk drives logically as a single
drive, which Bacula does not directly support.
However, it is possible to back up your data to multiple
disks as if they were a single drive
by linking the Volumes from the first disk to the
second disk.
<p>
For example, assume that you have two disks named
<b>/disk1</b> and <b>/disk2></b>. If you then create a
standard Storage daemon Device resource for backing up
to the first disk, it will look like the following:
<pre>
Device {
  Name = client1
  Media Type = File
  Archive Device = /disk1
  LabelMedia = yes;          
  Random Access = Yes;
  AutomaticMount = yes;      
  RemovableMedia = no;
  AlwaysOpen = no;
}
</pre>
Since there is no way to get the above Device resource
to reference both <b>/disk1</b> and <b>/disk2</b> we
do it by pre-creating Volumes on /disk2 with the following:
<pre>
ln -s /disk2/Disk2-vol001 /disk1/Disk2-vol001
ln -s /disk2/Disk2-vol002 /disk1/Disk2-vol002
ln -s /disk2/Disk2-vol003 /disk1/Disk2-vol003
...
</pre>
At this point, you can label the Volumes as 
Volume <b>Disk2-vol001</b>, <b>Disk2-vol002</b>, ...
and Bacula will use them as if they were on /disk1
but actually write the data to /disk2. The only     
minor inconvenience with this method is that you
must explicitly name the disks and cannot use
automatic labeling unless you arrange to have the
labels exactly match the links you have created.


<a name="MultipleClients"></a>
<h2>Considerations for Multiple Clients</h2>
If we take the above example and add a second Client, here
are a few considerations:
<ul>
  <li>Although the second client can write to the same set of
      Volumes, you will probably want to write to a different
      set. </li>
  <li>You can write to a different set of Volumes by defining
      a second Pool, which has a different name and a different
      <b>LabelFormat</b>.</li>
  <li>If you wish the Volumes for the second client to go into
      a different directory (perhaps even on a different filesystem
      to spread the load), you would do so by defining a second
      Device resource in the Storage daemon. The <b>Name</b>
      must be different, and the <b>Archive Device</b> could
      be different. To ensure that Volumes are never mixed from
      one pool to another, you might also define a different
      MediaType (e.g. <b>File1</b>).</li>
</ul>
                  
In this example, we have two clients, each with a different
Pool and a different number of archive files retained. They
also write to different directories with different Volume
labeling. 

<p>The Director's configuration file is as follows:
<pre>
Director {
  Name = my-dir
  QueryFile = "~/bacula/bin/query.sql"
  PidDirectory = "~/bacula/working"
  WorkingDirectory = "~/bacula/working"
  Password = dir_password
}

\# Basic weekly schedule
Schedule {
  Name = "WeeklySchedule"
  Run = Level=Full fri at 1:30
  Run = Level=Incremental sat-thu at 1:30
}

FileSet {
  Name = "Example FileSet"
  Include = compression=GZIP signature=SHA1 { 
    /home/kern/bacula/bin
  }
}

Job {
  Name = "Backup-client1"
  Type = Backup
  Level = Full
  Client = client1
  FileSet= "Example FileSet"
  Messages = Standard
  Storage = File1
  Pool = client1
  Schedule = "WeeklySchedule"
}

Job {
  Name = "Backup-client2"
  Type = Backup
  Level = Full
  Client = client2
  FileSet= "Example FileSet"
  Messages = Standard
  Storage = File2
  Pool = client2
  Schedule = "WeeklySchedule"
}

Client {
  Name = client1
  Address = client1
  Catalog = BackupDB
  Password = client1_password
  File Retention = 7d    
}

Client {
  Name = client2
  Address = client2
  Catalog = BackupDB
  Password = client2_password
}

\# Two Storage definitions permits different directories
Storage {
  Name = File1
  Address = rufus
  Password = local_storage_password 
  Device = client1                  
  Media Type = File                 
}

Storage {
  Name = File2
  Address = rufus
  Password = local_storage_password 
  Device = client2                  
  Media Type = File                 
}

Catalog {
  Name = BackupDB
  dbname = bacula; user = bacula; password = ""
}

Messages {
  Name = Standard
  ...
}

\# Two pools permits different cycling periods and Volume names
\# Cycle through 15 Volumes (two weeks)
Pool {
  Name = client1
  Use Volume Once = yes
  Pool Type = Backup
  LabelFormat = &quot;Client1-&quot;
  AutoPrune = yes
  VolumeRetention = 13d
  Maximum Volumes = 15
  Recycle = yes
}

\# Cycle through 8 Volumes (1 week)
Pool {
  Name = client2
  Use Volume Once = yes
  Pool Type = Backup
  LabelFormat = &quot;Client2-&quot;
  AutoPrune = yes
  VolumeRetention = 6d
  Maximum Volumes = 8
  Recycle = yes
}

</pre>

and the Storage daemon's configuration file is:

<pre>
Storage {
  Name = my-sd    
  WorkingDirectory = "~/bacula/working"
  Pid Directory = "~/bacula/working"
  MaximumConcurrentJobs = 10
}

Director {
  Name = my-dir
  Password = local_storage_password
}

\# Archive directory for Client1
Device {
  Name = client1
  Media Type = File
  Archive Device = /home/bacula/client1
  LabelMedia = yes;          
  Random Access = Yes;
  AutomaticMount = yes;      
  RemovableMedia = no;
  AlwaysOpen = no;
}

\# Archive directory for Client2
Device {
  Name = client2
  Media Type = File
  Archive Device = /home/bacula/client2
  LabelMedia = yes;          
  Random Access = Yes;
  AutomaticMount = yes;      
  RemovableMedia = no;
  AlwaysOpen = no;
}

Messages {
  Name = Standard
  director = my-dir = all
}
</pre>
 
<page_prev name="Recycling Your Volumes" url="recycling.html">
<page_next name="Using Pools to Manage Disk Volumes" url="pools.html">
