
#include "bacula_template.inc" title="Bacula Catalog Services"


<h1>Catalog Services</h1>

<h2>General</h2>
This chapter is intended to be a technical discussion of the
Catalog services and as such is not targeted at end users but
rather at developers and system administrators that want or
need to know more of the working details of <b>Bacula</b>.

<p>The <b>Bacula Catalog</b> services consist of the programs that  
provide the SQL database engine for storage and retrieval
of all information concerning files that were backed up and
their locations on the storage media.

<p>We have investigated the possibility of using the following SQL
engines for Bacula: Beagle, mSQL, GNU SQL, PostgreSQL, SQLite, Oracle,
and MySQL. Each presents certain problems with either licensing or
maturity.  At present, we have chosen for development purposes to use
MySQL, PostgreSQL and SQLite.  MySQL was chosen because it is fast,
proven to be reliable, widely used, and actively being developed.  MySQL
is released under the GNU GPL license.  PostgreSQL was chosen because it
is a full-featured, very mature database, and because Dan Langille did
the Bacula driver for it.  PostgreSQL is distributed under the BSD
license.  SQLite was chosen because it is small, efficient, and can be
directly embedded in
<b>Bacula</b> thus requiring much less effort from the system
administrator or person building
<b>Bacula</b>. In our testing SQLite has performed very well,
and for the functions that we use, it has never encountered
any errors except that it does not appear to handle databases
larger than 2GBytes.

<p>The Bacula SQL code has been written in a manner
that will allow it to be easily modified to support any
of the current SQL database systems on the market (for example:
mSQL, iODBC, unixODBC, Solid, OpenLink ODBC, 
EasySoft ODBC, InterBase, Oracle8, Oracle7, and DB2).
  
<p>If you do not specify either <b>--with-mysql</b> or 
<b>--with-postgresql</b> or
<b>--with-sqlite</b>
on the ./configure line, Bacula will use
its minimalist internal database. This database
is kept for build reasons but is no longer supported.
Bacula <b>requires</b> one of the three databases (MySQL,
PostgreSQL, or SQLite) to run.</p>

<h3>Filenames and Maximum Filename Length</h3>
In general, either MySQL, PostgreSQL or SQLite permit storing arbitrary long path
names and file names in the catalog database.  In practice, there still may be
one or two places in the Catalog interface code that restrict the
maximum path length to 512 characters and the maximum file name length
to 512 characters.  These restrictions are believed to have been
removed. Please note, these restrictions apply only to the Catalog
database and thus to your ability to list online the files
saved during any job. All information received and stored by the
Storage daemon (normally on tape) allows and handles arbitrarily
long path and filenames.

<h3>Installing and Configuring MySQL</h3>
For the details of installing and configuring MySQL,
please see the <a href="mysql.html">Installing and Configuring MySQL</a>
chapter of this manual.

<h3>Installing and Configuring PostgreSQL</h3>
For the details of installing and configuring PostgreSQL,
please see the <a href="postgresql.html">Installing and Configuring PostgreSQL</a>
chapter of this manual.

<h3>Installing and Configuring SQLite</h3>
For the details of installing and configuring SQLite,
please see the <a href="sqlite.html">Installing and Configuring SQLite</a>
chapter of this manual.

<h3>Internal Bacula Catalog</h3>
Please see the <a href="internaldb.html">Internal Bacula Database</a>
chapter of this manual for more details.

<h3>Database Table Design</h3>
All discussions that follow pertain to the MySQL database.
The details for the PostgreSQL and SQLite databases are essentially identical
except for that all fields in the SQLite database are stored
as ASCII text and some of the database creation statements
are a bit different.
The details of the internal Bacula catalog are not discussed
here.
<p>
Because the Catalog database may contain very large amounts of
data for large sites, we have made a modest attempt to normalize
the data tables to reduce redundant information. While reducing 
the size of the database significantly, it does, unfortunately,
add some complications to the structures.

<p>In simple terms, the Catalog database must contain a record
of all Jobs run by Bacula, and for each Job, it must maintain
a list of all files saved, with their File Attributes (permissions,
create date, ...), and the location and Media on which the file
is stored. This is seemingly a simple task, but it represents
a huge amount interlinked data. Note: the list of files and
their attributes is not maintained when using the internal
Bacula database. The data stored in the File records, which
allows the user or administrator to obtain a list of all
files backed up during a job, is by far the largest volume
of information put into the Catalog database.
<p>Although the Catalog database has been designed to handle
backup data for multiple clients, some users may want to
maintain multiple databases, one for each machine to be
backed up. This reduces the risk of confusion of accidental 
restoring a file to the wrong machine as well as reducing the
amount of data in a single database, thus increasing efficiency
and reducing the impact of a lost or damaged database.

<h2>Sequence of Creation of Records for a Save Job</h2>
Start with StartDate, ClientName, Filename, Path, Attributes,
MediaName, MediaCoordinates. (PartNumber, NumParts). In the
steps below, &quot;Create new&quot; means to create a new
record whether or not it is unique. &quot;Create unique&quot;
means each record in the database should be unique. Thus,
one must first search to see if the record exists, and only
if not should a new one be created, otherwise the existing
RecordId should be used.
<ol type="1">
    <li>Create new Job record with StartDate; save JobId</li>
    <li>Create unique Media record; save MediaId</li>
    <li>Create unique Client record; save ClientId</li>
    <li>Create unique Filename record; save FilenameId</li>
    <li>Create unique Path record; save PathId</li>
    <li>Create unique Attribute record; save AttributeId
       <br>store ClientId, FilenameId, PathId, and Attributes</li>
    <li>Create new File record
       <br>store JobId, AttributeId, MediaCoordinates, etc</li>
    <li>Repeat steps 4 through 8 for each file</li>
    <li>Create a JobMedia record; save MediaId</li>
    <li>Update Job record filling in EndDate and other Job statistics</li>
</ol>

<h2>Database Tables</h2>

<!--  Filename Table -->
<table cellspacing="2" cellpadding="2" border="3">
<tr> <th colspan="3" align=left>Filename</th> </tr>
<tr> <th>Column Name</th>  <th align="left">Data Type</th> <th align="left">Remark</th> </tr>
<tr> <td>FilenameId</td>  <td>integer</td>   <td>Primary Key</td> </tr>
<tr> <td>Name</td>        <td>Blob</td>      <td align=left>Filename</td>  </tr>
</table>
<p>The <b>Filename</b> table shown above contains the name of each file
backed up with the path removed.  If different directories or machines
contain the same filename, only one copy will be saved in this table.
<p>&nbsp;<p>

<!--  Path Table -->
<table cellspacing="2" cellpadding="2" border="3">
<tr> <th colspan="3" align="left">Path</th> </tr>
<tr> <th>Column Name</th> <th>Data Type</th>    <th>Remark</th> </tr>
<tr> <td>PathId</td>      <td>integer</td>      <td align=left>Primary Key</td> </tr>
<tr> <td>Path</td>        <td>Blob</td> <td align=left>Full Path</td>  </tr>
</table>
<p>The <b>Path</b> table contains shown above the path or directory
names of all directories on the system or systems.  The filename and any
MSDOS disk name are stripped off.  As with the filename, only one copy
of each directory name is kept regardless of how many machines or drives
have the same directory.  These path names should be stored in Unix path
name format.

<p>Some simple testing on a Linux file system indicates that separating
the filename and the path may be more complication than is warranted
by the space savings. For example, this system has a total of 89,097 files,
60,467 of which have unique filenames, and there are 4,374 unique paths.
<p>Finding all those files and doing two stats() per
file takes an average wall clock time of 1 min 35 seconds
on a 400MHz machine running RedHat 6.1 Linux.

<p>Finding all those files and putting them directly into a MySQL
database with the path and filename defined as TEXT,
which is variable length up to 65,535 characters takes
19 mins 31 seconds and creates a 27.6 MByte database.

<p>Doing the same thing, but inserting them into Blob
fields with the filename indexed on the first 30 characters
and the path name indexed on the 255 (max) characters takes
5 mins 18 seconds and creates a 5.24 MB database.  Rerunning
the job (with the database already created) takes about
2 mins 50 seconds.

<p>Running the same as the last one (Path and Filename
Blob), but Filename indexed on the first 30 characters
and the Path on the first 50 characters (linear search done there
after) takes 5 mins on the average and creates a 3.4 MB database.
Rerunning with the data already in the DB takes 3 mins 35 seconds.
<p>Finally, saving only the full path name rather than splitting
the path and the file, and indexing it on the first 50 characters
takes 6 mins 43 seconds and creates a 7.35 MB database.
<p></p>
<p>&nbsp;<p></p>

<!--   File Table -->
<table cellspacing="2" cellpadding="2" border="3">
<tr> <th colspan="3" align=left>File</th> </tr>
<tr> <th>Column Name</th>    <th>Data Type</th> <th align=center>Remark</th> </tr>
<tr>  <td>FileId</td>         <td>integer</td>    <td>Primary Key</td> </tr>
<tr> <td>FileIndex</td>      <td>integer</td>   <td>The sequential file number in the Job</td> </tr>
<tr> <td>JobId</td>          <td>integer</td>   <td>Link to Job Record</td></tr>
<tr> <td>PathId</td>         <td>integer</td>   <td>Link to Path Record</td></tr>
<tr> <td>FilenameId</td>     <td>integer</td>   <td>Link to Filename Record</td></tr>
<tr> <td>MarkId</td>         <td>integer</td>   <td>Used to mark files during Verify Jobs</td></tr>
<tr> <td>LStat</td>          <td>tinyblob</td>  <td>File attributes in base64 encoding</td>  </tr>
<tr> <td>MD5</td>            <td>tinyblob</td>  <td>MD5 signature in base64 encoding</td>  </tr>
</table>
<p>The <b>File</b> table shown above contains one entry for each file
backed up by Bacula.  Thus a file that is backed up multiple times (as
is normal) will have multiple entries in the File table.  This will
probably be the table with the most number of records.  Consequently, it
is essential to keep the size of this record to an absolute minimum.  At
the same time, this table must contain all the information (or pointers
to the information) about the file and where it is backed up.  Since a
file may be backed up many times without having changed, the path and
filename are stored in separate tables.
<p>This table contains by far the largest amount of information in 
the Catalog database, both from the stand point of number of records,
and the stand point of total database size. As a consequence, 
the user must take care to periodically reduce the number of File
records using the <b>retention</b> command in the Console program.
<p>&nbsp;<p>

<!--    Job Table  -->
<table cellspacing="2" cellpadding="2" border="3">
<tr> <th colspan="3" align="left">Job</th> </tr>
<tr> <th>Column Name</th>    <th>Data Type</th> <th>Remark</th> </tr>
<tr> <td>JobId</td>          <td>integer</td>   <td>Primary Key</td> </tr>
<tr> <td>Job</td>            <td>tinyblob</td>  <td>Unique Job Name </td>  </tr>
<tr> <td>Name</td>           <td>tinyblob</td>  <td>Job Name </td>  </tr>
<tr> <td>PurgedFiles</td>    <td>tinyint</td>   <td>Used by Bacula for purging/retention periods</td></tr>
<tr> <td>Type</td>           <td>binary(1)</td>      <td>Job Type: Backup, Copy, Clone, Archive, Migration</td></tr>
<tr> <td>Level</td>          <td>binary(1)</td>      <td>Job Level</td></tr>
<tr> <td>ClientId</td>       <td>integer</td>   <td>Client index</td></tr>
<tr> <td>JobStatus</td>      <td>binary(1)</td>      <td>Job Termination Status</td></tr>
<tr> <td>SchedTime</td>      <td>datetime</td>  <td>Time/date when Job scheduled</td>  </tr>
<tr> <td>StartTime</td>      <td>datetime</td>  <td>Time/date when Job started</td>  </tr>
<tr> <td>EndTime</td>        <td>datetime</td>  <td>Time/date when Job ended</td>  </tr>
<tr> <td>JobTDate</td>       <td>bigint</td>   <td>Start day in Unix format but 64 bits; 
                                                   used for Retention period.</td></tr>
<tr> <td>VolSessionId</td>   <td>integer</td>   <td>Unique Volume Session ID</td>  </tr>
<tr> <td>VolSessionTime</td> <td>integer</td>   <td>Unique Volume Session Time</td>  </tr>
<tr> <td>JobFiles</td>       <td>integer</td>   <td>Number of files saved in Job</td>  </tr>
<tr> <td>JobBytes</td>       <td>bigint</td>   <td>Number of bytes saved in Job</td>  </tr>
<tr> <td>JobErrors</td>      <td>integer</td>   <td>Number of errors during Job</td>  </tr>
<tr> <td>JobMissingFiles</td> <td>integer</td>  <td>Number of files not saved (not yet used)</td>  </tr>
<tr> <td>PoolId</td>         <td>integer</td>   <td>Link to Pool Record</td>  </tr>
<tr> <td>FileSetId</td>      <td>integer</td>   <td>Link to FileSet Record</td>  </tr>
<tr> <td>PurgedFiles</td>    <td>tiny integer</td>   <td>Set when all File records purged</td>  </tr>
<tr> <td>HasBase</td>      <td>tiny integer</td>   <td>Set when Base Job run</td>  </tr>
</table>
<p>The <b>Job</b> table contains one record for each Job run by Bacula. Thus
normally, there will be one per day per machine added to the database.
Note, the JobId is used to index Job records in the database, and it
often is shown to the user in the Console program. However, care must be
taken with its use as it is not unique from database to database. For example,
the user may have a database for Client data saved on machine Rufus and
another database for Client data saved on machine Roxie. In this case,
the two database will each have JobIds that match those in another
database. For a unique reference to a Job, see Job below.
<p>The Name field of the Job record corresponds to the Name resource
record given in the Director's configuration file. Thus it is a generic
name, and it will be normal to find many Jobs (or even all Jobs) with
the same Name.
<p>The Job field contains a combination of the Name and the schedule 
time of the Job by the Director. Thus for a given Director, even
with multiple Catalog databases, the Job will contain a unique name that
represents the Job.
<p>For a given Storage daemon, the VolSessionId and VolSessionTime
form a unique identification of the Job. This will be the case
even if multiple Directors are using the same Storage daemon.
<p>The Job Type (or simply Type) can have one of the following
values:
<table cellspacing="2" cellpadding="2" border="3">
<tr> <th>Value</th>    <th>Meaning</th> </tr>
<tr> <td>B</td>      <td>Backup Job</td> </tr>
<tr> <td>V</td>      <td>Verify Job</td> </tr>
<tr> <td>R</td>      <td>Restore Job</td> </tr>
<tr> <td>C</td>      <td>Console program (not in database)</td> </tr>
<tr> <td>D</td>      <td>Admin Job</td> </tr>
<tr> <td>A</td>      <td>Archive Job (not implemented)</td> </tr>
</table>
<p>
The JobStatus field specifies how the job terminated, and can
be one of the following:
<table cellspacing="2" cellpadding="2" border="3">
<tr> <th>Value</th>    <th>Meaning</th> </tr>
<tr> <td>C</td>      <td>Created but not yet running</td> </tr>
<tr> <td>R</td>      <td>Running</td> </tr>
<tr> <td>B</td>      <td>Blocked</td> </tr>
<tr> <td>T</td>      <td>Terminated normally</td> </tr>
<tr> <td>E</td>      <td>Terminated in Error</td> </tr>
<tr> <td>e</td>      <td>Non-fatal error</td> </tr>
<tr> <td>f</td>      <td>Fatal error</td> </tr>
<tr> <td>D</td>      <td>Verify Differences</td> </tr>
<tr> <td>A</td>      <td>Canceled by the user</td> </tr>
<tr> <td>F</td>      <td>Waiting on the File daemon</td> </tr>
<tr> <td>S</td>      <td>Waiting on the Storage daemon</td> </tr>
<tr> <td>m</td>      <td>Waiting for a new Volume to be mounted</td> </tr>
<tr> <td>M</td>      <td>Waiting for a Mount</td> </tr>
<tr> <td>s</td>      <td>Waiting for Storage resource</td> </tr>
<tr> <td>j</td>      <td>Waiting for Job resource</td> </tr>
<tr> <td>c</td>      <td>Waiting for Client resource</td> </tr>
<tr> <td>d</td>      <td>Wating for Maximum jobs</td> </tr>
<tr> <td>t</td>      <td>Waiting for Start Time</td> </tr>
<tr> <td>p</td>      <td>Waiting for higher priority job to finish</td> </tr>
</table>

<p>&nbsp;<p>


<!--   FileSet Table -->
<table cellspacing="2" cellpadding="2" border="3">
<tr> <th colspan="3" align="left">FileSet</th> </tr>
<tr> <th>Column Name</th>    <th>Data Type&nbsp;&nbsp;&nbsp;</th>    <th>Remark</th> </tr>
<tr> <td>FileSetId</td>      <td>integer</td>      <td>Primary Key</td> </tr>
<tr> <td>FileSet</td>        <td>tinyblob</td>   <td>FileSet name</td>  </tr>
<tr> <td>MD5</td>         <td>tinyblob</td>      <td>MD5 checksum of FileSet</td>  </tr>
<tr> <td>CreateTime</td>    <td>datetime</td>   <td>Time and date Fileset created</td>  </tr>
</table>
<p>The <b>FileSet</b> table contains one entry for each FileSet that is used.
The MD5 signature is kept to ensure that if the user changes anything inside
the FileSet, it will be detected and the new FileSet will be used. This
is particularly important when doing an incremental update. If the user deletes
a file or adds a file, we need to ensure that a Full backup is done prior
to the next incremental.
<p>&nbsp;<p>


<!--   JobMedia Table -->
<table cellspacing="2" cellpadding="2" border="3">
<tr> <th colspan="3" align="left">JobMedia</th> </tr>
<tr> <th>Column Name</th>   <th>Data Type&nbsp;&nbsp;&nbsp;</th>    <th>Remark</th> </tr>
<tr> <td>JobMediaId</td>    <td>integer</td>      <td>Primary Key</td> </tr>
<tr> <td>JobId</td>         <td>integer</td>      <td>Link to Job Record</td>  </tr>
<tr> <td>MediaId</td>       <td>integer</td>      <td>Link to Media Record</td>  </tr>
<tr> <td>FirstIndex</td>    <td>integer</td>     <td>The index (sequence number)
          of the first file written for this Job to the Media</td>  </tr>
<tr> <td>LastIndex</td>     <td>integer</td>     <td>The index
          of the last file written for this Job to the Media</td>  </tr>
<tr> <td>StartFile</td>     <td>integer</td>      <td>The physical media (tape)
          file number of the first block written for this Job</td></tr>
<tr> <td>EndFile</td>       <td>integer</td>      <td>The physical media (tape)
          file number of the last block written for this Job</td></tr>
<tr> <td>StartBlock</td>     <td>integer</td>      <td>The number of the first
          block written for this Job</td></tr>
<tr> <td>EndBlock</td>      <td>integer</td>      <td>The number of the last
          block written for this Job</td></tr>
<tr> <td>VolIndex</td>      <td>integer</td>      <td>The Volume use sequence number
          within the Job</td></tr>

</table>
<p>The <b>JobMedia</b> table contains one entry for each volume written for
the current Job. If the Job spans 3 tapes, there will be three JobMedia
records, each containing the information to find all the files for the
given JobId on the tape.
<p>&nbsp;<p>

<!--   Media Table   -->
<table cellspacing="2" cellpadding="2" border="3">
<tr> <th colspan="3" align="left">Media</th> </tr>
<tr> <th>Column Name</th>    <th>Data Type&nbsp;&nbsp;&nbsp;</th>    <th>Remark</th> </tr>
<tr> <td>MediaId</td>        <td>integer</td>  <td>Primary Key</td> </tr>
<tr> <td>VolumeName</td>     <td>tinyblob</td> <td>Volume name</td>  </tr>
<tr> <td>Slot</td>           <td>integer</td>  <td>Autochanger Slot number or zero</td>  </tr>
<tr> <td>PoolId</td>         <td>integer</td>  <td>Link to Pool Record</td>  </tr>
<tr> <td>MediaType</td>      <td>tinyblob</td>  <td>The MediaType supplied by the user</td>  </tr>
<tr> <td>FirstWritten</td>   <td>datetime</td>  <td>Time/date when first written</td>  </tr>
<tr> <td>LastWritten</td>    <td>datetime</td>  <td>Time/date when last written</td>  </tr>
<tr> <td>LabelDate</td>      <td>datetime</td>  <td>Time/date when tape labeled</td>  </tr>
<tr> <td>VolJobs</td>        <td>integer</td>   <td>Number of jobs written to this media</td>  </tr>
<tr> <td>VolFiles</td>       <td>integer</td>   <td>Number of files written to this media</td>  </tr>
<tr> <td>VolBlocks</td>      <td>integer</td>   <td>Number of blocks written to this media</td>  </tr>
<tr> <td>VolMounts</td>      <td>integer</td>   <td>Number of time media mounted</td>  </tr>
<tr> <td>VolBytes</td>       <td>bigint</td>   <td>Number of bytes saved in Job</td>  </tr>
<tr> <td>VolErrors</td>      <td>integer</td>   <td>Number of errors during Job</td>  </tr>
<tr> <td>VolWrites</td>      <td>integer</td>   <td>Number of writes to media</td>  </tr>
<tr> <td>MaxVolBytes</td>     <td>bigint</td>  <td>Maximum bytes to put on this media</td>  </tr>
<tr> <td>VolCapacityBytes</td> <td>bigint</td> <td>Capacity estimate for this volume</td>  </tr>
<tr> <td>VolStatus</td>         <td>enum</td>    <td>Status of media: Full, Archive, Append, Recycle,
       Read-Only, Disabled, Error, Busy</td>  </tr>
<tr> <td>Recycle</td>         <td>tinyint</td>    <td>Whether or not Bacula can recycle the Volumes: Yes, No</td></tr>
<tr> <td>VolRetention</td>     <td>bigint</td>    <td>64 bit seconds until expiration</td></tr>
<tr> <td>VolUseDuration</td>   <td>bigint</td>    <td>64 bit seconds volume can be used</td></tr>
<tr> <td>MaxVolJobs</td>     <td>integer</td>    <td>maximum jobs to put on Volume</td></tr>
<tr> <td>MaxVolFiles</td>     <td>integer</td>    <td>maximume EOF marks to put on Volume</td></tr>
</table>
<p>The <b>Volume</b> table (internally referred to as the Media table)
contains one entry for each volume, that is each tape, cassette (8mm,
DLT, DAT, ...), or file on which information is or was backed up.  There
is one Volume record created for each of the NumVols specified in the
Pool resource record.
<p>&nbsp;<p>

<!--   Pool Table   -->
<table cellspacing="2" cellpadding="2" border="3">
<tr> <th colspan="3" align="left">Pool</th> </tr>
<tr> <th>Column Name</th>     <th>Data Type</th> <th>Remark</th> </tr>
<tr> <td>PoolId</td>          <td>integer</td>   <td>Primary Key</td> </tr>
<tr> <td>Name</td>            <td>Tinyblob</td>  <td>Pool Name</td>  </tr>
<tr> <td>NumVols</td>         <td>Integer</td>   <td>Number of Volumes in the Pool</td>  </tr>
<tr> <td>MaxVols</td>         <td>Integer</td>   <td>Maximum Volumes in the Pool</td>  </tr>
<tr> <td>UseOnce</td>         <td>tinyint</td>   <td>Use volume once</td>  </tr>
<tr> <td>UseCatalog</td>      <td>tinyint</td>   <td>Set to use catalog</td>  </tr>
<tr> <td>AcceptAnyVolume</td> <td>tinyint</td>   <td>Accept any volume from Pool</td>  </tr>
<tr> <td>VolRetention</td>   <td>bigint</td>   <td>64 bit seconds to retain volume</td>  </tr>
<tr> <td>VolUseDuration</td>   <td>bigint</td>   <td>64 bit seconds volume can be used</td>  </tr>
<tr> <td>MaxVolJobs</td>   <td>integer</td>   <td>max jobs on volume</td>  </tr>
<tr> <td>MaxVolFiles</td>   <td>integer</td>   <td>max EOF marks to put on Volume </td>  </tr>
<tr> <td>MaxVolBytes</td>   <td>bigint</td>   <td>max bytes to write on Volume</td>  </tr>
<tr> <td>AutoPrune</td>      <td>tinyint</td>   <td>Yes/no for autopruning</td>  </tr>
<tr> <td>Recycle</td>         <td>tinyint</td>   <td>Yes/no for allowing auto recycling of Volume</td>  </tr>
<tr> <td>PoolType</td>        <td>enum</td>      <td>Backup, Copy, Cloned, Archive, Migration</td>  </tr>
<tr> <td>LabelFormat</td>     <td>Tinyblob</td>  <td>Label format</td>  </tr>
</table>
<p>The <b>Pool</b> table contains one entry for each media pool
controlled by Bacula in this database.  One media record exists for each
of the NumVols contained in the Pool.  The PoolType is a Bacula defined
keyword.  The MediaType is defined by the administrator, and corresponds
to the MediaType specified in the Director's Storage definition record.
The CurrentVol is the sequence number of the Media record for the
current volume.
<p>&nbsp;<p></p>

<!--    Client Table   -->
<table cellspacing="2" cellpadding="2" border="3">
<tr> <th colspan="3" align="left">Client</th> </tr>
<tr> <th>Column Name</th>  <th>Data Type</th>    <th>Remark</th> </tr>
<tr> <td>ClientId</td>     <td>integer</td>      <td>Primary Key</td> </tr>
<tr> <td>Name</td>         <td>TinyBlob</td>     <td>File Services Name</td>  </tr>
<tr> <td>UName</td>        <td>TinyBlob</td>      <td>uname -a from Client (not yet used)</td>  </tr>
<tr> <td>AutoPrune</td>      <td>tinyint</td>   <td>Yes/no for autopruning</td>  </tr>
<tr> <td>FileRetention</td>   <td>bigint</td>   <td>64 bit seconds to retain Files</td>  </tr>
<tr> <td>JobRetention</td>   <td>bigint</td>   <td>64 bit seconds to retain Job</td>  </tr>
</table>
<p>The <b>Client</b> table contains one entry for each machine backed up
by Bacula in this database. Normally the Name is a fully qualified domain name.
<p>&nbsp;<p></p>

<!--    Unsaved Files Table   -->
<table cellspacing="2" cellpadding="2" border="3">
<tr> <th colspan="3" align="left">UnsavedFiles</th> </tr>
<tr> <th>Column Name</th>  <th>Data Type</th>    <th>Remark</th> </tr>
<tr> <td>UnsavedId</td>     <td>integer</td>      <td>Primary Key</td> </tr>
<tr> <td>JobId</td>         <td>integer</td>     <td>JobId corresponding to this record</td>  </tr>
<tr> <td>PathId</td>        <td>integer</td>      <td>Id of path</td>  </tr>
<tr> <td>FilenameId</td>      <td>integer</td>   <td>Id of filename</td>  </tr>
</table>
<p>The <b>UnsavedFiles</b> table contains one entry for each file that was
not saved. Note! This record is not yet implemented.
<p>&nbsp;<p></p>


<!--    Counter Table   -->
<table cellspacing="2" cellpadding="2" border="3">
<tr> <th colspan="3" align="left">Counter</th> </tr>
<tr> <th>Column Name</th>  <th>Data Type</th>    <th>Remark</th> </tr>
<tr> <td>Counter</td>     <td>tinyblob</td>      <td>Counter name</td> </tr>
<tr> <td>MinValue</td>         <td>integer</td>  <td>Start/Min value for counter</td>  </tr>
<tr> <td>MaxValue</td>         <td>integer</td>  <td>Max value for counter</td>  </tr>
<tr> <td>CurrentValue</td>    <td>integer</td>  <td>Current counter value</td>  </tr>
<tr> <td>WrapCounter</td>      <td>tinyblob</td>   <td>Name of another counter</td>  </tr>
</table>
<p>The <b>Counter</b> table contains one entry for each permanent counter
defined by the user.
<p>&nbsp;<p></p>


<!--    Version Table   -->
<table cellspacing="2" cellpadding="2" border="3">
<tr> <th colspan="3" align="left">Version</th> </tr>
<tr> <th>Column Name</th>  <th>Data Type</th>    <th>Remark</th> </tr>
<tr> <td>VersionId</td>     <td>integer</td>      <td>Primary Key</td> </tr>
</table>
<p>The <b>Version</b> table defines the Bacula database version number.
Bacula checks this number before reading the database to ensure
that it is compatible with the Bacula binary file.
<p>&nbsp;<p></p>

<!--    Base Files Table   -->
<table cellspacing="2" cellpadding="2" border="3">
<tr> <th colspan="3" align="left">BaseFiles</th> </tr>
<tr> <th>Column Name</th>  <th>Data Type</th>    <th>Remark</th> </tr>
<tr> <td>BaseId</td>     <td>integer</td>      <td>Primary Key</td> </tr>
<tr> <td>BaseJobId</td>     <td>integer</td>      <td>JobId of Base Job</td> </tr>
<tr> <td>JobId</td>     <td>integer</td>      <td>Reference to Job</td> </tr>
<tr> <td>FileId</td>     <td>integer</td>      <td>Reference to File</td> </tr>
<tr> <td>FileIndex</td>     <td>integer</td>      <td>File Index number</td> </tr>
</table>
<p>The <b>BaseFiles</b> table contains all the File references
for a particular JobId that point to a Base file -- i.e. they
were previously saved and hence were not saved in the current
JobId but in BaseJobId under FileId. FileIndex is the index
of the file, and is used for optimization of Restore jobs to
prevent the need to read the FileId record when creating the
in memory tree.  This record is not yet implemented.
<p>&nbsp;<p></p>


<h3>MySQL Table Definition</h3>
The commands used to create the MySQL tables are as follows:
<pre>
USE bacula;
CREATE TABLE Filename (
  FilenameId INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
  Name BLOB NOT NULL,
  PRIMARY KEY(FilenameId),
  INDEX (Name(30))
  );

CREATE TABLE Path (
   PathId INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
   Path BLOB NOT NULL,
   PRIMARY KEY(PathId),
   INDEX (Path(50))
   );


CREATE TABLE File (
   FileId INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
   FileIndex INTEGER UNSIGNED NOT NULL DEFAULT 0,
   JobId INTEGER UNSIGNED NOT NULL REFERENCES Job,
   PathId INTEGER UNSIGNED NOT NULL REFERENCES Path,
   FilenameId INTEGER UNSIGNED NOT NULL REFERENCES Filename,
   MarkId INTEGER UNSIGNED NOT NULL DEFAULT 0,
   LStat TINYBLOB NOT NULL,
   MD5 TINYBLOB NOT NULL,
   PRIMARY KEY(FileId),
   INDEX (JobId),
   INDEX (PathId),
   INDEX (FilenameId)
   );


CREATE TABLE Job (
   JobId INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
   Job TINYBLOB NOT NULL,
   Name TINYBLOB NOT NULL,
   Type BINARY(1) NOT NULL,
   Level BINARY(1) NOT NULL,
   ClientId INTEGER NOT NULL REFERENCES Client,
   JobStatus BINARY(1) NOT NULL,
   SchedTime DATETIME NOT NULL,
   StartTime DATETIME NOT NULL,
   EndTime DATETIME NOT NULL,
   JobTDate BIGINT UNSIGNED NOT NULL,
   VolSessionId INTEGER UNSIGNED NOT NULL DEFAULT 0,
   VolSessionTime INTEGER UNSIGNED NOT NULL DEFAULT 0,
   JobFiles INTEGER UNSIGNED NOT NULL DEFAULT 0,
   JobBytes BIGINT UNSIGNED NOT NULL,
   JobErrors INTEGER UNSIGNED NOT NULL DEFAULT 0,
   JobMissingFiles INTEGER UNSIGNED NOT NULL DEFAULT 0,
   PoolId INTEGER UNSIGNED NOT NULL REFERENCES Pool,
   FileSetId INTEGER UNSIGNED NOT NULL REFERENCES FileSet,
   PurgedFiles TINYINT NOT NULL DEFAULT 0,
   HasBase TINYINT NOT NULL DEFAULT 0,
   PRIMARY KEY(JobId),
   INDEX (Name(128))
   );

CREATE TABLE FileSet (
   FileSetId INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
   FileSet TINYBLOB NOT NULL,
   MD5 TINYBLOB NOT NULL,
   CreateTime DATETIME NOT NULL,
   PRIMARY KEY(FileSetId)
   );

CREATE TABLE JobMedia (
   JobMediaId INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
   JobId INTEGER UNSIGNED NOT NULL REFERENCES Job,
   MediaId INTEGER UNSIGNED NOT NULL REFERENCES Media,
   FirstIndex INTEGER UNSIGNED NOT NULL DEFAULT 0,
   LastIndex INTEGER UNSIGNED NOT NULL DEFAULT 0,
   StartFile INTEGER UNSIGNED NOT NULL DEFAULT 0,
   EndFile INTEGER UNSIGNED NOT NULL DEFAULT 0,
   StartBlock INTEGER UNSIGNED NOT NULL DEFAULT 0,
   EndBlock INTEGER UNSIGNED NOT NULL DEFAULT 0,
   VolIndex INTEGER UNSIGNED NOT NULL DEFAULT 0,
   PRIMARY KEY(JobMediaId),
   INDEX (JobId, MediaId)
   );


CREATE TABLE Media (
   MediaId INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
   VolumeName TINYBLOB NOT NULL,
   Slot INTEGER NOT NULL DEFAULT 0,
   PoolId INTEGER UNSIGNED NOT NULL REFERENCES Pool,
   MediaType TINYBLOB NOT NULL,
   FirstWritten DATETIME NOT NULL,
   LastWritten DATETIME NOT NULL,
   LabelDate DATETIME NOT NULL,
   VolJobs INTEGER UNSIGNED NOT NULL DEFAULT 0,
   VolFiles INTEGER UNSIGNED NOT NULL DEFAULT 0,
   VolBlocks INTEGER UNSIGNED NOT NULL DEFAULT 0,
   VolMounts INTEGER UNSIGNED NOT NULL DEFAULT 0,
   VolBytes BIGINT UNSIGNED NOT NULL DEFAULT 0,
   VolErrors INTEGER UNSIGNED NOT NULL DEFAULT 0,
   VolWrites INTEGER UNSIGNED NOT NULL DEFAULT 0,
   VolCapacityBytes BIGINT UNSIGNED NOT NULL,
   VolStatus ENUM('Full', 'Archive', 'Append', 'Recycle', 'Purged',
    'Read-Only', 'Disabled', 'Error', 'Busy', 'Used', 'Cleaning') NOT NULL,
   Recycle TINYINT NOT NULL DEFAULT 0,
   VolRetention BIGINT UNSIGNED NOT NULL DEFAULT 0,
   VolUseDuration BIGINT UNSIGNED NOT NULL DEFAULT 0,
   MaxVolJobs INTEGER UNSIGNED NOT NULL DEFAULT 0,
   MaxVolFiles INTEGER UNSIGNED NOT NULL DEFAULT 0,
   MaxVolBytes BIGINT UNSIGNED NOT NULL DEFAULT 0,
   InChanger TINYINT NOT NULL DEFAULT 0,
   MediaAddressing TINYINT NOT NULL DEFAULT 0,
   VolReadTime BIGINT UNSIGNED NOT NULL DEFAULT 0,
   VolWriteTime BIGINT UNSIGNED NOT NULL DEFAULT 0,
   PRIMARY KEY(MediaId),
   INDEX (PoolId)
   );

CREATE TABLE Pool (
   PoolId INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
   Name TINYBLOB NOT NULL,
   NumVols INTEGER UNSIGNED NOT NULL DEFAULT 0,
   MaxVols INTEGER UNSIGNED NOT NULL DEFAULT 0,
   UseOnce TINYINT NOT NULL,
   UseCatalog TINYINT NOT NULL,
   AcceptAnyVolume TINYINT DEFAULT 0,
   VolRetention BIGINT UNSIGNED NOT NULL,
   VolUseDuration BIGINT UNSIGNED NOT NULL,
   MaxVolJobs INTEGER UNSIGNED NOT NULL DEFAULT 0,
   MaxVolFiles INTEGER UNSIGNED NOT NULL DEFAULT 0,
   MaxVolBytes BIGINT UNSIGNED NOT NULL,
   AutoPrune TINYINT DEFAULT 0,
   Recycle TINYINT DEFAULT 0,
   PoolType ENUM('Backup', 'Copy', 'Cloned', 'Archive', 'Migration', 'Scratch') NOT NULL,
   LabelFormat TINYBLOB,
   Enabled TINYINT DEFAULT 1,
   ScratchPoolId INTEGER UNSIGNED DEFAULT 0 REFERENCES Pool,
   RecyclePoolId INTEGER UNSIGNED DEFAULT 0 REFERENCES Pool,
   UNIQUE (Name(128)),
   PRIMARY KEY (PoolId)
   );


CREATE TABLE Client (
   ClientId INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
   Name TINYBLOB NOT NULL,
   Uname TINYBLOB NOT NULL,       /* full uname -a of client */
   AutoPrune TINYINT DEFAULT 0,
   FileRetention BIGINT UNSIGNED NOT NULL,
   JobRetention  BIGINT UNSIGNED NOT NULL,
   UNIQUE (Name(128)),
   PRIMARY KEY(ClientId)
   );

CREATE TABLE BaseFiles (
   BaseId INTEGER UNSIGNED AUTO_INCREMENT,
   BaseJobId INTEGER UNSIGNED NOT NULL REFERENCES Job,
   JobId INTEGER UNSIGNED NOT NULL REFERENCES Job,
   FileId INTEGER UNSIGNED NOT NULL REFERENCES File,
   FileIndex INTEGER UNSIGNED,
   PRIMARY KEY(BaseId)
   );

CREATE TABLE UnsavedFiles (
   UnsavedId INTEGER UNSIGNED AUTO_INCREMENT,
   JobId INTEGER UNSIGNED NOT NULL REFERENCES Job,
   PathId INTEGER UNSIGNED NOT NULL REFERENCES Path,
   FilenameId INTEGER UNSIGNED NOT NULL REFERENCES Filename,
   PRIMARY KEY (UnsavedId)
   );


CREATE TABLE Version (
   VersionId INTEGER UNSIGNED NOT NULL 
   );

-- Initialize Version            
INSERT INTO Version (VersionId) VALUES (7);

CREATE TABLE Counters (
   Counter TINYBLOB NOT NULL,
   MinValue INTEGER,
   MaxValue INTEGER,
   CurrentValue INTEGER,
   WrapCounter TINYBLOB NOT NULL,
   PRIMARY KEY (Counter(128))
   );
</pre>

<page_prev name="Bacula File Services" url="file.html">
<page_next name="Internal Component Designs" url="internals.html">
