Deployment to remote hosts

Samhain
Prev |  | Next

# Chapter 10. Deployment to remote hosts

# 10.1. Method A: The deployment system

samhain includes a system to facilitate deployment of the client to remote hosts. This system enables you to: build and store binary packages for different operating systems, install them, create baseline databases upon installation, update the server configuration, and maintain the client database required by the beltane web-based console.

The system comprises a shell script deploy.sh that will be installed in the same directory as the samhain/yule (by default, `/usr/local/sbin`), and a directory tree that will be installed below the samhain/yule data directory (see Section 10.1.2>). The script and the directory tree can be relocated freely. There is a configuration file `~/.deploy.conf` that is created in the home directory of the user when deploy.sh is run for the first time, where you can specify the default for the top level directory of the system.

| Note
| In the following, an architecture is just a label for some group of hosts, typically a particular operating system (or some particular installation thereof). As long as you know what the label means, you can choose it freely (alphanumeric + underscore). The architecture for a build/install host (i.e. the association between a host and the architecture-specific configuration data) is currently specified via a command-line option.

The system allows to use per-architecture customized build options, as well as per-host customized runtime configuration files.

By default, the system will search for a sufficiently advanced incarnation of dialog to provide a nice user interface. You can switch this off in favour of a plain console interface, if you prefer (or if you want to run the script non-interactively).

To use this system, you must first install it with the command:

    sh$ make install-deploy

| Installation tip
| This system is somewhat tied to the server (yule). While you can safely install it later, installing it together with the server will take care that the defaults are already correct. Upon first invocation a configuration file `~/.deploy.conf` will be written, where you can modify the defaults settings.

| Backward compatibility
| The deployment system has been completely revised in version 2.0 of samhain. It will not work with samhain versions below 2.0 (i.e. you cannot install them using this system). However, the default location and format of the client database (used by the beltane web-based console) has not changed. Installing the new version of the deploy system will not overwrite the old version (deploy.sh will be installed as deploy2.sh, if an old version is detected).

## 10.1.1. Requirements

1. You must have compiled and installed the server (yule) on the local host where you use the deploy system.

2. You must have installed the deployment system by using make deploy-install. This will install the script `deploy.sh` into the `sbindir` (default `/usr/local/sbin`, but depends on your configure options), and the deployment system into `localstatedir/install_name/profiles` (default `/var/lib/yule/profiles`, but depends on your configure options).

   If you already have installed the deprecated version 1 deployment system, the script will be installed as `deploy2.sh`.

3. For each architecture that you define, there must be (at least) one build host where development tools (C compiler, make, strip) are available to build the client executable.

4. On each remote where you want to build or install, you should be able to login as root with ssh using RSA authentication, such that ssh-agent can be used.

| Tip
| To use RSA-based authentication in a secure way, you may proceed as follows: Use ssh-keygen to create a public/private key pair. Don't forget to set a passphrase for the private key (ssh-keygen will ask for it). Copy the public key (`HOME/.ssh/identity.pub` for the ssh protocol version 1, `HOME/.ssh/id_rsa.pub` for ssh protocol version 2) to `HOME/.ssh/authorized_keys` on any remote host where you want to log in. Do not copy the private key `HOME/.ssh/identity` (ssh protocol version 1) or `HOME/.ssh/id_rsa` (ssh protocol version 2) to any untrusted host ! On your central host, execute the commands (use "ssh-agent -c" if you are using a csh-style shell):
    bash$eval `ssh-agent -s`
    bash$ssh-add
You can then ssh/scp without typing the passphrase again, until you exit the current shell.

## 10.1.2. Layout of the deployment system

    (localstatedir)/(install_name)/profiles/
                          |
                          |
                          |-- source ------------> (tarballs)
                          |
                          |-- configs -----------> (default configs)
                          |
                          |-- archpkg
                          |    |
                          |    |-- architecture -> (compiled package, setup script)
                          |
                          |-- hosts
                          |    |
                          |    |-- hostname -----> (optional host-specific config)
                          |
                          |-- libexec -----------> (scripts)
                          |
                          |-- private -----------> (gpg key)
                          |
                          |-- tmp

### 10.1.2.1. The configs subdirectory

The `configs` subdirectory holds for each architecture at least two files (example files will be placed there upon installation of the deployment system):

<architecture>.configure (required)

  The configure options for this architecture; one option per line, each enclosed in single quotes.

  If this file does not exist, it will be copied from `generic.configure`, which is created upon installation, and holds (only) some minimum options.

<architecture>.samhainrc (required)

  The default runtime configuration file for a client running on this architecture. It is possible to override this on installation with a file `hosts/<hostname>/samhainrc`.

<architecture>.preinstall (optional)

  The shell script to run before installing a client. Typically this script would shutdown the running client, if there is one.

  Defaults to `libexec/preinstall`.

<architecture>.postinstall (optional)

  The shell script to run after installing a client. This script receives the client password as first (and only) argument, and typically would set the password in the client binary.

  Defaults to `libexec/postinstall`.

<architecture>.initscript (optional)

  The shell script to initialize/update the baseline database file installing a client.

  Defaults to `libexec/initscript`.

### 10.1.2.2. The archpkg subdirectory

The `archpkg` directory holds for each architecture a subdirectory `archpkg/<architecture>`, where compiled binary installer packages are stored.

For each build, up to four files will be stored: (a) the binary installer package `samhain-<version>.<format>`, (b) the configure options used (`configure-<version>.<format>`), (c) the samhain-install.sh script generated during the build (`install-<version>.<format>`), and (only for packed executables) the client password set in the executable (`PASSWD-<version>.<format>`).

## 10.1.3. Customizing the system

### 10.1.3.1. Setting default options

If you want to change the default options, you can set some of them via a configuration file `~/.deploy.conf`, which is created upon the first invocation of deploy.sh.

### 10.1.3.2. Adding support for an architecture

To add support for another architecture <arch>, just create the two files `<arch>.configure` (configure options) and `<arch>.samhainrc` (runtime configuration) in the `configs` directory of the deployment system (see Section 10.1.2>).

Upon installation of the system, a template file `generic.configure` is created, which contains the minimum options for a client.

### 10.1.3.3. Per-architecture pre-/postinstallation scripts

The default scripts for preinstallation (shutting down the running client) and postinstallation (setting the client password, fixing the local configuration file), and the script for database initialization are located in the `libexec` directory. You can override them for some (or all) architectures by storing architecture-specific files `<arch>.preinstall`, `<arch>.postinstall`, `<arch>.initscrip` in the `configs` directory.

### 10.1.3.4. Per-host runtime configuration

If you want to override the runtime configuration file `configs/<arch>.samhainrc` on a per-host basis, you need to store a host-specific runtime configuration file as `hosts/<hostname>/samhainrc`, before you run deploy.sh install.

## 10.1.4. Using the deploy.sh script

| Tip
| When run for the first time, deploy.sh will create a configuration file `~/.deploy.conf` with some default configuration options. You may want to review this file. Note that you can override all options there with command-line options; the configuration file is just for convenience, if you don't like the defaults and don't want to type the corresponding option on the command line every time.

deploy.sh can be invoked in three ways:

    bash$deploy.sh --help
    This will provide a general overview.
    bash$deploy.sh --help command
    This will provide help on a specific command (where command can be any of:
    'clean', 'download', 'checksrc', 'build', or 'install'.
    bash$deploy.sh [options] command
    This will run 'command' with the specified options.

A log of the last run will be kept in `tmp/logfile.lastrun`

command can be any of the following:

info

  Provides information on installed clients, or available installer packages.

clean

  Removes source tarballs from the `source` subdirectory of the deploy system. Removes unused installer packages from the `archpkg/<arch>` subdirectories of the deploy system.

download

  Download a source tarball from the distribution site, verify the GnuPG signature (gpg must be installed), and install it into the `source` subdirectory of the deploy system. Requires one of: wget, curl, links, lynx, fetch, or lwp-request.

checksrc

  Check the GnuPG signatures of available source tarballs in the `source` subdirectory of the deploy system (gpg must be installed). Optionally delete tarballs with no/invalid signature.

build

  Build a binary installer package for the chosen architecture from one of the tarballs in the `source` subdirectory, and store it in the `archpkg/<architecture>` subdirectory (which will be created if it does not exist yet). Requires a file `<architecture>.configure` and a file `<architecture>.samhainrc` in the `configs` subdirectory.

install

  Copy a pre-built binary package (built with deploy.sh build) to a remote host, stop the client running there (if any), install the (new) client, update the server configuration file and reload the server, initialize the file signature database and fetch it from the remote host.

uninstall

  Remove a samhain client that was previously installed with deploy.sh install.

### 10.1.4.1. General options

-q | --quiet | --quiet=2 Produce output suitable for logging. Note that --quiet=2 implies --yes (see below).

-s | --simulate Print what would be done, but do not actually change the system.

-y | --yes Assume yes as answer to all prompts and run non-interactively.

-o <file> | --logfile=<file> Specify an output file for messages that would go to stdout otherwise. Has no effect on stderr (error messages).

-d <dialog> | --dialog=<dialog> Specify your preferred "dialog" clone (e.g. Xdialog). Use "no" to force plain text.

## 10.1.5. deploy.sh info

This command will show information for hosts in the client database (default), or for available binary installer packages.

### 10.1.5.1. Specific options

--packages Show information for available installer packages rather than for clients.

## 10.1.6. deploy.sh clean

This command will clean unused files: source tarballs in the `source` subdirectory, and unused installer packages in the `archpkg/<arch>` subdirectories.

### 10.1.6.1. Specific options

There are no specific options for this command.

## 10.1.7. deploy.sh download

This command will download a source tarball from the distribution website, verify its GnuPG signature, and install it into the `source` subdirectory. This command requires that either wget or lynx is in your PATH.

| Manual installation of source
| This note applies if you want to download source manually instead. Samhain distribution tarballs contain exactly two files: first, a source tarball with the source code, and second, its GnuPG signature. For installation into the `source` subdirectory, the distribution tarball must be unpacked, and both the source source tarball and its GnuPG signature moved into the `source` subdirectory.

### 10.1.7.1. Specific options

--version=<version> The version of samhain to download. The default is "current" to download the current version.

## 10.1.8. deploy.sh checksrc

This command will check the GnuPG signatures of source tarballs in the `source` subdirectory.

### 10.1.8.1. Specific options

--delete Delete source tarballs if PGP signature cannot be verified.

## 10.1.9. deploy.sh build

This command will create a temporary directory on a remote build host, copy the selected version of the source there, build the selected format of the binary installer package, retrieve and store the package into the `archpkg/<architecture>` subdirectory, and remove the temporary build directory.

For each build, up to four files will be stored: (a) the binary installer package `samhain-<version>.<format>`, (b) the configure options used (`configure-<version>.<format>`), (c) the samhain-install.sh script generated during the build (`install-<version>.<format>`), and (only for packed executables) the client password set in the executable (`PASSWD-<version>.<format>`).

| Package formats
| Note that the build host must provide the required tools if you want to build a package for the native package manager (i.e. deb, rpm, tbz2, depot (HP-UX), or solaris pkg). On RPM-based Linux distributions and Gentoo Linux, building of RPMs and tbz2s, respectively, should just work. Debian requires additional packages for building debs. The "run" binary package format does not require additional tools (it is a self-extracting tar package based on the makeself application, which is included in the samhain distribution). Use /bin/sh <package> --help for details.

### 10.1.9.1. Specific options

--host=<hostname> The build host.

--arch=<arch> The architecture to build for. This is used to get the "./configure" options from the file `configs/<arch>.configure`, and to store the binary package into the directory `archpkg/<arch>`.

--version=<version> The version of samhain you want to build. Must be in the `source` subdirectory.

--format=<run|rpm|deb|tbz2|depot|solaris-pkg> The format of the binary installer package. "run" is a portable (Unix) package based on makeself, "deb" is a Debian package, "tbz2" is a binary Gentoo Linux package, "rpm" is an RPM package, "depot" is an HP-UX binary package, and "solaris-pkg" for Sun Solaris.

--packed=<password> Build a packed executable, and set the client password before packing.

--user=<username> Login as <username> on the build host (defaults to root).

--add-path=<path> Append <path> to the PATH variable on the build host.

## 10.1.10. deploy.sh install

This command will create a temporary directory on a remote host, copy the selected version of the installer package, its corresponding samhain-install.sh script, the runtime configuration file, and the preinstall, postinstall, initscripts scripts there. It will then:

(A) run the `preinstall` script on the client, which shuts down the running samhain daemon (if there is any).

(B) install the binary installer package on the client.

(C) run the `postinstall` script on the client, which sets the client password (unless the binary is packed), and replaces the default runtime configuration file with the proper one. The latter step is required, because deploy.sh build builds from the pristine source, so the runtime configuration file in the installer package is just the default one.

(D) copy the proper client runtime configuration file to the server data directory (as `rc.<client_name>`), fix the server configuration file, and restart the server (which will fail non-fatally if the server is not running).

(E) run the `initscript` script on the client, which initializes (or updates) the baseline database.

(F) retrieve the baseline database, copy it to the server data directory (as `file.<client_name>`), and remove the temporary directory on the client.

| The runtime configuration file
| If `hosts/<hostname>/<arch>.samhainrc` or `hosts/<hostname>/samhainrc` exists, this will be used (in this order of preference), otherwise `configs/<arch>.samhainrc` will be used. If the latter does not exist, the command will fail.

| Transparent handling of particular build options
| The build options '--enable-stealth=..' is handled by determining the argument from the configure options that were used for the build, and preparing the runtime configuration file appropriately. I.e., you should provide a 'normal', plain-text configuration file. The build option '--with-nocl=..' is handled by determining the argument (which is required for database initialization) from the configure options that were used for the build, and passing it to the `initscript` script.

### 10.1.10.1. Specific options

--host=<hostname> The host on which to install.

--group=<foobar> The group to which you want to assign that client (default: none). This is used by the beltane web console to restrict access to users which are members of that group.

--arch=<arch> The architecture to install. This is used to get the installer package from the directory `archpkg/<arch>/`.

--version=<version> The version of samhain you want to install. An installer package for this version must exist in the `archpkg/<arch>/` subdirectory.

--format=<run|rpm|deb|tbz2|depot|solaris-pkg> The format of the binary installer package. "run" is a portable (Unix) package based on makeself, "deb" is a Debian package, "tbz2" is a binary Gentoo Linux package, "rpm" is an RPM package, "depot" is an HP-UX binary package, and "solaris-pkg" for Sun Solaris.

--yule_exec=<path> Path to the yule executable.

--yule_conf=<path> Path to the yule configuration file.

--yule_data=<path> Path to the yule data directory.

--no-init Do not initialize the file signature (baseline) database (and consequentially, do not replace the `file.<host>` file on server.

--no-rcfile Do not replace the `rc.<host>` file on server.

--no-start Do not start up the client after installation.

--local=<command> An optional command executed locally (i.e. on the server) twice (with the last argument set to 'first' and 'second', respectively. First is after client config file installation (i.e. before baseline database initialisation on the client), second is just before client startup. Will be called as command `hostname` `arch` `basedir` `yule_data` `first|second`.

## 10.1.11. deploy.sh uninstall

This command will remove a samhain client that was previously installed by using deploy.sh install.

### 10.1.11.1. Specific options

--host=<hostname> The host on which to uninstall.

## 10.1.12. Usage notes

| Warning
| On Solaris, the PATH environment variable on the remote host (where you build or deploy) may get set according to /etc/default/su, which may be different from what you would expect (noted by S. Bailey).

Prev | Home | Next
Packing the executable |  | Method B: The native package manager