Setting up a ZEO Cluster with session tracking. Revised version.

  Meta::

    Valid for:  Silva 0.9.x
    Author:     Jan-Wijbrand Kolman
    Email:      jw@infrae.com
    CVS:        $Revision: 1.5 $ $Date: 2003/04/07 09:40:52 $ 

  REVISION NOTES:

    Although the lastest version for the ZEO (Zope Enterprise Objects)
    software is version 2. This document still covers version 1. In so far
    this version of ZEO and/or related software versions is not available
    online anymore, Infrae can provide tar archives of theses versions.

    In the near future Infrae will start testing version 2 of ZEO (and its
    related software) and update this document accordingly.

  Introduction

    These notes were collected during a ZEO Cluster setup proces to test
    Silva in a clustered environment. Most information can be found at the
    www.zope.org website, although either slightly outdated or targeting a
    specific platform or usage.

    The document asumes knowledge of and experience with setting up Zope
    from sources, using the Zope Session managers, etc. It also assumes a
    UNIX-like OS. The instructions use '$ ' for the command line prompt and
    commands or code snippets which really should be typed on one line,
    will be split with a '\'.

    I hope the notes do provide enough instructions to setup a ZEO cluster
    and a highlevel overview and sufficient pointers to more information.
    Feedback and/or corrections is highly appreciated.

  Problem description

    A ZEO cluster provides building blocks to setup a scalable and highly
    available platform for CPU intensive webapplications.

    Typically a ZEO cluster consists of 1 or more ZEO Clients and 1 ZEO
    Server. Each Client, most probably running on a seperate network node,
    serves the actual webapplication. The Server provides access to one
    central ZODB (the object database storage) and is responsible for
    synchronizing the Clients throughout database changes. Each Client
    keeps a local object cache in RAM, to speed up object read actions.

    To evenly spread the application load over the different Clients a
    'front-end', or 'load balancing' server is put in place. End users
    connnect to the Balancer, which transparently maps the requests to one
    of the Clients. Typically the Client and Server nodes are kept on a
    private network, where this Balancer acts as a "bridge" between the
    public and this private network.

    Most web applications keep track of the end users "state" within the
    application. Webapplication may do so by "tagging" the end user web
    browser with a cookie. This cookie merely provides an identifying
    number to the webapplication. With this ID the application keeps a
    'session' object per end user. This session object is a generic
    container to store end user states within the application.

    Normally (in a non-clustered situation) this session object is not
    persisted into the ZODB, but kept in a RAM cache. Amongst other
    reasons, one is to prevent the ZODB from excesive growth, since for
    each change in object (and in case of session objects, this effectively
    is for each http request), data is appended to this ZODB.

    In a clustered situation however, if the Balancer maps a request to,
    say, Client node A in the network, this will set a session object in
    its RAM cache. A consecutive request might be mapped to a Client B,
    which does not have a session object in RAM identified by the end users
    cookie. This will obviously break the application state tracking.

    A solution is to actually persist the session object in a central
    database on the Server. To prevent the mentioned excesive growth, a
    seperate DB is setup which does not append, but just replaces the data
    on each (session) object change.

  Software versions used:

    * Python 2.1.3

    * Zope 2.5.1

    * ZEO version 1

     * the Berkely DB libraries

     * Berkeley DB bindings for Zope/Python

     * the External Mount product

  Rough instructions

      The steps involved roughly are:

        * Setup a ZEO Server, and, preferably more than one, ZEO
        Client(s) all able to use the ExternalMount product.

        * Setup a "scratch" Zope instance backed by a Berkeley DB with
        at least one user defined folder. This will create a 'bootstrap'
        Berkeley DB for later use.

        * Setup the Server to use both the "main" ZODB FileStorage and
        the BerkeleyStorage. This BerkeleyStorage will store the
        session objects. Put the "bootstrapped" Berkeley DB in the
        directory where the Server is instructed to find it.

        * Create an 'ExternalMethod' in one of the Clients 'Extensions'
        directory. This 'ExternalMethod' will later 'mount' the
        session database

        * Start the Cluster. Use the ZMI of one of the running Clients
        to add a 'Mount via ExternalMethod'. Point this Mount to the
        previously created ExternalMethod and specify the id of the
        created folder in the 'bootstrapped' session BD. This folder is
        mounted and available to store objects.

        * Create a 'Transient Object Container' in this mounted folder
        and instruct the 'session_data_manager' to store its session
        data in this 'Transient Container'. Since data in this container
        is actually centrally persisited in the Berkeley DB, these
        (session) objects are available to all Clients.

        * Setup a (load balancing) front end webserver (the Balancer) as
        a cental point of access to the Cluster.

        * Instruct the Clients (e.g. with the 'SiteRoot' product) to
        generate abolute URLs corresponding to the Balancers URL. This
        will make all requests go through the Balancer.

  Berkeley ZODB Storage

      The "out of the box" ZODB backend is called 'FileStorage'. By
      design, this backend is a versionable and undoable persistent
      storage for (Zope) objects. Each change to an object is appended
      to the database file, which, unless this database is 'packed'
      every now and then, continues to grow

      This standard FileStorage backend can be replaced by a Berkeley DB
      backend. This may be setup in a fully versionable and undoable
      fashion, but can also be setup in a so called 'Packless' fashion,
      which does not keep any version or undo information for the
      objects in the DB.

      A setup like this needs a working installation of the Berkeley DB
      version 3 libraries, Python bindings to these libraries and a Zope
      interfaces to be able to make use of this DB (see [A6], [A8], [A9]).

      A 'custom_zodb.py' (see [B1]) file in the Zope instance directory
      (see [A1]) instructs Zope to use this BerkeleyStorage instead of the
      standard FileStorage.

  Partitioned ZODB Storage (see [A4], [A5])

      To make the ZEO Server use two ZODB backends simultaneously (two
      'partitions' in the ZODB) a StorageConfig.py (see [B2]) is used (the
      filename actually is arbitrary). The Server is then started with
      no 'custom_zodb.py' file in place, but with extra startup
      arguments (see [B3]).

  Mounting Session Storage (see [A4])

      After the Cluster is started, the BerkeleyStorage can be mounted.
      Use the ZMI of one of the Clients. Add a 'Mount via
      ExternalMethod' object. It doesn't actually create a Zope object,
      but will invoke an ExternalMEthod (see [B5]).

      This procedure will ask for three parameters - 'Module',
      'Function' and 'Path'. The 'Module' is the name of the filename in
      which the 'Function' resides.

      The 'Path' parameter value should be the name of the folder, say
      'session_data', created inside the Berkeley DB using the 'scratch
      Zope'. By using this name, the BerkeleyStorage is mounted and
      accessible through the 'session_data' folder in the root of the
      Zope hiearchy.

      The standard 'session_data_manager' puts its session objects in a
      'Transient Object Container', which in turn is put inside the
      'temp_folder' in the Zope root. Objects in this temp_folder are
      only stored in RAM.

      Create a new 'Transient Object Container' inside the
      'session_data' folder and instrcut the 'session_data_manager' to
      use this transient container. Session objects are now persisted in
      the BerkeleyStorage and thus available throughout the Cluster.

  A: References

      [A1] Central Zope code base, multiple instances:
              http://www.zope.org/Members/4am/instancehome

      [A2] ZEO setup:
              http://www.zope.org/Members/kedai/UseZeoZope                                              

      [A3] www.zope.org cluster setup:
              http://www.zope.org/About/

      [A4] Session tracking & ZEO:

              http://www.zope.org/Members/randy/ZEO-Sessions

              http://www.zope.org/Members/jgrewen/ZEO%20with%20CoreSessionTracking
                  %20and%20a%20mounted%20Berkeley%20session%20DB

              http://www.zope.org/Members/mcdonc/HowTos/UseExternalMountWithCST

      [A5] Berkeley Storage Wiki and code:
              http://www.zope.org/Wikis/ZODB/BerkeleyStorage
              http://dev.zope.org/Wikis/DevSite/Proposals/BerkeleyStorage     

      [A6] ExternalMount product:
              http://www.zope.org/Members/hathawsh/ExternalMount

      [A7] MountedStorage:
              http://www.zope.org/Members/natsukashi/Products/MountedStorage

      [A8] Bindings:
              Zope:   http://www.zope.org/Products/bsddb3Storage
              Python: http://pybsddb.sourceforge.net/

      [A9] Berkeley DB:
              http://www.sleepycat.com/

  B: Code

      [B1] 'custom_zodb.py' to use BerkelyStorage::

        import os
        from bsddb3Storage.Packless import Packless

        env = os.path.join('var', 'bsddb3Storage')
        Storage = Packless(name='BerkeleyStorage', env=env)

      [B2] 'StorageConfig.py' ::

        import os
        import ZODB.FileStorage
        from bsddb3Storage.Packless import Packless
        main_storage = ZODB.FileStorage.FileStorage(
                                          os.path.join('var', 'Data.fs'),)
        session_storage = Packless(
         name='BerkeleyStorage', env=os.path.join('var', 'bsddb3Storage'))

      [B3] 'start_zeo' script::

        #!/bin/sh
        export INST_HOME=`pwd`
        export INSTANCE_HOME=`pwd`
        export ZOPE_HOME= [ path to zope install directory ]

        exec python $ZOPE_HOME/lib/python/ZEO/start.py \
          -p 9000 \
          -D \
          ZEO_SERVER_PID=var/ZEO_SERVER.pid \
          STUPID_LOG_FILE=var/ZEO_EVENTS.log \
          -S main=StorageConfig:main_storage \
          -S sessiondata=StorageConfig:session_storage \
          "$@"

      [B4] Mount_Session_Data.py' ExternalMethod::

        import ZODB, ZEO.ClientStorage
        def mountSessionData():
        return ZODB.DB(ZEO.ClientStorage.ClientStorage(
                           ('192.168.1.20', 9000), storage='sessiondata'))

      [B5] Add 'Mount via ExternalMethod' 'object', fill in parameters::

        Module = Mount_Session_Data
        Function = MountSessionData
        Path = [ name of mount folder ]

<br />
Copyright &copy; 2002, 2003 Infrae. All rights reserved.<br />
See also "LICENSE.txt":license.txt
