Twisted Documentation: Writing a New Plug-In for mktap

# Writing a New Plug-In for mktap

This document does not describe current Twisted best practice. The progress of this document is currently being tracked in issue 1139.

## Goals

This document describes serialized Python configuration objects which can be used for running Twisted applications.

Users of this document should be have a Twisted server application that they are going to deploy. In general, if you are going to write a mktap plugin, it is the last stage of developing and deploying a Twisted application. The majority of users will use the `.tac` Python configuration files without needing to use the mktap serialization. Therefore, people writing Twisted applications should review the Application howto before this document, and consider using the `.tac` mechanism described there.

## What is a Plug-In?

Python makes it very easy to dynamically load and evaluate programs. The plug-in system for Twisted, `twisted.python.plugin`, is a way to find (without loading) and then load plug-ins for particular systems.

See The Twisted Plugin system for more information on using `twisted.python.plugin`.

## Applications and plugins in Twisted

### Application objects

The central concept that a Twisted system administrator will work with are files that contain `Application` instances serialized in various formats optimized for different uses. `.TAP` files are optimized for speed of loading and saving, `.TAX` files are editable by administrators familiar with XML syntax, and `.TAS` files are generated Python source code, most useful for developers. The two command-line programs which work with these files are `mktap` and `twistd`. The `mktap` utility create `.TA*` files from simple command-line arguments, and the `twistd` daemon will load and run those files.

`Application` objects and related infrastructure are covered in more depth in the Application howto.

### TAP plugins

The most prevalent kind of plug-in is the `TAP` (Twisted Application builder) type. These are relatively simple to get started with. Let's look at an excerpt from Twisted's own plugins.tml for an example of registering one:

    # ...

    register("Twisted Web Automated TAP builder",
             "twisted.tap.web",
             description="""
             Builds a Twisted Application instance that contains a general-purpose
             web server, which can serve from a filesystem or application resource.
             """,
             type="tap",
             tapname="web")

    # ...

`plugins.tml` will be a list of calls to one function:

    register(name, module, type=plugin_type,
             description=user_description
             [, **plugin_specific_data])

- `name` is a free-form string, to be displayed to the user in presentation contexts (like a web page, or a list-box in a GUI).
- `module` is a string which must be the fully-qualified name of a Python module.
- `type` is the name of the system you are plugging in to. Be sure to spell this right, or Twisted won't find your plug-in at all!
- `**plugin_specific_data` is a dictionary of information associated with the plug-in, specific to the `type` of plug-in it is. Note that some plug-in types may require a specific bit of data in order to work.

Note the `tapname` parameter given in the example above. This parameter is an example of `**plugin_specific_data`. The parameter `tapname` is only used by `"tap"`-type modules. It indicates what name to use on the `mktap` command line. In English, this particular call to `register` means When the user types `mktap web`, it selects the module `twisted.tap.web` to handle the rest of the arguments.

Now that you understand how to register a plug-in, let's move along to writing your first one.

## Writing a mktap plugin: the Twisted Quotes example

As an example, we are going to work on a Quote of the Day application, `TwistedQuotes` described in the Designing Twisted Applications document.

### Set up the project directory

See the description of setting up the TwistedQuotes example.

Creating the extension to `mktap` is done through implementing a module that follows the `mktap` plug-in interface, and then registering it to be found and loaded by `twisted.python.plugin`. As described above, registration is done by adding a call to `register` in the file `TwistedQuotes/plugins.tml`

    from twisted.application import internet # services that run TCP/SSL/etc.
    from TwistedQuotes import quoteproto    # Protocol and Factory
    from TwistedQuotes import quoters       # "give me a quote" code

    from twisted.python import usage        # twisted command-line processing

    class Options(usage.Options):
        optParameters = [["port", "p", 8007,
                          "Port number to listen on for QOTD protocol."],
                         ["static", "s", "An apple a day keeps the doctor away.",
                          "A static quote to display."],
                         ["file", "f", None,
                          "A fortune-format text file to read quotes from."]]

    def makeService(config):
        """Return a service that will be attached to the application."""
        if config["file"]:                  # If I was given a "file" option...
            # Read quotes from a file, selecting a random one each time,
            quoter = quoters.FortuneQuoter([config['file']])
        else:                               # otherwise,
            # read a single quote from the command line (or use the default).
            quoter = quoters.StaticQuoter(config['static'])
        port = int(config["port"])          # TCP port to listen on
        factory = quoteproto.QOTDFactory(quoter) # here we create a QOTDFactory
        # Finally, set up our factory, with its custom quoter, to create QOTD
        # protocol instances when events arrive on the specified port.
        return internet.TCPServer(port, factory)

Twisted Quotes TAP construction module - listings/TwistedQuotes/quotetap.py

This module has to conform to a fairly simple interface. It must have a class called `Options` which is a subclass of `twisted.python.usage.Options`. It must also have a function `makeService(config)`, which will be passed an instance of the `Options` class defined in the module itself, `TwistedQuotes.quotetap.Options`. Command-line options given on the `mktap` command line fill in the values in `Options` and are used in `makeService` to make the actual connections between objects. `makeService` is expected to return an object implementing `IService`. This can be a `Service` subclass, a `MultiService` collection of sub-services, a `TCPServer` serving a protocol factory, and so on.

A more detailed discussion of `twisted.python.usage.Options` can be found in the document Using `usage.Options`.

Once the TwistedQuotes application design is complete, we pull it them together by writing a TML file which allows the `mktap` utility to find our protocol module.

    register("Quote of the Day TAP Builder",
             "TwistedQuotes.quotetap",
             description="""
             Example of a TAP builder module.
             """,
             type="tap",
             tapname="qotd")

Twisted Quotes Plug-in registration - listings/TwistedQuotes/plugins.tml

Now the QOTD server is ready to be instantiated! Let's start up a server and get a quote from it.

    % mktap qotd
    Saving qotd application to qotd.tap...
    Saved.
    % twistd -f qotd.tap
    % nc localhost 8007
    An apple a day keeps the doctor away.
    % kill `cat twistd.pid`

Let's walk through the above example. First, we run `mktap` specifying the Application type (`qotd`) to create. `mktap` reads in our `plugins.tml` file, instantiates an `Application` object, fills in the appropriate data, and serializes it out to a `qotd.tap` file. Next, we launch the server using the twistd daemon, passing `qotd.tap` as a command line option. The server launches, listens on the default port from `quotetap.py`. Next, we run `nc` to connect to the running server. In this step, the `QOTDFactory` creates a `Quoter` instance, which responds to our network connection by sending a quote string (in this case, the default quote) over our connection, and then closes the connection. Finally, we shutdown the server by killing it via a saved out process id file.

(`nc` is the netcat utility, which no UNIX system should be without.)

After reading this (and following along with your own example, of course), you should be familiar with the process of getting your own Twisted code with unique functionality in it running inside of a server. You should be familiar with the concept of a drop-in and a plug-in, and understand both how to create them and how to install them from other people on your system.

By following the rules set out at the beginning of this HOWTO, we have accidentally implemented another piece of useful functionality.

    % mktap
    Usage:    mktap [options] <command> [command options]

    Options:
      -x, --xml      DEPRECATED: same as --type=xml
      -s, --source   DEPRECATED: same as --type=source
      -e, --encrypted  Encrypt file before writing
      -p, --progress   Show progress of plugin loading
      -d, --debug      Show debug information for plugin loading
      -u, --uid=     [default: 1000]
      -g, --gid=     [default: 1000]
      -a, --append=  An existing .tap file to append the plugin to, rather than
                     creating a new one.
      -t, --type=    The output format to use; this can be 'pickle', 'xml', or
                     'source'. [default: pickle]
          --help     display this message
    Commands:
        ftp              An FTP server.
        im               A multi-protocol chat client.
        inetd
        mail             An email service.
        manhole          An interactive remote debugger service.
        news             News Server
        portforward      A simple port-forwarder.
        qotd             Example of a TAP builder module.
        socks            A SOCKSv4 proxy service.
        ssh
        telnet           A simple, telnet-based remote debugging service.
        toc              An AIM TOC service.
        web              A general-purpose web server which can serve from a
                         filesystem or application resource.
        words            A chat service.

Not only does our `Options` class get instantiated by `mktap` directly, the user can query mktap for interactive help! This is just one small benefit to using Twisted as it was designed. As more tools that use the `tap` style of plug-in, more useful functionality will become available from Twisted Quotes. For example, a graphical tool could provide not just help messages at the command line, but a listing of all available TAP types and forms for each, for the user to enter information.

Index

Version: 2.4.0