WebStone FAQ

| Compiling and Running WebStone: Summary and FAQ

This document answers frequently-asked questions about running WebStone.

- Meta-FAQ: What is this document? Where can I get a copy?
- How do I compile and install WebStone?
- How do I run WebStone?
- Common problems when running WebStone

- Out of swap space
- Error reading timing info

- I'm still having problems. Where can I get help?
- Legal issues

If you have comments about this document, please forward them to webstone@mindcraft.com.

## Meta-FAQ: What is this document? Where can I get a copy?

This is a list of answers to Frequently Asked Questions (FAQ) about WebStone. The latest copy is always available at http://www.mindcraft.com/webstone and via the WebStone mailing list.

How do I compile and install WebStone?

From source:

- Make sure the C compiler is in your shell's search path. If `w`hence cc`' or ``which cc`' dooes not give the name of the C compiler you wish to use, put the full path to your C compiler into the `$CC` environment variable.
- Unpack the source archive. On Unix and Unix-like systems this is usually something like:
  `gunzip < W25_src.tgz | tar xf -`
- Change directory to WebStone/src
- Make sure the directory is clean:
  `make clobber`
- Customize the source files:
  `./configure`
- Build and install the binaries:
  `make install`

The `make install` completes without errors on the systems where we've used it.

If your compiler does not understand Standard C function prototypes, WebStone will not compile.

If WebStone needs a library function that's not in one of the libraries known to WebStone, you'll see an error message something like:

`Unresolved external reference: foobar()`

Find out what library the missing function is in and add the library name to the ``LIBRARY='`definition in WebStone/src/Makefile.

From a binary archive:

- Unpack the binary archive:
  `gunzip < WS_sol251.tgz | tar xf -`

On Windows NT:

Unpack the source archive with WinZip.

The directory `WebStone\NT40` contains a workspace definition for compiling with Microsoft Developer's Studio and Makefiles for compiling using nmake.

Binary versions of the programs are also available for NT.

## How do I run WebStone?

WebStone includes a README file which may answer some of your questions. However, here's a brief overview.

1. Set up your test-bed.
2. Load WebStone onto your webmaster system.
3. Edit `testbed`.
4. Write a file list.
5. Start the benchmark.
6. Collect the results.

### WebStone GUI

To try the GUI, make sure you have a Web browser, and run `./webstone -gui` from the WebStone base directory. You won't have to hand-edit the `testbed` file, but you will have to edit `filelist` if you want to change the workload.

These are the steps to follow to run the GUI:

1. Set up your test-bed
2. Load WebStone onto your webmaster system.
3. `./configure`
4. `./webstone -gui`

If the GUI appears to hang, you can kill stray WebStone processes with `./webstone -kill`

If the GUI dies with lots of syntax errors, make sure your browser isn't trying to execute shell scripts. Comment out the line for "x-pl" in your system's mime.types file.

The WebStone GUI requires that Perl version 5.001e or later be installed on your system.

The WebStone GUI doesn't work properly on Windows NT yet. Use the webstone.bat script instead.

### Setting up your test bed

Your test bed should include, at minimum, two machines and a network. The first machine is your Web server. It can be any HTTP 1.0-compliant server. As far as WebStone is concerned, it's a black box.

You'll also need a webmaster system and one or more webclient systems. These can be Unix hosts or Windows NT hosts. The webmaster and the webclient may be the same machine, if desired. We've run up to 120 webclient processes and the webmaster on a single 32MB Indy, and up to 150 webclients on a single NT system.

You must establish a trust relationship between your webmaster and webclients. Each webclient must be set up so that the webmaster can use `rexec` to execute the WebStone on the client. This can be done with a guest account. It's also helpful if root can `rexec` and `rcp` to the webclients, and even to the web server. This requires editing the `/.rhosts` and `/etc/host.equiv` files. Here's an example, for a system named "webmaster":

`/.rhosts` (on each webclient)

    webmaster root

`/etc/hosts/equiv` (on each webclient)

    webmaster

To make best use of WebStone, your webmaster should be equipped with a C compiler, Perl, awk, and a Web browser. A data analysis program such as GnuPlot may also come in handy.

Connect the webclients, the webmaster, and the web server to a common network. To check your setup, load a browser on one of the webclients, and make sure it can connect to the Web server.

If your clients are running Windows NT, you'll have to install an rexec server on each client system. The shareware program from Ataman Software works well.

### Loading WebStone

Compile and/or install the WebStone distribution on your webmaster.

Common porting errors

- If you want to use gcc instead of cc, change the CC variable in `src/Makefile`.
- Many System V-based Unix implementations (such as Solaris 2.4) will need`LIBS = -lsocket -lnsl` in `src/Makefile`.
- Some users may also need to comment out the definition of `rexec` in `webmaster.c`

If you encounter other errors, please contact the WebStone mailing list.

Type `make install` to put the binaries in the `bin` directory.

When you run WebStone, the `distribute` script automatically copies the `webclient` binary to the other client systems. If you're running diverse clients (e.g., a couple Suns, a couple BSD hosts), you'll want to comment the `distribute` script out of `bin/runbench`, and distribute host-specific versions of `webclient` by hand.

If you make changes to `configure.in` so that the `configure` script supports another architecture, please send the changes to us at Mindcraft so we can add them to the master copy.

### Edit testbed

If you use the `webstone` and `bin/runbench` scripts to automate WebStone, you'll want to edit the `conf/testbed` file. `testbed` contains several configurable parameters that WebStone relies on. Here is an example:

    ### BENCHMARK PARAMETERS -- EDIT THESE AS REQUIRED
    ITERATIONS="3"
    MINCLIENTS="8"
    MAXCLIENTS="128"
    CLIENTINCR="8"
    TIMEPERRUN="30"

    ### SERVER PARAMETERS -- EDIT AS REQUIRED
    #PROXY=
    SERVER="www"
    PORTNO=80
    SERVERINFO=hinv
    OSTUNINGFILES="/var/sysgen/master.d/bsd"
    WEBSERVERDIR="/usr/ns-home"
    WEBDOCDIR="$WEBSERVERDIR/docs"
    WEBSERVERTUNINGFILES="$WEBSERVERDIR/httpd-80/config/magnus.conf $WEBSERVERDIR/httpd-80/config/obj.conf"

    # WE NEED AN ACCOUNT WITH A FIXED PASSWORD, SO WE CAN REXEC
    # THE WEBSTONE CLIENTS
    CLIENTS="webstone1 webstone2 webstone3 webstone4 webstone5"
    CLIENTACCOUNT=guest
    CLIENTPASSWORD=guest
    CLIENTINFO=hinv
    TMPDIR=/tmp

The first set of parameters means that the WebStone benchmark will run from 8 clients to 128 clients, in increments of 8. Each increment will run for 30 minutes, and the whole test will be repeated three times. This test suite would take roughly 24 hours to complete.

Why multiple iterations? The WebStone benchmark is a stochastic process so there will be variation from run to run, especially if your test file sets have large files or if you approach overloading the server. 3 iterations is about the minimum you should run just to see if there is variation and to gauge the amount of variation. the `TIMEPERRUN` needs to be long enough to establish a steady state and allow it to dominate the run. 30 minutes seems to be enough if the sizes of the files are small. You may want to run the benchmark longer per run to minimize variation if the files are large.

The second set of parameters means that we will test a server called "www" at port 80 (note that the port number may be changed to accommodate proxy servers or multiple servers on the same host). We will use four clients. Also, we specify the location of a system tuning file (on Sun Solaris, one could use /etc/system), and web server tuning files (specified for Netscape). These files will be copied into the `runs` subdirectories for later reference.

Finally, we specify the WebStone account on the clients. Here, we use the guest account, with a fixed password: guest.

### Write a file list

The basic WebStone tests expect a set of files to reside on the server to be retrieved by the `webstone` client programs. The file list tells WebStone which files to retrieve.

It's possible to use an arbitrary set of fixed-length files for WebStone. Although these files have the `.html` extension, they are used to represent files of many types. Basically we treat "bits-as-bits". You can use the programs in the `genfileset` subdirectory to create the needed set of files, and copy them onto your server:

        ./webstone -genfiles

The sample file list shipped with WebStone uses the files created by genfiles:

`# Sample filelist, abstracted from access logs
/file500.html 350 #500
/file5k.html 500 #5125
/file50k.html 140 #51250
/file500k.html 9 #512500
/file5m.html 1 #5248000`

This filelist consists of 5 different files. The number following the filename is the weight of this file in the distribution. All the weights are summed together and the frequency of each file is the weight of that file over the total weights.

For example, in this fileset the weights add up to 1000. So the the file500k.html page will occur 350 out of 1000 times, and the file5m.html will occur once every 1000 pages.

Note that the URI should be changed to a full URI when testing proxy servers, for example, if the proxy server is called proxy, but the actual server which stores the file is called seltzer1, you could use the following filelist:

`#Sample filelist, abstracted from access logs
http://seltzer1.sgi.com/file500.html 350 #500
http://seltzer1.sgi.com/file5k.html 500 #5125
http://seltzer1.sgi.com/file50k.html 140 #51250
http://seltzer1.sgi.com/file500k.html 9 #512500
http://seltzer1.sgi.com/file5m.html 1 #5248000`

This URI is the one which is passed to the proxy server, which in turn uses it to fetch the file from seltzer1.sgi.com. Notice that the particular files and the distribution are identical to the previous filelist. The other change which would need to be made for testing proxy servers is to have an entry "PROXY=proxy" in the testbed file and to specify the port where the proxy server listens for requests.

Wherever possible, use the same pages for WebStone that you will use in the real world. This means that you'll have a harder time comparing your results with published results, but your results will more accurately reflect your situation.

### Start the benchmark

type `./webstone`

The results of each run will be saved in a directory called `runs`. Note that the runbench script attempts to collect configuration information about your client and server configurations such as netstat results. You may see some error messages if your clients don't have netstat or other utilities.

### Collect the results

The WebStone summary statistics generated by `webmaster` are saved by `runbench` in a date stamped subdirectory of the `runs` directory in the current directory similar to:

        runs/950804_2304/run

The script wscollect is provided as a tool for collected the results of all of the runs and generating a tab delimited file with all of the results. This file can be read into a spreadsheet or read by other analysis programs.

        wscollect runs > runs.tabs

An additional script called `tabs2html` will take a tab delimited file and produce an HTML 3.0 style table of the results:

        tabs2html runs.tabs > runs.html

## Common problems when running WebStone

### Out of swap space

It's fairly common for the Web server under test to run out of swap space. As a rule of thumb, make sure that you have swap space equal to the number of server processes times the size of the largest test file.

For instance, if you're testing a 10MB file on a Netscape server with 64 processes, you'll need to have at least 640MB of swap space. N.B.: On SGI IRIX 5.x, you can substitute large amounts of virtual swap space, since Netscape doesn't actually use all the space it asks for.

See your operating system-specific administration guide for details on adding and configuring swap space.

### Error reading timing info

Question:

Running:

    webmaster -w webmaster -p 9990 -u flist -f config

on jan.near.net

outputs:

    Waiting for READY from 6 clients
    All READYs received
    Sending GO to all clients
    All clients started at Tue Aug 8 11:57:30 1995
    Waiting for clients completion
    Reading results
    .Error second reading timing info from one of the clients:
    Interrupted system call
    web child 1 did not respond. 3456 bytes read
    .Error second reading timing info from one of the clients:
    Interrupted system call
    web child 0 did not respond. 3456 bytes read

What does the second reading timing info contain? What might cause the second read to fail while the first passes?

Answer:

It's most likely that one of the WebStone clients died before it could report results to the webmaster. We've squashed many circumstances in which this happens, but bugs continue to appear, especially on systems we haven't tested.

We can't do much for this kind of problem without debugging traces. Edit `testbed`, and set the `DEBUG` parameter to `DEBUG=-d`, so that debugging info will be written to files named /tmp/webstone-debug.<PID>.

If you can replicate this problem with debugging turned on, please let us know. We'd love to examine the traces.

Another possible source of problems with reading timing info is when a page in the filelist did not get read by a client, but the webmaster was expecting to find it. This can happen when the test time, number of clients and filelist distribution are set up so that a file which gets read infrequently does not get read _yet_ before the test period ends.This will get ironed out in a later release of WebStone.

## I'm still having problems. Where can I get help?

Subscribe to the WebStone mailing list! Send a message to majordomo@webstone.mindcraft.com. The subject doesn't matter, but the content should be:

    subscribe webstone

You should receive a message shortly, confirming that you've been added to the mailing list. You can send to the whole list at webstone@mindcraft.com. The authors of WebStone read the list, and they'll do their best to help. Other list members may also be able to help.

If you have access to USENET News, you can also read and post to comp.benchmarks. As with any newsgroup, read the FAQ before posting!

There's also a mailing list devoted to the performance limits of the HTTP protocol. You can subscribe by sending e-mail to www-speed-request@tipper.oit.unc.edu with the text

    subscribe <your-email-address>

## Legal Stuff

This file and all files contained in the WebStone distribution are copyright © 1997, Mindcraft, Inc.

Copyright © 1998, Mindcraft, Inc.