CampusIO - a FlowScan module for reporting on campus traffic I/O

- NAME
- SYNOPSIS
- DESCRIPTION
- CONFIGURATION
- METHODS
- SEE ALSO
- BUGS
- AUTHOR
- VERSION

# NAME

CampusIO - a FlowScan module for reporting on campus traffic I/O

# SYNOPSIS

       $ flowscan CampusIO

or in flowscan.cf:

       ReportClasses CampusIO

# DESCRIPTION

CampusIO is a general flowscan report for reporting on flows of traffic in and out of a site or campus. It does this by processing flows reported by one or more routers at the network border. The site or campus may be an Autonomous System (AS), as is often the case for large universities, but this is not necessary. CampusIO can be used by smaller institutions and other enterprises as well.

`flowscan` will run the CampusIO report if you configure this in your flowscan.cf:

       ReportClasses CampusIO

# CONFIGURATION

CampusIO's configuration file is CampusIO.cf. This configuration file is located in the directory in which the flowscan script resides.

The CampusIO configuration directives include:

NextHops
  This directive is suggested if OutputIfIndexes is not defined. Defining NextHops causes `flowscan` to identify outbound flows by their nexthop value. NextHops is a comma-seperated list of IP addresses or resolvable hostnames, e.g.:

         # NextHops
         NextHops gateway.provider.net, gateway.other.net

  If neither NextHops nor `OutputIfIdexes` is defined, `CampusIO` will use the flows' destination addresses to determine whether or not they are outbound. This is a less reliable and more CPU intensive method than NextHops or `OutputIfIdexes`.

OutputIfIndexes
  This directive is suggested if NextHops is not defined. Defining OutputIfIndexes causes `flowscan` to identify outbound flows by their output interface value. OutputIfIndexes is a comma-seperated list of ifIndexes as determined using SNMP, e.g.:

         $ snmpwalk router.our.domain public interfaces.ifTable.ifEntry.ifDescr

  or by looking at the raw flows from Cflowd to determine the `$output_if`. e.g.:

         # OutputIfIndexes
         OutputIfIndexes 1, 2, 3

  If neither NextHops nor `OutputIfIdexes` is defined, `CampusIO` will use the flows' destination addresses to determine whether or not they are outbound. This is a less reliable and more CPU intensive method than NextHops or `OutputIfIdexes`.

LocalSubnetFiles
  This directive is required. It is a a comma-seperated list of files containing the definitions of ``local'' subnets. E.g.:

         # LocalSubnetFiles local_nets.boulder
         LocalSubnetFiles bin/local_nets.boulder

OutputDir
  This directive is required. It is the directory in which RRD files will be written. E.g.:

         # OutputDir /var/local/flows/graphs
         OutputDir graphs

LocalNextHops
  This is an ``advanced'' option which is only required if you are exporting and collecting flows from multiple routers to the same FlowScan. It is a comma-seperated list of IP addresses or resolvable hostnames.

  Specify all the local routers for which you have configured cflowd to collecting flows on this FlowScan host. This will ensure that the same traffic isn't counted twice by ignoring flows destined for these next-hops, which otherwise might look as if they're inbound flows. FlowScan will only count flows that represent traffic forwarded outside this set of local routers.

  E.g.:

         # LocalNextHops other-router.our.domain

TCPServices
  This directive is optional, but is required if you wish to produce the CampusIO service graphs. It is a comma-seperated list of TCP services by name or number. E.g., it is recommended that it contain at least the services shown here:

         # TCPServices ftp-data, ftp, smtp, nntp, http, 7070, 554
         TCPServices ftp-data, ftp, smtp, nntp, http, 7070, 554

UDPServices
  This directive is optional. It is a comma-seperated list of UDP services by name or number. E.g.:

         # UDPServices domain, snmp, snmp-trap

Protocols
  This directive is optional, but is required if you wish to produce the CampusIO protocol graphs. It is a comma-seperated list of IP protocols by name. E.g.:

         # Protocols icmp, tcp, udp
         Protocols icmp, tcp, udp

ASPairs
  This directive is optional, but is required if you wish to build any custom AS graphs. It is a list of source and destination AS pairs. E.g.:

         # source_AS:destination_AS, e.g.:
         # ASPairs 0:0
         ASPairs 0:0

  Note that the effect of setting ASPairs will be different based on whether you specified ``peer-as'' or ``origin-as'' when you configured your Cisco. This option was intended to be used when ``peer-as'' is configured.

  See the BGPDumpFile directive for other AS-related features.

Verbose
  This directive is optional. If non-zero, it makes `flowscan` more verbose with respect to messages and warnings. Currently the values `1` and `2` are understood, the higher value causing more messages to be produced. E.g.:

         # Verbose (OPTIONAL, non-zero = true)
         Verbose 1

NapsterSubnetFiles
  This directive is optional, but is required if you wish to produce the CampusIO service graphs. It is a comma-seperated list of files containing the definitions of ``Napster'' subnets. E.g.:

         # NapsterSubnetFiles (OPTIONAL)
         NapsterSubnetFiles bin/Napster_subnets.boulder

NapsterSeconds
  This directive is optional. It is the number of seconds after which a given campus host has communicated with a host within the ``Napster'' `subnet(s)` will no longer be considered to be using the Napster application. E.g. Half an hour:

         # NapsterSeconds (OPTIONAL)
         NapsterSeconds 1800

NapsterPorts
  This directive is optional. It a comma-seperated list of default TCP ports used by Napster. These will be used to determine the confidence level of whether or not it's really Napster traffic. If confidence is low, it will be reported as ``NapsterMaybe'' rather than ``NapUser'' traffic. E.g., reasonable values are:

         # NapsterPorts (OPTIONAL)
         NapsterPorts 8875, 4444, 5555, 6666, 6697, 6688, 6699, 7777, 8888

TopN
  This directive is optional. It's use requires the `HTML::Table` perl module. TopN is the number of entries to show in the tables that will be generated in HTML top reports. E.g.:

         # TopN (OPTIONAL)
         TopN 10

  If you'd prefer to see hostnames rather than IP addresses in your top reports, use the ip2hostname script. E.g.:

         $ ip2hostname -I *.*.*.*_*.html

ReportPrefixFormat
  This directive is optional. It is used to specify the file name prefix for the HTML or text reports such as the ``originAS'', ``pathAS'', and ``Top Talkers'' reports. You should use `strftime(3)` format specifiers in the value, and it may also specify sub-directories. If not set, the prefix defaults to the null string, which means that, every five minutes, subsequent reports will overwrite the previous. E.g.:

         # Preserve one day of HTML reports using the time of day as the dir name:
         ReportPrefixFormat html/CampusIO/%H:%M/

  or:

         # Preserve one month by using the day of month in the dir name (like sar(1)):
         ReportPrefixFormat html/CampusIO/%d/%H:%M_

BGPDumpFile
  This directive is optional and is experimental. In combination with TopN and ASNFile it causes FlowScan to produce ``Top ASN'' reports which show the ``top'' Autonomous Systems with which your site exchanges traffic.

  BGPDumpFile requires the `ParseBGPDump` perl module by Sean McCreary, which is supplied with CAIDA's CoralReef Package:

         http://www.caida.org/tools/measurement/coralreef/status.xml

  Unfortunately, CoralReef is governed by a different license than FlowScan itself. The Copyright file says this:

         Permission to use, copy, modify and distribute any part of this
         CoralReef software package for educational, research and non-profit
         purposes, without fee, and without a written agreement is hereby
         granted, provided that the above copyright notice, this paragraph
         and the following paragraphs appear in all copies.
         [...]

         The CoralReef software package is developed by the CoralReef
         development team at the University of California, San Diego under
         the Cooperative Association for Internet Data Analysis (CAIDA)
         Program. Support for this effort is provided by the CAIDA grant
         NCR-9711092, and by CAIDA members.

  After fetching the `coral` release from:

         http://www.caida.org/tools/measurement/coralreef/dists/coral-3.4.1-public.tar.gz

  install `ParseBGPDump.pm` in FlowScan's perl include path, such as in the `bin` sub-directory:

         $ cd /tmp
         $ gunzip -c coral-3.4.1-public.tar.gz |tar x coral-3.4.1-public/./libsrc/misc-perl/ParseBGPDump.pm
         $ mv coral-3.4.1-public/./libsrc/misc-perl/ParseBGPDump.pm $PREFIX/bin/ParseBGPDump.pm

  Also you must specify TopN to be greater than zero, e.g. 10, and the `HTML::Table` perl module is required if you do so.

  The BGPDumpFile value is the name of a file containing the output of `show ip bgp` from a Cisco router, ideally from the router that is exporting flows. If this option is used, and the specified file exists, it will cause the ``originAS'' and ``pathAS'' reports to be generated. E.g.:

         TopN 10
         BGPDumpFile etc/router.our.domain.bgp

  One way to create the file itself, is to set up rsh access to your Cisco, e.g.:

         ip rcmd rsh-enable
         ip rcmd remote-host username 10.10.42.69 username

  Then do something like this:

         $ cd $PREFIX
         $ mkdir etc
         $ echo show ip bgp >etc/router.our.domain.bgp # required by ParseBGPDump.pm
         $ time rsh router.our.domain "show ip bgp" >>etc/router.our.domain.bgp
            65.65s real     0.01s user     0.05s system
         $ wc -l /tmp/router.our.domain.bgp
          197883 /tmp/router.our.domain.bgp

  Once `flowscan` is up and running with BGPDumpFile configured, it will reload that file if its timestamp indicates that it has been modified. This allows you to ``freshen'' the image of the routing table without having to restart `flowscan` itself.

  Using the BGPDumpFile option causes `FlowScan` to use much more memory than usual. This memory is used to store a `Net::Patricia` trie containing a node for every prefix in the BGP routing table. For instance, on my system it caused the `FlowScan` process to grow to over 50MB, compared to less than 10MB without BGPDumpFile configured.

ASNFile
  This directive is optional and is only useful in conjunction with BGPDumpFile. If specified, this directive will cause the AS names rather than just their numbers to appear in the Top ASN HTML reports. Its value should be the path to a file having the format of the file downloaded from this URL:

         ftp://ftp.arin.net/netinfo/asn.txt

  E.g.:

         TopN 10
         BGPDumpFile etc/router.our.domain.bgp
         ASNfile etc/asn.txt

  Once `flowscan` is up and running with ASNFile configured, it will reload the file if its timestamp indicates that it has been modified.

# METHODS

This module provides no public methods. It is a report module meant only for use by `flowscan`. Please see the `FlowScan` module documentation for information on how to write a FlowScan report module.

# SEE ALSO

`perl(1),` FlowScan, SubNetIO, `flowscan(1),` Net::Patricia.

# BUGS

When using the BGPDumpFile directive, `ParseBGPDump` issues a bunch of warnings which can safely be ignored:

       Failed to parse table version from: show ip bgp
        at (eval 4) line 1
       Failed to parse router IP address from: show ip bgp
        at (eval 4) line 1
       Nexthop not found:    Network          Next Hop            Metric LocPrf Weight Path
       $ at (eval 4) line 1
       Metric not found:    Network          Next Hop            Metric LocPrf Weight Path
       $ at (eval 4) line 1
       Local Preference not found:    Network          Next Hop            Metric LocPrf Weight Path
       $ at (eval 4) line 1
       Weight not found:    Network          Next Hop            Metric LocPrf Weight Path
       $ at (eval 4) line 1
       Origin code not found:    Network          Next Hop            Metric LocPrf Weight Path
       $ at (eval 4) line 1
       Possible truncated file, end-of-dump prompt not found
        at (eval 4) line 1

I'm not keen on patching `ParseBGPDump` to fix this since its license isn't compatible with the GPL. We probably just need to hack up a complete replacement for `ParseBGPDump`.

When using the BGPDumpFile directive, `ParseBGPDump` sometimes mistakes the `Weight` for the first ASN in the path. This has the totally undesirable effect of producing a ``Top Path ASNs'' report that erroneously reports the weight as one of the Top ASNs! I assume this is an indication of the difficulty of parsing the output of `show ip bgp`, which apparently was meant for human consumption.

When using the ASPairs directive, CampusIO will create RRD files that have a `:` character in the file name. While RRDTool is able to create RRD files with those names, it is not able to graph from them. To work around this problem, create symbolic links in your OutputDir before attempting to graph from these files. For example:

       $ ln -s 0:n.rrd Us2Them.rrd
       $ ln -s n:0.rrd Them2Us.rrd

# AUTHOR

Dave Plonka

Copyright (C) 1998-2001 Dave Plonka. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

# VERSION

The version number is the module file RCS revision number ($Revision: 1.63 $) with the minor number printed right justified with leading zeroes to 3 decimal places. For instance, RCS revision 1.1 would yield a package version number of 1.001.

This is so that revision 1.10 (which is version 1.010), for example, will test greater than revision 1.2 (which is version 1.002) when you want to require a minimum version of this module.