Content-type: text/html
Manpage of rate
rate
Section: User Commands (1)
Updated: 11-August-2003
Index
Return to Main Contents
NAME
rate - Swiss-Army-knife traffic analyzer
SYNOPSIS
rate [generic options] <-R | -A | -T | -E> [mode-specific options]
rate -L <name>
rate [-h | -?]
rate [mode select option] [-h | -?]
DESCRIPTION
Rate
helps an administrator to figure out what is happening in his network at the moment.
Unlike
tcpdump(1),
rate
uses statistical and stream-oriented traffic analysing methods, and it
will never produce an output stream at a speed beyond human perception. The
output is less accurate, however.
Rate
features four different operating modes, designed to perform the following tasks:
estimating overall traffic rates
(the -R mode),
determining nodes generating the highest traffic
(the -A mode),
determining connections and flows generating the highest traffic
(the experimental -T mode) and
extracting strings from packets
(the bonus -E mode).
USAGE
Rate
accepts parameters in a standard, short
getopt(3)
syntax. There are several options common for all operation modes - these
options are described in the
GENERIC OPTIONS
section below. The operation mode itself is chosen by
one of special mode-selection options: -R, -A, -T or -E. After
a mode-selection option, no other generic options are allowed,
the only valid options are the mode-specific ones. Each operation
mode has its own set - see appropiate sections below.
After a successful startup, the application installs a packet
capturing handler, and starts to generate reports. The default report
generating policy is to dump a report to stdout every 1 second.
Always make sure
rate
has enough information on datalink layer protocol present on the
interface it was ordered to bind to. Consult the
-p
option description for details.
GENERIC OPTIONS
- -0 c
-
Replace every NUL character (ASCII 0) with c before doing regular expression
based filtering. Ignored if the
-x
option was not specified. The default is '@'.
- -c
-
Color (ANSI-compatible) output in modes that support it (currently: stream
analyzer and "abusers detection" mode).
- -f f
-
BPF filter expression to use. Using this option causes
rate
to ignore any packets NOT matching the specified BPF filter expression. For
a detailed description of BPF filter expressions syntax, consult the
tcpdump(1)
manual page.
- -g
-
Signal-based report generation policy. The reports are dumped
whenever rate receives a SIGUSR1 signal.
- -h -?
-
Print help.
rate
dumps a short help on available command-line options and quits, regardless
of other options.
- -i I
-
Bind to interface I. The default is 'eth0', which of course will cause a failure on
systems other than Linux. Make sure you specify the datalink prefix (see -p)
when you order rate to bind to an interface of an uncommon type.
- -k
-
Interactive report generation. The reports are dumped whenever
data is available on the standard input, which usually means you'll have
to press RETURN in order to generate a report.
- -l
-
Make stdout line-buffered. This option is useful when reports are redirected
(eg. using shell redirection) to a file.
- -L N
-
Loads a previously saved (using
-S
) option set. No other options are allowed when -L is being used.
- -n
-
Turns off asynchronous reverse DNS lookups.
Rate
will print numeric IPs rather than fully qualified domain names.
- -p N
-
Datalink layer header prefix length. Every (or at least almost every) known datalink
layer protocol prefixes a packet with its own header - which has to be stripped
before the actual data essential for rate (the IP protocol header) can be read.
Rate
is able to determine automatically how many bytes to skip only for the most common
datalink layer protocols (Ethernet, FDDI, TokenRing, loopback, PPP) - in other cases
the prefix length
must be specified using this option. It is EXTREMELY IMPORTANT to set the right value
- otherwise
rate
might print completely irrevelant reports and output invalid IP addresses. The default
is autosense, or if that fails - 14 bytes, which is the length of an
Ethernet
header.
- -q N
-
Orders
rate
to quit after outputting N reports.
- -r N
-
Time-based report generation policy. The reports will be dumped on
stdout every N seconds. This is the default (with N = 1).
- -s N
-
Capture at least N bytes. For performance reasons,
rate
does not acquire the whole packet from network, it just reads and processes first N
bytes. The default is 64 bytes, which might be not enough if you are using complicated BPF
expressions or filtering the packets using a regular expression. In such cases, it is
good to set the capture length to MTU on the interface. The value is automatically increased
to at least 1500 (which is the default MTU for an Ethernet interface) if one of -x, -E or -T
options is used. This option does NOT affect statistical data (amount of bytes, per-second byte rate)
collected by rate - the accounted packet size is always the 'real' one.
- -S N
-
Save the whole specified option set into a file in your home directory (~/.rate.modes)
under the name N. The option set might then be recalled using the
-L
option.
- -v
-
Print exact values. Normally,
rate
uses SI prefixes (like k - kilo, M - mega, G - giga, T - tera) to make
the printed numeric values more attractive for a human being. The -v option
disables this feature, causing
rate
to print exact values.
- -w
-
Clear the screen before printing each report. This assumes your terminal
is capable of understanding certain control sequences.
- -x r
-
Regular expression-based filtering. This option will cause
rate
to ignore packets that DO NOT match specified regular expression. Before
any tests, NUL characters occuring in a packet are replaced with
an other character, as specified in the -0 option (the default is '@').
Consult
regex(5)
manual for a detailed description of POSIX regular expressions.
In addition to standard regex syntax, you may use the
\r (CR), \n (LF), \t (TAB), \\ (\)
and \xNN (hex NN) special sequences.
MODE SELECTING OPTIONS
These options are used to choose between one of the four operation modes. It is necessary
to specify exactly one of them. After specyfing a mode selecting option, you may
pass the mode-specific options described below, depending on the selected mode.
No generic options are allowed though - all the generic ones have to be passed BEFORE
the mode selecting option.
- -R
-
Enable the rate estimation mode. See the
RATE ESTIMATION MODE
section for details.
- -A
-
Enable the "abusers detecion" mode. See the
ABUSERS DETECTION MODE
section.
- -T
-
Enable the stream analyzer mode. See the
STREAM ANALYZER MODE
section.
- -E
-
Enable the regular expression extractor mode. See the
STRING EXTRACTING MODE
section.
RATE ESTIMATION MODE
rate [...] -R -h
rate [...] -R [-b] [bpf filter expression]
In this mode,
rate
just shows overall traffic generated by packets matching the
specified (generic) filtering options (-f, -x). The reports in
this mode are just one-line long:
=> Currently XXX Bps/YYY pps, Average: ZZZ Bps/TTT pps
Where of course, XXX and YYY are the byte and packet per second
values, measured since the last report, while ZZZ and TTT are
the rates measured since the application started.
Available options:
- -b
-
Use bit units. The output is to be presented in bits (b) rather than bytes (B).
- -h
-
Help. Dumps a short hint on mode-specific options available for this mode, and
forces rate to quit.
ABUSERS DETECTION MODE
rate [...] -A -h
rate [...] -A [-a n] [-b] <-c c/p [-c c/p [-c c/p ...]]> [-dflt] [-O | -M] [-P | -B] [-T | -R] [bpf filter expression]
This mode is designed for hunting network nodes that generate highest
traffic. An independent set of counters is allocated for every host from every
specified IP subnet. On a report event, a list of hosts is composed, sorted
by one of the counters (depending on OMPBTR sorting options), and top entries (and
values of their counters) are displayed on stdout. Available options:
- -a N
-
Print N top nodes.
- -b
-
Use bit units. The output is to be presented in bits (b) rather than bytes (B).
- -c C/P
-
Consider nodes from this IP subnet. The network mask has to be specified in 'short', CIDR notation, eg.
10.0.0.0/8.
- -d
-
Alternative ("dump") output format. Instead of showing top N nodes,
rate
will dump the whole host list in a form that can be easily parsed by automated
tools. The output format is:
-
<host IP>:<overall input bytes #>:<overall output
bytes #>:<momentary input bytes #>:<momentary output bytes
#>:<overall packet input #>:<overall packet output
#>:<momentary packet input #>:<momentary packet output #>
- -f
-
Print spaces instead of "pipes" (|) as column separators. Normally,
rate
will use characters imitating a vertical line ('|') to separate columns in order to improve
readability. This option disables this feature.
- -l
-
Account local transfers too.
Rate
will also account 'local' transfers, ie. transfers between two hosts in IP classes specified with
-c
option. By default, such transfers are ignored.
- -t
-
Print total amount of transferred data instead of overall speed.
- -O
-
Consider overall transfer rates while sorting the host list.
- -M
-
Consider momentary transfer rates while sorting the host list (default).
- -P
-
Consider packet counters while sorting the host list.
- -B
-
Consider byte counters while sorting the list (default).
- -T
-
Consider output (TX) counters while sorting the list.
- -R
-
Consider input (RX) counters while sorting the list (default).
STREAM ANALYZER MODE
rate [...] -T -h
rate [...] -T [-m <memlimit> [-f <free>]] [-rtv] [-n
<n> | -s <n> | -p <n>] [-R | -S] [-M | -O] [-B | -P]
[-A | -D] [-b]
In this mode,
rate
will try to keep track of every TCP connection and UDP or ICMP stream present on the interface. Every detected
conversation is allocated its own set of traffic counters; besides,
rate
attempts to identify common protocols (like HTTP or FTP). The feature is still experimental, and consumes
HUGE amounts of system resources. Do not trust the -m and -f options, avoid leaving
rate -T
somewhere in the background and without supervision.
The reports might be generated in three different ways:
- host-oriented reports
-
a list of most active (or least active, depending on sorting options) hosts is printed,
with an optional list of conversations below every entry.
- stream-oriented reports
-
rate
print an overall list of most/least active streams.
- protocol-oriented reports
-
show the mostly utilized protocols, with an optional list of conversations classified
as a specific protocol below every entry.
Available options:
- -m M
-
Limit memory used by the conversation engine to M kilobytes. When the amount of allocated memory gets
beyond the set limit, a number of least active conversation is dropped. DO NOT trust this option.
- -f F
-
Drop F conversations after a memory overlimit. Default: 4000.
- -r
-
Output resource usage statistics (CPU, memory) before every report.
- -t
-
Use tabs instead of spaces to separate columns.
- -v
-
Increase verbosity level. This will print the guessed conversation protocol below every printed conversation
entry. Again, please keep on mind that the stream analyzer feature is highly experimental, and it still lacks
ability to recognize many common protocols. Only the basic ones (HTTP, FTP, FTP Data, POP3, SMTP) are supported
for now. The remaining streams are classified as "Unknown" and "Unidentified".
- -n N
-
Node-driven reports. Print a list of N most/least active nodes (and, optionally, conversations associated with them)
- -s N
-
Stream-driven reports. Print a list of N most/least active streams.
- -p N
-
Protocol-driven reports. Output a list of N top protocols.
- -b
-
Brief output. Supress the conversation lists - valid only for node and protocol driven reports.
Sorting options:
- -R
-
Consider received data counters. Makes sense only for node-driven reports. This is the default.
- -S
-
Consider sent data counters. As above, only for node-driven reports.
- -M
-
Consider momentary counters (default).
- -O
-
Consider overall counters.
- -B
-
Consider byte counters (default).
- -P
-
Consider packet counters.
- -D
-
Descending sort - the most active hosts/nodes/protocols first. This is the default.
- -A
-
Ascending sort - the least active hosts/nodes/protocols first.
STRING EXTRACTING MODE
rate [...] -E -h
rate [...] -E [-o <output format string>] [-i] [-e] <pattern>
This is a "bonus" mode - it has nothing to do with traffic analysis. Unlike other operation modes,
this one does not generate reports. It is designed for extracting strings from packets - whenever
a matching packet occurs, the extracted string is printed to stdout, regardless
of the report generation policy.
pattern
is a regular expression to be matched. Parts of the expression encosed in escaped braces
\( ... \) are printed to standard output, in order they appear in the expression if a packet
matches the pattern. If the -o (output format) option was specified, the
output format string
is printed instead, with \1 being substituted with the first subexpression in pattern,
\2 with the second, etc. The whole thing works much like sed 's///'. A typical application
would be investigating web pages currently visited by users:
# rate -f 'dst port 80' -E -o 'Host: \2 Browser: \1' -ie 'User-Agent: \([^\r\n]+\)\r\n.*Host: \([^\r\n]+\)\r\n'
The
-i
option enables printing source and destination addresses before each set of extracted strings.
As you can see, it is possible to use \n, \r, \t and \x in
pattern,
just like in the case of the -x generic option.
EXAMPLES
To determine overall traffic on eth3:
- # rate -i eth3 -R
-
To estimate bandwidth consumed by WWW traffic on ppp0, bit units:
- # rate -i ppp0 -f 'port 80' -Rb
-
To print top 20 nodes from 10.0.0.0/8 that are receiving the highest number of bytes (including transfers between
hosts inside the 10.0.0.0/8 network) every 5 seconds:
- # rate -i eth0 -r 5 -Aa 20 -lc 10.0.0.0/8
-
Same as above, but w/color output and screen clearing:
- # rate -i eth0 -r 5 -w -c -Aa 20 -lc 10.0.0.0/8
-
Show 10 nodes from 10.0.0.0/8 and 192.168.0.0/16 that generated the
largest overall number of packets, generate reports on a keypress
(RETURN), ANSI color output:
- # rate -i eth0 -ck -Ac 10.0.0.0/8 -c 192.168.0.0/16 -a 10 -OPT
-
Estimate ICMP traffic exchanged with the 10.0.0.0/8 subnet, wait for SIGUSR1, then dump the counters for each host in
192.168.0.0/24 to a file named FILE, then quit.
- # rate -i eth1 -f 'icmp and net 10.0.0.0/8' -q 1 -gl -A -c 192.168.0.0/24 -d > FILE
-
Show most active (considering transferred bytes) conversations detected
on eth3, color output on keypress; save the option set under the name
'streams':
- # rate -S streams -i eth3 -ckTrs 25 -MB
-
Recall the 'streams' operation mode:
- # rate -L streams
-
On a keypress, show a list of 3 nodes that have sent the highest amount of packets since the application started; color output:
- # rate -i eth3 -ckTrvn 3 -SOP
-
Show currently visited web pages:
- # rate -f 'dst port 80' -E -o 'Host: \2 Browser: \1' -ie 'User-Agent: \([^\r\n]+\)\r\n.*Host: \([^\r\n]+\)\r\n'
-
View addresses e-mails are being sent from:
- # rate -i eth1 -f 'dst port 25' -E -ie 'MAIL FROM: \([^\r\n]+\)'
-
BUGS AND LIMITATIONS
The
TRAFFIC ANALYZER
(-T) operation mode is experimental. It consumes large amounts of system resources. Memory
leaks in code that provides this feature are possible.
The commandline options are a bit complicated, but it seems it's impossible to do anything about that.
SEE ALSO
tcpdump(1),
regex(7),
pcap(3),
bpf(4)
AUTHOR
Mateusz Golicz <mteg@jaszczur.org>
Feel free to send comments, suggestions, bug reports, etc. The
author is not a native english speaker, and is aware of the fact that his english is far from
perfect. Because of that, reports on grammar or vocabulary mistakes in this manual are also welcome.
The asynchronous DNS resolver part was taken from
mtr
- a very handy traceroute replacement by Matt Kimball.
LICENSE
Copyright 2003 Mateusz Golicz. All rights reserved.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License, Version 2,
as published by the Free Software Foundation. A copy of this license is
distributed with this software in the file "COPYING".
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Read the
file "COPYING" for more details.
Index
- NAME
-
- SYNOPSIS
-
- DESCRIPTION
-
- USAGE
-
- GENERIC OPTIONS
-
- MODE SELECTING OPTIONS
-
- RATE ESTIMATION MODE
-
- ABUSERS DETECTION MODE
-
- STREAM ANALYZER MODE
-
- STRING EXTRACTING MODE
-
- EXAMPLES
-
- BUGS AND LIMITATIONS
-
- SEE ALSO
-
- AUTHOR
-
- LICENSE
-
This document was created by
man2html,
using the manual pages.
Time: 14:36:02 GMT, August 20, 2003