Linux 2.0 and 2.2 ip accounting package
What is ipac / Overview
ipac is an ip accounting package for linux. It collects, summarizes and nicly displays ip accounting data. The output of ipac can be a simple ascii table, an ascii graph or even images with graphs, showing traffic progression. ipac can be used for ip traffic analysis and for accounting purposes. For further information, read the current README which follows. At the bottom of the page is the latest version of the file CHANGES.
New major features in version 1.10 since 1.05:
Update from version 1.06 or older is recommended; for ipchains users (Linux kernel 2.2.x) update from version 1.08 or older is recommended. If you have problems with ipacsum which are mentioned in the CHANGES file in section 1.09 or 1.10, update to 1.10.
There is a development release of ipac 2, too. Go to Sourceforge to find it.
ipac development stagnation (2002/01/23)
Unfortunatelly, ipac development has stagnated since the last release due to limited time I have for it. It is even hard to reach me by email on ipac related matters. I still hope that I will have time for it in the future again, but I don't know if so and when.
ipac does not support 2.4 kernels and I have no idea when it will. There is however a branch named ipac-ng; the author claims that it does support ipchains (kernel 2.4). However, I have nothing to do with that.
ipac 2 development: First development release (2000/07/14)
The first development release of ipac is out. Check the sourceforge page for details.
ipac 2 development (2000/06/06)
Currently, there is a next major version of ipac, 2, under development. ipac 2 will have a general "storage backend API" interface. Storage backends can be "plugged in" at build time. They have to provide certain functionality like opening their database, storing records, deleting records or listing timestamps. ipac 2 will come with two standard storage backends: The old "plain file" backend which is the one used in ipac 1.x, and a new "gdbm" backend which uses gdbm files to store data and which is much faster than the plain file backend.
I hope that once an ipac 2 (beta) version is released, people who are interested in this may start to develop more backends. In the first row, this might be SQL based backends like mysql. One of the main goals would be to centralize accounting information and to be able to add traffic from different routers within the same rule. Probably the biggest problem to solve for SQL backends would be to handle error conditions such as unreachable database servers.
The current state of ipac 2 is still alpha, but shortly it will be beta. The main task for the moment is to write documentation, especially for the storage backend API; as soon as this is done, I will release the beta of ipac 2 and put the cvs tree on Sourceforge. I expect this to happen within the next two or three weeks.
ipac now uses Sourceforge (same with ssl) for development support. Sourceforge is a good place for patches (upload and download) and bug reports. In future, the Sourceforge cvs repository will be used, too.
There is a mailing list about ipac. The list is intended for announcements, discussions, suggestions, patches etc. - everything that relates to ipac. To subscribe to this mailing list, send mail to email@example.com with the word subscribe in the body of the message. The list is really low-traffic.
IPAC Version 1.10 (c) 1997 - 2000 Moritz Both For copyright notice see at the bottom of this file WHAT IS IT? ipac is a package which is designed to gather, summarize and nicely output the IP accounting data. ipac make summaries and graphs as ascii text and/or gif images with graphs. ipac... - is for Linux - runs on top of the ipfwadm or ipchains tool - needs certain kernel parts compiled in HOW DOES IT WORK? ipac consists of two scripts (shell and perl) and one C program: - ipacset reads a configuration file and sets up ip accounting for the kernel using ipfwadm or ipchains - fetchipac, executed from cron once in a while, reads the current ip accounting data assembled by the kernel and writes it to a new file - ipacsum summarizes the data from a set of files and, optionally, replaces these files by one. It displays the values as a simple table containing the sums, as png graph pictures or as ascii graph pictures. UPDATE If you are updating from an old version of ipac, read the file UPDATE. 2.1.* OR NEWER KERNEL If you update your kernel from a 2.0 to 2.1 or newer version, beware that you will suddenly need the ipchains (version 1.3.8 or newer), awk and mktemp tools. If these all existed when you installed ipac, there is no need for re-installing; however, that is the unlikely case. You should probably re-install it anyway to be on the safe side. Once ipac was installed with all the tools in place, you can switch kernels as you like without re-installing. Put differently, if the programs ipfwadm, ipchains, awk and mktemp are in the PATH when installing, you are fine for every kernel. (As long as it is Linux.) INSTALLATION / PRECONDITIONS ipac runs at least under Linux kernel 2.0.29, 2.0.33, 2.0.35, 2.2.3 and 2.2.5. It should run on any kernel above. You need perl 5. If you want to use ipacsum to create images, you need a perl library called "GD". If GD is not installed and you run ipacsum to make images, it will exit with an error. GD can be found at http://www.cpan.org/CPAN.html - follow the link to the perl module list and look for GD. After downloading, you must install GD as described within the package. The type of images ipacsum makes depends on the version of the perl GD library you have. If you use GD version 1.19 or older, you will be generating gif images. If you use GD version 1.20 or later, the image format will be png. png is preferred because there is no copyright / patent hassle. PRECONDITIONS FOR 2.0.* kernels You need a kernel which was compiled with the configuration option CONFIG_IP_ACCT set to y. You also need the front end to ip firewall and ip accounting, that is, the tool 'ipfwadm'. I used version 2.3.0. PRECONDITIONS FOR 2.1.* and 2.2.* kernels You need a kernel which was compiled with the configuration option CONFIG_IP_FIREWALL set to y. You also need the front end to ip firewall and ip accounting, that is, the tool 'ipchains'. I used version 1.3.8, and it was reported that older versions do not work. Finally, you will need awk and mktemp. * Beware: firewall packet filter scripts may interfere with ipacset when * using 2.1.* or 2.2.* kernels! If you have a script that sets up a packet * filter, read the ipacset man page, section BUGS! INSTALLATION / OVERVIEW To install: - Edit the file 'config'. - Type 'make'. - As root, type 'make install'. - Create the file 'ipac.conf' and execute ipacset (see below). - Put fetchipac into cron (see below). - Put ipacset into a startup file to set ip accounting after reboots (see below) - Make sure that the accounting data files are cleaned up properly (see below). INSTALLATION / CONFIG FILE, RUNNING IPACSET The ipac.conf file is '/etc/ipac.conf' if you don't change this path in config. ipac.conf controls what data is collected. Each line which begins with a '#' is ignored. All the other lines have the format Name of rule|direction|interface|protocol|source|destination where Name of rule Any string to identify this rule direction 'in' or 'out' or 'both' interface ip number or interface name or empty protocol 'tcp' or 'udp' or 'icmp' or 'all' source \ destination both as described in ipfwadm(8), or empty In the summaries, the 'Name of rule' string identifies the counter. Both the source and destination must be in ipfwadm syntax - consult the man page. The interface can be named (for example, eth0) or its IP number can be given. Linux IP accounting always counts at one interface or at any interface. The direction means in or out of this interface, or both directions. For a more complete explanation of this file, see the man page of ipacset(8). * You must run the ipacset script after changing the * ipac.conf file every time for the changes to take effect! An example ipac.conf file comes with the distribution. INSTALLATION / FETCHING ACCOUNTING DATA FROM KERNEL: FETCHIPAC IN CRON In order to collect the accounting data, you must put a line into a crontab to call fetchipac on a regular basis. The more often you call fetchipac, the less data is lost in case of a crash or reboot. It is unharmful to call fetchipac any time. I suggest to call it every 15 minutes. For example, put this into your /etc/crontab file: # Save IP accounting info every five minutes. */15 * * * * root /usr/local/bin/fetchipac INSTALLATION / AFTER REBOOTS Naturally, the kernel forgets about the ip accounting on reboots. To reset the ip accounting properly, you should put a line into a startup file to call ipacset. For example, in my /etc/rc.d/rc.local file, I put this: # Switch on ip accounting /usr/local/bin/ipacset READING IP ACCOUNTING SUMMARIES To get summaries, use ipacsum. Without arguments, ipacsum will print a sum for every rule in ipac.conf. It will evaluate every file it finds in the ip accounting data directory, thus, all data ever gathered by fetchipac will be used. ipacsum outputs a nicely formatted overview of all accounting rules which were in effect during the given period. The rules are identified by their names from the ipac.conf file. If a rule was added or deleted during that time, it is nevertheless shown. Other capabilities of ipacsum include generation of graph images, filter output by rule name and setting time frame of files to be evaluated. For a complete description of ipacsum, read the man page ipacsum(8). A small help screen will be displayed with ipacsum --help. CLEANING UP fetchipac generates a single file every time it runs. The more often fetchipac runs, the more files you get and the more exact will be your accounting info. For example, if you run fetchipac every five minutes, ipacsum will be able to display accurate data for every five minute period. Every time ipacsum runs, you can let it make a summary file for all the files read to replace them. This will decrease the needed disk space and the time ipacsum needs to calculate sums for this period. You lose accuracy, though, since all data files are summarized into one, meaning there will be no more information when exactly the traffic occurred, but only the sum for the whole period. In general, it makes sense to periodically summarize the info for a past period. For example, you could run these cron jobs cleanups: - run fetchipac every 15 minutes - every hour, summarize the files of the hour 48 hours ago - every day, summarize the files of the day 7 days ago - every week, summarize the files of the week 11 weeks ago - every year, summarize the files of the year 2 years ago With this scheme, you can have the data of the ip traffic with 15-minute-accuracy for the past two days. For the last week, you still can tell at which day the traffic passed your machine. Keeping the daily files for 14 weeks, you have a daily overview of the last three months. After that, you keep weekly files only. After two years, you sum up the data of the year into one file. The daily cron jobs could look like this: # Summarize ip accounting info: # every day, sum up the data of 7 days ago into one file. 1 0 * * * root /usr/local/bin/ipacsum -r -t "the day 7 days ago" >/dev/null # every hour, sum up the data of 48 hours ago into one file 2 * * * * root /usr/local/bin/ipacsum -r -t "the hour 48 hours ago" >/dev/null # every week, sum up the data of the week 11 weeks ago into one file 3 0 * * 0 root /usr/local/bin/ipacsum -r -t "the week 11 weeks ago" >/dev/null # every year, sum up the data of the year 2 years ago into one file 4 0 1 2 * root /usr/local/bin/ipacsum -r -t "the year 2 years ago" >/dev/null FURTHER DOCUMENTATION Read the man pages - ipacset(8), ipacsum(8) and fetchipac(8). CONTRIBUTIONS The directory contrib/ contains stuff that does not directly belong to ipac but is related. Further (well-documented and... "nice") contributions to the directory are welcome! Thanks to all who contributed with patches, comments or suggestions! UPDATES, BUG REPORTS, WHERE TO GET For new versions of ipac, look at http://www.comlink.apc.org/~moritz/ipac.html There is a mailing list about ipac, for discussion, patches, suggestions and announcements. To subscribe, send mail to <firstname.lastname@example.org> with the word "subscribe" (without the quotes) in the mail body. If you find a bug, please send me a report or a diff. See at the bottom of this file for the email address. COPYRIGHT Copyright (C) 1997 - 2000 Moritz Both This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. The author can be reached via email: email@example.com, or by snail mail: Moritz Both, Im Moore 26, 30167 Hannover, Germany. Phone: +49-511-1610129 $Id: README,v 1.32 2000/06/06 10:01:51 moritz Exp $ EOF
$Id: CHANGES,v 1.50 2000/06/06 10:01:49 moritz Exp $ V0.9 - initial revision V0.91: Added time zone arithmetics fixed bug in ipacsum: no input files and -r added CHANGES file corrected documentation V0.93: Improved -t time frame syntax Added graph support Added -f filter option V0.94: Fix: no longer produce readdir() warning -s and -e can take relative times, too minor fixes V0.95: Added official distribution site Added -d option to ipacsum Avoided warnings V0.96: - ipacsum: better handle corrupt data file content - ipacset + fetchipac: be more robust about missing permissions or RUNFILE don't call fetchipac on initial call at boot time handle all lock file cases correctly V0.97: - subst.c: fixed compile error under libc6 / RedHat 5.1 - ipacsum: fixed (nearly) endless loop when no start time was given and -g V0.98 (98/07/14): - changed default lock file directory from /tmp to /var/lock (security) - ipacset: discovered ipfwadm may add more than one line to the kernel tables with one call - this happens when a host name resolves to more than one ip address. As a result, all gathered data were messed up. Fixed ipacset to handle this properly. These changes to ipacset were done by Friedrich Lobenstock <firstname.lastname@example.org> [FL], thanks!: - Fixed a bug if there exist empty lines in /etc/ipac.conf - Added some basic syntax checking (empty parameters) - Added command line option '-D' for debugging - Not really added support for bidirectional rules. Just say 'io'(or 'both' as in ipfwadm(8)) instead of 'in' or 'out' in /etc/ipac.conf - README: Added hint to mailing list - support for long option names V0.99 (98/07/29) - added man pages - corrected many spelling errors (discovered ispell). Changed "--intervall" option of ipacsum to "--interval" :-). (Old version still works) - added gif image creation (all options beginning with --gif) - added --version V1.00 - improved ipacset error handling - fixed order of week days in ipacsum - added average value feature for gif pictures - --gif-caption-in-index / show max and average values in html index file - --show-run-progression / display a progression indicator while running - added "Resolution:" in index.html - support for empty interface spec in ipac.conf for "any interface" - --gif-asis, HTTP Expires: header lines and HTML META tags - can now define two or more rules with the same name. ipac shows the sum in its outputs V1.01 - fixed 'Illegal division by zero at /usr/local/bin/ipacsum line 763.' - minor sub-pixel fix in gif creation (probably never visible) V1.02 (1999/04/14) - ipacsum: --gif-average-dots - Linux 2.1.* and up kernel / ipchains compatibility - --gif-caption-in-index shows total traffic (for whole gif image) V1.03 (1999/05/10) - fixed bugs in ipacset and fetchipac which appeared with some shells - added handling of cases when something deletes some or all of ipac's settings in the kernel (ipacset --fix-chains, fetchipac corrupted kernel settings detection, "BUGS" section in ipacset(8)) V1.04 (1999/05/31) - bug fix concerning ipchains option --fix-chains (still had problems with firewall scripts) - bug fix for /bin/sh compatibility in ipacset V1.05 (1999/08/06) - bug fix ipacsum "this week", "last week", "the week n weeks ago" - bug fix no "in" rule resulting in "out" rules not counting V1.06 (2000/01/24) - added support for .png images while being backward compatible to old .gif images... if you have the GD library. Renamed all options '--gif-*' to '--png-*' in the documentation; the old versions still work. - --gif-average-curve now "smoother" - --fixed-quantity - --png-total - made pathsearch more compatible (hints from Thomas <email@example.com>) - added contrib/ directory with ipac.cgi by Dan Swan <firstname.lastname@example.org> V1.07 (2000/02/25) - added debug code to ipacsum which can be stripped off on install by an awk script in Makefile - introduced 'accuracy' to ipac. ipacsum's output is now The Correct Numbers, I believe. All older versions didn't have a real concept about calculating the correct numbers at the beginning and the end of a given time frame, resulting in incorrect values at the edges of a given time frame. Specifically, ipac would take each data file it found in the data file directory which was created within the given time frame; if there was no data file, it would take no data file; and sum them all up. If you had one data file for the whole day, created at 23:58 h, it would give you a good result if you asked about the whole day, but asked about 0:00 h (start time) to 23:50 h (end time), it wouldn't find any data and display nothing. Asked about 23:50 (start time) to 23:59 (end time), it would give you the whole day's data. As of this version, ipacsum really evaluates the correct values by considering all data from the given time frame plus the data from directly before and after the time frame. If the data for a given time frame is not specific enough, ipacsum computes an average value. Given the example above, asked about the whole day's data, ipacsum would consider the data from 23:58, but also the time of the data before this day (possibly with a time stamp of the day before, 23:58), and the data of the next file. So the values would be (a little) more accurate. Asked about the time until 23:50, the values would probably decrease a little. And asked about the time from 23:50 to 23:59, it also would only display this nine minutes' share of the collected values. Also, --replace now works in a conceptual meaningful way. The generated file's name is now derived from the time at the end of the time frame (before: from the newest file that was read). - rewrote fetchipac in C. It is now more robust, more verbose in case of errors and it *could* be extended to store its data into a database. (Also, it creates much smaller data files and uses much less system resources. And it does no longer use programs like ipchains, ipfwadm, awk and so on.) V1.08 (2000/04/29) - ipacsum bug fix --fixed-quantity and values >2^31 (showed negative numbers) - ipacsum bug fix dont try to read data files which are directories or which have an invalid file name - ipacsum bug fix dont reproduce old, deleted rules and old rule order on --replace runs. dont create empty meaningless files on --replace runs which may confuse ipacsum, saying "* = data incomplete..." - ipacsum bug fix --gif-total (option wasn't recognized) - ipacsum add --filename-prefix option - added 'ipac_and_mrtg' to contrib directory V1.09 (2000/06/06) - fetchipac: bug fix keep rule order in ipchains systems in any case - ipacsum: bug fix: don't make time frame in images longer than it should be in infrequent cases - ipacsum: bug fix: (again) dont make first value in images too high in rare cases V1.10 (2000/08/07) - added Suse rpm directory under contrib/ - ipacsum: --png-header-in-index option - ipacsum: corrected documentation and behavior concerning --png-index - ipacsum: --png-start, --png-end, --png-charset - ipacsum: x axis labels are now aligned to month and year borders - ipacsum: bug fix illegal division at line 1206/1185
Letzte Änderung dieser Seite: 14.03.2003 durch Moritz Both, Email: moritz at daneben.de (Leerzeichen und "at" durch das passende Klammeraffensymbol ersetzen) - © 2002-2003 Moritz Both, Alle Rechte vorbehalten - Impressum