Index of /code/PostgreSQL/iplog

	Name	Last modified	Size	Description
	Parent Directory		-
	LICENSE	24-Feb-2007 21:17	1.4K
	Makefile	24-Feb-2007 21:17	281
	README.html	05-Mar-2007 20:13	2.7K
	ether.h	24-Feb-2007 21:17	2.5K
	ip.h	24-Feb-2007 21:17	5.7K
	iplog.c	05-Mar-2007 20:04	32K
	iplog.db.drop.sh	24-Feb-2007 21:17	69
	iplog.db.init.sh	05-Mar-2007 20:04	1.3K
	ipproto.h	24-Feb-2007 21:17	4.7K
	tcp.h	24-Feb-2007 21:17	3.5K
	tcpdump-stdinc.h	24-Feb-2007 21:17	4.9K
	udp.h	24-Feb-2007 21:17	3.6K

iplog

Promiscuously monitor an interface for traffic traversing a specified gateway. Collect aggregate statistics about the monitored packets, and dump those statistics at a specified interval into a database.

There are a number of header files included from the tcpdump distribution. These are used to provide the data structures used to dissassemble packets captured with the pcap library.

The interesting files here include:

• iplog.c. The core application.
• iplog.db.init.sh. A simple PostgreSQL setup script.
• iplog.db.drop.sh. A simple PostgreSQL drop script.

A simple script to run this application might look like:

#! /bin/bash

/root/bin/iplog \
    --dbname iplog \
    --host localhost \
    --username iplog \
    --interval 15 \
    --interface eth1 \
    --gatewayip 138.110.100.1 \
    --gatewaymac 0004c0d32470 \
    < /root/bin/.secretpasswordfile

Data collection is keyed on source_ip + destination_ip + ip_protocol + destination_port. After 'count' packets have been collected, the keyed aggregate statistics are logged to PostgreSQL. The logging process runs in its own thread, so as not to block the collection process.

The logged data looks like this:

iplog=# select * from iplog limit 1;
-[ RECORD 1 ]----+------------------------------
logged           | 2007-03-05 18:33:27-05
source_ip        | 138.110.72.109/32
destination_ip   | 24.3.10.117/32
ip_protocol      | UDP
destination_port | 63293
timespan         | 00:15:00
packet_count     | 1
ethernet_caplen  | 104
ethernet_len     | 104
ip_total_size    | 90
ip_data_size     | 70
payload_size     | 62
id               | e54c528dff2a4b05a799f03f5312d1ed

As of 2007-02-24, this code is under active development. Mostly I'm trying to figure out the best way to optimize performance.

Note: you'll need to know a thing or two about PostgreSQL performance tuning if you want to shovel this much data...