01.11.10
Posted in Information, OpenSourceSoftware, Security, Sguil, Snort, daemonlogger, forensics, cxtracker, Suricata, fpcgui at 11:06 pm by Edward Bjarte Fjellskål
I started a little project of mine that I have been thinking about since the summer of 2008 (Also see this post). I saw that it was a problem finding vendors selling a cheap setup for a full packet capture solution. The recommendation was to set up a Linux server on your own, run tcpdump and spool pcaps to disk. Well, once you have all that data, you need some way to manage it. I thought about using sancp to index the connections, and tools like tcpxtract, foremost, dsniff, chaosreader, tcptrace and combine features from xplico to add some extra value and possibilities on top.
So I started my project back in september 09, calling it FPCGUI (Full Packet Capture Graphical User Interface). It is currently supporting daemonlogger/tcpdump/sancp for spooling pcaps with a wrapper script that puts pcaps in directories based on “year-month-date”. cxtracker/sancp can be used for connection profiling/tracking, writing session data to disk, where I have written fpc-session-loader.pl which picks up the session data files and inserts them to a mysql database. If I now have an interest in seeing all the traffic from one host, I can do a search in my webgui and get the data. I can do rather interesting queries on all the data from cxtracker/sancp, and get interesting results.
I use cxtracker in my setup, as it collects meta data on both IPv4 and IPv6 connections. I have also managed to store IPv4 and IPv6 addresses in the mysql database in a reasonable and usable way.
I have just finished writing a PHP webgui, where I can enter a search term, and get a list (or just a single session if I’m specific enough), click on the session of choice, and up pops a download dialog, where I can choose to open the pcap straight away in wireshark! The pcap of the specific session is carved out from the pcaps for the relevant period (days) when the session took place. More or less the same functionality you find in a Sguil stack setup. I wrote the php-gui in such a way, that it can take search terms via an URL, like “?srcip=10.10.10.10&srcport=80″ and so on, making it easier to integrate with other applications.
Example screenshot of what happens when you click on an event:
I have associated the pcap files with: ‘Content-Type: application/pcap-capture’ and set firefox to spawn wireshark for those files automatic 
So now I’m one step closer to having Full Packet Capture with my Sourcefire 3D system! Just need to find out what part of the 3D webgui code to hack, to add my custom <click here to get the pcap of the session that triggered the event> Of course I can enter the data manually, but I’m lazy, and I like to hack stuff
The project i hosted here. Any thoughts are more than welcome.
Permalink
10.23.09
Posted in OpenSourceSoftware, Security, cxtracker at 7:20 am by Edward Bjarte Fjellskål
I just compiled Phil Wood’s memory map enabled libpcap and compiled cxtracker against it.
From earlier test, comparing sancp and cxtracker, it seems that cxtracker is in the range of 25 to 30% less CPU intensive than sancp with regular libpcap.
Output from pidstat:
Average: PID %user %system %CPU CPU Command
Average: 28269 1.42 5.75 7.17 - sancp
Average: 27383 1.12 4.44 5.56 - cxtracker
Running cxtracker with mmap libpcap, the results are even better:
Average: PID %user %system %CPU CPU Command
Average: 5450 1.98 8.97 10.95 - sancp
Average: 5322 1.38 2.79 4.16 - cxtracker
As you see, over 60% less CPU usage!
The memory usage is of course higher, but thats the price you have to pay for less CPU usage…
The tests are done about 5 minutes after sancp and cxtracker is started, and the summary are generated from a 5 second interval with 60 counts with pidstat.
Permalink
10.17.09
Posted in Information, OpenSourceSoftware, Security, Sguil, forensics, cxtracker at 10:30 pm by Edward Bjarte Fjellskål
I started out writing cxtracker and PRADS in perl, as my C fu was kinda rusty, and perl programmers claimed that perl was fast enough… It turned out, that perl was not fast enough for my purpose, but it added to my perl skillz at least.
So I wanted to rewrite cxtracker and PRADS in C, and the last three weeks, I have re-learned C and rewritten cxtracker. It has been many late hours and small bugs have been annoying me way too much. But I have had the chance to play more with debuggers and C which I find very interesting and pleasing.
cxtracker can now be used instead of sancp in a sguil setup. cxtracker is also meant to be used in another little project I have going, FPCGUI (Full Packet Capture GUI), but thats another blog entry soon to come.
cxtracker also logs IPv6 traffic, something that sancp does not. Sguil does not eat IPv6 yet, so to use cxtracker with sguil, a bpf filer for not inspecting IPv6 traffic should be used.
On my sensors, cxtracker seems to use about 20-30% lesser CPU time than sancp, which makes me really happy! The memory footprint is the same as sancp.
The reason I started with cxtracker as my first C project, is that it is a corner stone in my other projects, FPCGUI and PRADS. In FPCGUI, cxtracker is used to track IP sessions and storing them to a DB. In PRADS, it also tracks sessions, so one can limit the amount of data to check for service signatures in (Just check xx first packets, or xxx bytes of data after initial connection).
cxtracker is hosted on github : http://github.com/gamelinux/cxtracker
git clone git://github.com/gamelinux/cxtracker.git
It uses libpcap and I have yet only tried to compile it on Ubuntu and Debian machines (x86 and x86_64).
To test it:
# libpcap and a build environment is needed.
$ git clone git://github.com/gamelinux/cxtracker.git
$ cd cxtracker/src/
$ make
$ ./cxtracker -h
USAGE:
$ cxtracker [options]
OPTIONS:
-i : network device (default: eth0)
-b : berkeley packet filter
-d : directory to dump sessions files in
-u : user
-g : group
-D : enables daemon mode
-h : this help message
-v : verbose
$ ./cxtracker -i eth0 -D -d /nsm_data/sensor-hostname/sancp/ -u nsm -g nsm -b ‘ip’
If you try out cxtracker, feedback is more than welcome!
Permalink
06.08.09
Posted in Information, OpenSourceSoftware, Security, Sguil, cxtracker at 7:12 pm by Edward Bjarte Fjellskål
I just tested my latest perl project, cxtracker, with sguil.
CxTracker (Connection Tracker) is a passive network connection tracker for profiling, history, auditing and network discovery. It can be used as an replacement for sancp in the sguil setup.
I started implementing this because I am thinking of making prads a bit more connection oriented. In stead of branching prads, I started a small perl script, and I quickly saw that it could do well as a standalone daemon in my sguil environment. So it now has its own life.
I will polish a bit more on it, and then start porting the functionality into prads for some performance testing. The idea, is that making prads aware of connections, one can do regexp on parts of the traffic, and not the whole traffic (to save cpu cycles on tcp/udp service/client detection).
Check it out/Clone it! Feedback is always wellcome!
“Know your connections!”
Permalink
