OSS and Security @ WORK
Back from vacation 
I did pack 2.8.6.0, but it never made it to the public before I went on vacations :/
You can find 2.8.6.1 here:
http://debs.gamelinux.org/snort/hardy/
-*> Snort! <*-
Version 2.8.6.1 IPv6 GRE (Build 39)
As Kacper stated here, PRADS 0.2.0 has been released!
You can download some debian packages here, or you can check out the GIT repo here.
Bug reports, issues, thoughts or any comments are very welcome!
Enjoy!
I haven’t seen any HOWTOs yet on how to use the feature of dividing up your network into multiple snort configuration files (Virtual Networks?). I have tried this on my sensors and it works great.
Before, one would solve the problem by firing up multiple instances of snort, each with their own sets of options/arguments. Now we only start one instance of snort with a default snort config, and including config files for each IP, IP-range or VLAN that one would like to monitor. The default snort config file is used as a fall-back if the traffic is not matched in one of the virtual configs.
example:
config binding: /etc/snort/vips/snort-0.conf net 192.168.0.0/24
config binding: /etc/snort/vips/snort-1.conf net 192.168.1.0/24
config binding: /etc/snort/vips/snort-2.conf net 192.168.2.0/24
config binding: /etc/snort/vips/snort-3.conf vlan 1337
So, you have a default /etc/snort/snort.conf and configure that as a fall-back configuration (Catch all traffic not handled by your virtual configs) and then add the statements above. You can then configure snort-0.conf, snort-1.conf, snort-2.conf and snort-3.conf to handle their respective traffic (Variables, rules, preprocessors etc).
In this case, if you have:
192.168.0.0/24 on eth1
192.168.1.0/24 on eth2
192.168.2.0/24 on eth3
vlan 1337 on eth4
you would need to bond them together and have snort listen on the bonded interface.
My gut feelings are that there are some performance and memory benefits firing up one instance of snort configured with virtual-networks, then firing up X instances of snort, but I have not done any tests.
Read more in the README.multipleconfigs in the doc/ directory of the Snort Tarball.
*I would like to hear thoughts from other playing with this feature*
Small PADS info:
I bumped the version of pads to 1.2.1 (My version) after applying a patch that fixes many issues as follow:
PADS did not enable warnings during compilation. Enabling that revealed
that it did not actually include header files declaring the functions it
used. Fixing this revealed a multitude of little bugs of varying
severity, including:
- Uninitialized variables
- Unused variables
- Using close() instead of fclose()
- Using a bstring as a string, rather then using bdata()
- Useless statements
- Return without argument, even though function must return something
- Assuming time_t is int
- Passing pointers to arrays instead of the array itself
Many thanks to Erwin Paternotte for submitting this patch in the work of getting pads to work on Hardened Gentoo 64bit.
I just had to comment on this…
Read the advisory here.
My short comment; If you install any type of Software, or use any kind of mechanical devices, or do anything in life, be sure too know what you are doing.
If you buy a car, and the car door is not locked when you are handed the keys, do still lock the doors if you don’t want people to come into your car!
Snippets from the advisory:
“In order to completely protect against the vulnerability (in the short term), Nth Dimension recommend turning off the server and replacing it with another reverse proxy such as Squid.”
That would be like stepping out of a Ferrari and crawling into twelve old Tractors… I don’t think people will do that Mr. Brown…
“Should this not be possible, Nth Dimension would strongly recommend that users confirm that the master process is not listening on an external network interface.”
This is so much easier to do than migrate to Squid or alike? And the right thing to do if you are not in a trusted environment. Again, do lock your car door.
“In the latter case, users should confirm that only trusted users have SSH access to the system.”
As a rule of thumb: You should NEVER have untrusted users on your systems if you value your data on it or the data accessible from it.
There are tons of information on how to harden a Operating System (OS). One of the first and most common step is to make sure the system does not listen on network ports that you don’t want it too. I feel that the advisory is bogus because it is a feature of Varnish.
The advisory should have been aimed at the distributions that have packages that don’t implement “non-clue friendly defaults”.
That said, there is nothing stopping me from sending out my passwords via email once I have installed a browser and I manage to log into my gmail account…
Subject: “Medium security hole in Mozilla Firefox”
Body: “I’ve identified a couple of security flaws affecting Mozilla Firefox (All versions) which may allow privilege escalation….”
“Grumpy day”
As Ubuntu 10.04 (Lucid Lynx) is the next LTS (Long Time Support) version of Ubuntu that is coming out soon (April 29, 2010), I have started to look at how sguil and my dot deb packages will work.
I installed Lucid Lynx yesterday and installed my server and sensor debs on it.
Some first notes:
* MySQL is not eating the create_sguildb.sql (Just remove the comments)
* Lucid (and Karmic) does not ship with tclx8.3(Installing the Hardy version worked just fine)
(I filled a bug report to Ubuntu, hoping to get tclx8.3 into the final release…)
So, from my first tests, it seems to work fine!
I have yet to test the sguil-client on Lucid, and also I did not get to test with extensive amount of traffic and operations on the Lucid test server.
So, looking promising
It has been a long journey, but after about two years, I finally got a way to populate Snorts host attribute table, automagically(tm)!
When I started this, my first option was to use nmap to scan the network to populate the information. This was scratched, as my goal was to be non intrusive and always up to date (The minute a new host popped up, I want to know). Scanning 65535 ports times two for each of the hosts Im monitoring is not an options also… I started to look at the Open Source tools out there, on which to use to get the information from. As I was familiar with p0f and PADS, I saw that they could do the job, but they needed some band-aid to work together, and they where lacking active development… p0f has a DB patch/version, and I already had PADS hooked up in Sguil, so I had the info in a DB, but not in the way I wanted it. So I started out on a journey to merge the two projects, enhance them, and try to speed things up a bit.
The project is still in development, but the main parts are done. It is useful in the way that it will print out information about detected hosts, like this in verbose mode (And yes, it also does IPv6):
2a02:c0:1002:100:21d:72ff:fe92:728,[syn:S4:64:1:40:M1440,S,T,N,W7:Z],[Linux:2.6 (newer, 7) IPv6],[link:IPv6/IPIP],[uptime:2hrs],[distance:0]
2a02:c0:1002:10::2,[synack:5712:63:1:40:M1440,S,T,N,W7:ZAT],[Linux:2.6 (newer, 7) IPv6],[link:IPv6/IPIP],[uptime:4069hrs],[distance:1]
2a02:c0:1002:100:21d:72ff:fe92:728,[ack:45:64:1:*:N,N,T:ZAT],[Linux:2.6],[uptime:2hrs],[distance:0]
2a02:c0:1002:10::2,[service:OpenSSH 5.1p1 (Protocol 2.0):22:6],[distance:1]
2a02:c0:1002:10::2,[ack:45:63:1:*:N,N,T:ZAT],[Linux:2.6],[uptime:4069hrs],[distance:1]
2a02:c0:1002:100:21d:72ff:fe92:728,[client:OpenSSH 5.1p1 (Protocol 2.0):22:6],[distance:0]
At the moment, it also makes a file in your /tmp/ dir, /tmp/prads-asset.log, which presents the info in the following way:
2a02:c0:1002:100:21d:72ff:fe92:728,0,56268,6,SYN,[S4:64:1:40:M1440,S,T,N,W7:Z:Linux:2.6 (newer, 7) IPv6:link:IPv6/IPIP:uptime:2hrs],0,1269420770
2a02:c0:1002:10::2,0,22,6,SYNACK,[5712:63:1:40:M1440,S,T,N,W7:ZAT:Linux:2.6 (newer, 7) IPv6:link:IPv6/IPIP:uptime:4069hrs],1,1269420770
2a02:c0:1002:100:21d:72ff:fe92:728,0,56268,6,ACK,[45:64:1:*:N,N,T:ZAT:Linux:2.6:uptime:2hrs],0,1269420770
2a02:c0:1002:10::2,0,22,6,SERVER,[ssh:OpenSSH 5.1p1 (Protocol 2.0)],1,1269420770
2a02:c0:1002:10::2,0,22,6,ACK,[45:63:1:*:N,N,T:ZAT:Linux:2.6:uptime:4069hrs],1,1269420770
2a02:c0:1002:100:21d:72ff:fe92:728,0,22,6,CLIENT,[ssh:OpenSSH 5.1p1 (Protocol 2.0)],0,1269420770
Input from the community on how to present the information/output for a best possible way for integration with other applications are welcome.
To try it out, this is what I believe is necessary to install on my Ubuntu machine to run it:
$ sudo aptitude install build-essential git-core libpcre3-dev libpcap0.8-dev
$ git clone http://github.com/gamelinux/prads.git
$ cd prads/src/ && make
$ # then to start it
$ sudo ./prads -i ethX -v
For populating the Snort host attribute table, there is a script in the tools dir, prads2snort.pl, which takes the prads-asset.log file and processes it.
Example:
$ perl prads2snort.pl -i prads-asset.log -o hosts_attribute.xml -v -f
The -v (verbose) mode prints out some details, which can be good to check to see if stuff seems to be detected correctly.
Snort supports reloading of the attribute table if you give it the signal 30. (kill -30 <snort-pid>). This means, that if you discover a difference in your host attribute table (Say you got a new http service some where, or a host has changed OS), you can swap out the attribute file with a new updated one, and tell snort to reload its attribute file without restarting snort! Darn cool if you ask me
You can read more about Snort and its host attribute table here, and you can read about another tool called Hogger here. Also, you should read the Snort documentation section 2.7 (around page 104/105) “Host Attribute Table”.
I would once again like to thank Michal Zalewski and Matt Shelton for their work on p0f and pads. I would also like to thank Martin Roesch & The Snort Team (And all the contributers) for a great application and all the effort they have put into Snort and its surroundings. (Virtually giving you the price for best Open Source security application 2000 - 2010!).
Attribute Table Loaded with 980 hosts
…
Attribute Table Reload Thread Starting…
Attribute Table Reload Thread Started, thread 363022672 (15333)
…
$ /bin/kill -30 15333
…
Swapping Attribute Tables.
…
$ /bin/kill 15333
…
===========================================
Attribute Table Stats:
Number Entries: 980
Table Reloaded: 1
===========================================
…
Saturday 18 Jun 2005 Matthew J. Shelton released PADS. PADS is a great tool, and the security industry really needs a good open source passive asset tool. But since 2005, PADS development has stopped, and there are no place to send new signature or patches/bugs too, and hope that they will get added/fixed. Also, logical, there are no new features being added…
I have used PADS in my Sguil setup, but have seen that it lacks stuff that I wanted to have there, and also, there has been some problems running PADS on newer operation systems. I have a copy of the pads-1.2-sguil-mods.tar.gz, and I added it to github yesterday, fixed some issues when writing data to the FIFO file for Sguil, added some patches, among vorants vlan patch. I compiled it on Ubuntu Hardy and Jaunty (x86_64), and it has been running fine for 12+ hours.
If you try out my version of PADS and have issues, I will try to solve them. I see there are some, in stuff that I don’t use, and if I one day find the urge, I’ll fix them on my own.
I should probably also mention, shamelessly again, that there is a project that takes PADS to the next level and then some more….
You can read about PRADS here and what more it can do for you.
Loglevel: INFO
I have packed snort 2.8.5.3 for Ubuntu Hardy and Jaunty:
http://debs.gamelinux.org/snort/hardy/
http://debs.gamelinux.org/snort/jaunty/
-*> Snort! <*-
Version 2.8.5.3 IPv6 (Build 124)
These are general pointers too things you want to dig into when you need to optimize Snort. If you are one of those who believe that Snort can’t go beyond 100Mbit/s and still not drop packets, you should read on. Comments/feedback/new tips/corrections on how to tune a Snort system is very welcome.
–[ Optimize the hardware ]–
This is always a moving target… And you need to keep yourself updated on the topic and pay attention when you buy your hardware. If someone in the community is maintaining a updated list of such hardware, give me a note!
Intel Network Interface Controllers(NIC) are the off the shelf choice of network adapters, 825NNXX PCI Express series with minimum TCP segmentation offload, TCP, UDP, IPv4 checksum offload, interrupt moderation, and maybe Bypass if you use inline mode/IPS.
If you want to pay someone that already has researched a bit (pure speculation from my side), then maybe Endace could be a choice. But if you first go there, then why not just go straight to Sourcefire (The makers of Snort).
(Matt Jonkman states that you can increase your Snort throughput up to a 16-fold increase if you introduce Endace platform’s acceleration features. Matt is the founder of Emerging Threats, and also deep into the OISF and the Suricata project)
At one time (early 2009), a discussion on IRC (Freenode) summed up in something like this:
“IICH8 southbridge, and 975G north bridge performing at 1066MHz, 8GB of 1333MHz DDR2 ram on a Intel quad core 3.2Ghz 8MB L2 cache processor running at 1333 MHz FSB and Intel 825NNXX PCI Express Gigabit Ethernet Controller.” - for a high end sniffer at that time.
Your whole system would benefit great from fast hard drives, as I/O too hard drives generally sucks juice, and locks up the system.
To sum it up:
Fast CPUs, fast RAM, fast buses, fast hard drives and a good network adapter.
–[ Optimize the Linux kernel ]–
In the file /etc/sysctl.conf - you should consider options like these:
# Just sniffing:
net.core.netdev_max_backlog = 10000
net.core.r mem_default = 16777216
net.core.rmem_max = 33554432
net.ipv4.tcp_mem = 194688 259584 389376
net.ipv4.tcp_rmem = 1048576 4194304 33554432
net.ipv4.tcp_no_metrics_save = 1
# IF also in Inline mode:
net.core.wmem_default = 16777216
net.core.wmem_max = 33554432
net.ipv4.tcp_wmem = 1048576 4194304 16777216
# Memory handling - not that important
vm.overcommit_memory=2
vm.overcommit_ratio = 50
–[ Optimize your network interface card ]–
Change the RX and TX parameters for the interfaces. The following command will display the current settings and the maximum settings you can bump them up to.
# ethtool -g ethX
To change settings, the command is something like this:
# Just sniffing
ethtool -G ethX rx
# and for inline mode, also add
ethtool -G ethX tx
Adding the command to /etc/rc.d/rc.local so that they are execute automatically when you boot would be a good idea.
–[ Optimize Snort ]–
Snorts performance is based on several factors.
1 - YOUR network!
2 - How snort is compiled
3 - Preprocessors enabled
4 - Rules
5 - Snort in general and snort.conf
–[ 1. YOUR network! ]–
Your network is a variable that is most likely not like any other networks. The amount of concurrent connections, packets and packet size flowing through, is most likely unique. Also, depending on the payload in your packets, Snort will perform differently. Also, if your $HOME_NET is one single host, compared to complex list of “networks” and “!networks”, Snort will spend more time figuring out what to do.
–[ 2. How snort is compiled ]–
First, I recommend only to compile Snort with the options that you need. I used to compile Snort in two different ways, one including options among “–enable-ppm and –enable-perfprofiling” and one without. But as my sensors are not suffering enough at the moment, I include them both by default, for easy access to preprocessor and rule performance data if I need too.
Also, I have not confirmed this, because its out of my budged reach, but the rumors are that Snort performs up to 30% better if it is compiled with an Intel C compiler (and probably run on pure Intel hardware).
If you use Phil Wood mmap libpcap and compile Snort with that, you will get some better performance in the packetcapture, giving you less dropped packets. I nice writeup/howto is found here.
–[ 3 - Preprocessors enabled ]–
How many and which preprocessors you have enabled is also playing a role on the total performance of your system. So if you can, you need to reduce the numbers of preprocessor to a minimum. Also you need to read the Snort documentation, and figure out the best settings that you can live with for each preprocessors that takes configuration options. The flow_depth parameter in the http_inspect preprocessor is a good example.
Here are two settings/views I switch between when profiling preprocessors:
config profile_preprocs: print 20, sort avg_ticks, filename /tmp/preprocs_20-avg_stats.log append
# And
config profile_preprocs: print all, sort total_ticks, filename /tmp/preprocs_All-total_stats.log append
You should now review the *stats.log files and make changes based on your interpretation, and profile again to see if things get better or worse.
–[ 4 - Rules ]–
The amount of rules also affects the performance of Snort. So tuning your rules to just enable the ones that you need is essential when aiming for performance.
Also, how a rule is performing on your network, might defer from how it performs in my network… That said, you need to profile your set off rules, and tweak or disable them so your system uses less overall “ticks”.
Here are two settings/views I switch between when profiling rules:
config profile_rules: print 20, sort avg_ticks, filename /tmp/rules_20-avg_stats.log append
# And
config profile_rules: print all, sort total_ticks, filename /tmp/rules_All-total_stats.log append
You will get a fairly good view of rules that needs/should/would benefit from tuning/disabling.
–[ 5 - snort in general and snort.conf ]–
* search-method
You should look into which search-method snort is using. The default search method is AC-BNFA (Aho-Corasick NFA - low memory, high performance). This is probably the best overall search method, but if you have the RAM for it, AC (Aho-Corasick Full - high memory, best performance) would be a better choice. Snort 2.8.6 added a new pattern matcher named AC-SPLIT. The new pattern matcher is optimized to use less memory and perform at AC speed. This would probably the choice for the future? Need to test right away
To enable it, add something like:
config detection: search-method ac-split, max-pattern-len 20,
search-optimize
* Latency-Based Packet Handling
If you have a problem with dropped packets, I would say over 1% on an average, I would recommend enabling Latency-Based Packet Handling. You should run some tests in your environment to find a value that works for you, but the general situation is like this:
If your Snort “Packet Performance Summary” is telling you that your “avg pkt time is 10 usecs” then Snort can inspect about 1000 packets in 10000 usecs. If a packet for some reason is using 10000 usec to get through Snort, you may have dropped/sacrificed 1000 other packets in that time frame, just to inspect this packet. So if you configure max-pkt-time to be 1000, Snort will stop inspecting packets that take more time than 1000 usec, and in this basic example leaving you with 100 dropped packets instead of 1000. You choose! (The example is not technical correct, as a packet can take over 10000 usec with out Snort dropping any packets at all (Imagine if there is only one packet going through snort that day…), but in my tests, this is more or less the real world outcome of enabling Latency-Based Packet Handling).
Example:
config ppm: max-pkt-time 10000, fastpath-expensive-packets, pkt-log
Other keywords you should be aware off in the Snort config, that I don’t want to go into details about, as I don’t have enough Snort-Fu about to stand firm, and the doc is rather lacking! I have a personal understanding of what they do, and how it effects performance etc. but if anyone has some nice writeup of the topics, please point me to it!! :
* Event Queue Configuration
* Latency-Based Rule Handling
–[ Additional notes ]–
Obviously, if you need to go as fast as possible, your system should not be used for lots of other different stuff. So keep your running processes/services too a minimum.
Snort is also, as far as I can tell, single threaded when it comes too packet inspection. There is a pdf here from Intel, explaining how Sensory Networks Software Acceleration Solutions boost performance of Snort and things alike, making them Multi-core enabled/aware.
That said, Snort benefits from sticking to one CPU, so using schedtool in a proper way, might help snort perform overall better. If you are running multiple instances of Snort on one multi-CPU server, you should use schedtool to stick each Snort process to its own physical CPU etc. Example:
$ man schedtool # and read about “AFFINITY MASK” and understand the difference between cpu-cores and hyper-threading etc.
$ schedtool <pid of snort> # Displays current settings
$ schedtool -a 0×01 <pid of snort> # Pin the snort process to one CPU (The first)
$ schedtool -M 2 -p 10 # Change the policy to SCHED_RR and set priority to 10 (0 highest, 100 lowest)
$ schedtool <pid of snort> # to verify your changes
Always when optimizing a system, you should have some sort of measuring system. I use Munin. I wrote some basic Munin plugins for Snort which monitors the most important stuff.
And as always,
“Measure, don’t speculate” — Unknown
“Premature optimization is the root of all evil” — Tony Hoare
This work is licensed under a Creative Commons License.
·
XHTML
·
CSS