Benjamin Smee - strerror@disciplina.net
1.1
I have been using FlowScan for years and have thought that it was one of the better applications around for displaying network data. The problem was that it was very difficult to setup, each time you did so requiring a significant amount of work (for an idea of what you have to do manually see this HOWTO). What I wanted to do was to automate most of the setup, so that administrators could focus on the configuration and nothing else. Hopefully with this HOWTO and the ebuilds I have achieved that.
Monitoring is essential for looking after any network. Fortunately there are a plethora of tools available for all kinds of different monitoring, however, one area which lacks is real time graphs. Many people immediately think of MRTG when you talk about real time network related graphs, and MRTG is great at what it does, but it has some limitations, most importantly that it doesn't really break down the traffic and its hard to get graphs stacking against other graphs to see things real time and clearly differentiated. At the very least it would require extensive configuration just to get the graph that you wanted, which might only be a once off thing. To this end tools like FlowScan were written. There is a lot of documentation on the architecture of FlowScan ( in particular I highly recommend the manual written for JKFlow which extensively covers all the alternatives as well as JKFlow) but I think it is important to outline the salient points of what we will be building.
Our network monitoring will be built to accept flows. These flows can be from a router (almost all routers and switches from all companies can send flows) or even from a linux box via utilities such as fprobe. The flow of data in our monitoring system will be as follows. We will be running flow-capture (part of flow-tools) on a port to accept flows sent to us, flow-capture will periodically write the flows to a desitination we will specify and at the same time make a symbollic link to the flow. Now we will run FlowScan that will process these flows every 30 seconds, the byproduct of which is that it will delete the flow, but because we will point FlowScan to the symbollic links instead of to the real data our actual data will remain intact. FlowScan then uses a reporting module to generate rrds which will be used to generate our graphs via calling a cgi written for that module.Finally the reporting module will also write out some summary information. FlowScan comes with 2 reporting modules, however, neither are any good by todays standards. Instead this HOWTO will focus on two additional reporting modules, CUFlow and JKFlow which can be used instead of the ones that come with FlowScan to generate more complex and powerful graphs.
To get your interest you can see some interesting graphs done with FlowScan and CUFlow here. Alternatively you can see a demo of JKFlow here. Please note that the JKFlow demo requires flash.
In order to get the most benefit from your network monitoring you should have a good understanding of the architecture of the network you are trying to monitor, that said for the purposes of ensuring it is all working you can just export flows from your local machine. The following applications are required for this HOWTO:
net-analyzer/flow-tools
net-analyzer/FlowScan
dev-perl/Cflow
dev-perl/CUFlow
dev-perl/JKFlow
The required applications for this HOWTO are now in portage. A simple: "emerge JKFlow" will install everything you need if you want to use JKFlow or "emerge CUFlow" if you prefer CUFlow.
You must pick which reporting module you will use as this is really the deciding factor for how you will configure the monitoring, so your main choice will be between CUFlow and JKFlow. It seems to me that JKFlow has far more functionality, flexibility and scaleability then CUFlow and it is the obvious choice for enterprise level network monitoring or any complex network configurations. Though JKFlow is probably the suprerior option, it has the negatives of being considerably harder to configure as well as generating larger files, which obviously in turn require more computational power and more disk space. Generally speaking if you can put the extra time into learning how to configure JKFlow I would recommend using it (the extra disk space and CPU overhead really are fairly small and won't be an issue unless you are aggregating many gigabytes per second of bandwidth) over CUFlow, but if you just want a simple setup with minimal fuss then CUFlow is for you.
Once you have emerged JKFlow then you will need to create a JKFlow.xml in :
/var/lib/flows/bin/
There are two example configuration files in there and the JKFlow manual has extensive examples and explanations to show to create your own.Once this is done you need to start up flow-capture, first edit the /etc/conf.d/flowcapture file to your tastes and then start it via "/etc/init.d/flowcapture start". Now assuming you are sending the machine flows, you should start to see them appear in /var/lib/flows/ft and symlinks created in /var/lib/flows. In a large configuration you will have many routers and machines exporting flows to you, but in order to test that the configuration is working you might want to emerge fprobe and then run a command like this:
fprobe -l /var/log/fprobe.log:1 -a YOUR_IP -i eth0 -fip YOUR_IP:2055
NOTE: I recommend creating a user for fprobe and running it in a chrooted environment if you are going to continue using it as your flow exporter.
Once you see that the flows are being captured we need to configure FlowScan to use JKFlow as its reporting module. This is done by editing /var/lib/flows/bin/flowscan.cf and adding this line:
ReportClasses JKFlow
You will also have to comment out any other ReportClasses lines. Once that is done it is time to start up FlowScan itself, this is done via "/etc/init.d/flowscan start".
At this point you should be receiving flows via flowcapture and processing them with FlowScan which in turn is calling JKFlow to do the analysis. All that is left to do is see the results. The ebuild will put the graphing utility into /var/www/localhost/cgi-bin by default, so if you want to create a separate vhost for this then you will need to move it. My vhost entry for this setup looks like:
<VirtualHost 100.100.100.100:80>
ServerAdmin webmaster@foo.net
ServerName flowscan.foo.net
DocumentRoot /var/www/localhost/htdocs
ScriptAlias /cgi-bin/ /var/www/localhost/cgi-bin/
<Directory />
AuthName "Foo Networks - Restricted Access"
AuthType basic
AllowOverride AuthConfig
Options ExecCGI FollowSymLinks Indexes
DirectoryIndex cgi-bin/JKGrapher.pl
Order allow,deny
Allow from all
</Directory>
ErrorLog /var/log/apache2/flowscan/error_log
CustomLog /var/log/apache2/flowscan/access_log combined
</VirtualHost>
The important part being that you enable ExecCGI.
Once that is done you should be able to go to:
http://flowscan.foo.net/cgi-bin/JKGrapher.pl
and voila ;)
This is almost identical to JKFlow, except that the configuration file, CUFlow.cf will require minimal editing and should almost work out of the box once you have replaced the IP's in it with your own networks. The steps are:
modify /var/lib/flows/bin/CUFlow.cf
modify /etc/conf.d/flowcapture
start flowcapture via "/etc/init.d/flowcapture start"
check that flows are being captured. Files in /var/lib/flows/ft should appear
modify /var/lib/flows/bin/flowscan.cf - add the line:
ReportClasses CUFlow
and remove all other ReportClasses lines.
start flowscan via "/etc/init.d/flowscan start"
While almost all cisco routers are able to export in netflow version 5 format (the format that flow-tools is capable of parsing), many other vendors no longer support this format and have instead moved to the open sflow format. Unfortunately I am unaware of any open source tools that will work well with sflow, but I have recently put sflowtools into portage which will convert sflows to netflows. This gives you an option for displaying data from things like foundry routers / switchs, juniper and many more.
One thing that might come in handy is the ability to redirect or replicate flows. This is achieved by using a tool that is part of flow-tools called "flow-fanout". With this tool it is possible to have a number of flow collectors around the network and still have them all redirect the flows back to one central server for analysis. Additionally it would be possible to configure the flow collector to duplicate the flows so that you could send one set of flows to say CUFlow for reporting and the other to JKFlow.
If you change the directory of where the flows are kept please note that you will have to change the following files:
/var/lib/flows/bin/linkme
/var/www/localhost/cgi-bin/{JKGrapher.pl,CUGrapher,pl}
/var/lib/flows/bin/flowscan.cf
/var/lib/flows/bin/JKFlow.pm (if you are using JKFlow)
When you first setup this monitoring solution bear in mind that it will take at least 5 minutes (using the configuration files I provided) for data to be collected and then up to another 30 seconds for it to be processed. Once that is done it will still only be ONE data point on the graph so don't expect pretty pictures immediately, it takes time to gather all the data.