Dear reader, I'm not updating these pages anymore. If you have tc or ip related questions, you can post them on the LARTC mailing list.

QOS configurator

Be warned: I'm not working anymore on these scripts. They proved to me that it is possible to write a QOS configurator. I'm probaly going to redo this is in php and maybe with a mysql backend.

What

I wanted to create a GUI for tc. So I merged all my existing scripts in 1 set of scripts. The result is a tc GUI interface to configure htb classes and u32/fw filters. No cbq support. I also have some scripts to monitor the tc setup.
You can use these scripts to created the needed tc commands or only to collect tc stats.

You can download the needed scripts/files in 1 archive: qos.tar.bz2 (see the download section at the end of this page)

The scripts are not finished, so good luck :)

TODO

The graphs are not like I wanted them. I'm experimenting with the rrdtool options to find the best way I can use them.
I also need a configurable overview of the graphs so you can graph what you want.
And I also have to spend some time on the java program for the only graphing.

Web interface installation

I have all the files installed in /qos. This path is hard decoded in the scripts so it's best you extract the scripts also in /qos. So for easy documentation, I suppose you did the same. You can try to install it in an other place, but you have to check each script to change the path.

Installation commands (execute as root, tested on Debian 3.0, apache as web-server) :

cd /
tar xvjf <location of downloaded qos.tar.bz2>
cp `which ip` /qos/bin
cp `which tc` /qos/bin
chmod +s /qos/bin/tc
chown -R <web-server id> /qos/bin/tc
chown <web-server id>: config tmp
echo "include /qos/config/httpd.conf" >> /etc/apache/httpd.conf
/etc/init.d/apache restart

With <web-server id> www-data on a Debian system. Other distributions uses nobody. You can find the user for your system if you run "ps -ef | grep apache" or "ps -ef | grep httpd". It's also possible that the other files (/qos/config/httpd.conf and /etc/init.d/apache) are located on an other place.

Snmp monitor script installation

!!! I rewrote these scripts. It supports a lot more classes, but it's also slower :( !!!

I have a script that can be used together with snmpd so you can get the tc stats on a remote machine. To do so, you have to install the snmpd daemon (apt-get install snmpd on a debian system). Open the file /etc/snmp/snmpd.conf and enable public read only access :

com2sec readonly  default         public

After that, try to query your snmp server with

snmpwalk <server name> public

If this is not working, check the snmp installation.

Next step is adding the following line to /etc/snmp/snmpd.conf :

pass .1.3.6.1.4.1.2021.255 /qos/bin/snmp_tc.pl

After this, restart snmp.
You can test this by executing

$ snmpwalk <server name> public .1.3.6.1.4.1.2021.255
enterprises.ucdavis.255 = "Network interface names"
Error: OID not increasing: enterprises.ucdavis.255  = enterprises.ucdavis.255
$ snmpwalk <server name> public .1.3.6.1.4.1.2021.255.0
enterprises.ucdavis.255.0 = "2"
Error: OID not increasing: enterprises.ucdavis.255.0  = enterprises.ucdavis.255.0

Or just run /qos/bin/get_snmp.pl and check the file /qos/www/snmp.html :)

snmp script installation bis

The second snmp script is to query the server each 5 minutes and retrieve the tc stats. You can do this with snmp so you can do this on an other server. But the snmp buffer is limited so if you have a lot of classes, it's possible this will not work. As workaround, you can create the graphs on your router so you don't need snmp to get the stats.

get_snmp.pl will query the tc stats, update the rrd file and create a html page with an overview of all available stats. You can run this script from crontab by creating the file /etc/crond.d/tcc and put this line in it :

0-55/5 * * * *  root su - -c /qos/bin/get_snmp.pl

If you provide "snmp" as parameter to the get_snmp.pl script, the script will try to get the tc stats remotely with snmp. The server that needs to be polled can be configured in /qos/modules/qos_config.pm.

After running the /qos/bin/get_snmp.pl script. You should find rrd files in /qos/rrds. These files contains the tc stats.

Config files

Some very important config files are stored in /qos/config. Some of them (main and htb) are only used within the web interfaces. The others are also used in the snmp scripts. The web-server needs read-write access in /qos/config chown -R <web-server id>: /qos/config). All these config files are updated by scripts, so you don't need to edit them.

An other important config file is /qos/modules/qos_config.pm. You have to edit this file to reflect your configuration. See the documentation in the file for more information about the options you can set.

tc.interface : contains all the information about the running configuration. The script /qos/bin/find_tree.pl can be used to update the config files.
main : central configuration file (used NIC's and the link bandwidth)
htb : saved HTB configuration
filter : saved filter configuration
u32 : u32 filter configuration
tc.sh : tc scripts generated from the HTB configuration when you click on "Commit" in the web interface

/qos/cgi-bin

This directory must be published with a web-server as /cgi-bin/qos. I provided the needed apache config changes in the dile /qos/config/httpd.conf.

filter.pl : filter configuration
graph.pl : grapph configuration, unfinished
help.pl : help script
htb.pl : htb interface, it can show the running configuration and change/show the saved htb configuration
left.pl : shows a menu
main.pl : main configuration script
tcrrd.pl : this script generates the graphs

scripts and binary's

In /qos/bin are some scripts and binary's stored.

find_tree.pl : script to find the running config, it's called from the web-interface if you click on "refresh" on the "Running" page or you can run it from the command line. It will update the tc.interface files in the config directory.
get_snmp.pl : script to query a router with snmp to retrieve the qos configuration and counters, update the rrd files and create a html page with an overview of the possible graphs.
snmp.pl : script to enhance snmpd so you can query qos parameters (can be used in combination with get_snmp.pl). You can also call this script directly from get_snmp.pl if you don't use snmp (for instance if you create the graphs on the router)
tc : binary with the s-bit set so the web-server can execute it (chmod +s /qos/bin/tc).
ip : binary

Other directories/files

/qos/modules/qos_config.pm : System-wide configuration file
/qos/modules/qos.pm : general perl functions
/qos/modules/htb.pm : general perl functions
/qos/tmp/images : place where the web-server can store the graphs (so it needs write access to it
/qos/cgi-bin : cgi scripts, must be available as /cgi-bin/qos in the web-server
/qos/rrds : place where the rrd files are stored
/qos/www : must be publisched with a web-server, it contains some html, css and images.

What I want to create ....

GUI (Graphical User Interface) to tc

Graphical interface to traffic shaping
Web-based so accessible from everywhere
Configuring HTB qdiscs and filters
Monitoring : real-time and historical

How

Web based interface build with perl cgi-bin scripts
Config file stored in flat file
Pre-configured filters/rules
Support for iptables so we can shape on natting boxes
Multiple NIC support
Support for external firewall script
Monitoring
- Daemon that reports data as snmp agent
- Script that uses snmp query's to collect data and put them in a rdd database
- This can be done on a remote host or on the local host
- Monitor the classes and filter, but also extra monitors so users can find the bottleneck
- Also real-time monitoring but how ? ? ? Anybody an idea ? ? ?

Downloads

Old text and scripts

RRD monitor (integrated in QOS configurator)

I wrote some scripts so you can make nice looking graphs of the output of tc. I transfer the acutally data with snmp so you can make the graphs on another pc.

snmp.pl : addon for snmpd so you can get the info of the htb/cbq classes
proces.pl : processing the output of the snmp-script, saving the rrd data and creating a html page with an overiew of the possible graphs, required to be run each 5 minutes
tcrrd.pl : creating the graphs and extra pages with overviews of available data, must be placed in the cgi-bin of your web-server

You can find an example here. This script monitors the two interfaces of my Linux firewall.

You need the rrds perl library. For debian, the package is called "librrds-perl". You can also check it by searching for a file "RRDs.pm". For me, it's installed in /usr/lib/perl5/RRDs.pm.
And you will need the snmp interface for perl. You can check it by searching for /usr/lib/perl5/SNMP.pm.

other scripts :

u32.pl: Defining u32 filters
htb.pl: Creating a htb structure
qos.pl: Second try to create a htb structure

snmp.pl

perl script that can be used to query remotely by extending snmp. Save it in /root and make it executable.

Installation : install snmpd and enable public read only access :

com2sec readonly  default         public

And add the following line :

pass .1.3.6.1.4.1.2021.255 /root/snmp.pl

After this, restart snmp.
You can test this by executing

~# snmpwalk server public .1.3.6.1.4.1.2021.255 enterprises.ucdavis.255 = " eth0 10:1 10: 1512463 2154 0 0 309 0 0 0 8902 8902 eth0 10:20 10:1 1434530 1460 0 0 1151 309 0 0 3300 8902 eth0 10: NONE 1512463 2154 0 0 NONE NONE NONE NONE NONE NONE eth0 10:30 10:1 25538 280 0 0 280 0 0 0 4134 9152 eth1 10:1 10: 0 0 0 0 0 0 0 0 1279 9191 eth1 10: NONE 288016 1756 0 0 NONE NONE NONE NONE NONE NONE eth0 10:10 10:1 52395 414 0 0 414 0 0 0 11326 9133 "

Numbers = NIC handle parent bytes packets dropped overlimits lended borrowed giants injects tokens ctokens
As debugging, you can check if all numbers are returned and also all classes and qdiscs.

proces.pl

It uses snmpget to get the data over the netwerk from the snmpd daemon. So you can install snmpd on your firewall and install a web-server on another machine.

This script must be run from the contrab each 5 minutes. It will create a HTML-page with an overview of the class it founds on the snmpd server. It will also creates and update the rrd databases. The place of the html page and the rrd files, can be set in the process.pl script. The rrd file location must also be set in the tcrrd.pl script.

If you want to debug the script, you can execute it on the command line. If there is a problem contacting the snmp-server, the script will die with the error. Same story for the rrd-tools. If there is a problem creating or updating the rrd databases, the script will tell it on the command line.

tcrrd.pl

This script must be put in the cgi-bin of the web-server and the web-server needs read access to the rrd-databases. It will create the graphs and additional html pages.

The proces.pl script expects this script found as /cgi-bin/tcrrd.pl. If this isn't the case, you will have to change the script. The location of the rrd-files must also be correct and the same as it's defined in proces.pl.