README

   1 ============================================================================
   2
   3 Prometheus QoS - steal fire from your ISP !
   4 "fair-per-IP" quality of service utility
   5 requires Linux kernel with HTB qdisc enabled
   6
   7 Copyright(C) 2005-2008 Michael Polak (xChaos) original source code
   8 Copyright(C) 2007-2008 Martin Svoboda (Ludva) iptables-restore, CLASSIFY
   9
  10 Credit: CZFree.Net+Netdave (idea), Aquarius (.rpm), Gandalf (.deb)
  11 ...and: Martin Devera (.cz) for his HTB qdisc (of course)
  12 ...and: Jakub Walczak (.pl) for providing feedback and patches
  13 ...and: Ing. Jiri Engelthaler (.cz) for bugfixes and Asus WL500 port
  14 ...and: Dial Telecom (our slightly expensive ISP) for chance to test it
  15
  16 Feedback: xchaos(at)arachne.cz
  17 Homepage: http://gpl.arachne.cz
  18 SVN tree: https://dev.arachne.cz/svn/prometheus
  19 ============================================================================
  20
  21 QoS (or Quality-of-service) is IPv4 traffic shaper replacement for Internet
  22 Service Providers (ISP). Dump your vintage hard-wired routers/shapers
  23 (C|sco, etc.) in favour of powerful open source and free solution !
  24
  25 Prometheus QoS generates multiple nested HTB tc classes with various rate
  26 and ceil values, and implements optional daily traffic quotas and data
  27 transfer statistics (as HTML). It is compatible with NAT, both asymetrical
  28 and symetrical, yet still provides good two-way shaping and prioritizing,
  29 both upload and download. Prometheus QoS allows both "hard shaping"
  30 (reducing HTB ceil value for aggressive downloaders) and "soft shaping"
  31 (keeping HTB ceil, but reducing HTB prio, probably optimal solution for
  32 normal users).
  33
  34 Prometheus iGW was written in C<<1, which means it compiles simply with
  35 GNU C Compiler, and doesn't require any external liberaries (except libc)
  36 and huge interpreter packages (like Perl or Java) to run. However, it
  37 depends on HTB algorithm hardcoded in Linux kernel. It is currently being
  38 tested in real-world enviroment to provide QoS services on 30 Mbps internet
  39 gateway and proxy being used by 2000+ PCs connected to gateway using
  40 CZFree.Net broadband community network.
  41
  42 Advantages over more straightforward traffic control scripts include
  43 HTB fine tuning features (rate and ceil magic), data transfer statistics,
  44 optional data transfer quotas, full NAT (both symetric and one way)
  45 compatibility and optinal sharing of bandwith by IPs in completely
  46 different subnets.
  47
  48 Performance and scaling - current release:
  49
  50 we run Prometheus QoS on Celeron 2.8 Ghz serving around 600 individual
  51 traffic classes (fine tuning is using five user-defined prometheus.conf
  52 keyword) and another 2000 IPs sharing bandwith with certain other IPs
  53 ("sharing-" keyword). Prometheus QoS is especially strong tool if you want
  54 IP's from different subnets to share the same traffic class.
  55
  56 With 30 Mbps (each way) total capacity of line, Cisco Catalyst 2950 on
  57 ISP side and up to cca 6000 packets per seconds, we were running into some
  58 problems with overall system load. We moved QoS from Athlon 1700 XP to
  59 Celeron 2.8 Ghz, and kept all SNAT related stuff (see optinal-tools directory)
  60 on Athlon 1.7 Ghz, which alowed for peak throughput up to 10000 pps.
  61
  62 Performance fine tuning - history:
  63
  64 With kernel version 2.4.20 and release 0.2 we started to experience problems
  65 at cca 1500 packets/sec. However, with new iptables indexing feature
  66 implemented in 0.3 release, system load seems to be approximately
  67 10 times lower. Same HW was later shaping 2000 packets/sec without problems,
  68 and it looked like comparable relatively low-end system should be able to do
  69 traffic shaping for at least 10000 packets/sec (well, if HotSaNIC was turned
  70 off, of course <g>). With 0.6 release and dynamicaly calculated iptables
  71 indexing scheme we made it up to 6000 packets/sec, and then we ran into some
  72 performance-related problems, which may be related to the fact we are doing
  73 SNAT of 1000+ individual IP addresses on the same machine which is doing
  74 also the QoS: something on the way seems to be limited to 34 Mbps HD
  75 (half-duplex, sum of upload and download) no matter what we try. Our ISP
  76 claims the fault is not on his side, so our next step will be to separate
  77 traffic shaping and massive SNAT (IP masquerading) and assign separate
  78 PC-based router to do each task.
  79
  80 Maximum performance observed with prometheus 0.6 and hashtable optimization
  81 of tables with individual SNAT targets was up to 9000 packets/sec at cca 40
  82 Mbps half-duplex (more then 20 Mbps fyull-duplex). However, this required
  83 massive optimization, including
  84 echo -n 65000 > /proc/sys/net/ipv4/ip_conntrack_max
  85 and
  86 echo -n 21600 > /proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_timeout_established
  87 and disabling of most userspace applications (like eg. hotsanic). At the
  88 same time, router machine and system was accumulating wide set of various
  89 performance related problems, which required us to reboot it at least
  90 mohtly.
  91
  92 Note: Some time ago it seemed that maximum index of tc classes was restricted
  93 to 10000 - but I haven't checked this again for quite a while.
  94 Another note: All the echo stuff in previous paragraph can be also achieved by
  95 adding following lines to /etc/sysctl.conf which is much cleaner way to do it:
  96 sys.net.ipv4.ip_conntrack_max=65000
  97 sys.net.ipv4.netfilter.ip_conntrack_tcp_timeout_established=21600
  98
  99 Future plans include also setting of individual daily limits on maximum
 100 pps (packets per second) rates allocated to individual IP addresses (this
 101 may be needed partly because of problems mentioned above).
 102
 103 ============================================================================
 104
 105 Prometheus QoS is free software; you can redistribute it and/or
 106 modify it under the terms of the GNU General Public License as
 107 published by the Free Software Foundation; either version 2.1 of
 108 the License, or (at your option) any later version.
 109
 110 Prometheus QoS is distributed in the hope that it will be useful,
 111 but WITHOUT ANY WARRANTY; without even the implied warranty of
 112 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
 113 General Public License for more details.
 114
 115 You should have received a copy of the GNU General Public License
 116 along with Prometheus QoS source code; if not, write to
 117 Michael Polak, Svojsikova 7, 169 00 Praha 6 Czech Republic