The master version is at http://info.ost.eltele.no/freeware/squidGuard/
This page was last modified on Wednesday, 03-Nov-1999 19:04:47 MET
Welcome to squidGuard
An ultrafast and free filter, redirector and access controller for Squid

Note1: this page is primarily written for squidGuard-1.0.x. Though most of it still applies to squidGuard-0.9.x.
Note2: To print all-in-one go here.
Note3: The FAQ is here.
Note4: The ANNOUNCE message is here.

Contents

Downloading

The release directory for squidGuard is accessible by both HTTP and FTP
The current production release is squidGuard-1.1.1
New: A regularly updated sample blacklist is now also available.

Se the News and changes and Known bugs and misfeatures sections for more info on the releases.

Introduction

squidGuard is a combined filter, redirector and access controller plugin for Squid. It is

squidGuard can be used to

Neither squidGuard nor Squid can be used to

*) 100,000 requests in 10seconds on a 500MHz Pentium with lists of
  5900 domains
  7880 urls
13780 total
    100,000 requests in 12seconds on a 500MHz Pentium with lists of
    5900 domains
200000 urls
205900 total
    I.e. domain and URL listsizes has neglectable performance effect..

**) squidGuard is not a porn or banner filter/blocker, but it is very well suited for these purposes too.

Capabilities

squidGuard has many powerful configuration options that lets you:

  1. define different time spaces based on any reasonable combination of

  2. group sources (users/clients) into distinct categories like "managers", "employees", "teachers", "students", "customers", "guests" etc. based on any reasonable combination of
  3. and optionally link the group to a given time space
  4. group destinations (URLs/servers) into distinct categories like "local", "customers", "vendors", "banners", "banned" etc. based on an unlimited number of unlimited lists of
  5. and optionally link the group to a given time space:
  6. rewrite/redirect URLs based on any reasonable combination of

  7. define access control lists (acl) based on any reasonable combination of the definitions above by

  8. have selective logging by optional log statements in the: ****)
  9. and optionally anonymized to protect the individuals (log anonymous "file")

*) Client access control based on domain name requires enabling reverse lookups (log_fqdn on) in squid.conf.
**) Client access control based on user id requires enabling RFC931/ident in squid.conf. Note: The RFC931/ident configuration is changed in squid-2.2 and the RFC931/ident support is broken in squid-2.2 at least up to STABLE2. We currently recommend using squid-2.1.PATCH2 in production if RFC931 is used.
***) Note: Visible redirects (302:new-url) are not supported by some interim versions of Squid (presumably 1.2-2.0).
****) Note: squidGuard is smart enough to open only one filedescriptor per logfile (i.e. not necessarily one per log statement); per spawned process of course. Though logging to too many different files may exeed your system's concurrent filedescriptor limit.

Portability

squidGuard should compile right out of the box on any modern brand of UNIX with a development environment and a recent version (2.X) of the Berkeley DB library. squidGuard is developed on Sun Solaris-2.6 with gcc-2.7.2.3, bison-1.25, flex-2.5.4. We also test regularly on Linux/RedHat-5.2 with gcc-2.8.1 and Digital UNIX/OSF1-4.0 with gcc-2.7.2.3. We always use our most recent copy of the Berkeley DB.

Users have reported success on at least, but not limited to:

Please report missing list entries.

Installation guide

The installation guide is in a separate document.

Configuration

The Configuration is in a separate document.

A glimpse behind the curtain

How squidGuard comunicates with Squid

squidGuard uses Squid's standard redirector interface. When Squid is used with squidGuard:

How squidGuard decides what to do

For each request squidGuard will:

What makes squidGuard so fast and robust

The main design goals for squidGuard were, and still are:

The efficiency is achieved by:

The robustness is achieved by:

Special notes, features and oddities

Please note:

News and changes

Major news and changes in
1.1.1:
1.1.0:
1.1.0.beta1:
1.0.0:
1.0.0.beta3:
1.0.0.beta2:
1.0.0.beta1:

Se the change log for the details and minor changes

Known bugs and misfeatures

We have had squidGuard runing on our Squid proxy for more than a year now with about 5-11000 listed domains, 6-14000 listed URLs, 3 complex regular expressions and 20-25 acls whereof some with timeconstrains. So far we haven't seen a single crash, coredump or hang caused by squidGuard in production. Although squidGuard has proven rock solid there still are some known minor problems:

To do or not to do

That is the question.. Here are some possible development issues we have discussed:

Put host addresses in IP lists into B-tree
This should dramaticly reduce the performance penalty of long lists of random client addresses. We should still keep IP ranges in sequentially linked lists since ranges of random width are hard to index efficiently.

Add syslog support
Maybe we should add syslog support as an alternative for the squidGuard.log.

Include support for classless IP ranges in the domainlists
We have to find a good storage method and modify the domain comparator slightly to select IP range mode if the URL host is an IP address. IP range comparision is allready in use in the client group match so it shouldn't be to hard.

Benchmarking alternative regexp libraries
Configure should automaticly select the best regexp library; native or GNU/FSF.

Reconfigure on a HUP signal directly to squidGuard
In the beginning where we used only one redirector process, this seemed like a good idea. I really don't know why one should use more squidGuard redirectors than CPUs on the system but that seems to be recommended. So therefor we have got second thoughts about this since it raises a real challenge to find a way to reconfigure all the redirectors with one signal. Either this should be done by having the good Squid guys implement a signal handler for a dedicated signal that is passed to the redirector processes or that makes Squid send a HUP signal to its redirectors. Alternatively this has to be done by some kind of IPC. Maybe SysV IPC semaphores is an option here, but this is likely to reduce the portability and increase the complexity of squidGuard.

Preventing users from accessing blocked sites by using their IP address after a manual nslookup
This is partially done by the new optional !in-addr flag. An alternative path with non blocking reverse lookups before group matching have been left since it complicates things too much at the present. An alternative is to use a simple Perl script to include both the domainname and the IP addresses in the lists

A shared database engine
When running multiple squidGuard processes it is of course a waist of RAM to have each processes hold a complete private copy of the destination databases in memory. This is now solved with the new support for prebuilt on-disk databases. Though there are other possible solutions to avoid this too:

-- Comments and contributions are welcome.

FAQ - Frequently Asked/Answered Questions

The FAQ is in a separate document.

Links to related documentation

Here is a guide to related documentation

A happy user's squidGuard page :-)
Fabrice Prigent, Network Administrator at the University of Toulouse, France, has written an excellent page about squidGuard in both French and English. He has done a neutral comparison of known free redirectors for Squid. He also has links to some URL/domain lists/patterns you may find useful in conjunction with Squid and squidGuard.
The official Squid 1.1 Release Notes:
The Squid programmers/authors original documentation on Squid's redirector interface.
The official Squid redirector FAQ:
Frequently Answered Questions about Squid redirectors.
The official Squid ralated software FAQ:
Pointers to Squid redirector packages and more.
The official Squid FAQ:
Answers to almost all your questions about Squid.
Squid related documentation:
Pointers to even more Squid related documentation.

Useful software links

Here is a guide to more or less free software tools you may or may not need.

squidGuard:
Well, that is what this is all about :-)
Squid:
If you don't have Squid you don't need squidGuard either. You may prefer to download our copy if you are in Europe or have trouble reaching the Squid home page.
libdb.a - Berkeley DB - The Berkeley Database library version 2.X:
squidGuard requires this library. You must install a version 2.X of DB if your system does not already have it or it has a version 1.X installed (Version 1.X and 2.X may coexist on a system). The Berkeley Database library can be used free of charge with squidGuard since squidGuard is GPL. You may prefer to download our copy if you are in Europe or have trouble reaching Sleepycat.
Note: All versions of the Berkeley Database library are called betas. In contrast to the normal use of the notion beta this does not mean that the Berkeley Database library is in unstable prerelease state. Quite contrary, it has proven to be very stable and can safely be used in production quality software.
Gzip - GNU's replacement for compress:
You need gzip/gunzip/zcat to unzip the squidGuard distribution. If you ever plan to work with freeware for UNIX-like environments then gzip/gunzip/zcat is a must. You may prefer to download our copy if you are in Europe or have trouble reaching The Free Software Foundation or finding a good mirror.
Gmake - GNU make:
If you don't have a development environment with a "make" compatible build tool and stuff listed below, then the Free Software Foundation is your best friend. The Free Software Foundation makes free versions of everything you need to set up a development environment for UNIX-like systems. Besides the University of California, Berkeley/Berkeley Software Design, the free software from the Free Software Foundation is the main reason there are things called 386bsd, NetBSD, FreeBSD, Linux, etc. etc. Thanks to the constant open source revision and refinement via the Internet over the years since 1983, the Free Software Foundation in most cases makes faster, better and less buggy reincarnations of most UNIX-like tools an libraries than the commercial vendors do and the code runs on almost any UNIX-like platform you can think of so you can set up a more uniform environment on a multivendor fleet. Therefor we recommend using GNU tools wherever you can! You may prefer to download our copy if you are in Europe or have trouble reaching The Free Software Foundation or finding a good mirror.
Gcc - The GNU C Compiler:
If you don't have an ANSI C compiler then the Free Software Foundation is your best friend. Gcc makes faster and less buggy code than many commercial compilers and runs on almost any UNIX-like platforms you can think of, so we recommend using it anyway unless your commercial compiler is really clever. squidGuard is developed mainly with gcc-2.7.2.3. (The new gcc-2.8.x may be fine too.) You may prefer to download our copy if you are in Europe or have trouble reaching The Free Software Foundation or finding a good mirror.
Bison - The GNU/FSF parser generator, Bison:
Byacc - The Berkeley Yacc parser generator:
If you for some strange reason don't have a "yacc" (Yet Another Compiler Compiler) compatible parser generator then The GNU/FSF parser generator, (bison) and The Berkeley Yacc parser generator (byacc) are your friends. You need yacc, byacc or bison to compile src/sg.y into src/y.tab.c and src/y.tab.h. Bison makes faster parsers than most commercial versions of yacc, so we recommend using it anyway as long as its copyright isn't a problem for you (You may not use Bison to build commercial software). If you can't use Bison and lack yacc, use byacc which has a more liberal copyright. (Though the parser speed doesn't matter much for squidGuard since it is used for parsing the config file only). You may prefer to download our copies of bison and/or byacc if you are in Europe or have trouble downloading from the sources.
Flex - The Fast Lexical Analyzer Generator:
If you for some strange reason don't have a "lex" compatible lexical analyzer generator then the Fast Lexical Analyzer Generator (flex) is your friend. You need lex/flex to compile src/sg.l into src/lex.yy.c. Flex makes faster scanners than many (most/all?) commercial versions of lex, so we recommend using it anyway. (Though the lexer speed doesn't matter much for squidGuard since it is used for parsing the config file only). squidGuard is developed mainly with flex-2.5.4a. You may prefer to download our copy if you are in Europe or have trouble reaching The Free Software Foundation or finding a good mirror.
Regex - the GNU regex library:
Rx - A fast replacement for the GNU regex library:
squidGuard must be linked with a regular expression library that supports regcomp() and regexec(). If your system for some strange reason don't have this the GNU regex or rx libraries may be good alternatives. We haven't done any comparison of these libraries yet, but have successfully linked squidGuard with both. These may be faster than many commercial variants anyway, so you may consider using one of them even if your system have a compatible library. Finding a fast regular expression executer, regexec(), may be critical for good performance if you run squidGuard with many and/or complex regular expressions. The performance of the regular expression compiler regcomp() is less critical since all expression compilation is done at load time, not when handling requests. You may prefer to download our copies if you are in Europe or have trouble reaching The Free Software Foundation or finding a good mirror.
Apache - The most popular web server on the Internet:
Although your redirectors could point to nonexisting URLs, it would be a better idea in most configurations to redirect to existing URLs. So you probably need a web server. If you don't have a web server we strongly recommend Apache. You can install it on the proxy server or a nearby server. You may prefer to download our copy if you are in Europe or have trouble reaching the apache.org or finding a good mirror.
Perl - The popular all in one language:
You don't really need Perl to run squidGuard, but it comes very handy when making CGI scripts, HTML parsers, URL collectors, database compactors etc. etc. You may prefer to download our copy if you are in Europe or have trouble reaching the perl.com or finding a good mirror.
Identd - our free ident daemon for M$ Windoze:
Pidentd - a free Portable Ident Daemon for Unix:
To be able to use user IDs in the client rules definitions the actual clients must support RFC931/RFC1413. Pidentd is for Unix and Identd for M$ Windoze. Feel free to download our copy of pidentd or our official distribution of identd.
squidlog2combined:
squidlog2combined is a free tool that lets you convert native Squid accesslogs to the Apache combined log format, suitable for log analyzers like Webalizer and others. It has some really nice options.
Webalizer -a free web server log file analyzer:
Webalizer is a fast, free web server log file analysis program. It produces highly detailed, easily configurable usage reports in HTML format, for viewing with a standard web browser. With Squid you may find it usefull in combination with squidlog2combined and Apache's logresolve. Feel free to download our copy.

Note: ftp.ost.eltele.no is not mirroring other sites. We just put a copy there, to have it handy, of what we manually download and find useful. The copies are unchanged from the original but may have been compressed with gzip (.gz) if the original was compressed with the old compress (.Z).

Copyright

squidGuard is covered by the GNU General Public Licence (GPL). The copyright boils down to:

For more copyright details you have to read the GNU General Public Licence (GPL).

Feedback

If you find a bug, have a bug fix, a contribution, a question or a comment on squidGuard feel free to send it to squidguard@ost.eltele.no

The preferred bug fix format is context diff:
diff -c origfile fixedfile
or
diff -cr origdir fixeddir
Please explain what problem your diff fixes.

Success reports are very welcome too.. They makes us even more enthusiastic!

Authors

squidGuard was designed by Pål Baltzersen and implemented by Lars Erik Håland
at ElTele Øst AS and made freely available by permission of ElTele Øst AS.

This page is written and maintained by Pål Baltzersen.
Please report any errors, broken links, missing stuff, typos, misspellings, wrong grammar, norwenglish phrases etc. you discover.

Contributors

Thanks to:
Gary Lindstrom, Fabrice Prigent. Jiri A. Randus. Joseph Lesko.