Recompile Squid with SSL Bump

https://docs.diladele.com/administrator_guide_4_0/system_configuration/https_filtering/recompile_squid.html

Recompile Squid with SSL Bump

Warning

This administrator guide is now obsolete. Please take a look at latest stable administrator’s guide at Web Safety - ICAP Web Filter for Squid.

Default Squid version present in most Linux distributions is compiled in such a way that does not let it perform HTTPS inspection and SSL filtering out of the box. In order to fix this we need to manually recompile Squid from source. This article will show how to do it using Ubuntu 14.04 LTS.

Note

Squid 3.3.8 in CentOS 7 is fully capable of HTTPS filtering out of the box. If running CentOS / RedHat 7 is acceptable in your environment it is recommended to follow this installation guide.

Note

All scripts and files mentioned in this article may be downloaded from this page. It may be helpful to look through them while reading this article.

Install Build Tools

Before compiling it is considered a good practice to bring the operation system to a most recent state. This can be done by running the following commands in the terminal.

$ sudo apt-get update && sudo apt-get upgrade && sudo reboot

In order to build Squid from source we need to install some build tools and fetch the sources of Squid and various dependent packages from Ubuntu repository. This does not need to take place on the production server, it is possible to build Squid on one machine and install the resulting binaries on others.

$ sudo apt-get install devscripts build-essential fakeroot libssl-dev
$ apt-get source squid3
$ sudo apt-get build-dep squid3

After completion of these commands sources of Squid are unpacked into squid3-3.3.8 folder. To set this folder as current and modify configure options in debian/rules to include necessary compiler switches (–enable-ssl and –enable-ssl-crtd) run the following command.

$ patch squid3-3.3.8/debian/rules < rules.patch

The patch is part of the scripts archive at this page. It looks like the following.

--- rules 2013-11-15 11:49:59.052362467 +0100
+++ rules.new 2013-11-15 11:49:35.412362836 +0100
@@ -19,6 +19,8 @@
 DEB_CONFIGURE_EXTRA_FLAGS := --datadir=/usr/share/squid3 \
    --sysconfdir=/etc/squid3 \
    --mandir=/usr/share/man \
+   --enable-ssl \
+   --enable-ssl-crtd \
    --enable-inline \
    --enable-async-io=8 \
    --enable-storeio="ufs,aufs,diskd,rock" \

One file in source code of Squid proxy needs to be adjusted too (src/ssl/gadgets.cc). This change is needed to prevent Firefox error sec_error_inadequate_key_usage that usually occurs when doing HTTPS filtering with latest Firefox browsers. If you use only Google Chrome, Microsoft Internet Explorer or Apple Safari this step is not required.

$ patch squid3-3.3.8/src/ssl/gadgets.cc < gadgets.cc.patch

The gadgets.cc.patch is part of the scripts archive at this page. It looks like the following.

--- squid3-3.3.8/src/ssl/gadgets.cc 2013-07-13 15:25:14.000000000 +0200
+++ squid3-3.3.8/src/ssl/gadgets.cc.new 2013-12-01 12:58:36.047007719 +0100
@@ -257,7 +257,7 @@
 mimicExtensions(Ssl::X509_Pointer & cert, Ssl::X509_Pointer const & mimicCert)
 {
     static int extensions[]= {
-        NID_key_usage,
+        //NID_key_usage,
         NID_ext_key_usage,
         NID_basic_constraints,
         0

Then build the installation packages using the following command.

$ cd squid3-3.3.8 && dpkg-buildpackage -rfakeroot -b

Configure ICAP Filtering and SSL Bumping

The Squid packages we have compiled previously need to be installed on the system. To perform installation run the following commands.

$ sudo apt-get install ssl-cert
$ sudo apt-get install squid-langpack
$ sudo dpkg --install squid3-common_3.3.8-1ubuntu6_all.deb
$ sudo dpkg --install squid3_3.3.8-1ubuntu6_amd64.deb
$ sudo dpkg --install squidclient_3.3.8-1ubuntu6_amd64.deb

In order to recreate original SSL certificates of the remote web sites during HTTPS filtering Squid uses a separate process named ssl_crtd that needs to be configured like this.

$ sudo ln -s /usr/lib/squid3/ssl_crtd /bin/ssl_crtd
$ sudo /bin/ssl_crtd -c -s /var/spool/squid3_ssldb
$ sudo chown -R proxy:proxy /var/spool/squid3_ssldb

Modify Squid configuration file in /etc/squid3/squid.conf to integrate it with Diladele Web Safety as ICAP server. The squid.conf.patch is part of the scripts archive at this page. It looks like the following.

--- squid.conf  2014-06-01 16:05:26.306759233 +0200
+++ squid.conf.new  2014-06-01 16:04:34.718757596 +0200
@@ -918,11 +918,11 @@
 # Example rule allowing access from your local networks.
 # Adapt to list your (internal) IP networks from where browsing
 # should be allowed
-#acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
-#acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
-#acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
-#acl localnet src fc00::/7       # RFC 4193 local private network range
-#acl localnet src fe80::/10      # RFC 4291 link-local (directly plugged) machines
+acl localnet src 10.0.0.0/8  # RFC1918 possible internal network
+acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
+acl localnet src 192.168.0.0/16  # RFC1918 possible internal network
+acl localnet src fc00::/7       # RFC 4193 local private network range
+acl localnet src fe80::/10      # RFC 4291 link-local (directly plugged) machines

 acl SSL_ports port 443
 acl Safe_ports port 80   # http
@@ -1073,7 +1073,7 @@
 # Example rule allowing access from your local networks.
 # Adapt localnet in the ACL section to list your (internal) IP networks
 # from where browsing should be allowed
-#http_access allow localnet
+http_access allow localnet
 http_access allow localhost

 # And finally deny all other access to this proxy
@@ -1476,8 +1476,11 @@
 #
 #

+# ssl-bump settings managed by Diladele Web Safety for Squid Proxy
+include "/opt/qlproxy/etc/squid/squid.acl"
+
 # Squid normally listens to port 3128
-http_port 3128
+http_port 3128 ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/opt/qlproxy/etc/myca.pem

 #  TAG: https_port
 #  Usage:  [ip:]port cert=certificate.pem [key=key.pem] [mode] [options...]
@@ -2156,7 +2159,7 @@
 #  For more information use:
 #    /usr/lib/squid3/ssl_crtd -h
 #Default:
-# sslcrtd_program /usr/lib/squid3/ssl_crtd -s /var/lib/ssl_db -M 4MB
+sslcrtd_program /usr/lib/squid3/ssl_crtd -s /var/spool/squid3_ssldb -M 4MB

 #  TAG: sslcrtd_children
 #  The maximum number of processes spawn to service ssl server.
@@ -2644,7 +2647,7 @@
 #  NOTE: connect_retries (default: none) can make each of these
 #  possible forwarding paths be tried multiple times.
 #Default:
-# forward_max_tries 10
+forward_max_tries 25

 #  TAG: hierarchy_stoplist
 #  A list of words which, if found in a URL, cause the object to
@@ -2698,7 +2701,7 @@
 #  local memory as they need. For more details about the shared memory
 #  cache, see memory_cache_shared.
 #Default:
-# cache_mem 256 MB
+cache_mem 1024 MB

 #  TAG: maximum_object_size_in_memory  (bytes)
 #  Objects greater than this size will not be attempted to kept in
@@ -2706,7 +2709,7 @@
 #  accessed frequently in memory to improve performance whilst low
 #  enough to keep larger objects from hoarding cache_mem.
 #Default:
-# maximum_object_size_in_memory 512 KB
+maximum_object_size_in_memory 1024 KB

 #  TAG: memory_cache_shared  on|off
 #  Controls whether the memory cache is shared among SMP workers.
@@ -2963,7 +2966,7 @@
 #

 # Uncomment and adjust the following to add a disk cache directory.
-#cache_dir ufs /var/spool/squid3 100 16 256
+cache_dir ufs /var/spool/squid3 14000 16 256

 #  TAG: store_dir_select_algorithm
 #  How Squid selects which cache_dir to use when the response
@@ -4652,7 +4655,7 @@
 #  during shutdown mode.  Any active clients after this many
 #  seconds will receive a 'timeout' message.
 #Default:
-# shutdown_lifetime 30 seconds
+shutdown_lifetime 3 seconds

 # ADMINISTRATIVE PARAMETERS
 # -----------------------------------------------------------------------------
@@ -4721,6 +4724,8 @@
 #  names with this setting.
 #Default:
 # Automatically detect the system host name
+visible_hostname qlproxy
+

 #  TAG: unique_hostname
 #  If you want to have multiple machines with the same
@@ -5902,7 +5907,7 @@
 #  This clause supports both fast and slow acl types.
 #  See http://wiki.squid-cache.org/SquidFaq/SquidAcl for details.
 #Default:
-# Prevent any cache_peer being used for this request.
+always_direct allow all

 #  TAG: never_direct
 #  Usage: never_direct allow|deny [!]aclname ...
@@ -6036,7 +6041,7 @@
 #  TAG: icap_enable  on|off
 #  If you want to enable the ICAP module support, set this to on.
 #Default:
-# icap_enable off
+icap_enable on

 #  TAG: icap_connect_timeout
 #  This parameter specifies how long to wait for the TCP connect to
@@ -6113,7 +6118,7 @@
 #Example:
 #icap_preview_enable off
 #Default:
-# icap_preview_enable on
+icap_preview_enable on

 #  TAG: icap_preview_size
 #  The default size of preview data to be sent to the ICAP server.
@@ -6148,7 +6153,7 @@
 #  Whether or not Squid should use persistent connections to
 #  an ICAP server.
 #Default:
-# icap_persistent_connections on
+icap_persistent_connections on

 #  TAG: adaptation_send_client_ip  on|off
 #  If enabled, Squid shares HTTP client IP information with adaptation
@@ -6157,7 +6162,7 @@
 #
 #  See also: adaptation_uses_indirect_client
 #Default:
-# adaptation_send_client_ip off
+adaptation_send_client_ip on

 #  TAG: adaptation_send_username on|off
 #  This sends authenticated HTTP client username (if available) to
@@ -6167,7 +6172,7 @@
 #  icap_client_username_encode option and is sent using the header
 #  specified by the icap_client_username_header option.
 #Default:
-# adaptation_send_username off
+adaptation_send_username on

 #  TAG: icap_client_username_header
 #  ICAP request header name to use for adaptation_send_username.
@@ -6265,8 +6270,9 @@
 #  deprecated but supported for backward compatibility.
 #
 #Example:
-#icap_service svcBlocker reqmod_precache icap://icap1.mydomain.net:1344/reqmod bypass=0
-#icap_service svcLogger reqmod_precache icap://icap2.mydomain.net:1344/respmod routing=on
+icap_service qlproxy1 reqmod_precache icap://127.0.0.1:1344/reqmod bypass=0
+icap_service qlproxy2 respmod_precache icap://127.0.0.1:1344/respmod bypass=0
+
 #Default:
 # none

@@ -6470,7 +6476,26 @@
 #        See also: icap_service and ecap_service
 #
 #Example:
-#adaptation_access service_1 allow all
+acl qlproxy_icap_edomains dstdomain "/opt/qlproxy/etc/squid/icap_exclusions_domains.conf"
+acl qlproxy_icap_etypes rep_mime_type "/opt/qlproxy/etc/squid/icap_exclusions_contenttypes.conf"
+
+adaptation_access qlproxy1 deny qlproxy_icap_edomains
+adaptation_access qlproxy2 deny qlproxy_icap_edomains
+adaptation_access qlproxy2 deny qlproxy_icap_etypes
+
+# access to IP addresses in the local network is not scanned with qlproxy ICAP
+acl icap_bypass_to_localnet dst 10.0.0.0/8      # RFC1918 possible internal network
+acl icap_bypass_to_localnet dst 172.16.0.0/12   # RFC1918 possible internal network
+acl icap_bypass_to_localnet dst 192.168.0.0/16  # RFC1918 possible internal network
+
+adaptation_access qlproxy1 deny icap_bypass_to_localnet
+adaptation_access qlproxy2 deny icap_bypass_to_localnet
+
+adaptation_access qlproxy1 allow all
+adaptation_access qlproxy2 allow all
+
+
+
 #Default:
 # Allow, unless rules exist in squid.conf.

@@ -6739,7 +6764,7 @@
 #Default:
 # ignore_unknown_nameservers on

-#  TAG: dns_v4_first
+#  TAG: ndns_v4_first
 #  With the IPv6 Internet being as fast or faster than IPv4 Internet
 #  for most networks Squid prefers to contact websites over IPv6.
 #
@@ -6752,7 +6777,7 @@
 #    connectivity is used (and tested), potentially hiding network
 #    problems which would otherwise be detected and warned about.
 #Default:
-# dns_v4_first off
+dns_v4_first on

 #  TAG: ipcache_size (number of entries)
 #  Maximum number of DNS IP cache entries.

Apply the patch with the following commands.

$ sudo cp /etc/squid3/squid.conf /etc/squid3/squid.conf.default
$ sudo patch /etc/squid3/squid.conf < squid.conf.patch
$ sudo /usr/sbin/squid3 -k parse

From now on Squid is capable of HTTPS filtering and we may continue filtering adjustments from Web UI of Diladele Web Safety.

Put Squid 3 Package on Hold

To prevent automatic updates of the rebuilt Squid3 package run the following command.

$ sudo apt-mark hold squid3 squid3-common

Then automatic updates of squid3 package from Ubuntu maintainers will not overwrite custom compiled Squid binary. You may need to redo the compilation steps again if Squid developers find serious security flaws or errors in the code though.

Note

If you would like to get the virtual appliance that has Squid recompiled please take a look at http://www.quintolabs.com/virtual.php. All scripts and files mentioned in this article may be downloaded from this page.