【转发】Build Squid with SSL Bump and ICAP Client
原文文档:http://docs.diladele.com/administrator_guide_3_4/installation_and_removal/filtering_https.html
上面这个地址更新了,用这个吧:https://docs.diladele.com/howtos/build_squid_ubuntu16/index.html
感觉Squid的配置乱七八糟的,不同版本参数,配置方法都不一样,所以我为了测HTTPS+ICAP的功能,就找了这个3.3.8版本的配置方法,不过这其中仍然有些参数丢失了。所以如果你不设置ACL的话,用我这个配置应该可以。我这个配置是在原配置上增加了:
ssl_bump server-first all
sslproxy_cert_error allow all
sslproxy_flags DONT_VERIFY_PEER
这三个参数。
我的配置文档: http://files.cnblogs.com/files/bonjov1/Ubuntusquid3.3.8.rar
This article will tell you how to compile, setup and configure Squid proxy capable of filtering encrypted HTTPS connections using Diladele Web Safety ICAP content filtering server. Being able to look into HTTPS contents greatly increases your ability to control what is allowed and accepted within your network while keeping inappropriate contents away.
Why do We Need to Filter HTTPS
HTTPS protocol was designed to provide secure means of communications between internet browser and remote web servers. In order to achieve this goal HTTPS protocol encrypts data passing through established connections so that it cannot be decrypted in reasonable amount of time thus preventing anyone from sniffing the contents interchanged over this connection. This protocol was primarily invented to enable safe and secure communication between the user and financial sites or government institutions over the insecure medium such as the Internet.
Recently more and more web sites started to use HTTPS encrypted communications to increase online privacy of users. Google who as first enabled HTTPS for all its searches by default probably initiated this trend. Although there are no doubts that HTTPS encryption is a good thing for safety on the wire we must take into account that it also creates several problems for controlled networks typically found at home or offices. The main problem here is the essence of the HTTPS protocol itself – no one except the browser and the web server is able to see and thus filter transferred data. This may not always be desired. Contents that are usually blocked suddenly become immediately accessible by anyone. As an example imagine a school network where minors can see questionable content by just mistyping a search term in Google. Moreover the law often forces administrators in educational institutions to block access to such content (e.g. CIPA for educational environments) and encrypted access to web sites makes it nearly impossible to fulfill such an obligation.
In order to overcome these limitations it is advised to setup HTTPS filtering of web contents with help of SSL bump feature of Squid proxy server and Diladele Web Safety web filter.
How It Works
In order to filter web requests user’s browser needs to be explicitly directed to use the proxy that is deployed in the same network. It is also possible to set the transparent proxy but we are not going to explain how this is done in this tutorial because steps involved are quite different from explicit proxy setup.
When a user tries to navigate to a web site, browser sends the request to proxy server, asking it to get the requested page on his behalf. The proxy establishes a new connection to the remote site and returns the response to browser. If normal HTTP is used then proxy is able to see the original contents of the response and filter it. In case of HTTPS the flow of data is a little different. Browser asks the proxy to establish a virtual tunnel between itself and remote server and then sends encrypted data through the proxy. Domain name to which a virtual tunnel is being established is usually known, so proxy is able to block this virtual tunnel when it finds out that domain name belongs to a prohibited category. Unfortunately this is not a complete solution as there are a lot of sites on the Internet which are general in nature (like Google or YouTube) but allow you to easily navigate to something undesired.
To improve the quality of web filtering and get access to contents in encrypted connections, browsers in the network may be setup to trust proxy to act on their behalf for establishing HTTPS connections, filtering them and passing the allowed data to clients while blocking everything that is not allowed. Although this assumption is too strict to be implemented in public networks, it is easily doable in controlled home, educational or corporate environments where administrators act as sole owners of network devices and may force any trusting rules. After established trust browser is able to ask proxy to connect to a remote site in a safe manner with HTTPS, proxy is able to decrypt the traffic, filter it, encrypt it again and pass it to browser. As browser trusts the proxy it continues working with filtered HTTS without any errors or warnings.
Unfortunately most of the Squid versions included in most common Linux/FreeBSD distributions do not contain compile switches necessary for successful HTTPS filtering. Proxy administrator needs to recompile Squid proxy, reinstall and reconfigure it with additional list of options. Although this process is not very complex, it is complex enough and it would be helpful to have all the necessary steps exactly described. We will provide such exact instructions for the latest version of one of the most popular Linux distributions –Ubuntu Server 13.10.
Build Squid with SSL Bump and ICAP Client
Before compiling it is considered a good practice to bring the operation system to a most recent state. This can be done by running the following commands in the terminal.
sudo apt-get update && sudo apt-get upgrade && sudo reboot
In order to build the Squid from source we need to install some build tools and fetch the sources of Squid and various dependent packages from Ubuntu repository. This does not need to take place on the productionserver, it is possible to build Squid on one machine and install the resulting binaries on others.
sudo apt-get install devscripts build-essential fakeroot libssl-dev
sudo apt-get source squid3
sudo apt-get build-dep squid3
Running the following command unpacks Squid source package together with all system integration scripts and patches provided by Ubuntu developers.
dpkg-source -x squid3_3.3.8-1ubuntu3.dsc
Sources are unpacked intosquid3-.3.8 folder. We need to set this folder as current and modify configure options in debian/rules to include compiler switches (–enable-ssl and –enable-ssl-crtd) necessary for HTTPS filtering.
patch squid3-3.3.8/debian/rules < rules.patch
The rules.patch file should look like this.
--- rules 2013-11-15 11:49:59.052362467 +0100
+++ rules.new 2013-11-15 11:49:35.412362836 +0100
@@ -19,6 +19,8 @@
DEB_CONFIGURE_EXTRA_FLAGS := --datadir=/usr/share/squid3 \
--sysconfdir=/etc/squid3 \
--mandir=/usr/share/man \
+ --enable-ssl \
+ --enable-ssl-crtd \
--enable-inline \
--enable-async-io=8 \
--enable-storeio="ufs,aufs,diskd,rock" \
One file in source code of Squid Proxy needs to be adjusted too ( src/ssl/gadgets.cc ). This change is needed to prevent Firefox error sec_error_inadequate_key_usage that usually occurs when doing HTTPS filtering with latest Firefox browsers. If you use only Google Chrome, Microsoft Internet Explorer or Apple Safari this step is not required.
Download patches here
patch squid3-3.3.8/src/ssl/gadgets.cc < gadgets.cc.patch
Where gadgets.cc.patch looks like this.
--- gadgets.cc 2013-07-13 09:25:14.000000000 -0400
+++ gadgets.cc.new 2013-11-26 03:25:25.461794704 -0500
@@ -257,7 +257,7 @@
mimicExtensions(Ssl::X509_Pointer & cert, Ssl::X509_Pointer const & mimicCert)
{
static int extensions[]= {
- NID_key_usage,
+ //NID_key_usage,
NID_ext_key_usage,
NID_basic_constraints,
0
Then we build the package using the following command. After a while this command builds all required *.DEB packages.
cd squid3-3.3.8 && dpkg-buildpackage -rfakeroot -b
Install Diladele Web Safety
SSL Bumpng feature alone is not enough to block questionable web content. We also need the filtering server that could be paired with Squid. We will use Diladele Web Safety (DDWS) formerly known as QuintoLabs Content Security for the filtering and blocking part. It is an ICAP daemon capable of integrating existing Squid proxy and providing rich content filtering functionality out of the box. It may be used to block illegal or potentially malicious file downloads, remove annoying advertisements, prevent access to various categories of the web sites and block resources with explicit content.
We will use version 3.0 of qlproxy that is in release candidate state and will probably be finally released this month. It was designed specifically with HTTPS filtering in mind and contains rich web administrator console to perform routine tasks right from the browser.
By default, DDWS comes with four polices preinstalled. Strict policy contains web filter settings put on maximum level and is supposed to protect minors and K12 students from inappropriate contents on the Internet. Relaxed policy blocks only excessive advertisements and was supposed to be used by network administrators, teachers and all those who do not need filtered access to web but would like to evade most ads. Third policy is tailored to white list only browsing and the last group contains less restrictive web filtering settings suitable for normal web browsing without explicitly adult contents shown.
In order to install Diladele Web Safety for Squid Proxy, download package for Ubuntu 13.10 from Diladele B.V. web site at http://www.quintolabs.com using browser or just run the following command in terminal.
wget http://updates.diladele.com/qlproxy/binaries/3.0.0.3E4A/amd64/release/ubuntu12/qlproxy-3.0.0.3E4A_amd64.deb
Administration console of Diladele Web Safety is built using Python Django framework and is usually managed by Apache webserver. To install packages required for correct functioning of web UI run the following commands in the terminal.
sudo apt-get install python-pip
sudo pip install django==1.5
sudo apt-get install apache2 libapache2-mod-wsgi
Install the DEB package and perform integration with Apache by running the following commands.
sudo dpkg --install qlproxy-3.0.0.3E4A_amd64.deb
sudo a2dissite 000-default
sudo a2ensite qlproxy
sudo service apache2 restart
Please note you may need to uncomment the #Require all granted line in /etc/apache2/sites-available/qlproxy.conf if you get Access Denied errors trying to access Diladele Web Safety’s Web UI. This is due to the fact that Apache configuration settings have changed between Ubuntu 12 and Ubuntu 13. Luckily has to be done only once.
Configure Squid for ICAP Filtering and HTTP Bumping
The Squid packages we have compiled previously need to be installed on the system. To perform installation run the following commands.
sudo apt-get install ssl-cert
sudo apt-get install squid-langpack
sudo dpkg --install squid3-common_3.3.8-1ubuntu3_all.deb
sudo dpkg --install squid3_3.3.8-1ubuntu3_amd64.deb
sudo dpkg --install squidclient_3.3.8-1ubuntu3_amd64.deb
In order to recreate original SSL certificates of the remote web sites during HTTPS filtering Squid uses a separate process named ssl_crtd that needs to be configured like this.
sudo ln -s /usr/lib/squid3/ssl_crtd /bin/ssl_crtd
sudo /bin/ssl_crtd -c -s /var/spool/squid3_ssldb
sudo chown -R proxy:proxy /var/spool/squid3_ssldb
Finally, modify Squid configuration file in /etc/squid3/squid.conf to integrate it with Diladele Web Safety as ICAPserver. Due to the size of the patch file it’s text not included into the article directly but is part of the download archive.
sudo cp /etc/squid3/squid.conf /etc/squid3/squid.conf.default
sudo patch /etc/squid3/squid.conf < squid.conf.patch
sudo /usr/sbin/squid3 -k parse
From now on Squid is capable of HTTPS filtering and we may continue filtering adjustments from Web UI of Diladele Web Safety.
Navigate to http://<YOUR PROXY IP ADDRESS>/ and login with default name root and password P@ssw0rd. Select Settings / HTTPS Filtering / Filtering Mode. Diladele Web Safety may either filter specific HTTPS sites or all of them with exclusions. Total filtering is more tailored to providing very safe network environments.
Select the desired mode, click Save Settings , add target domains or exclusions as you like and then restart ICAP server by clicking on the green button in the top right corner as indicated on the following screenshots.
【转发】Build Squid with SSL Bump and ICAP Client的更多相关文章
- Recompile Squid with SSL Bump
https://docs.diladele.com/administrator_guide_4_0/system_configuration/https_filtering/recompile_squ ...
- 五大开源 Web 代理服务器横评:Squid、Privoxy、Varnish、Polipo、Tinyproxy
https://linux.cn/article-7119-1.html Web 代理软件转发 HTTP 请求时并不会改变数据流量.它们可以配置成透明代理,而无需客户端配置.它们还可以作为反向代理放在 ...
- Linux实战教学笔记42:squid代理与缓存实践(一)
第1章 Squid介绍 1.1 缓存服务器介绍 缓存服务器(英文意思cache server),即用来存储(介质为内存及硬盘)用户访问的网页,图片,文件等等信息的专用服务器.这种服务器不仅可以使用户可 ...
- 代理服务器squid简介
Squid 是一个高性能.开源的代理缓存服务器和 Web 缓存进程,支持 FTP.Internet Gopher.HTTPS 和 SSL 等多种协议.它通过一个非阻塞的.I/O 事件驱动的单一进程处理 ...
- squid代理与缓存(上)
squid代理与缓存(上) 1. Squid介绍 1.1 缓存服务器介绍 缓存服务器(英文意思cache server),即用来存储(介质为内存及硬盘)用户访问的网页,图片,文件等等信息的专用服务器. ...
- 学习Squid(一)
第1章 Squid介绍 1.1 缓存服务器介绍 缓存服务器(英文意思cache server),即用来存储(介质为内存及硬盘)用户访问的网页,图片,文件等等信息的专用服务器.这种服务器不仅可以使用户可 ...
- squid 学习笔记
Squid学习笔记 1.安装前的配置 编译安装之前需要校正的参数主要包括File Descriptor和Mbuf Clusters. 1.File Descriptor 查看文件描述符的限制数目: u ...
- CentOS 7 Squid代理服务器正向代理-透明代理
Squid是Linux系统中最常用的一款开源代理服务软件,主要提供缓存加速和应用层过滤控制的功能,可以很好的实现HTTP.FTP.DNS查询以及SSL等应用的缓存代理 透明代理:提供与传统代理相同的功 ...
- 高效配置Linux代理服务器 Squid介绍
作为一种免费的网络操作系统,Linux越来越受到广大网络爱好者的欢迎,目前Internet上运行的主机有相当一部分采用的就是Linux,而且中国已经把Linux作为政府上网的指定网络操作系统.种种迹象 ...
随机推荐
- pip更改国内源
国内源: 阿里云 http://mirrors.aliyun.com/pypi/simple/中国科技大学 https://pypi.mirrors.ustc.edu.cn/simple/豆瓣(dou ...
- 交换机基础配置之stp生成树实验
实验的要求是以上面的拓扑图为例,查看当前根桥的所在并把三成交换机switch1指定为根桥 我们可以用show spanning-tree来查看stp的信息 输入后会显示两部分 上面一部分为当前交换机自 ...
- JQuery制作网页—— 第一章 JavaScript基础
1. JavaScript(弱类型语言):是一种描述性语言,也是一种基于对象(Object)和事件驱动(Event Driven)的,并具有安全性能的脚本语言. 特点:1.主要用来在HTML页面中添加 ...
- POJ 1981 最大点覆盖问题(极角排序)
Circle and Points Time Limit: 5000MS Memory Limit: 30000K Total Submissions: 8346 Accepted: 2974 ...
- 初见spark-02(RDD及其简单算子)
今天,我们来进入spark学习的第二章,发现有很多事都已经开始变化,生活没有简单的朝自己想去的方向,但是还是需要努力呀,不说鸡汤之类的话了, 开始我们今天的spark的旅程 一.RDD是什么 rdd的 ...
- No module named appium
在脚本中会有:from appium import webdriver 第一次运行时可能会遇到这样的error:No module named appium 之所以会报这样的error是因为没有装cl ...
- 6,Flask 中内置的 Session
Flask中的Session非常的奇怪,他会将你的SessionID存放在客户端的Cookie中,使用起来也非常的奇怪 1. Flask 中 session 是需要 secret_key 的 from ...
- 15.6,redis主从同步
redis主从同步 原理:1. 从服务器向主服务器发送 SYNC 命令.2. 接到 SYNC 命令的主服务器会调用BGSAVE 命令,创建一个 RDB 文件,并使用缓冲区记录接下来执行的所有写命令.3 ...
- CodeForces 873F Forbidden Indices 后缀数组
忘了当时怎么做的了,先把代码贴上,保存一下后缀数组模板. #include <cstdio> #include <cstring> #include <algorithm ...
- 什么时候会报unrecognized selector的异常?
当调用该对象上某个方法,而该对象上没有实现这个方法的时候, 可以通过“消息转发”进行解决,如果还是不行就会报unrecognized selector异常 objc是动态语言,每个方法在运行时会被动态 ...