转:Fuzzing Apache httpd server with American Fuzzy Lop + persistent mode
Fuzzing Apache httpd server with American Fuzzy Lop + persistent mode
Posted by javier on 20 June 2017
Intro
Recently, I reported CVE-2017-7668 (Apache Server buffer-over-read). This is a cross-post from my personal blog where I explain how to fuzz network programs with AFL by porting techniques learned in honggfuzz into AFL.1 After a small chat with Dominic he asked me to re-post it here which, for me it’s an honour to do so!
The reported CVE was obtained with code analysis and instrumentation of the right parts of the code (mainly core and parsing) – First, with honggfuzz I got the initial dirty test cases and then, through radamsa generated a few thousands mutations and finally AFL with the technique described here.
Goal
When stumbling upon the great American Fuzzy Lop and trying its awesome deterministic fuzzing capabilities and instrumentation soon we find out that this fuzzer was built to fuzz programs that take input from the command line (or standard input) instead of a network socket. Because of that, I thought I would give it a try and make AFL fuzz against Apache’s httpd server. First the AFL way by adding a new option to Apache’s command line and the second way, by using the persistence fuzzing (afl-clang-fast) by shamelessly copying the way Robert Swiecky fuzzes Apache with honggfuzz.
Takeaways for the reader
- Learn to fuzz network based programs with AFL
- Code to start fuzzing Apache with AFL in no time
- A push in the interwebz fuzzing race
Let’s do it!
Setup part 1
I will be using a Debian GNU/Linux 8 64bit with the kernel 4.9.0-0.bpo.2-rt-amd64. You don’t really need that setup. All that you need is an operating system (under a virtual machine or not) that can compile and run AFL with the afl-clang feature.
But, before getting into any compilations/installations/fuzzing, I encourage you to set an organised folder structure that suits you best but, in case you haven’t got one already, I am sharing mine.
Under the Fuzzing folder I have:
- Victims – For the target programs that we are about to fuzz
- Fuzzers – AFL, honggfuzz, radamsa, etc. go here
- Testcases – The samples we are going to feed the fuzzer to throw against our Victim
- Sessions – For storing the fuzzing sessions
- Compilers – To store compilers such as clang-4.0 and binaries needed to compile
Getting clang-4.0 and llvm-tools
Getting pre-built binaries for clang-4.0 and the llvm-tools is fairly easy if you have Debian or Ubuntu. You can get these from here http://releases.llvm.org/download.html. In my case the clang+llvm-4.0.0-x86_64-linux-gnu-debian8.tar.xz tarball.
If you are following the structure mentioned above, you can cd into your Compilers folder and drop the tarball there, extract it and then add the binaries folder to your path by adding the following line to the end of your ~/.bashrc file (~/.profile nor /etc/environment worked for me – It seems that you need to logout and login for these changes to take place).
PATH="$HOME/Fuzzing/Compilers/clang+llvm-4.0.0-x86_64-linux-gnu-debian8/bin:$PATH"
Now issuing the which command on a new shell we should have the following output:
$ which clang
$ /home/javier/Fuzzing/Compilers/llvm-clang-binaries/clang+llvm-4.0.0-x86_64-linux-gnu-debian8/bin/clang
Compiling and Installing AFL
Compiling AFL should be pretty straight forward but, for the lazy, you can just copy paste these commands and you should be ready to go:
sudo apt install build-essential
wget http://lcamtuf.coredump.cx/afl/releases/afl-latest.tgz
tar xzf afl-latest.tgz
cd afl*
make && sudo make install && echo-e "\n[+] AFL ready to fuzz at $(which afl-fuzz)"
That’s it, the binary afl-fuzz should be in your path now ready to be unleashed.
Compiling and installing Apache
First move to a folder where we are about to download Apache server and all the dependencies needed. In my case, the folder is at ~/Fuzzing/Victims/apache_afl/.
Before compiling Apache we are going to need the Apache Portable Runtime (APR), APR Utils and support for HTTP/2 through nghttp2.
No lazyness this time, go download:
- APR and APR Util
- nghttp
- The following dependencies:
- sudo apt install pkg-config
- sudo apt install libssl-dev
- The latest pcre-8.x (DON’T DOWNLOAD PCRE2!!)
Now we need to get the latest Apache build, which I recommend you do from their subversion repository by doing so:
sudo apt install subversion
svn checkout http://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x httpd-2.4.x
Now if you downloaded and unpacked everything, you should have a similiar output from ls -l command:
drwxr-xr-x 28 javier javier 4096 Apr 9 23:12 apr-1.5.2
drwxr-xr-x 20 javier javier 4096 Apr 9 23:13 apr-util-1.5.4
-rwxr-xr-x 1 javier javier 1445 May 6 20:56 compile_dependencies_with_flags.sh
drwxr-xr-x 11 javier javier 4096 Apr 29 01:22 httpd-2.4.x
drwxr-xr-x 14 javier javier 4096 Apr 9 23:14 nghttp2-1.21.0
drwxr-xr-x 9 javier javier 12288 Apr 9 22:53 pcre-8.40
I made the following script to compile and link it all since I found myself often changing flags for the compiler and it was too time consuming compiling each dependency one by one with its own flags. Get it with:
wget https://gist.githubusercontent.com/n30m1nd/14418fd425a3b2d14b64650710fae301/raw/e1cff738eb1ffaa55cb8a1a66bb1a2b06ed7f97e/compile_httpd_with_flags.sh
Before editing any files yet lets run the bash script and see that we can compile everything cleanly without any missing dependencies whatsoever:
CC="clang" CXX="clang++" PREFIX="/usr/local/apache_clean_test/" ./compile_dependencies_with_flags.sh
Please see the next asciinema for reference of a nice compilation run (takes a while to load).
Fuzzing Apache with AFL through an inut file
As you might know by now, AFL in its basic usage feeds a file into the target program through its “argv” array in the following form:(通过argv参数传递文件名)
afl-fuzz -i testcases/ -o session_1/ -- ./victim -v -f @@
The problem with Apache is that it doesn’t have such functionality so we will have to patch it our own way.
Patching Apache
Taking into account the aforementioned problem, we need to write some lines into Apache’s main.c file to make it able to read files from input.(注意补丁方式:在apache main函数中加入读取文件的函数,相当于提供了一种直接从文件读取并运行的能力!)
You can patch Apache with the following patch file here. Now apply it by cd‘ing into the base path of Apache httpd’s source code and issuing the following command:
patch -p0 -i apatching_apache_for_AFL_fuzzing.diff
I am not going to cover all the patch in detail but some parts are worth mentioning.
The first and only time that I have seen the following technique was by Robert Swiecky, an information security researcher at Google when fuzzing Apache with honggfuzz. It is pretty clever and pretty obvious once you see the way it is done. It basically consists of launching a new thread inside Apache that will create a connection to the web server itself and send our fuzzed input; all happening within the same unique process so we can get all the instrumentation data into AFL. Clever! Right?
To achieve this it uses the unshare function that disassociates parts of this thread’s context from the others without the need of creating a new process. Specifically, the network and mount namespaces are separated. This is done so we can have several processes with the same settings (listening on the same loopback interface and port with the help of netIfaceUp on line 44 of the patch file and writing logs to /tmp on line 75) running at the same time on each process we launch.
We can see that the unshare(unsh();) function is indirectly called on line 188 previous to firing the new thread that will receive the fuzzed input at line 189.
The process of reading a file through the “-F” switch starts on line 156 and when the file is read into a buffer, this buffer is passed onto the function responsible to launch the new thread (189) that will, in turn, send the fuzzed file inside the SENDFILE function on line 119.
unshare() allows a process (or thread) to disassociate parts of its
execution context that are currently being shared with other
processes (or threads). Part of the execution context, such as the
mount namespace, is shared implicitly when a new process is created
using fork(2) or vfork(2), while other parts, such as virtual memory,
may be shared by explicit request when creating a process or thread
using clone(2). The main use of unshare() is to allow a process to control its shared
execution context without creating a new process.
unshare允许进程控制与其它进程或线程共享的上下文。
Fuzzing Apache
Yes! We are ready now! Let’s compile Apache:
CC="afl-clang" CXX="afl-clang++" PREFIX="/usr/local/apache_afl_blogpost/" ./compile_dependencies_with_flags.sh
If you are familiar with AFL and how it works you probably have your own testcases to feed it with, in case you don’t the following video shows how to launch AFL and create two very simple test cases – remember that we need to be root or use “sudo sysctl -w kernel.unprivileged_userns_clone=1” in order to use the unshare function AND MORE IMPORTANT TO LAUNCH APACHE WITH THE “-X” FLAG AND “-m none -t 5000” FOR AFL SO IT CAN BOOT APACHE:
Well, that was not too fast, was it? 5 execs per second on my laptop… how can we speed things up a bit?
Setup part 2
Compiling afl-clang-fast
Remember we downloaded clang-4.0 and the llvm-tools before and set it in our path? This is where it comes most handy. Inside your AFL folder, navigate to the llvm_mode and run make and sudo make install in the root folder of AFL. What we have just done is compiling an experimental feature of AFL that will run a certain number of fuzzed inputs against a program without having to run the whole program per fuzzing input.
Patching Apache for Persistence
Following the same dance as before, download this patch(与单个不同,怎么没有读文件呢?), patch it and ready to fuzz!
Fuzzing Apache with AFL on Persistent mode
Let the video speak for itself but again remember the previously mentioned “-X” flag for Apache server and the “-m none -t 5000” flags for AFL:
Update:
As pointed by Robert himself, you don’t need to run everything as root as I did in the examples which, obviously imposes security risks. You can make use of the following command line (this one to be run as root :P) to let non-root users make use of the unshare function:
patch -p0 -i apatching_apache_for_AFL_fuzzing.diff
Conclusions
We have learned how to effectively fuzz server programs such as web servers by using Robert’s technique of launching a different thread and different context through the unshare function.
While relatively fast, it is not as fast as honggfuzz, which can go up to 20k iterations per second with 8 processes running. Also after a few days of fuzzing AFL’s stability goes way down 50% because of the multithreading that Apache is implemented on and so, any reported crashes, can be false positives or would either be needing the last iterations launched which AFL lacks of.
It is left as an exercise to the reader to implement into Apache a way to save the last 1k sent inputs into a file and to think which other ways would improve stability and/or speed. Hint: Don’t instrument everything.
Shouts to the Apache Security Team, it was a pleasure to work with such an efficient team.
Happy pwning! x)
For further doubts, ramblings or whatever you can contact me on javier at domain’s name
转:Fuzzing Apache httpd server with American Fuzzy Lop + persistent mode的更多相关文章
- 使用mod_cluster进行apache httpd server和jboss eap 6.1集群配置
本文简单介绍,使用mod_cluster进行apache httpd server和jboss eap 6.1集群配置.本配置在windows上测试通过,linux下应该是一样的.可能要稍作调整.后面 ...
- Apache Httpd Server 2.2升级2.4
Apache Httpd Server 2.2升级2.4 (2 votes, average: 5.00 out of 5) 2,302 views 2012 年 3 月 20 日Web服务器.服务器 ...
- Apache httpd Server 配置正向代理
背景 代理(Proxy),位于客户端与实际服务端之间,当客户端需要请求服务端内容时,先向代理发起请求,代理将请求转发到实际的服务器,再原路返回.也可以在代理服务器设置缓存,将实际服务器上不常变化的内容 ...
- Apache httpd Server Notes
1. httpd启动.停止以及重启 启动: apachectl –f $PATH_TO_CONF/httpd.conf 停止及重启 apachectl –k stop/restart/graceful ...
- 查看apache httpd server中加载了哪些模块
说明: 有的时候,需要查看当前apache中都加载了哪些模块,通过以下命令进行查看 [root@hadoop1 httpd-]# bin/apachectl -t -D DUMP_MODULES Lo ...
- Apache httpd.conf配置文件 2(Main server configuration)
### Section 2: 'Main' server configuration # # The directives in this section set up the values used ...
- centos7 apache httpd安装和配置django项目
一.安装httpd服务 apache在centos7中是Apache HTTP server.如下对httpd的解释就是Apache HTTP Server.所以想安装apache其实是要安装http ...
- tomcat集群学习记录1--初识apache http server
起因 平时开发的时候我都是用eclipse把代码部署到本地tomcat的,当然只会去跑1台tomcat啦... 偶尔有时候解决问题需要去公司测试环境找日志.连上公司测试环境以后发现竟然有2台weblo ...
- 【转】Linux下apache/httpd服务启动与停止
apache服务,或者说httpd服务,如何启动,如何开机启动. 转来转去,找不到原文.. 操作系统环境:红帽5,具体如下:# uname -a Linux machine1 2.6.18-164.e ...
随机推荐
- RabbitMQ 命令
1.RabbitMQ 服务启动与关闭 同时启动RabbitMQ 节点和erlang服务 普通方式启动./rabbitmq-server守护线程方式启动./rabbitmq-server –detach ...
- java获去json所有对象
public static void main(String args[]){ JSONObject json1=JSONObject.fromObject("{'username' : ' ...
- 轻量级HTTP服务器Nginx(配置与调试Nginx)
原创作品,允许转载,转载时请务必以超链接形式标明文章 原始出处 .作者信息和本声明.否则将追究法律责任.http://ixdba.blog.51cto.com/2895551/790611 Nginx ...
- String StrigBuffer StringBuilder 浅解
1.String是最基本的字符串类,用于表示字符串. 特点:对象内容不可变,但可以通过指向不同的对象来“表示”不同的内容. 使用场景:如果不涉及到内容改变,可以使用String. 注意:如果想将Str ...
- ZooKeeper内部构件
引言 这个文档包含关于ZK内部工作的信息.目前为止,它讨论了这些主题: 原子广播 日志 原子传播 ZK的核心是一个原子的通信系统,它使所有的服务端保持同步. 保证.属性和定义 通过使用ZooKeepe ...
- ZooKeeper食谱(八)
使用ZooKeeper构造高级别应用的指南 在这个文章中,你将会发现使用ZooKeeper来实现高级别功能的指南.所有的它们在客户端上被实现而不需要ZooKeeper特别的支持.希望社区将注意到这些约 ...
- Android 之 Spinner 键值对的绑定(转)
很多时候我们会在下拉菜单中绑定一个值,但是 Spinner本身不提供这样的服务 于是在网上找了N久,终于找到一个简单易用的方案;废话不多说,直接上菜了 首先要定义一个Item类,有以下要注意的: ...
- 基于 Docker 的 Zabbix 微服务系统
zabbix 官网提供一个镜像 [ zabbix-appliance ], 可以直接拉起一个 zabbix-server. 但是数据库无法分离出来. 本实践使用 zabbix 官方提供的 Docker ...
- Remmarguts' Date(POJ2449+最短路+A*算法)
题目链接:http://poj.org/problem?id=2449 题目: 题意:求有向图两点间的k短路. 思路:最短路+A*算法 代码实现如下: #include <set> #in ...
- 将文件内容导入到MySQL中
1.作用 把文件系统的内容导入到数据库中 2.语法 load data infile "文件名" into table 表名 fields terminated by " ...