Windows API Hooking in Python

catalogue

.  相关基础知识
. Deviare API Hook Overview
. 使用ctypes调用Windows API
. pydbg
. winappdbg
. dll injection
. process monitor with WMI
. sobek-hids

0.  相关基础知识

0x1: Python 程序和 C 程序的整合

为了节省软件开发成本，软件开发人员希望能够缩短的软件的开发时间，希望能够在短时间内开发出稳定的产品。Python 功能强大，简单易用，能够快速开发应用软件。但是由于 Python 自身执行速度的局限性，对性能要求比较高的模块需要使用效率更高的程序语言进行开发，例如 C 语言，系统的其他模块运用 Python 进行快速开发，最后将 C 语言开发的模块与 Python 开发的模块进行整合。在此背景下，基于 Python 语言与 C 语言的各自特点，用 C 语言来扩展现有的 Python 程序，显得很有意义

利用 ctypes 模块整合 Python 程序和 C 程序

ctypes 是 Python 的一个标准模块，它包含在 Python2.3 及以上的版本里。ctypes 是一个 Python 的高级外部函数接口，它使得 Python 程序可以调用 C 语言编译的静态链接库和动态链接库。运用 ctypes 模块，能够在 Python 源程序中创建，访问和操作简单的或复杂的 C 语言数据类型。最为重要的是 ctypes 模块能够在多个平台上工作，包括 Windows，Windows CE，Mac OS X，Linux，Solaris，FreeBSD，OpenBSD
利用 Python 本身提供的 ctypes 模块可以使 Python 语言和 C 语言在源代码层面上进行整合，在 Python 程序中可以定义类似 C 语言的变量

ctypes type	c type	Python type
c_char	char	1-character string
c_wchar	wchar_t	1-character unicode string
c_byte	char	int/long
c_ubyte	unsigned char	int/long
c_short	short	int/long
c_ushort	unsigned short	int/long
c_int	int	int/long
c_uint	unsigned int	int/long
c_long	long	int/long
c_ulong	unsigned long	int/long
c_longlong	__int64 or long long	int/long
c_ulonglong	unsigned __int64 or unsigned long long	int/long
c_float	float	float
c_double	double	float
c_char_p	char * (NUL terminated)	string or None
c_wchar_p	wchar_t * (NUL terminated)	unicode or None
c_void_p	void *	int/long or None

Python 访问 C 语言 dll

通过 ctypes 模块，Python 程序可以访问 C 语言编译的 dll

from ctypes import windll 

def callc():
    # load the some.dll
    somelibc = windll.LoadLibrary(some.dll)
    print somelibc. helloworld() 

if __name__== “__main__”:
    callc()

hitman hook了内核的createprocess事件(串行hook)，拿到事件后通过文件特征的方式进行匹配

Relevant Link:

http://www.ibm.com/developerworks/cn/linux/l-cn-pythonandc/

1. Deviare API Hook Overview

viare is a professional open source hooking engine for instrumenting arbitrary Win32 functions, COM objects, and functions which symbols are located in program databases (PDBs). It can intercept unmanaged code in 32-bit and 64-bit applications. It is implemented as a COM component, so it can be integrated with all the programming languages which support COM, such as C/C++, VB, C#, Delphi, and Python.
Several Fortune 500 companies are using Deviare technology for application virtualization, packaging, and troubleshooting, and for computer security. Computer science researchers are also using Deviare to conduct malware and reverse engineering studies
Deviare offers a unique “programmer friendly API” which resolves the complexities associated with binary instrumentation so that even software engineers without expertise in the field can use it. Deviare takes care of code injection, parameter marshalling, and inter-process communication. We created Deviare API in 2007 and continually improve it. Intercepting applications is a complex task. We test multiple application environments to ensure that the end user has a trouble-free experience. Deviare also has a focus on performance handling thousands of hooks with little footprint.
Code instrumentation is used in several other areas like: tracing and debugging, sandboxing and browser security, malware analysis, video conference recording, and gaming.

该框架使用的是process inline hook的思路，通过writeprocessmemory方式修改指定进程的指定api入口为hook function，待hook function执行完毕后再跳转回原始的被劫持函数

0x1: Deviare's Design

windows上的ring3 hook方案需要使用系统级API(Kernel32.dll、ntdll.dll导出函数)，而这些API必须使用C代码去调用，为了能让应用层的VB/Python/C#等代码调用到这些功能，一个好的方法是使用COM实现这些功能，然后暴露出API接口给python调用

With Deviare you can program a hook handler in your own process to get called when any API function is called in the remote process in this way:

Deviare supports COM technology to let you write hook handlers in any high-level language like VB, VB.NET, C#, Python, Delphi, etc.
If you need many hooks and you need extreme performance you can implement your hooks inside the remote process in this way:

0x2: Install

. Install Python 2.7.
. Register DeviareCOM.dll and DeviareCOM64.dll (if running under an x64 platform).
http://www.initdll.de/deviarecom64.dll/en-download-62999.html
http://www.initdll.de/deviarecom.dll/en-download-62982.html
regsvr32 DeviareCOM.dll
regsvr32 DeviareCOM64.dll

pushd C:\Windows\System32
右键以管理员方式启动cmd.exe，然后调用regsvr32方可成功

. Download and install Python Win32 Extensions
http://sourceforge.net/projects/pywin32/files/pywin32
NOTE: Be careful to download the appropriate version for your platform and Python version.
NOTE : If you get a message stating that you haven't installed Python in your system yet (but you did and you are completely sure that you downloaded the correct Python Win32 Extensions installer) 

Relevant Link:

https://github.com/nektra/deviare2
http://www.nektra.com/products/deviare-api-hook-windows/doc-v2/interface_deviare2_1_1_d_nkt_hook_events.html
http://www.nektra.com/products/deviare-api-hook-windows/design/
https://github.com/srw/windows-api-hooking-in-python-with-deviare-sample
http://blog.nektra.com/main/category/products/deviare-products/

2. 使用ctypes调用Windows API

0x1: user32.dll -> SetWindowsHookExA():  获取键盘、鼠标事件

Installs an application-defined hook procedure into a hook chain. You would install a hook procedure to monitor the system for certain types of events. These events are associated either with a specific thread or with all threads in the same desktop as the calling thread.

HHOOK WINAPI SetWindowsHookEx(
  _In_ int       idHook,
  _In_ HOOKPROC  lpfn,
  _In_ HINSTANCE hMod,
  _In_ DWORD     dwThreadId
);

. idHook [in]: The type of hook procedure to be installed. This parameter can be one of the following values
    ) WH_CALLWNDPROC: Installs a hook procedure that monitors messages before the system sends them to the destination window procedure.
    ) WH_CALLWNDPROCRET: Installs a hook procedure that monitors messages after they have been processed by the destination window procedure.
    ) WH_CBT: Installs a hook procedure that receives notifications useful to a CBT application.
    ) WH_DEBUG: Installs a hook procedure useful for debugging other hook procedures.
    ) WH_FOREGROUNDIDLE: Installs a hook procedure that will be called when the application's foreground thread is about to become idle. This hook is useful for performing low priority tasks during idle time.
    ) WH_GETMESSAGE: Installs a hook procedure that monitors messages posted to a message queue.
    ) WH_JOURNALPLAYBACK: Installs a hook procedure that posts messages previously recorded by a WH_JOURNALRECORD hook procedure
    ) WH_JOURNALRECORD: Installs a hook procedure that records input messages posted to the system message queue. This hook is useful for recording macros.
    ) WH_KEYBOARD: Installs a hook procedure that monitors keystroke messages.
    ) WH_KEYBOARD_LL: Installs a hook procedure that monitors low-level keyboard input events.
    ) WH_MOUSE: Installs a hook procedure that monitors mouse messages.
    ) WH_MOUSE_LL: Installs a hook procedure that monitors low-level mouse input events.
    ) WH_MSGFILTER: Installs a hook procedure that monitors messages generated as a result of an input event in a dialog box, message box, menu, or scroll bar.
    ) WH_SHELL: Installs a hook procedure that receives notifications useful to shell applications.
    ) WH_SYSMSGFILTER: Installs a hook procedure that monitors messages generated as a result of an input event in a dialog box, message box, menu, or scroll bar. The hook procedure monitors these messages for all applications in the same desktop as the calling thread. 

. lpfn [in] : A pointer to the hook procedure. If the dwThreadId parameter is zero or specifies the identifier of a thread created by a different process, the lpfn parameter must point to a hook procedure in a DLL. Otherwise, lpfn can point to a hook procedure in the code associated with the current process.

. hMod [in]: A handle to the DLL containing the hook procedure pointed to by the lpfn parameter. The hMod parameter must be set to NULL if the dwThreadId parameter specifies a thread created by the current process and if the hook procedure is within the code associated with the current process.

. dwThreadId [in]: The identifier of the thread with which the hook procedure is to be associated. For desktop apps, if this parameter is zero, the hook procedure is associated with all existing threads running in the same desktop as the calling thread.  

SetWindowsHookExA()只能抓到窗口、鼠标、键盘的事件(和UI有关的)
Relevant Link:

https://msdn.microsoft.com/en-us/library/windows/desktop/ms644990(v=vs.85).aspx
http://stackoverflow.com/questions/31379169/setting-up-a-windowshook-in-python-ctypes-windows-api
https://github.com/schurpf/pyhk/blob/master/pyhk.py
http://www.cnblogs.com/oubo/archive/2011/08/25/2394555.html
http://www.cs.unc.edu/Research/assist/developer.shtml
http://reverseengineering.stackexchange.com/questions/2715/api-hooking-using-dll-injection-with-python-c-types

3. pydbg

A pure-python win32 debugger interface.

0x1: qHooK

qHooK is very simple python script (dependent on pydbg) which hooks user defined Win32 APIs in any process and monitor then while process is running and at last prepare a CSV report with various interesting information which can help reverse engineer to track down / analyse unknown exploit samples / shellcode.

Relevant Link:

https://github.com/OpenRCE/pydbg
https://github.com/debasishm89/qHooK
http://blog.csdn.net/cheng_tian/article/details/7652058
https://github.com/debasishm89/qHooK
https://raw.githubusercontent.com/debasishm89/qHooK/master/qHooK.py
https://my.oschina.net/dkexcellent/blog/38717

4. winappdbg

The WinAppDbg python module allows developers to quickly code instrumentation scripts in Python under a Windows environment.
It uses ctypes to wrap many Win32 API calls related to debugging, and provides an object-oriented abstraction layer to manipulate threads, libraries and processes, attach your script as a debugger, trace execution, hook API calls,
handle events in your debugee and set breakpoints of different kinds (code, hardware and memory). Additionally it has no native code at all, making it easier to maintain or modify than other debuggers on Windows.

0x1: More examples

Set a debugging timeout

Sometimes you’ll want to set a maximum time to debug your target, especially when fuzzing or analyzing malware. This is an example on how to code a custom debugging loop with a timeout. It launches the Windows Calculator and stops when the target process is closed or after a 5 seconds timeout.

from winappdbg import *
from time import time

# Using the Debug object in a "with" context ensures proper cleanup.
with Debug( bKillOnExit = True ) as dbg:

    # Run the Windows Calculator (calc.exe).
    dbg.execl('calc.exe')

    # For the extra paranoid: this makes sure calc.exe dies
    # even if our own process is killed from the Task Manager.
    System.set_kill_on_exit_mode(True)

    # The execution time limit is  seconds.
    maxTime = time() + 

    # Loop while calc.exe is alive and the time limit wasn't reached.
    while dbg and time() < maxTime:
        try:

            # Get the next debug event.
            dbg.wait()  #  second accuracy

            # Show the current time on screen.
            print time()

        # If wait() times out just try again.
        # On any other error stop debugging.
        except WindowsError, e:
            if e.winerror in (win32.ERROR_SEM_TIMEOUT,
                              win32.WAIT_TIMEOUT):
                continue
            raise

        # Dispatch the event and continue execution.
        try:
            dbg.dispatch()
        finally:
            dbg.cont()

Dump the memory of a process

import os
import sys
import zlib
import winappdbg
from winappdbg import win32

try:
    import sqlite3 as sqlite
except ImportError:
    from pysqlite2 import dbapi2 as sqlite

# Create a snaphot of running processes.
system = winappdbg.System()
system.request_debug_privileges()
system.scan_processes()

# Get all processes that match the requested filenames.
for filename in sys.argv[:]:
    print "Looking for: %s" % filename
    for process, pathname in system.find_processes_by_filename(filename):
        pid  = process.get_pid()
        bits = process.get_bits()
        print "Dumping memory for process ID %d (%d bits)" % (pid, bits)

        # Parse the database filename.
        dbfile   = '%d.db' % pid
        if os.path.exists(dbfile):
            counter =
            while :
                dbfile = '%d_%.3d.db' % (pid, counter)
                if not os.path.exists(dbfile):
                    break
                counter +=
            del counter
        print "Creating database %s" % dbfile

        # Connect to the database and get a cursor.
        database = sqlite.connect(dbfile)
        cursor   = database.cursor()

        # Create the table for the memory map.
        cursor.execute("""
            CREATE TABLE MemoryMap (
                Address INTEGER PRIMARY KEY,
                Size    INTEGER,
                State   STRING,
                Access  STRING,
                Type    STRING,
                File    STRING,
                Data    BINARY
            )
        """)

        # Get a memory map of the process.
        memoryMap       = process.get_memory_map()
        mappedFilenames = process.get_mapped_filenames(memoryMap)

        # For each memory block in the map...
        for mbi in memoryMap:

            # Address and size of memory block.
            BaseAddress = mbi.BaseAddress
            RegionSize  = mbi.RegionSize

            # State (free or allocated).
            if   mbi.State == win32.MEM_RESERVE:
                State   = "Reserved"
            elif mbi.State == win32.MEM_COMMIT:
                State   = "Commited"
            elif mbi.State == win32.MEM_FREE:
                State   = "Free"
            else:
                State   = "Unknown"

            # Page protection bits (R/W/X/G).
            if mbi.State != win32.MEM_COMMIT:
                Protect = ""
            else:
                if   mbi.Protect & win32.PAGE_NOACCESS:
                    Protect = "--- "
                elif mbi.Protect & win32.PAGE_READONLY:
                    Protect = "R-- "
                elif mbi.Protect & win32.PAGE_READWRITE:
                    Protect = "RW- "
                elif mbi.Protect & win32.PAGE_WRITECOPY:
                    Protect = "RC- "
                elif mbi.Protect & win32.PAGE_EXECUTE:
                    Protect = "--X "
                elif mbi.Protect & win32.PAGE_EXECUTE_READ:
                    Protect = "R-X "
                elif mbi.Protect & win32.PAGE_EXECUTE_READWRITE:
                    Protect = "RWX "
                elif mbi.Protect & win32.PAGE_EXECUTE_WRITECOPY:
                    Protect = "RCX "
                else:
                    Protect = "??? "
                if   mbi.Protect & win32.PAGE_GUARD:
                    Protect += "G"
                else:
                    Protect += "-"
                if   mbi.Protect & win32.PAGE_NOCACHE:
                    Protect += "N"
                else:
                    Protect += "-"
                if   mbi.Protect & win32.PAGE_WRITECOMBINE:
                    Protect += "W"
                else:
                    Protect += "-"

            # Type (file mapping, executable image, or private memory).
            if   mbi.Type == win32.MEM_IMAGE:
                Type    = "Image"
            elif mbi.Type == win32.MEM_MAPPED:
                Type    = "Mapped"
            elif mbi.Type == win32.MEM_PRIVATE:
                Type    = "Private"
            elif mbi.Type == :
                Type    = ""
            else:
                Type    = "Unknown"

            # Mapped file name, if any.
            FileName = mappedFilenames.get(BaseAddress, None)

            # Read the data contained in the memory block, if any.
            Data = None
            if mbi.has_content():
                print 'Reading %s-%s' % (
                    winappdbg.HexDump.address(BaseAddress, bits),
                    winappdbg.HexDump.address(BaseAddress + RegionSize, bits)
                )
                Data = process.read(BaseAddress, RegionSize)
                Data = zlib.compress(Data, zlib.Z_BEST_COMPRESSION)
                Data = sqlite.Binary(Data)

            # Output a row in the table.
            cursor.execute(
                'INSERT INTO MemoryMap VALUES (?, ?, ?, ?, ?, ?, ?)',
                (BaseAddress, RegionSize, State, Protect, Type, FileName, Data)
            )

        # Commit the changes, close the cursor and the database.
        database.commit()
        cursor.close()
        database.close()
        print "Ok."
print "Done."

Relevant Link:

https://pypi.python.org/pypi/winappdbg/1.5
https://sourceforge.net/p/winappdbg/mailman/message/23259562/
http://winappdbg.sourceforge.net/index.html
http://winappdbg.sourceforge.net/MoreExamples.html

5. dll injection

To enable system wide hooking on NT/XP, a DLL will have to be loaded into the target process. InjectLibrary() injects a DLL into an already running process.
The injection system stays resident until the system is rebooted, or a call to UnInjectLibrary() is made. When using the dynamic library, target processes must be able to locate both the DLL to be injected

6. process monitor with WMI

0x1: proces creation event monit

The Win32_Process WMI class represents a process on an operating system.

[Dynamic, Provider("CIMWin32"), SupportsCreate, CreateBy("Create"), SupportsDelete, DeleteBy("DeleteInstance"), UUID("{8502C4DC-5FBB-11D2-AAC1-006008C78BC7}"), DisplayName("Processes"), AMENDMENT]
class Win32_Process : CIM_Process
{
  string   CreationClassName;
  string   Caption;
  string   CommandLine;
  datetime CreationDate;
  string   CSCreationClassName;
  string   CSName;
  string   Description;
  string   ExecutablePath;
  uint16   ExecutionState;
  string   Handle;
  uint32   HandleCount;
  datetime InstallDate;
  uint64   KernelModeTime;
  uint32   MaximumWorkingSetSize;
  uint32   MinimumWorkingSetSize;
  string   Name;
  string   OSCreationClassName;
  string   OSName;
  uint64   OtherOperationCount;
  uint64   OtherTransferCount;
  uint32   PageFaults;
  uint32   PageFileUsage;
  uint32   ParentProcessId;
  uint32   PeakPageFileUsage;
  uint64   PeakVirtualSize;
  uint32   PeakWorkingSetSize;
  uint32   Priority = NULL;
  uint64   PrivatePageCount;
  uint32   ProcessId;
  uint32   QuotaNonPagedPoolUsage;
  uint32   QuotaPagedPoolUsage;
  uint32   QuotaPeakNonPagedPoolUsage;
  uint32   QuotaPeakPagedPoolUsage;
  uint64   ReadOperationCount;
  uint64   ReadTransferCount;
  uint32   SessionId;
  string   Status;
  datetime TerminationDate;
  uint32   ThreadCount;
  uint64   UserModeTime;
  uint64   VirtualSize;
  string   WindowsVersion;
  uint64   WorkingSetSize;
  uint64   WriteOperationCount;
  uint64   WriteTransferCount;
};

code example

import win32con
import win32api
import win32security

import wmi
import sys
import isapi

def log_to_file(message):
    fd = open('process_monitor_log.csv', 'ab')
    fd.write("%s\r\n" % message)
    fd.close()

log_to_file("time,user,executable,commandline,pid,parent,pid,privileges")

c = wmi.WMI()

process_watcher = c.Win32_Process.watch_for('creation')

while True:
    try:
        new_process = process_watcher()
        cmdlline = new_process.CommandLine
        print cmdlline
    except:
        pass

wmi本身无法获取注册表和文件系统的操作事件，文件系统变动需要使用windows的原生api ReadDirectoryChangesW来实现

Relevant Link:

https://books.google.com.hk/books?id=9MS9BQAAQBAJ&pg=PA139&lpg=PA139&dq=windows+python+api+hook+createprocess&source=bl&ots=tYgj5EYO1D&sig=f8fDHsUanrebdAsWrg_GfWVBIFw&hl=zh-CN&sa=X&ved=0ahUKEwiSq9Luw9zQAhUHOCYKHeWyDlcQ6AEIUTAH#v=onepage&q=windows%20python%20api%20hook%20createprocess&f=false
https://books.google.com.hk/books?id=9MS9BQAAQBAJ&pg=PA139&lpg=PA139&dq=windows+python+api+hook+createprocess&source=bl&ots=tYgj5EYO1D&sig=f8fDHsUanrebdAsWrg_GfWVBIFw&hl=zh-CN&sa=X&ved=0ahUKEwiSq9Luw9zQAhUHOCYKHeWyDlcQ6AEIUTAH#v=onepage&q&f=false
https://technodesk.wordpress.com/2009/09/25/python-monitor-process-cpu-using-wmi/
https://msdn.microsoft.com/en-us/library/aa394372%28v=vs.85%29.aspx
http://python.jobbole.com/86349/
https://code.google.com/archive/p/sobek-hids/downloads
https://msdn.microsoft.com/en-us/library/aa393035(v=vs.85).aspx
https://pypi.python.org/pypi/watchdog
https://github.com/gorakhargosh/watchdog
http://stackoverflow.com/questions/20610005/how-to-watch-the-windows-registry-for-changes-with-python
https://msdn.microsoft.com/en-us/library/aa394394(v=vs.85).aspx

7. sobek-hids

Sobek-Hids is a python based Host IDS system that is capable of monitor: * Registry Changes * File Activity * Process Creation * Printing Jobs * External Drives (USB Disk Plugs) * Shared Resources * Windows Accounts * Logon * Firewall Changes

Relevant Link:

Copyright (c) 2016 LittleHann All rights reserved