Written by bonziski — https://github.com/Yannis-Zribi
CONTENTS
0 dll fundamentals
1 what is dll hooking
2 inline hooking
3 iat hooking
4 the pe file structure
5 detecting inline hooks
6 unhooking inline hooks
7 detecting iat hooks
8 unhooking iat hooks
──[ 0. DLL Fundamentals ]──
A Dynamic Link Library (DLL) is a shared binary that exposes functions any
process can call at runtime. The Linux equivalent is a .so.
What DLLs buy you:
- Smaller executables — code lives in the DLL, not in every binary
- Less RAM — one DLL loaded once, shared across callers
- Easier maintenance — patch the DLL, everyone benefits
Anyone can build and sign their own DLL. Windows uses the signature to check
integrity and stop tampering (unsigned or tampered DLLs don't load when DSE is
on).
──[ 1. What is DLL Hooking ]──
DLL hooking is how EDRs intercept Win32 API calls. When a hooked function gets
called, execution detours through the EDR's code before (or instead of) reaching
the real one.
The EDR correlates the call with process context, flags suspicious patterns,
logs the event. Two canonical examples:
notepad.exe writes to a file — normal, ignored
notepad.exe spawns cmd.exe — suspicious, flagged
Two main hooking techniques in the wild.
──[ 2. Inline Hooking ]──
The EDR patches the function directly in memory. The first bytes get overwritten
with a JMP to the EDR's trampoline.
On disk the DLL is untouched (and signed — the kernel won't load a modified
signed DLL). The hook only exists in memory.
NtReadVirtualMemory — clean prologue (on disk):
4C 8B D1 MOV R10, RCX
B8 3F 00 00 00 MOV EAX, 0x3F
...
NtReadVirtualMemory — hooked (in memory):
E9 XX XX XX XX JMP <EDR function>
Every call to NtReadVirtualMemory jumps into the EDR unconditionally.
──[ 3. IAT Hooking ]──
The Import Address Table (IAT) is a per-module table of function pointers. When
your exe calls MessageBoxA, it reads the pointer from its IAT and jumps. The IAT
gets filled in by the loader at startup.
IAT hooking swaps a pointer in this table for the EDR's function address. Only
the module whose IAT is patched is affected; other processes aren't (unlike
inline hooking, which patches the shared in-memory DLL).
Before hooking:
IAT[MessageBoxA] → user32!MessageBoxA (0x7FF812345678)
After IAT hook:
IAT[MessageBoxA] → edr.dll!HookMsgBox (0x7FF900001234)
──[ 4. The PE File Structure ]──
Detecting and unhooking both need the PE (Portable Executable) format, used for
.exe, .dll, .sys and the rest.
DOS Header — legacy; holds e_lfanew (offset to NT Headers)
DOS Stub — prints "This program cannot be run in DOS mode" on old OS
NT Headers — signature + FileHeader + OptionalHeader
Section Headers — array describing each section (.text, .rdata, .idata…)
Locating the Import Directory:
NTHeaders → OptionalHeader → DataDirectory[1] → VirtualAddress
The .idata section holds the Import Descriptor Table: one entry per imported
DLL, with two parallel arrays — the ILT (pre-resolution names) and the IAT
(post-resolution addresses).
──[ 5. Detecting Inline Hooks ]──
Compare the first N bytes of a function between its in-memory copy and the
on-disk copy of the DLL. Mismatch = patched.
// Read N bytes from memory
HMODULE h = GetModuleHandleA("ntdll.dll");
FARPROC f = GetProcAddress(h, "NtReadVirtualMemory");
memcpy(memBuf, (BYTE*)f, N);
// Read N bytes from the file on disk
// (locate the function's RVA via the export directory,
// convert to file offset, then ReadFile)
ReadFile(hFile, diskBuf, N, &read, NULL);
// Compare
for (SIZE_T i = 0; i < N; i++) {
if (memBuf[i] != diskBuf[i]) { /* HOOKED */ }
}
The on-disk copy is authoritative — a signed DLL can't be modified without
breaking its signature.
Automated detection with pe-sieve (https://github.com/hasherezade/pe-sieve):
pe-sieve64.exe /pid <PID>
Excerpt:
Hooked: 1
...
8cea0;MessageBoxA->7ff8ed4a0178;5
The .tag file gives the function name and the JMP target. Open the dumped DLL in
CFF Explorer, jump to the RVA, decode the bytes to confirm the JMP.
──[ 6. Unhooking Inline Hooks ]──
Once you spot a hook, patch the bytes back to the clean on-disk version:
// Compare, and if different, restore from the disk copy
if (memcmp(memBuf, diskBuf, N) != 0) {
WriteProcessMemory(
GetCurrentProcess(),
(LPVOID)f,
diskBuf,
N,
NULL
);
}
WriteProcessMemory handles page permissions internally on modern Windows. If it
fails with ACCESS_DENIED, use VirtualProtect to make the page writable, write,
then restore the original protection.
──[ 7. Detecting IAT Hooks ]──
Walk the IAT of every loaded module, compare the recorded function address to
the expected one (resolved from the canonical DLL), flag any mismatch.
1. Snapshot all loaded modules (CreateToolhelp32Snapshot).
2. For each module, parse DOS → NT → Optional → Import Descriptor.
3. For each imported DLL in the descriptor:
a. Walk the ILT to get function names.
b. Read the corresponding IAT entry (runtime address).
c. Call GetModuleHandleExA(GET_MODULE_HANDLE_EX_FLAG_FROM_ADDRESS)
to find which DLL owns that address.
d. If the owner DLL differs from the expected import DLL (and isn't
a legit forwarder), IAT hook.
HMODULE provider = NULL;
GetModuleHandleExA(
GET_MODULE_HANDLE_EX_FLAG_FROM_ADDRESS |
GET_MODULE_HANDLE_EX_FLAG_UNCHANGED_REFCOUNT,
(LPCSTR)cur, &provider);
char provName[MAX_PATH];
GetModuleFileNameA(provider, provName, MAX_PATH);
if (_stricmp(BaseName(provName), expectedDllName) != 0)
/* HOOKED */
Whitelist legitimate forwarders (kernelbase.dll, api-ms-* virtual DLLs, etc.) to
avoid false positives.
Automated detection:
pe-sieve64.exe /pid <PID> /iat 3
Output:
IAT Hooks: 1
20150;[user32.MessageBoxA #657]->7ff91330114f[...dllhookiat...]
Owner DLL "dllhookiat" isn't user32.dll — confirmed IAT hook.
──[ 8. Unhooking IAT Hooks ]──
Put the IAT entry back to the legit function address:
// Resolve the real address
HMODULE hReal = GetModuleHandleA("user32.dll");
FARPROC expected = GetProcAddress(hReal, "MessageBoxA");
// The IAT entry address (pointer to the function pointer)
void *iatEntry = &ftTable[i].u1.Function;
DWORD old;
// Temporarily make the IAT page writable
VirtualProtect(iatEntry, sizeof(uintptr_t), PAGE_READWRITE, &old);
// Patch
ftTable[i].u1.Function = (uintptr_t)expected;
// Restore protection
VirtualProtect(iatEntry, sizeof(uintptr_t), old, &old);
// Flush cached instructions
FlushInstructionCache(GetCurrentProcess(), iatEntry, sizeof(uintptr_t));
After this, calls to MessageBoxA from this module go straight to user32.dll. EDR
is no longer in the call path for this IAT entry.