skip to content
HexPharoah Hexpharoah
Table of Contents

My Journey with Packers and Unpacking

The first time I opened a packed binary in Ghidra, the decompiler showed me garbage. Almost no imports, high entropy everywhere, and a tiny code section that seemed to do nothing meaningful. I did not know it at the time, but I was looking at an unpacking stub - the real code was hidden inside.

Learning to unpack binaries opened up a whole new level of analysis for me. Here is how I got there.


What I Learned About Packers

A packer transforms an executable so that its original code is hidden. The packed binary contains:

  1. A small unpacking stub (the loader I was seeing)
  2. The compressed or encrypted original binary

At runtime:

Packed binary starts -> Stub decompresses/decrypts payload ->
Writes it to memory -> Jumps to the Original Entry Point (OEP)

The key insight: if I can find that moment when it jumps to OEP, I can dump the unpacked code from memory.


My First Unpack - UPX

I started with UPX because everyone said it was the easiest. They were right.

Terminal window
# I could literally just do this
upx -d packed.exe
# Done. Unpacked.

But I wanted to understand the process manually. So I loaded a UPX-packed binary in x64dbg and followed the execution:

  1. It started with a PUSHAD (saves all registers)
  2. A decompression loop ran for a while
  3. Then POPAD (restores registers)
  4. A single JMP to the original code

That JMP target was the OEP. I set a breakpoint there, dumped the process with Scylla, and had my first manually unpacked binary. The feeling was great.


How I Identify Packed Binaries Now

Entropy Check

This is my first indicator. Packed or encrypted data has high entropy:

Terminal window
rabin2 -S suspicious.exe
# What I look for:
# Normal .text section: entropy 5.0-6.5
# Packed/encrypted: entropy 7.0-8.0

Import Table

Packed binaries have almost no imports - usually just:

  • LoadLibraryA
  • GetProcAddress
  • VirtualAlloc / VirtualProtect

That is because the real imports get resolved at runtime after unpacking.

Section Names

Dead giveaways I have seen:

  • .UPX0, .UPX1 (UPX)
  • .aspack (ASPack)
  • .vmp (VMProtect)
  • Random garbage characters (custom packers)

Tools I Use for Detection

Terminal window
# Detect It Easy - my go-to
die suspicious.exe
# Also check with pestudio for quick triage

The ESP Trick - My Reliable Method

This technique works on most simple packers and I use it constantly:

1. Load packed binary in x64dbg
2. Run to entry point
3. First instruction is usually PUSHAD or equivalent
4. Step over it once
5. Note the ESP value
6. Right-click ESP -> Follow in Dump
7. Set hardware breakpoint on access for those bytes
8. Press F9 (run)
9. It breaks after POPAD - the packer is done
10. Step forward - next JMP or CALL is to OEP
11. Follow it, now I am at the real entry point
12. Dump with Scylla

I practiced this on maybe 20 different samples before it became second nature.


Fixing the Import Table

After dumping, the binary is usually broken. The imports do not work because the IAT (Import Address Table) still points to the packer’s runtime-resolved addresses.

My fix process:

  1. In Scylla, set the OEP I found
  2. Click “IAT Autosearch” - it finds the import table in memory
  3. Click “Get Imports” - it resolves function names
  4. Fix any invalid entries manually
  5. Click “Fix Dump” - patches my dumped file

After this, the unpacked binary runs and loads in Ghidra/IDA properly with all imports visible.


Harder Packers I Have Dealt With

ASPack

Slightly harder than UPX. The ESP trick still works but the unpacking loop is longer and has a few fake jumps that confused me initially. I learned to recognize the final JMP by its distance - it jumps far away from the stub section into .text.

Custom XOR Packers

I see these a lot in CTFs and basic malware. The pattern:

// What the stub does
for (int i = 0; i < size; i++) {
payload[i] ^= key;
}
// Jump to decrypted payload

I handle these by breaking on VirtualProtect (the stub needs to make the decrypted memory executable) and checking what is in the target buffer after the call.

Multi-Stage Packers

Some malware unpacks in layers:

Stage 1: XOR decrypt -> Stage 2: Decompress (LZMA) -> Stage 3: Resolve imports -> OEP

I dealt with a sample that had 3 stages. I had to dump and analyze each layer separately. Memory breakpoints on execute were my friend here - every time execution hit a new region, I knew a new stage started.

Themida

The hardest I have faced. Themida combines packing with:

  • Anti-debug that detects x64dbg even with ScyllaHide
  • Anti-dump that erases PE headers after unpacking
  • Code virtualization on top of the unpacking

I needed to use TitanHide (kernel-level) to hide my debugger, and even then it took multiple attempts. For Themida, I now check if someone has already written an unpacker script before spending days on it.


Anti-Unpacking Tricks I Encountered

Timing Checks

// Packer checks if execution is too slow (debugger attached)
LARGE_INTEGER start, end;
QueryPerformanceCounter(&start);
// ... unpacking code ...
QueryPerformanceCounter(&end);
if ((end - start) > threshold)
exit(0); // detected debugger

I patch these out or use ScyllaHide to fake the timing values.

IsDebuggerPresent

The classic. Some packers check this multiple times throughout unpacking. ScyllaHide handles it, but I also learned to patch the PEB directly:

In x64dbg:
PEB -> BeingDebugged byte -> set to 0

Anti-Dump

Some packers erase the PE header from memory after unpacking. When I try to dump, the header is zeroed out. My fix: I take a copy of the header before running the packer, then paste it back into my dump.


My Current Workflow

1. Identify: DIE, entropy check, import count
2. Snapshot VM
3. Load in x64dbg with ScyllaHide active
4. Try ESP trick first (works 70% of the time)
5. If that fails, break on VirtualAlloc/VirtualProtect
6. Find OEP
7. Dump with Scylla
8. Fix IAT
9. Verify unpacked binary loads in Ghidra with imports
10. Revert VM snapshot

Tools I Rely On

TaskTool
Identify packerDIE, pestudio
Debugx64dbg + ScyllaHide
DumpScylla
Fix importsScylla IAT fixer
Hide debuggerScyllaHide, TitanHide
Auto-unpack (simple)upx -d, unipacker
Entropyrabin2, pestudio

What I Wish I Knew Earlier

  • The ESP trick works on most things. Learn it first.
  • Do not fight anti-debug manually - use ScyllaHide from the start.
  • Take VM snapshots before every run. I corrupted samples by accident early on.
  • The IAT fix step is where most people give up. Scylla makes it manageable.
  • Write down the OEP address. I forgot it once after a long session and had to redo everything.
  • Start with UPX, then ASPack, then custom packers. Build up gradually.

Unpacking is a patience game but it gets faster with practice. Now when I see a packed binary, I feel excited instead of frustrated.


Until Next -- Tata Bye Bye!!
jaswanth

Related Posts