skip to content
HexPharoah Hexpharoah
Table of Contents

How I Approach Malware Analysis

I got into malware analysis through CTFs. The forensics challenges hooked me - figuring out what a suspicious binary does felt like solving a crime. Over time I built a structured workflow that I follow for every sample. Here is how I do it.


My Setup

First thing I learned the hard way: never run malware on your host machine. I set up a dedicated analysis environment:

  • A Windows 10 VM in VirtualBox with snapshots (I revert after every analysis)
  • REMnux as my Linux analysis VM
  • FakeNet-NG for simulating network responses
  • Host-only networking with no internet access

I snapshot the clean state and revert after every session. Took me one close call to make this a habit.


Static Analysis - My First Pass

I always start static. No execution, just looking at what the file tells me before I let it run.

File Identification

Terminal window
# What am I looking at?
file suspicious.exe
# Get hashes for lookup
sha256sum suspicious.exe
# Quick check on VirusTotal
# If 50+ engines flag it, I already know it is bad
# But I still want to know WHAT it does

Strings - My Favorite Starting Point

Strings reveal so much. I run this on every sample:

Terminal window
strings suspicious.exe | less
# For Windows malware, wide strings are common
strings -el suspicious.exe
# When strings are obfuscated, FLOSS recovers them
floss suspicious.exe

What I look for:

  • IP addresses and domains - tells me where it phones home
  • Registry paths - shows me how it persists
  • File paths - where it drops stuff
  • API names - what it is capable of
  • Debug strings the author forgot to remove (happens more than you think)

PE Header Check

For Windows binaries, I check the PE structure:

Terminal window
# Imports tell me capabilities
rabin2 -i suspicious.exe
# High entropy sections = packed or encrypted
rabin2 -S suspicious.exe

My red flags:

  • Only imports LoadLibrary and GetProcAddress (resolves everything at runtime to hide)
  • Section entropy above 7.0 (packed)
  • Weird section names (.UPX, .vmp, random garbage)
  • Import of VirtualAlloc + WriteProcessMemory + CreateRemoteThread (process injection)

Loading in Ghidra

If static strings and headers look interesting, I load it in Ghidra:

  • Find the entry point, trace main logic
  • Look for anti-analysis checks early in execution
  • Map out what functions do based on API calls
  • Check for encrypted config blobs (common in RATs)

Dynamic Analysis - Watching It Run

Once I understand the structure, I let it execute in my sandbox and watch what happens.

Process Monitoring

I launch ProcMon before executing the sample and filter for the process:

What I watch for:
- File creates/writes (dropping payloads, logs)
- Registry modifications (persistence)
- Network connections (C2 communication)
- Process creation (spawning child processes, injection)

Process Hacker gives me a live view of the process tree, loaded DLLs, and open handles.

Network Capture

I run Wireshark in the background:

Terminal window
# Or tcpdump on REMnux
tcpdump -i eth0 -w malware_capture.pcap

FakeNet-NG simulates responses so the malware thinks it is online. This way I see:

  • DNS queries for C2 domains
  • HTTP beacons (regular interval callbacks)
  • Data being exfiltrated
  • The protocol it uses to talk to its server

Debugging Key Moments

When I need to understand specific behavior, I attach x64dbg:

  • Set breakpoints on interesting APIs (CreateFile, RegSetValue, send)
  • Step through decryption routines to get plaintext configs
  • Catch the moment it injects into another process
  • Find the unpacked code if it is packed

Memory Analysis

Sometimes the real payload only exists in memory. I use Volatility to analyze dumps:

Terminal window
# List processes
volatility -f memory.dmp --profile=Win10x64 pslist
# Find injected code
volatility -f memory.dmp --profile=Win10x64 malfind
# Check network connections
volatility -f memory.dmp --profile=Win10x64 netscan

This caught me several injected DLLs that never touched disk.


What I Document

For every sample I analyze, I record:

Indicators of Compromise (IOCs)

  • Hashes (MD5, SHA256)
  • C2 IPs and domains
  • Dropped file paths
  • Registry keys modified
  • Mutex names (used to prevent double-infection)

Capabilities

  • Persistence mechanism
  • C2 protocol
  • Data it steals
  • Lateral movement techniques
  • Evasion methods

Behavior Timeline

1. Execution starts, checks if debugger present
2. Decrypts embedded config (XOR key: 0x41)
3. Drops payload to %TEMP%\svchost.exe
4. Creates registry run key for persistence
5. Connects to 192.168.x.x:443
6. Sends system info (hostname, username, OS version)
7. Waits for commands in a loop

Writing Detection Rules

After analysis, I write YARA rules so the sample gets caught next time:

rule Sample_RAT_Variant {
meta:
author = "jaswanth"
description = "Detects RAT variant found during analysis"
date = "2026-04"
strings:
$mutex = "Global\\xRAT_mutex"
$config_xor = { 41 41 41 41 }
$api1 = "CreateRemoteThread"
$api2 = "VirtualAllocEx"
$reg = "Software\\Microsoft\\Windows\\CurrentVersion\\Run"
condition:
uint16(0) == 0x5A4D and
3 of them
}

Tools I Use Daily

WhatTool
File identificationfile, DIE, pestudio
Stringsstrings, FLOSS
DisassemblyGhidra, IDA
Debuggingx64dbg
Process monitoringProcMon, Process Hacker
NetworkWireshark, FakeNet-NG
Memory forensicsVolatility
DetectionYARA
SandboxFlareVM, REMnux

What I Learned Along the Way

  • Start safe. Always sandbox. No exceptions.
  • Document as you go. I forgot details too many times early on.
  • Do not skip static analysis. It gives you a map before you explore.
  • When stuck, step back and check strings again. Authors leave clues.
  • Every sample teaches something. Even commodity malware has tricks worth noting.

Malware analysis is detective work. I enjoy the puzzle of figuring out what someone built and how to stop it.


Until Next -- Tata Bye Bye!!
jaswanth

Related Posts