Precision AV signature analysis for detection engineering and purple-team research

Built by NINJAS — offensive security & detection engineering team

MalCheck is a modern C++20 tool that pinpoints the exact byte regions inside a file that trigger antivirus detections. Unlike basic scanners that only report whether a file is malicious, MalCheck answers the question detection engineers care about most:

Which bytes caused the detection, and where do they live in the file?

Use MalCheck to:

• Test corporate security products in authorized lab environments
• Reverse-engineer AV signatures for detection rule writing (YARA, Sigma, custom rules)
• Validate detection coverage during purple-team exercises
• Compare file variants to understand what changed between clean and flagged builds
• Document findings with structured output for your SOC and detection teams

MalCheck goes beyond ThreatCheck/DefenderCheck with multi-signature discovery, structured reporting, and an extensible engine architecture designed for production detection workflows.

MalCheck uses a binary search over byte ranges against a real AV engine to isolate the minimal triggering region:

┌─────────────────────────────────────────────────────────┐
│  Full file  ──►  Detected?  ──►  YES                    │
│                                                         │
│  Split in half ──► Test each half ──► Narrow range      │
│       │                                                 │
│       ▼                                                 │
│  Repeat until exact byte boundaries are found           │
│       │                                                 │
│       ▼                                                 │
│  Signature #1: offset 0x4F1A0 – 0x4F1C8 (40 bytes)     │
│  Section: .text  │  Pattern: 48 8B 05 ?? ?? ?? ??      │
└─────────────────────────────────────────────────────────┘

Algorithm steps:

1. Load the target file into memory
2. Confirm the selected engine detects the full sample
3. Binary-search the maximum removable prefix while the suffix still triggers → signature start
4. Binary-search the minimum suffix from that start that still triggers → signature end
5. Optionally repeat on remaining bytes to find all signatures (--all-signatures)

git clone https://github.com/security-attack/MalCheck MalCheck
cd MalCheck

# Bootstrap vcpkg (first time only)
git clone https://github.com/microsoft/vcpkg.git
.\vcpkg\bootstrap-vcpkg.bat

cmake -B build -S . `
  -DCMAKE_TOOLCHAIN_FILE=.\vcpkg\scripts\buildsystems\vcpkg.cmake `
  -A x64

cmake --build build --config Release

# Basic scan with Windows Defender
.\build\Release\MalCheck.exe sample.exe

# Verbose output + find all signatures
.\build\Release\MalCheck.exe sample.exe --engine defender --all-signatures --verbose

# Adjust hex context window
.\build\Release\MalCheck.exe sample.exe --context 128 -v

MalCheck.exe <file> [options]

Arguments:
  file                    Sample file to analyze (required)

Options:
  -e, --engine <name>     Scanner engine: defender (default: defender)
  -v, --verbose           Enable detailed logging and JSON preview
      --no-color          Disable ANSI colors in console output
      --all-signatures    Find all signature regions in the file
      --context <bytes>   Hex dump context size (default: 64)
  -h, --help              Show help message

[info] MalCheck v1.0 - NINJAS
[info] Target: payload.exe (487,424 bytes)
[info] File type: PE
[info] Engine: Windows Defender (MpCmdRun)
[info] Algorithm: Binary search
[info] Full file detected. Starting signature hunt...
[debug] Scan iteration 1: 0x0003B700 - 0x00076E00 -> DETECTED
[debug] Scan iteration 2: 0x0004A000 - 0x00059280 -> DETECTED
[debug] Scan iteration 3: 0x0004E800 - 0x00051000 -> CLEAN
[info] [+] Signature #1 found: 0x0004F1A0 - 0x0004F1C8 (40 bytes)
[info] Pattern: 48 8B 05 ?? ?? ?? ?? 48 89 ?? E8 ?? ?? ?? ??
[info] Context: before: ... | signature: 48 8B 05 ... | after: ...
[info] [Summary]
[info] Total signatures: 1
[info] Time: 8.3s

{
  "file": "payload.exe",
  "size": 487424,
  "file_type": "PE",
  "engine": "Windows Defender (MpCmdRun)",
  "algorithm": "binary",
  "signatures": [
    {
      "id": 1,
      "offset_start": 324000,
      "offset_end": 324040,
      "size": 40,
      "hex": "488B05...",
      "ascii": "H\\x8B\\x05...",
      "context": "before: ... | signature: ... | after: ..."
    }
  ],
  "scan_time_ms": 8347
}

Run MalCheck against internal tools or test payloads in an isolated lab to confirm your endpoint product catches the right bytes — not just the right file hash.

Extract the exact hex pattern and offset from MalCheck output, then translate it into detection rules:

# Example workflow
MalCheck.exe suspicious.dll --verbose > findings.txt
# Use offset + hex pattern to craft YARA rule (Phase 3: auto-export)

When red-team payloads get flagged, MalCheck tells blue team what triggered — enabling precise tuning without blind rule changes.

Compare a clean build vs. a flagged build to isolate the delta bytes responsible for the detection.

MalCheck/
├── CMakeLists.txt
├── vcpkg.json
├── include/
│   ├── core/
│   │   ├── scanner.hpp           # IAVScanner interface
│   │   ├── signature_finder.hpp  # Binary search engine
│   │   └── result.hpp            # Result<T> error handling
│   ├── engines/
│   │   └── defender_engine.hpp   # MpCmdRun.exe wrapper
│   └── utils/
│       ├── logger.hpp            # spdlog wrapper
│       ├── hex_dump.hpp          # Hex formatting
│       └── file_io.hpp           # File I/O helpers
├── src/
│   ├── core/
│   ├── engines/
│   ├── utils/
│   └── main.cpp
├── tests/
│   └── unit_tests.cpp
├── docs/
│   ├── assets/
│   │   └── malcheck-banner.png
│   └── README.md
└── plugins/                      # Phase 3: custom engine SDK

• SOLID — engines implement IAVScanner; finder depends only on the interface
• RAII — temporary scan files are always cleaned up
• Exception-safe — Result<T> for explicit error propagation
• Thread-safe — Defender scans are mutex-serialized (MpCmdRun is not concurrent-safe)
• Extensible — plugin DLL architecture planned for Phase 3

# Build and run unit tests
cmake --build build --config Release --target malcheck_tests
.\build\Release\malcheck_tests.exe

# Or via CTest
ctest --test-dir build -C Release --output-on-failure

Test coverage (Phase 1):

• CMake + vcpkg project
• IAVScanner interface
• Defender engine (MpCmdRun)
• Binary-search signature finder
• CLI with --engine defender
• Multi-signature detection
• Unit tests

• AMSI engine (AmsiScanBuffer / AmsiScanString)
• ClamAV integration (libclamav)
• PE parser (section mapping: .text, .data, .rsrc)
• Mutation engine (XOR, base64, NOP slides)
• JSON file export (--output)
• Delta analysis (--delta)

• Progress bar with ETA
• Resume capability (--resume)
• YARA rule generator (--export-yara)
• IDA Python export (--export-ida)
• Plugin SDK + sample plugin
• HTML report with hex viewer

Managed via vcpkg.json:

MalCheck is a defensive security research tool developed by the NINJAS team.

• Purple-team exercises with written approval
• Malware analysis in isolated lab environments
• AV/EDR signature research for detection engineering
• Internal security product validation (authorized scope)
• Rule writing and detection tuning

• Deploying real malware or bypassing AV to attack systems
• Testing on production endpoints without authorization
• Circumventing security controls outside approved scope
• Any activity violating local laws or organizational policy

You are solely responsible for ensuring your use complies with applicable laws and your organization's security policies.

🥷 NINJAS
Offensive Security · Detection Engineering · Purple Team

Internal team tool — use responsibly in authorized lab environments only.

For questions, feature requests, or Phase 2 approval, contact the NINJAS team.