Build a Phishing Detection Lab: Analyze Real Headers, Domains and Lookalikes with free MHT and Python

TL;DR: What Is a Phishing Detection Lab and How Do You Build One?

A phishing detection lab is a free, local analysis environment that combines Python-based email header parsing, Certificate Transparency log lookups, and domain-similarity scoring to dissect suspicious emails safely—no commercial tooling required. The full workflow takes an afternoon to stand up: grab phishing samples as .eml or .mht files, parse Authentication-Results and the Received chain with Python’s standard library, query crt.sh for lookalike certificates, and fuse everything into a 0–10 risk score. Every component runs offline or against passive, public data sources—meaning you get analyst-grade capability without touching attacker infrastructure.

Phishing remains the initial access vector for the overwhelming majority of breaches. CISA and the FBI’s Internet Crime Complaint Center consistently rank phishing at or near the top of reported cybercrime categories in their annual Internet Crime Reports (ic3.gov). Yet most security professionals learn to analyze phishing on the job—by accident, clicking “View Source” on a live threat in their production mailbox. That’s the wrong training environment. This article walks you through building the right one.

Lab Setup: Tools You Need (All Free)

Traditional email security analysis has a well-established enterprise playbook—secure email gateways, sandbox detonation appliances, commercial threat intelligence feeds. A home phishing detection lab throws most of that out the window. You need exactly four things:

• Python 3.10+ — the standard-library email module does 80% of the header work for free.
• A virtual environment — python -m venv phishlab keeps dependencies isolated.
• Three pip packages: requests for Certificate Transparency queries, dnspython for DNS record checks, and rapidfuzz (or python-Levenshtein) for fuzzy domain matching.
• Sample emails as .eml or .mht files—covered in the next section.

python -m venv phishlab
source phishlab/bin/activate  # Windows: phishlabScriptsactivate
pip install requests dnspython rapidfuzz

Run everything inside an isolated VM. VirtualBox is free; a minimal Ubuntu or Windows VM works fine. The isolation matters more than the guest OS.

Getting Safe Phishing Samples Without Risking Your Network

You’re not going to fish in your own inbox. Use curated sample repositories instead. Several reputable sources distribute phishing emails for research and training:

• Nazphish and the Nazario phishing corpus — Jose Nazario’s classic research corpus of real captured phishing emails, widely mirrored.
• PhishTank (phishtank.org) — community-verified phishing URLs, useful for reconstructing realistic test cases.
• CTF archives — events like DEF CON CTF qualifiers and platforms such as Hack The Box distribute sanitized phishing artifacts.

Handle every sample with the same paranoia you’d apply to live malware:

• Download only inside the isolated VM; never on your host.
• Disable all preview panes. Rendering HTML email executes remote content—tracking pixels at minimum, exploit payloads at worst.
• Never click links, never open attachments, never “unsubscribe.” Unsubscribing confirms a live target to the attacker.
• Work with raw files. .eml and .mht formats preserve headers and bodies as static text, which is exactly what analysis requires.

Don’t forward “interesting” samples to colleagues or paste content into online sandboxes that require submission of live URLs—you’d be handing attacker infrastructure fresh telemetry.

Parsing Email Headers with Python (SPF, DKIM, DMARC)

Every phishing email tells you its own story in its headers—you just have to read them. Python’s email module parses RFC 5322 messages cleanly:

from email import policy
from email.parser import BytesParser

with open("sample.eml", "rb") as f:
    msg = BytesParser(policy=policy.default).parse(f)

print("From:", msg["From"])
print("Return-Path:", msg["Return-Path"])
print("Authentication-Results:", msg["Authentication-Results"])
for h in msg.get_all("Received", []):
    print("Received:", h)

The header that matters most is Authentication-Results, which carries three verdicts:

• SPF — did the sending IP appear in the envelope sender’s authorized IP list in DNS?
• DKIM — does the cryptographic signature over selected headers and body validate?
• DMARC — does the visible From domain align (exact or organizational) with the SPF/DKIM-validated domain, and what policy (none, quarantine, reject) applies?

Read the chain critically. spf=pass only proves the sending server was authorized for the envelope domain—it says nothing about the display From header. Attackers routinely pass SPF from their own throwaway domain while forging From: ceo@yourcompany.com. That’s why DMARC alignment exists, and why a pass on all three from the correct domain is the only clean bill of health. The M3AAWG and DMARC.org documentation (dmarc.org/overview) are the authoritative references here.

Tracing the Received Header Chain to Spot Spoofing

Received headers append top-down as mail transits the network—which means you read them bottom-up to reconstruct the true origin. The bottommost (earliest) Received header is the closest thing to ground truth about where the message entered the internet; each header above it was added by a subsequent hop and is correspondingly easier to forge.

What you’re looking for:

• Geographic and ASN absurdity. A “corporate” mail claiming origin from a residential IP in a country where the organization has no presence.
• Client-to-server mismatch. The from clause of one hop should plausibly match the by clause of the next. Gaps mean injected headers.
• Forged internal hops. Attackers insert fake Received lines naming your Exchange servers to make the message look internal. Real internal hops carry consistent timestamps and TLS handshake details (version=TLS1.2, cipher=...).
• Envelope/display divergence. Received reveals the true MAIL FROM domain—compare it against the display From.

Bottom-up reading plus the Authentication-Results verdicts catches the majority of commodity phishing. What it can’t catch is a fully registered attacker-owned domain that passes all authentication—enter domain analysis.

Checking Domains Against Certificate Transparency Logs

Certificate Transparency (CT) is a public, append-only log ecosystem mandated by Chrome and Safari: every publicly trusted TLS certificate must be logged. Attackers provisioning phishing infrastructure overwhelmingly fetch a free Let’s Encrypt or ZeroSSL certificate for their lookalike domain—and that certificate lands in CT logs within minutes, queryable by anyone.

The crt.sh service exposes CT data over a simple JSON API. Querying it from your lab is one requests call:

import requests

def ct_lookup(domain):
    r = requests.get(
        "https://crt.sh/",
        params={"q": domain, "output": "json"},
        timeout=30,
    )
    r.raise_for_status()
    certs = r.json()
    return [{
        "name": c["name_value"],
        "issuer": c.get("issuer_name", ""),
        "issued": c["entry_timestamp"][:10],
    } for c in certs]

Interpretation guide: a lookalike domain with a TLS cert issued days before the phishing email arrived is a near-certain signal of staged infrastructure. A legitimate partner domain’s cert issued years ago with a known CA is benign. CT logs are the passive reconnaissance gift that keeps giving—no packets touch the attacker.

Detecting Lookalike Domains: Homoglyphs, Typosquats and Fuzzy Matching

Domain impersonation comes in three families, and your lab should check all of them against a watchlist of domains you care about:

• Typosquats — single-character edits: paypa1.com (digit one), microsft.com (dropped letter), arnazon.com (keyboard-adjacent swap). Detected with edit distance.
• Homoglyphs — Unicode characters that render identically to ASCII: the Cyrillic а (U+0430) versus Latin a (U+0061). рaypal.com with a Cyrillic р is a different domain that looks identical. Detected via Unicode confusables normalization (Unicode Technical Standard #39).
• TLD swaps and combo attacks — microsoft-secure.com, microsoft.co, microsoft-login.net. Detected with substring and token analysis plus Levenshtein scoring.

The standard technique for homoglyph detection is skeleton folding—mapping confusable characters to a canonical form so that a homoglyph domain collapses to the real one:

from unicodedata import normalize, name

def skeleton(domain: str) -> str:
    d = normalize("NFKD", domain).lower()
    out = []
    for ch in d:
        if ch.isascii() and ch.isalnum():
            out.append(ch)
        else:
            try:
                n = name(ch)
                # crude confusable fold: strip script prefixes
                out.append(n.split()[-1][0] if n else ch)
            except ValueError:
                out.append(ch)
    return "".join(out)

For production-grade confusable mapping, use the official Unicode confusables.txt data file (unicode.org) rather than a hand-rolled fold—the OWASP Authentication Cheat Sheet (cheatsheetseries.owasp.org) covers IDN/homograph risk in its session-verification context.

Building a Python Lookalike Scoring Engine

Now fuse everything. The engine takes a candidate domain, compares it against your watchlist, and produces a 0–10 risk score:

import socket
import requests
import dns.resolver
from rapidfuzz import fuzz

WATCHLIST = ["microsoft.com", "paypal.com", "yourcompany.com"]

def dns_alive(domain):
    try:
        return bool(dns.resolver.resolve(domain, "A"))
    except Exception:
        return False

def ct_hits(domain):
    try:
        r = requests.get("https://crt.sh/",
                         params={"q": domain, "output": "json"},
                         timeout=30)
        return len(r.json()) if r.ok else 0
    except requests.RequestException:
        return 0

def score_domain(candidate: str) -> dict:
    score, reasons = 0, []
    for legit in WATCHLIST:
        ratio = fuzz.ratio(candidate, legit)          # typosquat
        skel_ratio = fuzz.ratio(skeleton(candidate),  # homoglyph
                                skeleton(legit))
        if 80 <= max(ratio, skel_ratio) < 100:
            score += 4
            reasons.append(f"High similarity to {legit} "
                           f"({max(ratio, skel_ratio):.0f}%)")
        elif ratio == 100 and candidate != legit:
            score += 6
            reasons.append(f"Homoglyph match of {legit}")

    if "-" in candidate and any(w.split('.')[0] in candidate
                                for w in WATCHLIST):
        score += 2; reasons.append("Brand+hyphen combo pattern")
    if candidate.split('.')[-1] in ("tk","top","xyz","icu","click"):
        score += 1; reasons.append("High-abuse TLD")

    if dns_alive(candidate):
        score += 1; reasons.append("Domain resolves (live infra)")
        certs = ct_hits(candidate)
        if certs:
            score += 2 if certs < 5 else 3
            reasons.append(f"{certs} CT log cert(s) issued")
    return {"domain": candidate, "score": min(score, 10),
            "reasons": reasons}

if __name__ == "__main__":
    print(score_domain("micros0ft-login.com"))

The weighting logic: similarity alone (4 points) is suspicious but not damning—legitimate partners share substrings. Similarity plus live DNS plus fresh CT certificates is a phishing campaign signature, and the score reflects that compounding. Anything scoring 7+ deserves escalation; 5–6 warrants monitoring; below 5 is noise.

Testing the Lab Against Real Phishing Emails

Wire the analyzer into a pipeline: extract domains from header fields (Return-Path, Reply-To, link hrefs in the HTML body) of a sample .eml, then run each through the scorer. Expected output on a realistic Business Email Compromise sample:

$ python analyze.py bec_sample.eml
[*] From: "CEO <ceo@yourcompany-cfo.com>"
[*] Return-Path: bounce@mailer-xy.top
[*] Authentication-Results: spf=pass (sender IP is authorized)
    smtp.mailfrom=mailer-xy.top; dkim=none; dmarc=fail (p=none)
[*] Received chain origin: 45.xx.xx.xx (AS-UNKNOWN, residential)
[!] Domain "yourcompany-cfo.com"
    score: 9/10
      - High similarity to yourcompany.com (87%)
      - Brand+hyphen combo pattern
      - Domain resolves (live infra)
      - 2 CT log cert(s) issued (most recent: 3 days ago)
VERDICT: PHISHING (high confidence)

Reading the results: the DMARC failure combined with a forged display From is your header-level kill, and the 9/10 domain score with a freshly issued certificate confirms staged infrastructure. Together, headers plus domain analysis close the two halves of the detection problem—is this email what it claims? and is the sending domain what it appears?

Automating the Pipeline and Next Steps for Blue Teams

Wrap the pipeline in a CLI with argparse—phishlab analyze sample.eml --watchlist corp.txt --json—and you have a tool that slots directly into SIEM-style workflows: pipe JSON verdicts into Splunk or an ELK index, alert on scores ≥ 7, and retain raw parsed headers as evidence. Extensions worth building next:

• VirusTotal API (free tier) enrichment for URLs and file hashes in attachments.
• WHOIS age checks — domains registered within 30 days of the email are heavily weighted phishing indicators.
• MX record analysis — lookalike domains using free mail providers or no MX at all tell different stories.
• Screenshotless detonation — fetch link targets via curl with a null user agent inside the VM, checking for credential-kit HTML markers, never via a browser.

For deliberate practice, hunt CTF challenges tagged with email analysis on platforms like Hack The Box and TryHackMe, and re-analyze the Nazario corpus monthly—your scoring engine will improve every pass. The skills transfer directly: the same header forensics and domain-intelligence techniques you’re building here are what corporate SOC teams run at scale.

Frequently Asked Questions

Is it legal to analyze phishing emails in a home lab?

Yes—parsing email headers, querying Certificate Transparency logs, checking DNS records, and computing string similarity are all passive techniques against public data. That’s legal in essentially every jurisdiction. What you must never do is actively interact with attacker infrastructure beyond safe, passive checks: no clicking links, no submitting credentials to see what happens, no port-scanning the phishing host, and no triggering more email to the attacker’s infrastructure. Stay passive and you stay legal.

Why use Certificate Transparency logs for phishing detection?

Because attackers need TLS certificates for convincing phishing pages, and every publicly trusted certificate is mandatorily logged in public CT logs. Services like crt.sh make those logs queryably free. When a lookalike domain shows a certificate issued days before a phishing email arrives, you’ve caught infrastructure staging in the act—without sending a single packet to the attacker.

What is a homoglyph domain attack?

A homoglyph attack uses visually identical Unicode characters to impersonate a trusted domain—for example, Cyrillic а (U+0430) in place of Latin a (U+0061). The domain рaypal.com with a Cyrillic р renders identically to the real one but resolves to attacker infrastructure. Detection relies on Unicode confusables normalization per UTS #39.

What Python libraries do I need for this phishing lab?

The standard-library email module handles header parsing; add requests for crt.sh API queries, dnspython for DNS record checks, and a fuzzy-matching library like rapidfuzz or python-Levenshtein for typosquat and homoglyph scoring. Total cost: zero.

What is an MHT file and why use it for phishing samples?

MHT (MIME HTML) is a single-file web archive that bundles an email’s headers, HTML body, and embedded resources into one static file. That makes it ideal for phishing samples: it preserves the full evidence—headers and rendered content—offline, with no live links executing and no remote content fetching during analysis.