Quick Answer — In July 2023, researchers at Hoxhunt flagged WormGPT, a GPT-J-derived large language model marketed on criminal forums as “a blackhat’s ChatGPT”: no ethics filters, no refusal, purpose-built for BEC spam and phishing. The subscription bot (€500/month, per researcher screenshots) was the first widely-documented fraud-focused LLM — proof that generative AI’s cost curve had fallen far enough that offensive automation became a service industry, and the moment enterprise phishing-defense assumptions had to be rewritten from “spot the broken grammar” to “detect the fluent, personalized, machine-scaled lure”.
What happened
- The product: WormGPT was built on the open-source GPT-J 6B architecture (2021-vintage), trained — per its author’s own marketing — on malware-related data, with no alignment tuning whatsoever. Its selling points were the absence of refusals and “all illegal content” available on demand.
- The pitch: Forum threads showed the author demonstrating BEC email generation, phishing-kit copy, and malicious code snippets; pricing tiers ran from forum access to subscription API via Telegram bot (€500/month reported by Hoxhunt’s screenshots of the sales thread).
- The lifecycle: Hype peaked within weeks; by August 2023 the author claimed to be shutting down (a “commitment to a legal project”), amid speculation the exit was driven by law-enforcement heat, scammers scamming scammers, and the plain fact that jailbroken mainstream models often outperformed it.
- The market response: Security vendors (Abnormal, SlashNext, others) published detection analyses; “WormGPT” became shorthand for the whole criminal-LLM category that followed — FraudGPT, EvilGPT, WolfGPT and dozens of copycats.
Why a 6B model mattered
| Factor | Mainstream ChatGPT (2023) | WormGPT |
|---|---|---|
| Base capability | GPT-3.5/4-class | GPT-J 6B — 2021 open-source |
| Alignment | RLHF refusals | None — trained for offense-friendly outputs |
| Access | ToS-governed API | Telegram bot, forum subscription |
| Cost to operator | Enterprise pricing | GPU rental for fine-tune + hosting |
| Actual moat | Capability | Lack of refusals only — capability rented from open source |
Timeline
| Date | Event |
|---|---|
| 2021 | GPT-J 6B released by EleutherAI — the base model WormGPT would later ride |
| 2023-03→05 | WormGPT sales threads appear on underground forums; subscription bot matures |
| 2023-07-05 (approx) | Hoxhunt researchers publish the analysis that named it publicly |
| 2023-07-13 (our peg) | Global press coverage peaks; “blackhat’s ChatGPT” enters the vocabulary; vendors ship detection notes |
| 2023-08 | Author announces shutdown; FraudGPT and clones fill the vacuum overnight |
| 2024 → | Criminal-LLM category consolidates into subscription “attack platforms” combining LLM lures with infrastructure |
Defensive lessons
- The grammar tell is dead. WormGPT-class tools exist precisely to fix the typo-ridden BEC email; detection must move to intent, context, and channel anomalies — never to prose quality.
- Open weights are a dual-use commodity. Any 2021-era open model could be repurposed; assume adversaries rent frontier-adjacent capability at commodity prices and plan controls for fluency, not brokenness.
- Criminal SaaS has marketing departments. The screenshots, tiers, and “shutting down” drama were textbook underground commerce behavior — threat-intel teams should track criminal products like commercial vendors (changelogs, pricing, SLAs).
- Hype ≠ capability. Independent tests found WormGPT often worse than jailbroken GPT-4-class models; risk communication should separate the category’s trajectory from any single product’s prowess.
The economics of a criminal foundation model
The uncomfortable arithmetic: GPT-J’s weights were free, fine-tuning cost a few hundred dollars of GPU time, a Telegram bot is infrastructure anyone can rent, and the addressable market — BEC losses ran in the billions per year per FBI IC3 tallies — meant even a tiny conversion rate on a €500/month subscription printed money. Nothing about WormGPT required nation-state resources; it required patience and a payment processor. That is the durable strategic fact: the marginal cost of fluent offensive automation collapsed to commodity levels in 2023, and every defensive budget line since has been catching up to that reality.
Why it still matters in 2026
WormGPT was the category-defining proof that offense would industrialize generative AI the moment weights and GPU hours became cheap. By 2026, the “fraud LLM” is no longer a curiosity: deepfake video calls, voice clones of executives, and agentic phishing kits are table stakes, and the defensive industry WormGPT spawned — LLM-lure detection, behavioral mail analysis, human-in-the-loop verification for money movement — is now core enterprise budget. The name shut down in a month; the market it demonstrated never did.
What was WormGPT actually built on?
GPT-J, a 6-billion-parameter open-source model from EleutherAI, dating to 2021 — deliberately chosen because weights were free, hosting was cheap, and there was no vendor ToS to violate. Everything dangerous about it was the application layer wrapped around an ordinary model.
Did it work?
As a product, marginally: demonstrated BEC emails were fluent but generic, and buyers on criminal forums complained about quality. As a signal, enormously: it drew the defensive industry’s attention and budgets to LLM-generated phishing years before agentic attacks made the threat mainstream-scale.
Is the category still around?
Yes — consolidated. Standalone “evil chatbots” faded; their capabilities merged into full attack platforms (phishing-kit builders with LLM copy generation, deepfake modules, and infrastructure rental bundled as one subscription). WormGPT’s shutdown marked the shift from novelty products to integrated criminal platforms, which is precisely the consolidation legitimate SaaS went through a decade earlier.
Why peg this to July 13, 2023?
That’s the date press coverage of the Hoxhunt research reached global scale — the moment “criminal LLM” became a board-briefing topic. The sales threads predate it by months, which is itself a lesson: criminal tooling adoption runs on forum time, and defenders only see it when it breaks into press time.
Part of the hmmnm.com security-timeline series — one event per month, 2021–2024, indexed here.
