Write Sigma Rules That Actually Fire: A Detection-as-Code Lab with SigmaCLI and splunk-react

TL;DR: How Do You Make Sigma Rules Actually Fire?

Sigma has become the de facto open standard for generic detection rules—think of it as the YARA of log analysis, championed by the SigmaHQ community and now maintained under the Sigma project with pySigma as its reference implementation. Yet every SOC has a graveyard of Sigma rules that “validated fine” and never fired once in the SIEM. The problem isn’t the format. The problem is that most teams treat rule-writing as an authoring task instead of an engineering discipline. This guide walks you through building a Detection-as-Code lab—authoring, validating, converting, and testing Sigma rules end-to-end with sigma-cli and a Splunk backend—so your detections actually fire.

Why Sigma Rules Fail in Production

Rules that pass schema validation but never trigger in your SIEM almost always fail in one of three ways:

• Wrong logsource mapping. The rule declares a logsource of category: process_creation / product: windows, but your Splunk deployment ingests Sysmon Event ID 1 events into an index with field names, sourcetypes, or index patterns that the conversion pipeline doesn’t know about. The generated SPL queries an empty index or nonexistent fields. Silence.
• Overly strict field conditions. A condition combining Image|endswith, CommandLine|contains, and a parent process filter will match a captured event in a lab but not in production telemetry where field casing, quoting, or process paths differ slightly. Sigma rules built only from SigmaHQ’s corpus inherit assumptions about telemetry you may not share.
• Untested conversion output. Teams run sigma convert, paste the SPL into Splunk, see it’s syntactically valid, and ship it. Nobody checks whether the query returns rows for a known-bad event. Valid SPL that returns zero results is still a dead rule.

The fix is the same one software engineering arrived at decades ago: version control, automated validation, compilation, and testing. Detection-as-Code is that pipeline applied to detection engineering—and it’s increasingly expected practice. CISA and NSA’s joint guidance on logging and detection maturity, and the push behind open standards like Sigma listed on CISA’s cybersecurity resources, all point the same direction: detections belong in CI, not in a SIEM GUI.

Lab Setup: Installing sigma-cli and the Splunk Pipeline

sigma-cli is the command-line tool that wraps pySigma. Backends—including Splunk—are installed as separate Python packages:

pip install sigma-cli pysigma-backend-splunk

Verify the backend registered:

sigma list checkers
sigma list pipelines
sigma list backends

Use a repository layout that mirrors a software project:

sigma-lab/
├── rules/
│   └── windows/
│       └── suspicious_process_creation.yml
├── pipelines/
│   └── splunk_sysmon.yml      # your logsource-to-index mapping
├── tests/
│   └── events/                # captured or synthetic EVTX/JSON events
├── .github/workflows/ci.yml
└── README.md

The pipelines/ directory is where most labs skimp—don’t. A custom processing pipeline is what translates generic Sigma logsources into your environment’s actual index names, sourcetypes, and field transformations. We’ll build one below.

Sigma Rule Anatomy: Logsource, Detection, and Condition

Every Sigma rule has three load-bearing sections. Here’s a working rule, annotated:

title: Suspicious PowerShell Download Cradle via Encoded Command
id: 7b8d9c4a-1e2f-4c3b-9a5d-6f7e8a9b0c1d
status: experimental
description: Detects PowerShell spawning with encoded download commands,
  consistent with T1059.001 (PowerShell) payload retrieval.
references:
  - https://attack.mitre.org/techniques/T1059/001/
author: your-team
date: 2025/01/10
logsource:
  category: process_creation
  product: windows
detection:
  selection_img:
    Image|endswith: 'powershell.exe'
  selection_cmd:
    CommandLine|contains|all:
      - '-enc'
      - 'FromBase64String'
      - 'DownloadString'
  condition: selection_img and selection_cmd
falsepositives:
  - Administrative scripts using encoded commands
level: high
tags:
  - attack.execution
  - attack.t1059.001

Logsource declares what telemetry the rule expects—here, Windows process creation events (Sysmon Event ID 1 or Security 4688). Detection holds named selections of field/value matchers; the |contains|all modifier requires every list item to appear in the command line. Condition combines selections with boolean logic. Everything else—status, level, tags, ATT&CK references—is metadata that drives triage and reporting. Note the id is a stable UUIDv4: never change it when editing a rule, or you’ll orphan your tuning history.

Writing Your First Rule Against a Real Log Source

Now make the logsource concrete. In Splunk, Sysmon Event ID 1 events typically land with sourcetype="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational" and fields like Image, CommandLine, ParentImage. Create pipelines/splunk_sysmon.yml:

name: splunk_sysmon_mapping
priority: 10
logsources:
  windows-process-creation:
    category: process_creation
    product: windows
    conditions:
      index: sysmon
      sourcetype: XmlWinEventLog:Microsoft-Windows-Sysmon/Operational
fieldmappings:
  Image: Image
  CommandLine: CommandLine

This is the direct answer to the mapping question: the pipeline tells pySigma that a generic process_creation / windows logsource means “search index=sysmon with this sourcetype” in your environment. If your fields are lowercase or renamed during ingestion (say, process_path instead of Image), fix it here—not by editing every rule.

Validating Rules with sigma check

Before converting anything, run the built-in validators:

sigma check rules/

This catches YAML schema violations, malformed UUIDs, duplicate rule IDs, empty selections, unknown field modifiers, and condition logic errors. Sample output on a broken rule:

Checking rules/windows/broken.yml
  [SigmaDetectionItemConditionError] Condition 'endwith' is not a valid modifier
  [SigmaRuleStatusError] 'test' is not a valid status value
Found 2 errors in 1 rules

The distinction matters because it answers a common confusion: sigma check validates rule structure and logic against the Sigma spec. It does not touch your SIEM and knows nothing about your field names. sigma convert translates validated rules into backend queries using a processing pipeline that carries your environment-specific mapping. Validate first, convert second, test third—skipping steps here is exactly how dead rules get shipped.

Converting Sigma to Splunk SPL with sigma convert

With the backend installed and the pipeline in place:

sigma convert -t splunk -p splunk_sysmon_mapping rules/windows/suspicious_process_creation.yml

Output (formatted for readability):

index=sysmon sourcetype="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational"
  Image="*powershell.exe"
  CommandLine="*-enc*"
  CommandLine="*FromBase64String*"
  CommandLine="*DownloadString*"

Inspect this SPL carefully. The pipeline injected the correct index and sourcetype; the modifiers became Splunk wildcard expressions. If the output references fields your data doesn’t contain, or wraps things in the wrong case, the fix belongs in the pipeline, not the rule. pySigma’s backend ecosystem—Splunk, Elasticsearch/Lucene, Microsoft Sentinel (KQL), Loki, and others installable via pip—means the same validated rule compiles for every backend you support.

Testing That the Rule Actually Fires

Conversion correctness is not detection correctness. You need a positive test: an event that must fire the rule.

1. Produce or replay the event. In a lab VM with Sysmon installed, run the attack primitive: powershell -enc a base64 string containing DownloadString. Or replay a captured EVTX/log line into Splunk into the lab index.
2. Run the generated SPL in Splunk against a time range covering the event. You’re looking for at least one result row matching your test event—not just a query that executes without error.
3. Run a negative test. Run benign PowerShell activity; confirm the rule doesn’t fire. Detection without false-positive control is alert fatigue with extra steps.
4. Tune. If the rule fires on everything, tighten conditions (add parent process filters, use |startswith over |contains where appropriate). Re-run check, convert, and both tests after every change.

Automate this where you can: capture the test event as JSON in tests/events/, and have CI (or a small pytest harness using the Splunk SDK) assert the rule’s SPL returns that event. This mirrors the unit-test discipline of any software project and is the core promise of Detection-as-Code.

Wiring Sigma Detections into a CI Pipeline

A GitHub Actions job that blocks merges on invalid rules:

name: sigma-ci
on: [pull_request]
jobs:
  validate-and-convert:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - run: pip install sigma-cli pysigma-backend-splunk
      - name: Validate rules
        run: sigma check rules/
      - name: Convert to Splunk
        run: |
          mkdir -p dist
          sigma convert -t splunk -p splunk_sysmon_mapping 
            --output dist/splunk_queries.txt rules/
      - name: Fail on empty conversion output
        run: |
          if [ ! -s dist/splunk_queries.txt ]; then
            echo "Conversion produced no queries"; exit 1
          fi
      - uses: actions/upload-artifact@v4
        with:
          name: splunk-queries
          path: dist/

At minimum, CI must: validate every changed rule with sigma check, convert for every target backend (a rule that converts cleanly for Splunk but breaks for Sentinel fails the merge), and run your positive/negative test harness against captured events. Teams with a live lab SIEM add a deployment step that pushes validated queries on merge to main—full continuous delivery for detections.

Versioning, Tuning, and False-Positive Management

Treat rule changes like code changes:

• Stable UUIDs, evolving rules. Keep the id fixed across edits so tuning history and alert correlation survive. Use the related field when a rule genuinely supersedes another.
• Status lifecycle. Ship as experimental, promote to stable only after a soak period with measured false-positive rates. Deprecate explicitly rather than deleting silently.
• False positives as first-class artifacts. Every documented falsepositives entry is institutional memory. When an analyst dismisses an alert, the fix should be a rule PR—tightened condition or documented exclusion—not a one-off SIEM filter nobody will find again.
• Tags and references. Map rules to MITRE ATT&CK techniques (attack.t1059.001) so coverage gaps become measurable. The ATT&CK framework is the shared vocabulary your detection backlog needs.

Extending the Lab: More Backends and Rule Pipelines

Once the Splunk loop works, the same repo extends naturally. Install pysigma-backend-elasticsearch or the Sentinel backend and add a conversion job per target—CI fails if any backend breaks. Write additional pipelines per log source (Sysmon vs. Zeek vs. cloud audit logs), each declaring the index/sourcetype conditions and field mappings for that source. For practice, point the lab at real attacker TTPs: run Atomic Red Team or a purple-team exercise, capture telemetry, and challenge yourself to write rules that fire—then tune until false positives approach zero. This is also exactly the exercise structure of most blue-team CTFs, where writing detections for T1059.001 or T1003-style behavior is scored directly.

Key Takeaways for Blue-Team Practitioners

• Author in pySigma-compatible YAML; keep stable UUIDs and a status lifecycle.
• Map logsources to your environment with processing pipelines—never hand-edit generated queries.
• Validate with sigma check, convert with sigma convert, and inspect the SPL before shipping.
• Every rule needs a positive test (known-bad event fires) and a negative test (benign activity doesn’t).
• CI gates merges: check, convert for all backends, test—then deploy on merge.
• Tune via pull requests, document false positives, and tag ATT&CK techniques to measure coverage.

Detection engineering earns its “engineering” the same way software did: version control, automated gates, and tests that prove the thing works. Build the lab once, and every rule you ship afterward carries the evidence that it actually fires.

Frequently Asked Questions

What is Detection-as-Code?

Treating detection rules like software: stored in version control, validated, tested, and deployed through CI/CD. Rules get peer review, automated quality gates, and reproducible deployments instead of hand-editing queries in a SIEM console.

Do I need a full Splunk instance to follow this lab?

A local or free Splunk instance is ideal for end-to-end testing, but conversion and validation work without one—sigma check and sigma convert run entirely offline. You only need the SIEM to confirm the rule fires against real events.

Why does my Sigma rule convert but return no results?

Usually a logsource-to-index/field mapping mismatch, or conditions too strict for your telemetry. Fix the mapping in your processing pipeline and test the generated SPL against captured events where you know the rule should fire.

Which backends does sigma-cli support?

Any pySigma backend installable via pip—including Splunk, Elasticsearch, Microsoft Sentinel, Loki, and others listed in the pySigma documentation. Each backend is a separate package you install alongside sigma-cli.

Can I use Sigma rules for CTF challenges?

Yes—writing rules that fire on attacker TTPs is a common blue-team CTF exercise, and it maps well to MITRE ATT&CK techniques, which most challenges score against.

Related reading

• Why Planes Don't Get Hacked — And Where the Next Aviation Cyber Risk Really Is
• What Is Prompt Injection and How to Prevent It: A Complete Exam Prep Guide

Published by Hmmnm

Hands-on cybersecurity tutorials, CVE breakdowns, and guided learning paths — written and lab-tested by the Hmmnm team.

Learning Paths · About Hmmnm · Editorial policy