Try Sigma rules against your own logs

This how-to assumes you already ship logs somewhere central, for example Grafana Loki or Elasticsearch as set up in the monitoring topic, and that you have Python 3.9 or newer on the machine where you will run the converter. Nothing here touches any system other than your own log store. By the end you will have converted one community rule into a query, run it, and seen either zero results or a list of events to look at.

1. Install the converter

sigma-cli is the official command-line front end to the pySigma library. Install it in an isolated environment with pipx (or in a virtualenv with pip) and confirm it runs:

pipx install sigma-cli
sigma version
sigma plugin list

The plugin list shows the backends (target query languages) and pipelines (field-name mappings) that are available. Install the backend for your platform; the examples below use Loki, but Elasticsearch, Splunk, Microsoft Sentinel, OpenSearch and many others are installed the same way:

sigma plugin install loki            # for Grafana Loki (LogQL)
# sigma plugin install elasticsearch  # for Elastic (Lucene / ES|QL)
# sigma plugin install splunk

2. Get the rule library

git clone --depth 1 https://github.com/SigmaHQ/sigma.git
ls sigma/rules/

The rules/ directory is split by platform and then by log source. For a first test choose a rule that matches a log you definitely collect. If you ship Linux authentication logs, the rules/linux/ tree is a good place; if you ship web server access logs, look under rules/web/. Open the rule in an editor and read its title, description, logsource block and level; that tells you what it is looking for and whether you have the right kind of data.

3. Convert one rule

The general form is sigma convert -t BACKEND -p PIPELINE RULEFILE. The pipeline tells the converter how to translate Sigma’s standard field names into the names and labels your platform uses. Each backend ships with one or more pipelines; list them and pick the closest match:

sigma list pipelines
sigma convert -t loki -p loki_promtail_sysmon sigma/rules/linux/builtin/auth/lnx_auth_susp_failed_logons_single_source.yml

The output is a query in your platform’s language. If the converter complains that a field cannot be mapped, that is expected: it means your logs name things differently from the rule’s assumptions. You can write a small custom pipeline YAML file that renames fields and pass it with -p ./my-pipeline.yml; the pySigma documentation describes the format. Loki in particular often needs you to adjust the stream selector (the part in curly braces) to the labels you actually use, such as {job="auth"}.

4. Run the query and look at the result

Paste the query into your platform’s query interface: Grafana Explore for Loki, Kibana Discover or Dev Tools for Elastic, the search bar for Splunk. Widen the time range to a few days. There are three possible outcomes:

• Zero results. Good, but confirm the query is actually reaching your data by loosening it (for example, removing one condition) until it returns something. A query that can never match is not a detection.
• A few results. Open them. Most first hits are legitimate administration, a monitoring check or a cron job. Note what caused them; this is how you learn what a rule needs to exclude in your environment.
• Thousands of results. The rule is too broad for your data or the field mapping is wrong. Check the pipeline before assuming you have an incident.

5. Turn it into an alert (optional)

Once a converted query returns sensible results, add it as an alert rule in your platform: a Grafana alert rule on the Loki query, a Kibana rule in Elastic, or a saved scheduled search in Splunk. Keep the Sigma file in your own Git repository next to the converted query and the pipeline, so the next person can see where the rule came from and regenerate it after an upgrade. Converting the whole library in one go is possible with sigma convert on a directory, but start with a handful of rules you understand; a flood of untuned alerts is quickly ignored.

Common problems

• sigma: command not found after pipx install: open a new shell or run pipx ensurepath.
• Backend 'x' not found: the plugin is not installed, or it was installed into a different Python environment than sigma-cli.
• Conversion errors about unsupported modifiers: some rules use features a given backend cannot express; skip those rules or try another backend. The --skip-unsupported flag converts a directory while ignoring them.
• Field names do not match: this is the normal case; write a pipeline rather than editing the rule, so you can still pull updates from the library.

Official documentation

• Sigma documentation, including the rule specification and sigma-cli guide
• pySigma documentation (backends and processing pipelines)
• SigmaHQ/sigma-cli on GitHub