rsigma engine daemon

Run as a long-running daemon with hot-reload, health checks, and Prometheus metrics.

Synopsis

rsigma engine daemon [OPTIONS] --rules <RULES>

Description

Loads rules and pipelines, opens an event source, evaluates events as they arrive, fans the detections out to one or more sinks, and stays alive until it receives SIGTERM/SIGINT. Reloads rules and pipelines on file change, SIGHUP, or POST /api/v1/reload. Exposes Prometheus metrics, REST control endpoints, and OTLP log ingestion on the same --api-addr.

This is the long-running counterpart of engine eval. Use it when you need state to survive restarts, hot-reload across rule changes, or a Prometheus-scrapeable detection engine.

For narrative coverage see Streaming Detection. For NATS-specific operations (auth, replay, consumer groups, DLQ) see NATS Streaming.

Flags

Required

Configuration

Event input

Output sinks and DLQ

Each --output sink runs its own bounded queue and worker: results are delivered with bounded exponential-backoff retry, and a result is sent to the DLQ only after retries are exhausted. Fan-out is isolated up to each sink’s queue depth, so a slow sink does not immediately stall the others; a slow durable sink eventually applies backpressure, the cost of preserving at-least-once delivery. An acknowledgment is released to the source only once every sink has committed the result, and ?on_full=drop opts a sink out of that contract in favor of never stalling.

Acknowledgment guarantees differ by transport. NATS is durable at-least-once: results un-acked at shutdown are redelivered on restart. The HTTP and OTLP push receivers are at-most-once on ingest, since the request is accepted as soon as the event is queued.

Pipelines and dynamic sources

Post-evaluation enrichment

The enrichers file accepts max_concurrent_enrichments: <N> at the top level (default 16) plus a list of enricher entries, each declaring kind: detection | correlation, a primitive type: (template / lookup / http / command), an inject_field, and primitive-specific keys (template, url / headers / cache_ttl, command, source / extract / default, …). Cross-namespace template references are rejected at startup with a clear error pointing at the offending field.

The webhooks file (also settable as the daemon.output.webhooks path list) declares a list of webhooks: entries, each with an id, kind: detection | correlation, a url, optional headers / body templates, and optional timeout, retry, rate_limit, scope, and queue_size. Secrets are referenced via ${ENV_VAR} rather than stored in the file. See Webhooks for the schema and the Slack/Teams/Discord/PagerDuty recipe catalog.

API server

Authentication

Off by default: without --api-token-env or a daemon.api.auth config block, every route is open and network position (TLS/mTLS, reverse proxy, loopback, or a unix:// socket) is the only guard.

For anything beyond the single admin token, configure the daemon.api.auth block (config-file-only, like the tap and tail tuning keys): named roles as resource:action permission lists, per-token role assignment with env-resolved secrets, and anonymous_permissions for requests without a token (for example ["metrics:read"] keeps Prometheus scraping token-free). Built-in roles: reader, operator, ingest, admin. The per-route permission table, role definitions, and failure semantics (401 vs 403, the rsigma_api_auth_failures_total counter) are documented in HTTP API: Authentication; the threat-model discussion is in Security: Daemon API authentication.

daemon:
  api:
    auth:
      anonymous_permissions: ["metrics:read"]
      tokens:
        - name: grafana
          role: reader
          token_env: RSIGMA_API_TOKEN_GRAFANA
        - name: shipper
          role: ingest
          token_env: RSIGMA_API_TOKEN_SHIPPER

On a non-loopback TCP listener, pair authentication with TLS (below) or a TLS-terminating proxy so tokens are never sent in cleartext.

TLS (requires the daemon-tls build feature)

When TLS is configured, the daemon terminates TLS in-process for every protocol on --api-addr (HTTP REST API, /metrics, OTLP/HTTP, OTLP/gRPC). The negotiation advertises both h2 and http/1.1 via ALPN so legacy REST clients and modern gRPC clients share one socket.

Hot-reload: every reload trigger funnels through the daemon’s central debounced reload task, so a single POST /api/v1/reload (cross-platform, including Windows), kill -HUP <pid> (Unix), or a YAML file change picked up by the file watcher rotates rules, pipelines, enrichers, and the TLS certificate in one pass. The active rustls::ServerConfig is swapped atomically via Arc<ArcSwap<…>>, so new handshakes pick up the rotated material without dropping inflight TLS connections. Failed reloads keep the previous certificate active, bump rsigma_reloads_failed_total, and log an error so a typo in the cert path cannot black-hole the listener.

Observability: the /metrics endpoint exposes rsigma_tls_certificate_expiry_seconds (signed; negative once the cert has expired) and rsigma_tls_active_connections. A single WARN is logged at startup (and after every reload) if the active certificate expires within 30 days.

See TLS deployment for a deeper dive, including ACME / sidecar reverse proxy alternatives that this feature replaces.

Correlation behavior

State persistence

Throughput

NATS (requires the daemon-nats build feature)

The auth methods are mutually exclusive. See NATS Streaming for the full operational guide.

Performance (advanced)

Field observability (advanced)

These schema flags may also be supplied via the daemon.schema block in a config file (observe, routing, config, on_unknown); a flag always wins over the file.

These logsource flags may also be supplied via the daemon.logsource_routing block in a config file (enabled, field_map, event_logsource); a flag always wins over the file.

See Observability: detection coverage for the operator workflow, and HTTP API for the endpoint payloads.

Live event tap

The daemon serves GET /api/v1/tap (the endpoint behind rsigma engine tap), which records a bounded window of the live event stream as a replayable NDJSON fixture. It is disabled by default because it can exfiltrate raw event traffic; enable it with daemon.tap.enabled: true and expose it only behind mTLS.

The other keys are config-file-only under daemon.tap (there is no flag for them):

The tap can exfiltrate raw events; expose the admin API only behind mTLS and redact sensitive fields. See Security: live event tap.

Live detection tail

The daemon also serves GET /api/v1/detections/stream (the endpoint behind rsigma engine tail), which streams live detections as NDJSON. It is disabled by default; enable it with daemon.tail.enabled: true.

The other keys are config-file-only under daemon.tail:

Triage feedback loop

The daemon serves POST/GET /api/v1/dispositions, which ingest analyst verdicts and expose a per-rule false-positive ratio. It is disabled by default; enable it with --enable-dispositions, daemon.dispositions.enabled: true, or a configured pull source.

The tuning keys are config-file-only under daemon.dispositions:

See the Triage Feedback Loop guide.

Examples

Minimal daemon: stdin → stdout

rsigma engine daemon -r rules/

Reads NDJSON from stdin, writes detections to stdout. Default API on 0.0.0.0:9090.

HTTP ingest with persistent state

rsigma engine daemon -r rules/ \
    --input http \
    --state-db /var/lib/rsigma/state.db \
    --pipeline ecs_windows

Accepts POST /api/v1/events for ingest; correlation state survives restarts.

NATS source + sink + DLQ

NATS_CREDS=/etc/rsigma/nats.creds \
rsigma engine daemon -r /etc/rsigma/rules/ \
    --input "nats://nats.internal:4222/events.>" \
    --output "nats://nats.internal:4222/detections" \
    --dlq "file:///var/log/rsigma/dlq.ndjson" \
    --state-db /var/lib/rsigma/state.db \
    --buffer-size 50000 \
    --batch-size 128 \
    --drain-timeout 30 \
    --nats-require-tls \
    --api-addr 0.0.0.0:9090

Multi-output fan-out

rsigma engine daemon -r rules/ \
    --output stdout \
    --output "file:///var/log/rsigma/detections.ndjson" \
    --output "nats://nats.internal:4222/detections.urgent"

Export detections to an OpenTelemetry collector

Each result becomes one OTLP log record (Sigma level mapped to OTLP severity, rule title as the body, the full result as attributes). Requires a daemon-otlp build.

# OTLP/gRPC (default port 4317)
rsigma engine daemon -r rules/ --input http \
    --output "otlp://otel-collector:4317"

# OTLP/HTTP protobuf with gzip (default port 4318)
rsigma engine daemon -r rules/ --input http \
    --output "otlphttp://otel-collector:4318?compression=gzip"

# OTLP/gRPC over TLS, verifying the collector against a private CA
rsigma engine daemon -r rules/ --input http \
    --output "otlps://otel-collector:4317?ca=/etc/rsigma/tls/ca.pem"

# OTLP/HTTP over mutual TLS (client cert + key)
rsigma engine daemon -r rules/ --input http \
    --output "otlphttps://otel-collector:4318?ca=/etc/rsigma/tls/ca.pem&client_cert=/etc/rsigma/tls/client.pem&client_key=/etc/rsigma/tls/client.key"

# Post alerts to webhooks while keeping a durable NATS record
rsigma engine daemon -r rules/ --input http \
    --output "nats://nats.internal:4222/detections" \
    --webhook /etc/rsigma/webhooks/ \
    --dlq "file:///var/log/rsigma/dlq.ndjson"

TLS uses the bundled public (webpki) roots by default; pass ?ca= to verify against a private CA instead. Supplying both ?client_cert= and ?client_key= enables mutual TLS. Use ?tls_domain= to override the verified server name when dialing by IP.

HTTPS with mutual TLS

rsigma engine daemon -r rules/ \
    --input http \
    --api-addr 0.0.0.0:9090 \
    --tls-cert /etc/rsigma/tls/server.crt \
    --tls-key  /etc/rsigma/tls/server.key \
    --tls-client-ca /etc/rsigma/tls/clients-ca.crt

Clients connecting to https://daemon:9090/v1/logs (OTLP/HTTP) or https://daemon:9090/api/v1/events (REST) must present a certificate signed by clients-ca.crt or the handshake is rejected. Rotate the server cert with cp new.crt /etc/rsigma/tls/server.crt && kill -HUP $(pidof rsigma) on Unix, or cp new.crt … && curl -X POST https://daemon:9090/api/v1/reload on any platform (including Windows, where SIGHUP does not exist).

Forensic replay from a NATS sequence

rsigma engine daemon -r rules/ \
    --input "nats://localhost:4222/events.>" \
    --replay-from-sequence 1001 \
    --state-db /var/lib/rsigma/replay-state.db \
    --timestamp-fallback skip

--timestamp-fallback skip prevents wall-clock contamination of correlation windows when replaying old events.

Health and readiness

Full HTTP API reference: HTTP API. All metric definitions: Prometheus metrics.

Shutdown

SIGTERM and SIGINT trigger a graceful drain bounded by --drain-timeout. In-flight events are processed and acknowledged before the daemon exits. With --state-db, the final correlation state snapshot is written during shutdown.

Exit codes

See also

• Streaming Detection for the daemon walkthrough.
• NATS Streaming for auth, replay, consumer groups, and DLQ details.
• OTLP Integration for the OTLP receiver and agent recipes.
• Performance Tuning for --bloom-prefilter, --cross-rule-ac, --batch-size, and --buffer-size.
• Observability for the RUST_LOG targets, tracing spans, and metric alerting recipes.
• engine eval for the one-shot evaluation counterpart.