Wikiwand AI

Draft:Detection-as-Data (DaD)

Cybersecurity and data engineering paradigm From Wikipedia, the free encyclopedia


Detection-as-Data (DaD) is a cybersecurity and data engineering paradigm in which threat detection logic, signals, and outcomes are modeled and managed as structured, queryable data within data platforms rather than as static rules or alerts. The approach integrates principles from data engineering, machine learning, and detection engineering to enable scalable, reproducible, and explainable security analytics.

  • Comment: Probably notable, but needs more citations PeriodicEditor (talk) 20:50, 25 April 2026 (UTC)

The concept has been discussed in the context of modern data platforms and AI-driven security systems, particularly as organizations adopt data lakehouse architectures and shift toward data-centric approaches to cybersecurity. The term has been associated with work by Luiz Zanardo, among others exploring data-centric detection models.

Overview

Traditional security information and event management (SIEM) systems rely on rule-based or correlation-based detection models that generate alerts from predefined conditions.[1] These systems typically treat detections as transient events derived from log data.

Detection-as-Data proposes an alternative model in which:

  • Detection logic is implemented as data transformations
  • Detection outputs are stored as persistent datasets
  • Detection pipelines are recomputable and version-controlled

This approach aligns with broader industry trends toward data-centric architectures, including data lakehouses and data mesh systems.

Origins and development

The concept of treating detections as structured data has emerged alongside the evolution of large-scale data platforms and cloud-native analytics. In the early 2020s, practitioners in both cybersecurity and data engineering began exploring approaches that integrate detection logic directly into data processing pipelines.

The term Detection-as-Data has been referenced in discussions of modern detection engineering practices and has been associated with contributions by Luiz Zanardo, particularly in the context of integrating detection systems with data lakehouse architectures and AI-driven analysis.

Core concepts

Detection as a first-class data entity

In Detection-as-Data systems, detections are stored as structured records containing:

  • Source telemetry and signals
  • Transformation logic (e.g., queries, models)
  • Contextual metadata
  • Detection outcomes (e.g., risk scores, classifications)

This contrasts with traditional SIEM alerts, which are often ephemeral and lack full lineage.

Data pipeline-based detection

Detection logic is implemented using data pipeline paradigms, including:

  • SQL-based transformations
  • Stream processing
  • Graph-based correlation
  • Vector similarity search

These pipelines are often executed on distributed data platforms.

Reproducibility and replayability

Because detections are derived from stored data transformations:

  • Historical detections can be recomputed
  • New detection logic can be applied to past data
  • Detection performance can be evaluated over time

Explainability and lineage

Detection-as-Data systems maintain full lineage across:

  • Input data sources
  • Intermediate transformations
  • Final detection outputs

Relationship to existing approaches

SIEM and correlation engines

SIEM platforms aggregate and correlate events to generate alerts.[2] However, they typically rely on predefined logic and may struggle to detect novel attack patterns.

Detection-as-Data extends this model by:

  • Treating detections as datasets rather than alerts
  • Enabling recomputation and iteration
  • Integrating machine learning and semantic analysis

Detection engineering

Detection engineering focuses on building and maintaining detection logic as code. Detection-as-Data expands this concept by incorporating data engineering practices, including data versioning and lineage.

Anomaly detection

Detection-as-Data systems often incorporate techniques from anomaly detection, which identifies deviations from expected behavior.

Architecture

A typical Detection-as-Data architecture includes:

  • Data ingestion: Collection of logs, telemetry, and event streams
  • Data processing: Normalization, enrichment, and feature extraction
  • Detection layer: Execution of detection pipelines using SQL, machine learning, graph analytics, or vector search
  • Storage: Persistent storage of detection datasets in data lakes or warehouses
  • Consumption: Access via dashboards, APIs, and automated response systems

Role of artificial intelligence

Detection-as-Data systems frequently incorporate artificial intelligence techniques, including:

  • Embedding-based representations of events
  • Clustering and similarity analysis
  • Large language models for contextual reasoning

These techniques enable semantic analysis of security events and support adaptive detection strategies.

Advantages

  • Scalability through distributed data platforms
  • Flexibility in detection logic
  • Improved explainability through lineage tracking
  • Continuous improvement via recomputation and iteration

Challenges

  • Large data volume and storage requirements
  • Increased system complexity
  • Governance and compliance considerations
  • Emerging tooling ecosystem

Use cases

  • Advanced persistent threat detection
  • Insider threat analysis
  • Fraud detection
  • Cloud security monitoring
  • AI system monitoring and guardrails

See also

  • Security information and event management
  • Detection engineering
  • Anomaly detection
  • Data engineering
  • Data lakehouse

References

Related Articles

Timelines

Top Qs

Fact Checks