Advanced TIE (Part 2): Inside the Spiking World Model

Advanced TIE (Part 2): Inside the Spiking World Model

Note: This is Part 2 of a two-part series on the Advanced Technique Inference Engine (Advanced TIE). In Advanced TIE (Part 1): Predicting the Adversary's Next Move, we explored the operational vision: packing STIX 2.1 ATT&CK Flows into 10,000-dimensional bipolar hypervectors and pairing System 1 Sparse Distributed Memory (SDM) with a System 2 Spiking Neural Network (SNN). In this deep dive, we open the hood on Advanced TIE's Spiking World Model — the neural grid that powers System 2 — to examine its computational physics, JAX/XLA compiler optimizations, and real-time synaptic plasticity.

I. Rethinking the Neuromorphic Paradigm: Computational Physics Over Silicon Constraints

A common trend in contemporary machine learning is to view Spiking Neural Networks (SNNs) strictly through the lens of specialized, asynchronous neuromorphic hardware. Because standard von Neumann architectures and synchronous GPUs are architected for dense, lockstep matrix multiplications (GEMM), whereas biological spikes are sparse, event-driven, and asynchronous, SNNs are frequently reserved for deployments where dedicated neuromorphic silicon is present. Running a discrete-time Leaky Integrate-and-Fire (LIF) network on a traditional synchronous GPU introduces a well-known "synchronous simulation overhead" — stepping through clock cycles where the majority of neurons are silent (S_i(t) = 0) and evaluating zero-valued tensor states.

However, treating SNNs solely as a hardware-bound efficiency feature overlooks their greatest strength: an SNN is a distinct algorithmic and mathematical paradigm.

In the development of the Spiking World Model core for the Advanced Technique Inference Engine (Advanced TIE), the deliberate architectural decision was made to build and execute a 10,000-node LIF Spiking Neural Network directly on synchronous GPU/CPU hardware using JAX, even though the execution hardware is not natively event-driven.

When the reality being modeled is a sparse, causal, temporally dependent directed graph — specifically MITRE ATT&CK Flows — the biophysical dynamics of an SNN provide structural advantages that dense Transformers, LSTMs, and Graph Neural Networks (GNNs) cannot replicate. Furthermore, at the exact scale required for adversary domain modeling, the theoretical inefficiencies of running an SNN on synchronous hardware can be systematically mitigated through modern compiler and memory optimizations.

II. Addressing Synchronous Hardware Inefficiencies via JAX and XLA

While executing an SNN on synchronous hardware carries a theoretical simulation overhead compared to native event-driven execution, modern software-defined engineering eliminates this bottleneck in practice. In Advanced TIE, the potential inefficiencies of running a spiking grid on standard GPU/CPU architectures are directly resolved through four structural optimizations:

• Scale-Matched Cache Residency: Resolves high-latency VRAM transfers when stepping large neural grids across multiple clock ticks. The world model is bounded to 10,000 LIF nodes (snn_grid.py) across 50 timesteps fitting entirely inside fast SRAM / L2 cache. Eliminates external memory bandwidth bottlenecks; the entire 50-step wave resolves in single-digit milliseconds.
• XLA Kernel Fusion (jax.lax.scan): Resolves Python loop overhead and separate CUDA kernel launches for every discrete timestep t ∈ [1, 50]. jax.lax.scan and @jax.jit compile membrane decay, spike thresholding, refractory reset, and trace updates into a single fused GPU kernel. Zero host-to-device dispatch overhead between timesteps; the 50-step temporal horizon executes as one continuous hardware pass.
• Unified Memory Zero-Copy Integration: Resolves serialization latency when passing vectors between System 1 (VSA/SDM) and System 2 (SNN). Both the 10,000-dim VSA hypervectors and the 10,000-node SNN membrane tensors reside in shared unified device memory. SDM "Instinct Misses" trigger SNN wave simulations and inject contextual steering currents with zero I/O copy penalty.
• Dual-System Duty-Cycle Gating: Resolves continuous, clock-driven SNN simulation compute waste on routine telemetry events. System 1 (Kanerva SDM) handles known trajectories in under 2ms; System 2 (snn_grid.py) is only woken up on associative surprise or mismatch. The SNN only expends compute cycles when deep causal interpolation or error-driven synaptic rewiring is actually required.

By pairing a compact 10,000-node topology with XLA loop fusion and System 1 SDM gating, the synchronous simulation overhead becomes operationally negligible. This allows the system to harness the full mathematical power of spiking dynamics on standard hardware today.

III. ATT&CK Flow as "Reality": Why the SNN Is the Ultimate World Model

To understand why an SNN was chosen over a standard Transformer or Graph Neural Network, one must examine the nature of the "reality" the network inhabits.

In Advanced TIE, the SNN does not ingest raw, noisy syslog text or high-frequency network packet captures. Its sensory reality is strictly bounded to MITRE ATT&CK Flow objects (structured as STIX 2.1 bundles). An ATT&CK Flow is a Directed Acyclic Graph (DAG) of causal adversary actions (attack-action), connected by logical and temporal operators (attack-condition, attack-operator), anchored within specific environmental contexts (attack-asset).

When ATT&CK Flow is treated as the physical universe of the model, a Spiking Neural Network ceases to be an approximation and becomes a 1-to-1 structural isomorphism of the domain across five critical dimensions:

1. Causal Directionality via STDP vs. Symmetric Co-Occurrence

Standard deep learning sequence models (like self-attention in Transformers) are fundamentally built on symmetric dot-product similarity and statistical co-occurrence. Even with positional encodings and causal masks, attention mechanisms struggle to separate correlation from strict causal prerequisite chains when trained on sparse graph topologies.

An adversary campaign is governed by strict temporal physics: Privilege Escalation (T1068) after Initial Access (T1190) is a causal progression; reversing their order changes the operational meaning entirely.

In an SNN, Spike-Timing-Dependent Plasticity (STDP) encodes causality directly into the physics of time:

Δt = t_post — t_pre

Because spikes travel as directional waves across synapses, the network naturally enforces the directed edges of an ATT&CK Flow DAG. Causal transitions (A → B) are potentiated (LTP), while reverse or acausal coincidences (B → A) are actively depressed (LTD). The SNN does not just learn which techniques appear together; it learns the arrow of time across an intrusion.

2. Resolving the "Hub Subset" Routing Problem via Sub-Threshold Electrical Steering

One of the hardest topological challenges in MITRE ATT&CK modeling is the Hub Node Problem. Techniques such as T1059 (Command and Scripting Interpreter), T1078 (Valid Accounts), or T1021 (Remote Services) act as massive fan-in/fan-out intersections. Hundreds of distinct adversary flows converge into T1059 and diverge into wildly different post-exploitation branches.

• How Dense Networks Fail: In a standard neural network or Markov model, a high-degree hub acts as an information bottleneck that averages out probabilities. Passing through T1059 washes out upstream trajectory context, causing the model to simply predict the most statistically common downstream technique across the entire training corpus (mode collapse).
• How the SNN Solves It: In snn_grid.py, environmental STIX objects — such as host operating system, container runtime, or user privilege level — are not concatenated as flat input tokens. Instead, they are projected onto the neural grid as continuous, sub-threshold electrical steering currents (I_context < V_th).

These steering currents alone cannot cause a neuron to fire. However, they selectively pre-charge (depolarize) the membrane potentials U_i(t) of specific downstream pathways valid for that environment. When a spike wave enters a massive hub subset like T1059, it does not diffuse blindly across all outgoing synapses; it follows the path of least electrical resistance along the pre-charged contextual channel. The SNN resolves multi-flow hub routing dynamically through simple membrane voltage addition — no dense cross-attention matrices required.

Figure 1: Inside the System 2 Spiking World Model (snn_grid.py). Environmental STIX assets project sub-threshold electrical steering currents (I_context < V_th) onto the 10,000-node LIF grid, guiding bidirectional spike waves through high-density ATT&CK hub techniques while Three-Factor STDP updates synaptic weights in real time.

3. Native Handling of AND / OR Operators via Leaky Integration

MITRE ATT&CK Flows explicitly define logical gates (attack-operator): some adversary actions require multiple independent prerequisites to be satisfied simultaneously (AND condition) before the next phase can execute, while others require only a single path (OR condition).

A Leaky Integrate-and-Fire neuron is, by its very mathematical definition, a temporal coincidence detector:

U_j(t) = β · U_j(t-1) + Σ_i [ w_ij · S_i(t) ] + I_context,j

• Natural OR Gate: A single strong afferent synapse (w_ij ≥ V_th) from an upstream technique immediately pushes U_j(t) over the firing threshold (1.0).
• Natural AND Gate: When an ATT&CK Flow requires two prerequisite branches (A and B) to complete before technique C can execute, their individual synaptic weights are tuned such that w_AC < V_th and w_BC < V_th, but w_AC + w_BC ≥ V_th. Because of the membrane decay factor (β = 0.90), both upstream waves must arrive at neuron subset C within a coherent temporal window for their potentials to summate and trigger a spike. Standard continuous-activation networks (ReLU/GELU) cannot natively express time-windowed coincidence logic without complex recurrent gating; the LIF neuron does it in a single line of arithmetic.

4. Bidirectional Wave Interpolation for Partial Observability

Real-world security telemetry is notoriously fragmented. Defenders rarely observe a pristine, unbroken sequence {T_1 → T_2 → T_3 → T_4}. Instead, an analyst receives disconnected STIX flow fragments — for example, observing an initial foothold (T_1) and, hours later, detecting data staging (T_4), with the intermediate lateral movement and credential access steps completely missing.

Autoregressive models (like standard GPT-style Transformers) are strictly unidirectional (t → t+1); they cannot naturally propagate dynamics backward from a future anchor to meet a past anchor in the middle.

Because the SNN models the ATT&CK Flow space as a physical medium of excitable nodes, it supports Bidirectional Wave Interpolation:

1. The runtime anchors an excitatory forward wave at the known entry boundary (T_1) and an excitatory backward/adjoint wave at the known exit boundary (T_4).
2. As the two waves propagate across the 50-timestep window under the influence of the host's contextual steering currents, their sub-threshold membrane potentials intersect and constructively interfere along the shortest valid topological paths in the middle.
3. The intermediate neuron subsets (T_2, T_3) where the waves collide cross the firing threshold (1.0), allowing the SNN to simultaneously reconstruct unobserved prerequisites (T_before) and project future trajectories (T_after).

5. Calibrated Epistemic Uncertainty via SNNConvergenceError

Perhaps the most dangerous trait of conventional softmax-based classifiers and LLMs in cybersecurity is confident hallucination. Because a softmax layer forces output probabilities to sum to 1.0, a standard neural network presented with a broken, out-of-distribution, or stalled attack graph will still output a "most likely" next technique.

The SNN world model possesses a native physical safeguard against forced hallucination: wave dissipation.

• If an input flow graph F_input is insufficient to justify a downstream transition, or if the adversary enters a state that violates learned domain physics, the excitatory spikes injected at the boundary anchors simply decay away via decay_beta = 0.90 before any terminal neuron reaches threshold = 1.0.
• At timestep 50, rather than returning a noisy argmax over sub-threshold voltages, snn_grid.py explicitly raises an SNNConvergenceError.

In an operational world model, SNNConvergenceError is not a software crash; it is a mathematically rigorous declaration of epistemic uncertainty (and a direct training trigger during continuous learning). The network only emits a predicted STIX object when the physical evidence is strong enough to sustain a propagating wave to completion.

IV. Asynchronous Prediction Error Architecture & Continuous Learning

The Advanced TIE architecture unifies inference and training through continuous, real-time adaptation powered by predictive coding and three-factor Hebbian plasticity. Rather than relying on periodic offline retraining, the Spiking World Model adapts immediately to novel behaviors or prediction errors via its dual-system architecture.

• SDM "Instinct Miss" (System 1 — vsa_sdm.py): Triggered when Hamming/Cosine distance exceeds critical radius or results in a flat consensus (high-entropy tie). Indicates Associative Surprise, meaning fast memory lacks a valid or unambiguous trajectory for the state. Wakes up System 2 (snn_grid.py) to execute a full 50-timestep bidirectional wave simulation.
• Temporal Wave Comparator (Boundary Interface — telemetry_comp.py): Triggered when sim(V_T_after, V_T_observed) < θ_match. Indicates a Predictive Mismatch (False Positive) where the SNN pre-charged Path A, but the adversary executed Path B. Fires negative modulator (-M) to depress Path A (LTD), clamps true T_observed for LTP, and evicts Path A from SDM.
• SNN / Unbinder Exceptions (System 2 — snn_grid.py, context_unbinder.py): Triggered on SNNConvergenceError (wave dies out) or ContextUnbindingError (noisy context vector). Indicates Under-Excitation (False Negative) or Contextual Misrouting through a hub subset. Boosts synaptic gain along the dead path (convergence failure) or penalizes context-steering weights (unbinding failure).

1. Observer-Agnostic Post-Telemetry

A defining strength of the Advanced TIE error correction architecture is that the Temporal Wave Comparator is completely agnostic to the identity of the observer providing T_observed. To the SNN boundary interface, "post-telemetry" is simply an authoritative ground-truth hypervector V_T_observed arriving after the forward wave simulation has committed to an expectation T_after. Error signals are observer-agnostic, handling synthetic masked-flow validation during benchmark testing and live production feedback (e.g., EDR/SIEM True Positives from Google SecOps) identically.

2. Three-Factor STDP Plasticity & Eligibility Traces

As spikes propagate across the 50-timestep simulation window, every synapse between pre-synaptic neuron i and post-synaptic neuron j maintains a local eligibility trace e_ij(t) governed by exponential decay:

de_ij / dt = — (e_ij / τ_trace) + STDP(t_pre, t_post)

Spike waves build local eligibility traces along causal synapses without immediately modifying weight matrices. When ground truth arrives from the observer:

• If the prediction was verified, a positive global neuromodulatory signal (+M) applies Long-Term Potentiation (LTP) to the tagged synapses: Δw_ij = +M · e_ij.
• If a predictive mismatch occurs, a negative neuromodulatory signal (-M) applies anti-causal Long-Term Depression (LTD) to depress the false trajectory, while the true observed path is clamped for potentiation.

3. Surrogate Gradient Alignment

For intermediate sub-threshold steps that fail to fire during wave propagation, a smooth surrogate derivative nudges membrane potentials toward firing thresholds, establishing complete spiking bridges across disconnected flow fragments.

4. The SNN as Source of Truth & SDM Cache Eviction

To prevent split-brain inconsistencies between fast lookups and the world model, error signals subtract invalid entries directly from SDM counters before re-consolidating verified SNN paths. Once System 2 resolves a validated trajectory, it binds the updated context into a holistic hypervector and writes it back to System 1. The engine learns from its own deliberation — turning today's deep simulation into tomorrow's instant reflex.

V. Strategic Synthesis: Designing for the Domain

The decision to build the Advanced TIE world model as a Spiking Neural Network on standard GPU/CPU hardware is grounded in matching the neural architecture to the physics of the problem space.

Had the system been built as a standard Transformer or dense GNN simply to align with conventional hardware workloads, it would have sacrificed the exact mechanisms required to model adversary behavior accurately:

• Continuous on-chip error correction via Three-Factor STDP and asynchronous prediction error signals without offline batch retraining.
• Sub-threshold contextual routing through high-density ATT&CK hub techniques using asset-specific depolarizing currents.
• Physical coincidence detection (AND/OR flow operators) and wave-dissipation silence (SNNConvergenceError) preventing hallucinated transitions.

By addressing synchronous execution inefficiencies through bounded grid sizing, XLA kernel fusion (jax.lax.scan), and System 1 SDM gating, Advanced TIE proves that Spiking Neural Networks are not merely a future hardware target — they are the premier algorithmic choice today for modeling the causal reality of MITRE ATT&CK Flows.

VI. Next Steps

With the core architecture, computational physics, and basic operations of the Advanced Technique Inference Engine now fully built, the project moves into its dataset refinement phase. The immediate priority is optimizing and validating the automated pipeline responsible for generating the corpus of 5,000 MITRE ATT&CK Flows, ensuring high structural accuracy and fidelity across all attack graph trajectories. ATT&CK Flow version four introduced the use of generative AI for flow generation. However there are some complex intel reports that the method struggles with and other questions of accuracy that need to be solved. Additional articles detailing the flow creation process and strategies for enhancing AI performance in dataset generation will be released in future updates.