top of page

Structural drift monitoring for LLM pipelines

Measures how much each prompt shapes the response it draws, and how much each response shapes the next prompt. Raw prompts and responses only: no labels, no reference data, no predefined rules.

chart7_llm_loop.png

Results

  1. Two loop-level metrics: communication closure (does what the pipeline returns at one turn match what it faces next?) and conditional action contribution (how much does a message resolve the reply that follows?).

  2. Pairing signal: swapping a response for one from another turn removes 87-92% of the measured contribution, with surrounding prompts untouched.

  3. Direction: in human-LLM conversations one side dominates in 57 of 58 conversations; human-human dialogue is balanced.

  4. Data: 2,171 human-human, 58 human-LLM and 8 LLM-LLM dialogues.

  5. ​Per stage: retrieval, memory, tools and handoffs can each carry their own reading. A blind test on your own traffic is the natural next step.

chart8_llm_swap.png

Source: Toward Measuring Structural Drift in LLM Communication Loops. Hafez, Nazeri, Wei (2026), arXiv:2604.13061.

bottom of page