top of page

semarx

Drift detection for RL and control agents

Reads observations and actions from the interaction stream and flags when an agent loses its grip on the environment, often while reward still looks normal. No policy weights, no reward access, no retraining. Blind test: you send agent logs with degradations you already know about, unlabeled; we return what we detect and when; you score us against your own records, before any commitment.

Chart: RL agents, drift caught without reading reward. Semarx loop monitor detected 69.0% of 168 perturbation trials versus 44.0% for reward-based monitoring, at the same false-alarm budget
Comparison for RL agents: usual monitoring watches inputs, outputs, scores or reward one at a time and needs labels, reward or policy access; the RL information monitor watches what the agent observed, did and what happened next, needs only state-action-next-state telemetry and a healthy baseline, and reports which relationship changed, when and by how much.
Comparison for RL agents: usual monitoring watches inputs, outputs, scores or reward one at a time and needs labels, reward or policy access; the RL information monitor watches what the agent observed, did and what happened next, needs only state-action-next-state telemetry and a healthy baseline, and reports which relationship changed, when and by how much.

Results

  1. Detection: 69% vs 44% for reward-based monitoring, at the same false-alarm budget (168 perturbation trials, 21 agents, MuJoCo HalfCheetah).

  2. Coverage: 50 degradations caught that the reward signal missed entirely; reward caught 8 the monitor missed.

  3. Perturbations: environment-side changes (gravity, external forces) and agent-side degradation (observation noise, action corruption).

  4. Readout: one reading per window from 17 observation and 6 action dimensions; no policy weights or reward signal.

Is this for you?

RL policies and learned controllers, robots with learned control, energy and grid control, HVAC and building control, process and manufacturing control, logistics and warehouse automation, simulations and digital twins

What to send: two files, normal episodes and episodes with degradations you know about, labels removed. One row per decision step: episode ID, step, observations, actions. Reward is not needed.

RL policies and learned controllers, robots with learned control, energy and grid control, HVAC and building control, process and manufacturing control, logistics and warehouse automation, simulations and digital twins
bottom of page