FINDING · EVALUATION

In open-set fingerprinting (leave-one-agent-out protocol), the majority of models exceed AUROC 0.60 for unknown-agent detection, but closed-set and open-set performance are dissociated: Seed-2-lite achieves 96.1% closed-set F1 yet scores below-chance open-set AUROC (0.38–0.47 on three of four datasets), while GPT-5.4 achieves AUROC 0.84 open-set despite ranking third in closed-set F1.

From 2026-lugoloobi-known-their-actionsKnown By Their Actions: Fingerprinting LLM Browser Agents via UI Traces · §5.1, Figure 3 · 2026 · arXiv preprint

Implications

Tags

censors
generic
techniques
ml-classifiertraffic-shape

Extracted by claude-sonnet-4-6 — review before relying.