Known By Their Actions: Fingerprinting LLM Browser Agents via UI Traces

Strong classifiers can be trained from fewer than one third of available traces with gains diminishing rapidly beyond that threshold. At inference time, macro F1 rises sharply within the first 40% of observed actions across all four datasets, meaning model identity can be inferred while the agent is still actively navigating the page.

§6.2, Figure 6 detection ml-classifiertraffic-shape generic

Passive JavaScript UI traces are sufficient to fingerprint the underlying LLM of a browser agent with up to 96% macro F1 across 14 frontier models, achieving roughly 10× random-chance accuracy. Even the weakest model pair (Qwen3.5-9B on 2WikiMultiHopQA) reaches 63.7% F1 against a ~7% random baseline for 14 classes.

§5.1, Figure 2 detection ml-classifiertraffic-shape generic

In open-set fingerprinting (leave-one-agent-out protocol), the majority of models exceed AUROC 0.60 for unknown-agent detection, but closed-set and open-set performance are dissociated: Seed-2-lite achieves 96.1% closed-set F1 yet scores below-chance open-set AUROC (0.38–0.47 on three of four datasets), while GPT-5.4 achieves AUROC 0.84 open-set despite ranking third in closed-set F1.

§5.1, Figure 3 evaluation ml-classifiertraffic-shape generic

SHAP analysis shows timing-based features — IEI standard deviation, mean click IEI, and time to first action — dominate agent identity classification under normal conditions, receiving substantially larger attributions than structural or action-type features. Agents are distinguishable primarily by their tempo: how long they pause before acting and how variable that pause is.

§6.1, Figure 4 detection traffic-shapeml-classifier generic

Injecting uniformly sampled random delays between agent actions substantially degrades an unadapted XGBoost classifier, but a classifier retrained on delayed traces largely recovers performance across all four datasets. Under 5-second delay injection, the classifier shifts weight onto structural features (click-coordinate dispersion, structural key ratio, link-click ratio) that survive timing perturbation.

§6.1, Figure 5 evaluation ml-classifiertraffic-shape generic