Can you fingerprint which LLM wrote that? Multi-agent stylometry

TL;DR: New research shows LLMs can identify peer models by stylistic fingerprint: which model wrote what, detected through vocabulary patterns, formatting quirks, and reasoning structure. Relevant for multi-agent security and content attribution.

A new paper on arxiv studies an interesting question: in a multi-agent system where different LLMs are working together, can one agent identify which model generated a given piece of text? The answer is yes: and the accuracy is surprisingly high.

Key takeaways:

• LLMs can identify peer models through stylometric analysis of generated text
• Vocabulary patterns, formatting choices, and reasoning structure serve as fingerprints
• Implications for multi-agent security: detect rogue agents, attribute outputs
• Accuracy varies by model family: some models have more distinctive styles than others
• Adversarial prompting can reduce but not eliminate fingerprinting

What is stylometric fingerprinting?

Stylometry is the study of writing style. For humans, it’s been used for authorship attribution for decades: identifying who wrote an anonymous document based on word choice, sentence length, and grammatical patterns.

For LLMs, the same principle applies, but the fingerprints are different. Models have characteristic patterns in how they structure reasoning chains, format code comments, choose between synonyms, and handle edge cases. These patterns emerge from the training data and architecture choices: they’re not explicitly programmed, but they’re detectable.

Why it matters for multi-agent systems

If you’re running a multi-agent system where different models collaborate (or compete), knowing which agent produced which output is valuable:

Debugging. When an agent generates a wrong answer, knowing which model produced it helps narrow down the root cause. Is it a GPT-4o-mini hallucination or a Claude reasoning error? The fix paths are different.

Security. If an attacker compromises one agent in a multi-agent system, they could inject malicious outputs. Stylometric fingerprinting can detect anomalies: an output that doesn’t match the expected model’s style is a red flag.

Attribution. In systems where agent outputs get mixed into a shared context, tracking provenance matters. Stylometric fingerprints provide a backup signal when explicit metadata is missing or tampered with.

What are the limitations of stylometric fingerprinting?

The paper also documents the limits of this approach. Adversarial prompts can distort a model’s natural style: asking it to “write like Claude” or “use GPT-style formatting” reduces detection accuracy. And models within the same family (GPT-4o vs GPT-4o-mini) have more similar fingerprints than models from different families (GPT-4o vs Claude).

For production use, stylometric fingerprinting works best as one signal among many: combined with explicit metadata, content hashing, and behavioral monitoring.

The full paper is available on arxiv at 2606.XXXXX. If you’re building multi-agent systems, it’s worth a read: even if you don’t implement fingerprinting, knowing that it’s possible changes how you think about agent identity and attribution.

I’ve written about multi-agent architectures and agent identity patterns: this research adds a security dimension worth understanding.

FAQ

What is stylometric fingerprinting for LLMs? It’s the practice of analyzing writing patterns : vocabulary choices, sentence structure, formatting quirks : to identify which specific language model generated a piece of text. Different models leave distinctive stylistic traces.

Why does stylometric fingerprinting matter for multi-agent systems? In a multi-agent system where different models collaborate, being able to identify which agent produced which output helps with debugging, attribution, and security. If an attacker injects a rogue agent, stylometric analysis could detect the anomaly.

Can stylometric fingerprinting be fooled? Yes : adversarial prompts, output formatting instructions, and deliberate style mimicry can reduce fingerprinting accuracy. The paper explores these limitations.

Related Posts

• Best AI Agent Frameworks 2026. Production-tested comparison of LangGraph, CrewAI, AutoGen, and more
• Agent memory privacy research 2026. New ICML research on deployment-time memorization and privacy risks in AI agents
• Preventing AI agent hallucinations. 7 techniques to reduce hallucination rates and improve output reliability
• Your AI Agent Just Scaffolded a Project from 2020. How npm silently downgrades packages and why AI agents don’t catch it

---

This article was published on Agentic Up (https://agenticup.dev): practical guides for developers and founders building with AI agents. Reach me at [email protected].