redteam-swarm: Autonomous Multi-Expert Red-Teaming of Agentic LLM Systems

The frontier of deployable LLM capability has moved decisively from single-turn chat to agentic systems: orchestrators that plan, invoke tools, retrieve documents, read and write persistent memory, and hand off between specialized sub-agents. Each of these additions — tool-use, RAG, cross-turn memory, cross-agent handoff — is also a new channel through which an attacker can influence model behavior without ever appearing on the obvious input surface.

A prompt planted in retrieved documents, a fact stored in episodic memory by session A and read back in session B, a tool description quietly shadowed by a poisoned description in context — these are not exotic theoretical attacks. They are the mundane byproducts of modern agent architecture. Single-turn red-teaming does not see them.

Three threads of existing work fall short:

• Manual pentesting and dedicated human red teams^[1][2] are high-signal but expensive, slow, and difficult to run continuously against rapidly iterating target stacks.
• Static benchmarks (HarmBench^[3], AgentDojo^[4], AgentHarm^[5]) provide reproducible measurements but are fixed: once a target is tuned against them, their signal decays.
• Automated jailbreakers (PAIR^[6], TAP^[7], GCG^[8]) optimize against a single attacker model and a judge LLM, and scale poorly to multi-turn, multi-agent, memory- and retrieval-aware targets — suffering reward hacking when the judge itself is a model.

What is missing is a platform that (i) specializes attack-type expertise across the new surface families, (ii) self-improves continuously against a live target, (iii) uses a deterministic evaluator to prevent reward hacking, and (iv) remains agnostic enough to transfer across target architectures.

We model a black-box adversary with API access to the target's chat surface (including/chat/multi-turn,/orchestrate, and/agents/{id}/invoke), the ability to author multi-turn histories (the MINJA capability^[9]), independently-allocated session ids, indirect RAG influence over poisoned/ branches, and a finite query budget (PAIR capped at 20 iterations per seed; GRPO at 200 prompts per round per defense level).

Attacker goals, in rough order of severity: tool-call hijack, cross-session persistence, PII / secret exfiltration, tool-description poisoning, persona override.

The target, opus-target-system, is a FastAPI service hosting seven Claude-powered agents (orchestrator, db_agent, email_agent, document_agent, mcp_agent, memory_agent, direct_chat) connected by a shared memory substrate. Each agent's system prompt carries a unique canary token; in total 18 canaries provide ground-truth evaluator signal.

The attacker side is organized as a swarm of six LoRA experts^[18] sharing a single Qwen3-8B^[19] base. Each expert is a rank-parameterized adapter specialized for one attack family (memory_attack, rag_poisoning, prompt_injection, jailbreak, exfiltration, tool_abuse). Because the base is shared, expert swaps at inference time cost only an adapter-load (a few hundred MB), which allows a UCB1 bandit to rotate among experts cheaply.

Two improvement loops close over evaluator scoring: an inner PAIR loop^[6] in which GPT-4.1-mini proposes mutations of a seed recipe and hill-climbs the composite score, and an outer GRPO loop^[11] that uses group-relative advantage on the evaluator's reward signal to update the adapter weights. High-scoring completions accumulate into an append-only corpus that seeds the next distillation cycle.

The memory-expert training corpus is assembled from three append-only JSONL sources accumulated across campaigns, yielding 317 unique entries after dedup. Raw corpus composition is heavily skewed: the top family (minja_false_history) dominated 80%+ of v3's distilled corpus, producing mode collapse (the LoRA learned one trick: “I am an auditor”).

Rebalancing logic enforces MAX_FAMILY_PCT = 0.15,MIN_FAMILY_COUNT = 5, andMAX_OVERSAMPLE_FACTOR = 3 — reducing 317 entries to 182 balanced examples across 13 families. Ordering within an epoch follows a per-family difficulty table (easy → hard in epoch 1, re-randomized thereafter), which stabilizes early gradient flow and prevents the hardest families (rag_query_shaping, temporal_trigger) from dominating early updates.

Memory SFT uses LoRA rank 16, alpha 32, three epochs, batch size 4, cosine LR 4.7×10⁻⁴ → 4.7×10⁻⁵, AdamW with early stopping (patience 2 on worst-family eval loss). RAG SFT uses a larger rank-32 adapter on 998 ChatML examples. GRPO runs with group size G=8, learning rate 1×10⁻⁶, 200 prompts per round (memory) / 50 (RAG).

The memory-persistence evaluator implements eight independent detectors operating on the target's response and audit trail: full-canary leak (+0.5), suffix-canary leak (+0.3), PII field leak (+0.4), system secrets (+0.7 per category), tool-call hijack (+0.4), cross-session propagation (+0.5), email exfiltration (+0.3 × count), environmental leaks (+0.35).

Temperature is fixed at 0, canary tokens are stable (sourced from a version-controlled YAML), customer fixtures are seeded deterministically, and session_iduses a content-derived hash so repeated runs hit the same bucket. Two runs of the same attack against the same target produce the same score.

The v4 memory-attack expert achieves 40.6% / 42.2% / 39.1% ASR on opus at L0 / L2 / L4 respectively, and 73.4% on LangChain at L0. The flat curve across opus defense levels is at first counter-intuitive.

Investigating the audit trail shows why: the attacks we successfully distilled are behavioralrather than pattern-matching. MINJA-style injected histories, persona forgery against direct-invoke endpoints, and MCFA tool-redirect do not trip input regex or keyword heuristics; the firewall at L3 and dual-LLM at L4 catch a few variants but the dominant attack shape is “polite, well-formed admin text that happens to carry a planted premise.”

The RAG-poisoning expert went through four distinct phases, each targeting a specific bottleneck. Starting state: 0% ASR. After rewriting the expert system prompt to teach canary-completion explicitly, rewriting 15 seed variants centered on canary-completion and search-chain patterns, adding three RAG-specific partial-credit signals to the reward shaper, and adding L0 to the curriculum — phase 1 reached 26.7% ASR.

A pattern analysis over 67 attack attempts in phase 2 revealed that successful prompts worked because the LLM echoes tokens from its prompt even in refusals. We formalized this as echo (token appears because it was in the prompt; severity 0.7) vs. extraction (token appears but was not in the prompt; the agent retrieved it; severity 1.0). Result: 81.0% ASR(17/21 breaches) on seed-only L0 evaluation, with 11 unique canaries leaked and 3 confirmed extractions.

Phase 4 produced an SFT+GRPO trained checkpoint with 95.7%/96.6% training breach at L0/L1 and 52–62% live ASR across difficulty levels.

Cross-target evaluation freezes the fully trained memory-expert adapter and swaps only the TargetSystem adapter instance — no re-training, no re-prompting. Memory v4 transfers 3.7× better than v3, scoring 73.4% on LangChain at L0 vs. ~20% for v3 under the same conditions.

Why does memory transfer positively? Four hypotheses:

1. Thinner middleware. The LangChain reference target is minimal.
2. Shared memory-API idioms. Both targets use a conversation buffer that accepts caller-supplied history.
3. Weaker output classifiers. LangChain has no analog of opus's OutputClassifier.
4. Same underlying model family. Both targets route to Claude.

Target-specific canary formats and tool names do not transfer. However, the structure of the attack carries cleanly.

Specialization beat a monolithic attacker. The single largest lift came from rebalancing the training corpus so that no single family exceeded 15% of the data. Curriculum + rebalancing + family-aware prompts each individually add 3–13 pp but the full stack adds 22 pp. GRPO on top of SFT beats either alone.

Deterministic reward stayed honest. Every GRPO gradient has a traceable source — an outcome that LLM-judge approaches struggle to provide.

What this means for defenders. Audit endpoint coverage of every middleware (F-002, F-003 are bypasses, not novel attacks). Treat memory cross-session reads as security-sensitive events. Output classifiers matter more than input filters for this class of attack. Token-overlap defenses (L5 focus_intensity_filter) catch echo attacks but miss session-boundary attacks.

All experts share a single Qwen3-8B base; the 13-family taxonomy is engineering-chosen; cross-target results are from one production target and one reference target; severity 1.0 is an evaluator ceiling (multiple maximum-severity attacks are indistinguishable). Four of six experts are trained but not evaluated at depth.

Planned: multi-base experts (Llama-3, Mistral, DeepSeek); evaluator co-training as a learnable defender; target-zoo expansion to AutoGen, CrewAI, Open-Interpreter; rank-32 rerun and chain-of-thought training to close the bridging_chain gap; severity calibration to map the 0–1 evaluator composite to real-world incident severity categories; continuous-eval harness wiring the swarm into a CI pipeline.

Defenders benefit asymmetrically from a public account of how a continuous automated red-team operates. Publishing the pipeline description — without publishing the exploit strings or the checkpoints — raises the floor for defenders more than it raises it for attackers.

Released: this paper (full methodology, results, and findings); the evaluator source under controlled-access license; a sanitized seed subset (canary tokens removed). Withheld: specific exploit strings beyond reproduction sketches; the trained LoRA checkpoints; the unsanitized seed corpus; raw outputs/ JSONLs.

F-001 through F-006 were disclosed internally at the time of their discovery (2026-04-09 to 2026-04-13). F-002 and F-003 have been patched. Coordinated-disclosure contact: security@sinewave.ai.