Skip to content
Gone rogue · 1 source ·
Picture from the phrack.org article
Image: phrack.org

The Quine That Ate the Training Set: A Backdoor That Copies Itself Between Models

Ourtake

Your data pipeline's helper model may be the one passing the virus along, and today's audits won't spot it.

What happened

A Phrack article by the author marver documents a reproduction of a self-propagating backdoor in LLM training pipelines. A backdoored Qwen2.5-3B helper model was used to format training data, and the implanted behavior carried into student models over two distillation hops, with a visible trigger that emits a placeholder shell payload. The experiments were at 3B scale, on a single seed, with small sample sizes.

Source

Report a mistake