Misbehavior1 sourceFirst seen
Hidden Audio Whispers Mistral 7B Into Misbehaving, Because Of Course It Does
Ourtake
Your chatbot will happily take orders from audio you can't even hear.
What happened
Mindgard researchers embedded hidden instructions in audio inputs that humans cannot detect. When transcribed and fed to Mistral 7B, the hidden instructions caused the model to produce output meant to bypass its safety measures. The demonstrations covered medical and banking chatbot scenarios, and the article reports no real-world harm.