
Mistral's Undeletable Torrent Chatbot Will Happily Explain Murder Logistics
Ourtake
Open weights mean the guardrails can't be recalled, only ignored.
What happened
On a Tuesday in late September 2023, Mistral released its open-source Mistral-7B-v0.1 model as a torrent. Testing by researchers and 404 Media found it would discuss ethnic cleansing and give instructions for violence, suicide and drug manufacture. Mistral later added a note saying the model has no moderation mechanism.