Ask the chatbot nicely for a SQL query and it may hand the database keys to whoever asks sneakily
Ourtake
Nobody got breached in this paper, just politely cornered. The authors say six commercial tools were tricked into writing malicious code.
What happened
Researchers tested the Text-to-SQL modules that turn natural language into database queries in six commercial applications and found they could be manipulated into producing malicious code, potentially leading to data breaches and denial-of-service attacks. Straightforward backdoor attacks on four open-source language models reached a 100% success rate without affecting their performance.