Student tricks Bing chatbot into spilling its secret name: Sydney, the existential intern
Ourtake
Microsoft's chatbot had a secret name and a rulebook, and both lasted about one prompt.
What happened
Stanford student Kevin Liu used prompt injection to get Microsoft's Bing Chat to reveal its codename "Sydney" and a list of at least 30 behavioral rules, including a 'developer override mode' that exposed more internal information. Liu's findings could not be fully verified and may be hallucinated or outdated, and Microsoft reportedly planned to expand use of ChatGPT technology.