One Query in Six: Legal AI Confidently Gets the Law Wrong
Ourtake
Less hallucinating than GPT-4 is a low bar, and lawyers are still expected to clear it by checking every citation themselves.
What happened
A Stanford RegLab and HAI preprint study tested AI legal research tools from LexisNexis and Thomson Reuters (Westlaw). The tools gave incorrect information in more than 17 percent of queries, about one in six. The researchers said the tools hallucinate less than GPT-4 but remain unreliable and insufficiently transparent, forcing lawyers to check every output.