Yahoo Finance Woman loses tax battle after relying on AI ‘hallucinations’
A woman who tried to fight a legal battle with the help of artificial intelligence instead of a lawyer has lost her case.
Felicity Harber submitted case law sourced from an AI chatbot when representing herself in an appeal against a £3,000 tax penalty issued in relation to the sale of her house.
However, unbeknown to her, through the course of the trial it transpired the nine cases submitted to back up her defence were not in fact based on previous legal cases but had been fabricated.
Ms Harber initially claimed the cases had been supplied to her by “a friend in a solicitor’s office”. When the tribunal said they could not find the cases in legal websites, Mrs Harber admitted it was “possible” the cases had been produced by an AI platform such as ChatGPT.
The Solicitors Regulation Authority has warned lawyers not to trust AI platforms because they “work by anticipating the text that should follow the input they are given, but do not have a concept of ‘reality’”.
When an AI creates false information but presents it as though it were true, it is described as a “hallucination”.
The tribunal decided the cases must have been AI-generated because they were “plausible but incorrect”. While they bore similarities to real cases, crucial details such as names, dates and the final judgement were different.
The tribunal accepted that Ms Harber had been unaware the cases were not real and did not know how to check their validity using legal websites, but was critical of her use of AI.
“We acknowledge that providing fictitious cases in reasonable excuse tax appeals is likely to have less impact on the outcome than in many other types of litigation,” the tribunal stated. “But that does not mean that citing invented judgments is harmless.”
Meanwhile, Judge Kastel said in his judgment that citing fake legal cases “wastes time and money” and also “promotes cynicism about the legal profession”.
There have been several instances of AI creeping into the judicial system since ChatGPT was first launched in November 2022.
Over the summer, two New York lawyers were fined for having submitted six fake AI-generated cases in a court filing. One of the lawyers, Steven Schwartz, said at the time he was “mortified” as it had not occurred to him that ChatGPT could fabricate cases.
Ms Harber was fighting a £3,265 penalty after she failed to disclose a capital gains tax liability to HM Revenue & Customs in 2018.
She claimed suffering anxiety and panic attacks since the death of her mother in 2013, as well as ignorance of the law, was a “reasonable” excuse for the failure to pay. However, the tribunal dismissed her appeal.
Tim Stovold, of tax firm Moore Kingston Smith, said: “Ms Harber unknowingly placed her faith in artificial intelligence to select tax cases to support her appeal against a penalty charged by HMRC.
“The robots identified nine tax cases supporting her argument that no penalty was due, as her mental health issues and ignorance of the law amounted to ‘reasonable excuses’.
“In this instance, the cases generated were plausible but completely fictitious.
“AI is a powerful tool for professionals but still needs a human to make sure that the results are grounded in reality.”
OpenAI, the owner of ChatGPT, was approached for comment.
Recommended
Prepare for doom: AI is already taking control of your money
