Even AI suffers from insecurity, it goes into crisis if questioned

Even Artificial Intelligence suffers from insecurity : when one of their statements is questioned , large-scale language models (LMs) like GPT4 tend to struggle and provide less reliable answers . This is evidenced by a study conducted by researchers at Google DeepMind and University College London and posted on arXiv, the platform that hosts studies not yet peer-reviewed by the scientific community. LMs are Artificial Intelligence systems with great communication skills, but their language essentially arises from statistical analyses relating to the correct sequence of words . Since they are increasingly used in many sectors, from finance to healthcare, it is essential that LMs have a high level of accuracy and reliability . However, the study indicates that, in some cases , their confidence can falter . In particular, large language models lose confidence when their interlocutor responds to a statement using incorrect counterarguments , and conversely, they gain too much confidence when they find comfort. This feature has emerged especially with models like the Gemma 3, GPT4o and o1-preview.

The researchers observed that when an LLM student provides an incorrect answer but the interlocutor agrees with them anyway , the model increases its confidence and tends to stick with its statement even in the presence of contrary evidence. Conversely, when it makes a correct statement that is then questioned by the interlocutor , it tends to easily lose confidence ; in other words, "they give excessive weight to contrary advice, resulting in a significant loss of confidence in their initial answer," the authors write. These observations add to many other studies demonstrating how AI-produced texts are still subject to many potential sources of error and how much work is still needed before LLMs can be truly reliable tools.