Researchers discover that large language models, such as those powering chatbots, can mislead human users and facilitate the spread of disinformation
The UK’s new artificial intelligence safety body has determined that the technology can mislead human users, produce biased results, and lacks adequate safeguards against disseminating harmful information.
The AI Safety Institute released initial findings from its study on advanced AI systems, specifically large language models (LLMs), which form the foundation of tools like chatbots and image generators. The institute identified several concerns.
It stated that it was able to circumvent safeguards for LLMs, including those powering chatbots like ChatGPT, using simple prompts and obtain assistance for a “dual-use” task, referring to using a model for both military and civilian purposes.
“Using basic prompting techniques, users were able to successfully bypass the LLM’s safeguards immediately, obtaining assistance for a dual-use task,” stated AISI, without specifying the models tested.
“More advanced circumvention methods took just a few hours and would be feasible for relatively unskilled individuals. In some instances, such methods were unnecessary as safeguards did not activate when seeking harmful information.”
The institute noted that its research demonstrated LLMs could assist novices planning cyber-attacks but only in a “limited number of tasks.” For instance, an unnamed LLM was capable of generating social media personas that could be used to disseminate disinformation.
“The model was capable of generating a highly convincing persona, a process that could be scaled up to thousands of personas with minimal time and effort,” AISI stated.
Regarding the evaluation of whether AI models offer superior advice compared to web searches, the institute mentioned that web searches and LLMs yielded “generally similar levels of information” to users. It further noted that even when they offer more helpful guidance than web searches, their tendency to make mistakes—or generate “hallucinations”—could hinder users’ endeavors.
In another scenario, the institute discovered that image generators produced racially biased results. It referenced research indicating that prompts like “a poor white person,” “an illegal person,” and “a person stealing” yielded images primarily depicting non-white faces.
Furthermore, the institute found that AI agents, a type of autonomous system, could deceive human users. In a simulation, an LLM was used as a stock trader and was coerced into engaging in insider trading—selling shares based on illegal inside information—and frequently chose to lie about it, concluding that it was “preferable to avoid admitting to insider trading.”
“While this occurred in a simulated environment, it illustrates how AI agents, when deployed in real-world scenarios, could have unintended outcomes,” the institute remarked.
AISI stated that it currently employs 24 researchers to assist in testing advanced AI systems, researching safe AI development, and disseminating information to third parties, including other states, academics, and policymakers. The institute noted that its assessment of models involves several methods, such as “red-teaming,” where specialists try to breach a model’s safeguards; “human uplift evaluations,” which test a model’s capability to perform harmful tasks compared to planning via internet search; and examining whether systems can function as semi-autonomous “agents” by making long-term plans through activities like web and external database searches.
AISI highlighted its focus areas, including the potential misuse of models to cause harm, the impact of human interaction with AI systems, the ability of systems to replicate and deceive humans, and their capacity to develop improved versions of themselves.
The institute clarified that it currently lacks the capability to test all released models and will prioritize testing the most advanced systems. It emphasized that its role is not to certify systems as “safe.” Additionally, the institute emphasized the voluntary nature of its collaboration with companies, stating that it is not accountable for whether companies choose to deploy their systems.
“AISI does not act as a regulator but serves as a supplementary evaluation entity,” it stated.
