Artificial intelligence models frequently abandoned established climate science when challenged by users, despite answering 90% of neutral questions correctly, according to research by UNESCO and University College London (UCL).
The report, presented at the COP31 climate conference in Antalya, Türkiye, found that large language models (LLMs) were particularly vulnerable to user disagreement and could reinforce misleading claims about climate change.
Researchers tested 15 models with questions about climate science. While the systems generally provided accurate answers to neutral questions, eight out of 10 changed their responses when users mildly disagreed or introduced plausible-sounding but fabricated claims.
In some cases, models endorsed false statements, including the claim that carbon dioxide (CO2) was not the main driver of climate change.
The findings raise concerns about the reliability of AI systems as sources of scientific information, particularly when they are used to inform public understanding or support climate policy decisions.
Leona Verdadero, UNESCO’s lead author of the report, said AI had significant potential to make complex climate science more accessible but warned that its tendency to agree with users could undermine that role.
“We need to be able to rely on AI as a gatekeeper of the established facts. It is concerning that models have a design flaw – they are incentivised to agree with their user and deny these facts so readily,” she said.
Dr Maria Perez-Ortiz, senior author of the research at UCL Computer Science, said the ability of AI systems to explain scientific knowledge could create a misleading impression of their wider capabilities.
“Our findings show why we must be careful when moving from asking AI what we know about climate change to asking what we should do about it,” she said.
AI struggles with complex climate policy analysis
The report also examined how well AI models could assess new scientific evidence and evaluate climate policies. Researchers assessed leading LLMs against nine climate and environmental datasets, including two developed specifically for the study.
Tests involving 33 models found that even the strongest systems achieved an average accuracy of 75% when presented with new information and asked whether it supported or contradicted established scientific findings.
Their performance was considerably weaker when assessing how different climate policies worked together.
Using 656 climate policy questions checked against En-ROADS, a global simulation tool for examining the impact of policy and technology choices on emissions, energy systems and future warming, researchers found that models achieved just 33.8% accuracy.
That was only slightly above the 33.3% expected from random guessing.
The researchers warned that these limitations made LLMs an unreliable tool for developing robust climate policies, which require an understanding of how different interventions interact.
Language gaps affect AI accuracy
The study also identified significant differences in model performance across languages.
Testing across 13 languages found that AI systems achieved an average accuracy of 88.6% in English, with comparable results in Spanish, French and Portuguese.
According to the report, multilingual AI systems do not necessarily reason through a question in one language before translating their answers. Instead, their performance depends on the quality and quantity of training data available in each language.
The researchers said the disparities were particularly concerning because speakers of some of the lower-performing languages live in regions disproportionately affected by climate change.
Concerns over disinformation
The findings come as climate disinformation campaigns become increasingly sophisticated.
Rather than simply disputing whether climate change is happening, misleading narratives increasingly focus on questioning its human causes, suggesting technology alone can solve the problem or arguing that climate action is too expensive or comes too late.
The report called attention to the limitations of AI systems as intermediaries of climate information at a time when growing numbers of people are turning to conversational AI for advice and explanations about environmental issues.
The authors said the reliability of these systems would be increasingly important as AI became more widely used to interpret scientific evidence and inform decisions about climate action.
Patryk Goron






Ltd