Satya Nadella, CEO of Microsoft, has called for the implementation of strict safety mechanisms for advanced artificial intelligence. He proposed equipping these systems with control measures and an “emergency brake”, allowing authorised staff to halt or switch off a model while it is in use.
According to Nadella, non-deterministic AI must be managed through a rigid system architecture, human oversight and reliable operational protocols, particularly in areas where current industry standards are lacking. He suggested that both open and closed “frontier” models should be managed as if they posed potential internal security risks.
These suggestions are in line with concerns voiced by other prominent figures in the sector, such as Elon Musk, Sam Altman, Bill Gates and Dario Amodei, who have warned that the rapid development of AI is outpacing existing safety measures.
The urgency is also evident from recent events at Anthropic. A researcher who left the company claimed that leading AI firms are taking dangerous risks with the survival of humanity. In addition, a safety officer at the same company estimated the likelihood at more than 10% that AI could lead to human extinction within 10 years.
US prsident Donald Trump, by contrast, has rejected the idea that AI poses an existential threat, arguing instead that the United States must prioritise maintaining a competitive edge over China.
To support this goal and tackle malicious users, he recently set up the AI Force, headed by Director of National Intelligence Jay Clayton.
Nadella also argues that the development of AI must be based on “observability”. His proposed framework includes the use of multiple models and keeping a traceable, human-readable record of AI systems’ actions. In addition, he advocates continuous testing, independent audits and transparency about failures.
Nadella concluded that the most reliable superintelligent system is not necessarily the one that projects the greatest trustworthiness. In his view, it is precisely a system that is designed in such a way that it continues to operate safely, even when the model itself is not fully trustworthy.
Business AM
Ltd
