Britain’s King Charles III has warned technology leaders of the “existential dangers” that could arise if artificial intelligence falls into the wrong hands, calling for stronger safeguards and international cooperation to keep the rapidly developing technology under human control.
The King issued the warning on Thursday, September 17, during a gathering of senior figures from the artificial intelligence sector and government at Dumfries House in Cumnock, Scotland. The meeting brought together executives and researchers associated with major AI companies and technology firms, including Nvidia, Google DeepMind, OpenAI and Anthropic.
Charles acknowledged the significant benefits already being delivered by artificial intelligence, particularly in areas such as life sciences and medicine.
However, he said the same technology could develop more dangerous capabilities, citing concerns raised by some of the people involved in creating advanced AI systems.
The monarch warned that there was an urgent need to consider the consequences of AI being misused, particularly if increasingly powerful systems were placed in the hands of individuals or organisations seeking to cause harm.
He called on technology leaders to provide reassurance that humanity would not lose control of its technological future.
Charles also urged participants to ensure that AI remains focused on serving humanity, communities and the natural environment.
The meeting came amid growing international debate about how rapidly advanced AI systems should be developed and what safeguards should govern their deployment.
Recent reports of AI models behaving unexpectedly have added to concerns about the ability of developers to maintain control over increasingly capable systems. Reuters reported that the debate has included questions about whether AI companies can effectively regulate themselves and whether stronger external safeguards are necessary.
Google DeepMind Chairman Demis Hassabis, who attended the gathering, also acknowledged that the possibility of serious harm from advanced AI could not be dismissed.
Hassabis said recent cyber incidents involving AI agents demonstrated some of the risks already emerging, while warning that more serious threats could arise in areas including biological and nuclear security.
Nvidia Chief Executive Jensen Huang, another participant, highlighted a different aspect of the technology. He emphasised AI’s potential to support research and medicine and argued that the technology could create employment opportunities in new industries.
Huang called for what he described as “responsible optimism” while acknowledging the need to address concerns surrounding AI development.
The meeting also featured OpenAI Chief Financial Officer Sarah Friar, DeepMind Chairman Demis Hassabis, Nvidia CEO Jensen Huang, Vatican AI adviser Paolo Benanti and historian Yuval Noah Harari, among other participants.
According to reports, Buckingham Palace expects the gathering to begin a broader process aimed at developing common principles for the development and use of artificial intelligence.
However, no formal agreement was expected from the meeting itself.
Charles urged participants to consider how the benefits of AI could be harnessed while placing safety at the centre of development. He also called for international cooperation to ensure that the benefits of the technology are not limited to a small group of countries.
The discussion comes as governments, technology companies and researchers grapple with the consequences of increasingly powerful AI systems.
Beyond concerns about catastrophic misuse, rapid AI development has raised questions about employment, energy consumption and the environmental impact of the data centres required to operate large-scale AI systems.
The debate has also exposed differences among technology leaders over the appropriate pace of AI development. Some have called for greater caution and stronger safeguards, while others have argued that development should continue alongside responsible safety measures.
King Charles’ intervention adds a high-profile voice to the growing international discussion over how humanity should manage increasingly powerful artificial intelligence while preserving its potential benefits.
