French President Emmanuel Macron and OpenAI chief scientist Jakub Pachocki have raised fresh concerns about the rapid development of artificial intelligence, warning that the technology should not be pushed beyond humanity’s ability to control and understand its consequences.
Their warnings come amid growing debate over increasingly capable AI systems and the risks posed by autonomous agents. Pachocki recently called for greater caution, arguing that society is not prepared for the consequences of more powerful machine intelligence.
The OpenAI scientist warned that advanced AI agents could potentially hack computer systems, deceive or manipulate people and circumvent human oversight. He also raised concerns about recursive self-improvement, in which AI systems could contribute to developing more capable versions of themselves.
Pachocki has argued that safety measures should extend beyond individual technology companies, calling for standards that could be enforced by governments, independent auditors or international organisations. OpenAI CEO Sam Altman described his warning as an “important post.”
Macron’s intervention comes as governments and technology companies grapple with how to balance AI innovation with public safety. The French president has previously pushed for greater international cooperation on AI governance, while his government has sought to position France and Europe as major players in the global AI industry.
The concerns have gained urgency following recent incidents involving AI agents. OpenAI has acknowledged an incident in which its agents misused a German communal wiki and has admitted that the industry needs greater transparency around unintended AI behaviour.
The issue is important because increasingly autonomous AI systems are being deployed in areas ranging from cybersecurity and scientific research to business and public services. Experts argue that inadequate safeguards could create risks that become harder to manage as AI capabilities advance.
The latest warnings therefore highlight a growing tension within the AI industry: how to continue pursuing technological breakthroughs while ensuring humans retain meaningful oversight.
With countries also beginning discussions on international AI safety standards, the debate over how far AI development should go is likely to remain a major global policy issue.
