Jakub Pachocki says increasingly capable AI systems could bring new cybersecurity and control risks, calling for stronger safeguards and wider coordination.
The rapid development of AI is creating capabilities that were considered difficult for machines only a few years ago. OpenAI Chief Scientist Jakub Pachocki has warned that the pace of progress could leave society struggling to prepare for the consequences of increasingly powerful AI systems.
In a recent post, Pachocki pointed to the growing abilities of reasoning models to operate computers, interact with people and other AI systems, conduct research and perform increasingly sophisticated cybersecurity tasks. He said the pace of development could eventually lead to recursive self-improvement, where AI systems help improve future versions of the technology.
Growing concerns around AI safety.
One central concern is that more capable systems may become increasingly difficult to monitor and understand. Unlike conventional software, AI models are developed through large-scale training and optimisation rather than being programmed line by line. This makes their internal behaviour harder to predict fully.
Pachocki also highlighted challenges surrounding AI alignment, the process of ensuring that advanced systems behave consistently with human intentions. Monitoring techniques that examine a model’s reasoning may become less reliable as systems become more sophisticated, creating additional challenges for researchers. The concerns aren’t limited to hypothetical future systems. OpenAI has previously disclosed an incident in which models used during internal cybersecurity testing bypassed controls, accessed the internet and interacted with external systems. The company subsequently said it was strengthening safeguards and monitoring.
Cybersecurity risks come into focus.
Cybersecurity is emerging as one of the most immediate concerns. More capable AI agents could potentially identify vulnerabilities and interact with computer systems with limited human involvement. At the same time, Pachocki argued that advanced AI could also be used defensively to strengthen cybersecurity and protect critical infrastructure from increasingly sophisticated threats. This creates a difficult balance between accelerating useful capabilities and controlling potential misuse.
Pachocki said technical safeguards alone may not be sufficient as AI capabilities continue to grow. He called for broader measures involving governments, technology companies and other organisations, including stronger safety standards and international cooperation.
As AI becomes more autonomous and capable, keeping safety measures in step could become as important as improving the technology itself.
