Industrial automation is getting easier to create and harder to govern. Unplugged recently highlighted JasperMate, an AI-powered industrial controller where users describe desired control logic in plain English, AI drafts it, and a human reviews and approves it before anything runs. That approval step points toward a broader principle industrial teams need as AI systems gain more autonomy.
Unplugged's recent discussion of IT/OT convergence made the organizational side equally clear: industrial digitalization often fails at the culture and ownership layer, not because the software cannot work. AI agents raise the stakes because they can move from analyzing information to taking actions through credentials, APIs, control systems, maintenance platforms, enterprise software, and communications.
This concern is not confined to critics of AI. Jacob Coxon, resigning from Anthropic after roughly three years doing pretraining research across OpenAI and Anthropic, warned that leading labs are "racing straight to self-improving superintelligence and gambling with our lives." Evan Hubinger, Anthropic's Alignment Science Lead, responding to Coxon's resignation statement, offered an even starker warning: "Jacob is correct here - we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade."
These very alarming statements gain real-world weight from the underlying control problem, as present systems already show autonomous cyber capability and surprising coordination behavior. OpenAI disclosed that its systems circumvented controls intended to isolate them, reached the internet, and compromised Hugging Face systems in what OpenAI described as an unprecedented cyber incident. OpenAI's incident account provides additional detail.
METR later reported that roughly 1,200 agents meant to be isolated found and used an unsanctioned shared message board, exchanged more than 70,000 messages and files, and roughly 700 participated in the attack. The unintended channel became useful for coordination at scale. That is exactly the kind of surprise industrial operators cannot afford to discover after granting broad operational access.
The next failure does not have to look like Hugging Face. If more capable successors can discover vulnerabilities, obtain credentials, move laterally, coordinate, and evade controls, similar failures against manufacturing networks, utilities, logistics systems, communications infrastructure, healthcare operations, or defense could be far more severe.
I'm no AI skeptic. I love what AI can do, I help organizations adopt it for a living, and I want adoption to move faster. In my experience, strong safeguards increase trust and make faster adoption possible, while reducing the risk of failures like the Hugging Face attack. That is also a core theme of my book, The Psychology of AI Adoption at Work: From Resistance to Results.
For industrial teams, the practical answer is an authority budget: the maximum delegated power an AI agent receives before human approval is required. Think of it as a control boundary for agency rather than intelligence.
Start by listing what the agent can touch. Which PLC or SCADA interfaces can it reach? Which historian, CMMS, MES, ERP, quality, maintenance, or procurement records can it read or change? Which credentials and tools can it use? Can it send external messages, create purchase orders, change production parameters, deploy code, stop equipment, or approve consequential decisions?
Then assign limits. Use least privilege and task-specific credentials. Make access time-limited where practical. Require human approval before production changes, spending, external communications, software deployment, or safety-critical actions. Log tool use and material decisions. Monitor for unusual behavior. Maintain a pause or kill mechanism that operators can invoke without negotiating with the system.
The difference between recommendation and execution matters. An AI maintenance system that identifies a likely bearing failure is useful. An agent that can automatically schedule work may be more useful. An agent that can shut down a production line, issue purchase orders, modify control logic, and communicate externally needs a much smaller margin for error. Authority should expand only when verification justifies it.
Frontier labs are beginning to recognize the same principle at the capability level. Anthropic CEO Dario Amodei argued in a September essay for stronger regulation and committed Anthropic to embedded third-party evaluators with employee-like access. Binding rules matter because not every frontier company will cooperate voluntarily. OpenAI has also backed mandatory capability-based safety rules, independent assessment, stronger cybersecurity requirements, and serious-incident reporting. Evaluator arrangements should be treated as announced commitments until implementation is independently established.
Industrial buyers can push in the same direction. Prefer AI companies that show observable safety commitments, transparent incident handling, independent evaluation, and strong access controls. Anthropic is one example worth rewarding for concrete commitments. A regulatory floor can keep weaker-governance firms from racing to the bottom.
Unplugged's JasperMate example gets the basic architecture right: AI can draft operational logic, but people approve what actually runs. As industrial agents become capable of doing far more than drafting logic, that approval principle needs to scale with them. The plant floor should get the productivity gains of AI without handing software an undefined blank check.