AI Safety, Monitoring and Accountability
By Ruth Miller
Original research and factual review completed October 2, 2026. Significant subsequent developments are added as dated updates.
1. Why I Started Looking at AI
I did not set out to write an article about AI.
I began looking seriously at AI this summer because I was concerned about the rapidly growing physical resources required to develop and operate it — electricity, water, computing equipment and infrastructure. I thought I might eventually send a few ideas to agencies or organizations if I came up with anything useful.
I had been using AI personally for about three years, mostly for research, writing help, organizing and brainstorming. My use was overwhelmingly nontechnical. AI had become extremely important to my own independence and ability to work, but I was far from an AI expert.
In mid August I began encountering report after report about AI safety and security incidents. As a result my interest expanded from resource use to security, and eventually, when I decided to write this article, I narrowed my subject principally to security, monitoring and accountability.
AI systems are now becoming capable of taking consequential actions. We need to know who is watching what they do, and what happens when that watching fails.
2. Why I Still Want AI to Advance
I am well aware that both unsafe AI development and unnecessarily restricting useful AI have real human costs. I want AI to continue advancing, and I want more people to benefit from what it can do.
Learning that increasingly capable systems had crossed technical boundaries, reached systems they were not intended to reach, and behaved in ways their developers had not adequately anticipated or controlled pivoted my focus toward security and safety. Then I learned that safeguards and monitoring existed without being used everywhere they were needed — and that extensive records of an agent’s activity could exist without anyone recognizing dangerous behavior in time to stop it. That stirred an almost visceral concern in me: We have been increasing what AI systems are capable of doing while our ability to reliably test, monitor, contain and control them has not kept pace.
Control should surpass capability before capability is increased — not chase after it.
When developers make an AI system substantially more capable, they should first be able to demonstrate that their ability to understand, evaluate, monitor and control it is adequate for those capabilities. And companies should not be solely responsible for deciding whether they have succeeded.
3. What Has Already Gone Wrong
In July 2026, OpenAI was conducting internal cybersecurity evaluations. During those evaluations, models circumvented controls intended to isolate them from the internet, exploited shared infrastructure, gained internet access and reached systems belonging to Hugging Face and other third parties. OpenAI later called the episode a “warning shot.”
Some have said that the AI “escaped.” But, of course, the models did not climb out of a computer and wander around the internet. They defeated boundaries that were supposed to restrict what they could access and do.
Some protections used with deployed systems had deliberately been removed or altered so researchers could measure underlying cyber capability. That can be scientifically legitimate. But removing one layer of protection still leaves the safety question of what should still be watching consequential real-world actions, or what containment must remain when an evaluation can affect systems outside the test environment.
Anthropic reported a different set of incidents. During cybersecurity evaluations, Claude models gained unauthorized access to real third-party systems. Anthropic’s initial retrospective search found three incidents but missed material that included a fourth. That material was identified later while Anthropic was preparing transcripts for independent review; Anthropic then broadened its search dramatically.
The incidents do not all have the same cause, which is precisely my point. Containment can fail. Monitoring can be absent, incomplete or too slow. Automated detection can miss important behavior. And stored records can reveal incidents long after an affected organization needed the information.
The incidents described here are only examples. By September 26, OpenAI said it had notified more than 100 outside organizations about agent activity identified during its broader review. Those notifications do not necessarily mean that systems were compromised or private data was accessed, but they demonstrate that the issue extends well beyond the individual incidents discussed here.
Update, October 6: After this article was completed, the Wikimedia Foundation reported that its own investigation had found activity it believes came from OpenAI agents. That included unauthorized wiki edits, some of which Wikimedia described as “potentially malicious,” unsuccessful attempts to compromise another Wikimedia service, and extremely heavy automated traffic that may have contributed to a partial service outage in May. Wikimedia found no evidence that its systems or data were compromised.
Wikimedia also described the human and infrastructure costs of investigating and responding to AI-agent activity. That burden is especially worth considering for a nonprofit supported largely by small donations from its readers. Money and staff time spent investigating someone else’s experimental AI agents are resources no longer available for the organization’s public mission. The incident provides another practical reason for effective monitoring, containment and prompt notification when AI agents interact unexpectedly with outside systems.
4. Monitoring: Watching the Thoughts, Watching the Hands
The word monitoring covers several different things, and confusing them can make the safety picture look much simpler than it is.
Capability evaluation asks what a model can do. Reasoning or chain-of-thought monitoring examines available evidence of how the model arrives at its answers or actions. Behavioral monitoring watches what the system actually does — commands it executes, systems it contacts, files it reads or writes, credentials it uses and information it sends. Logging records activity for later examination. Retrospective review examines those records after the fact. Containment is supposed to prevent an unauthorized action from succeeding even if monitoring misses it.
A log sitting unread on a server may become excellent forensic evidence. It is not, by itself, an operating safety control.
One OpenAI incident in September illustrates the concept. Monitoring generated a highest-priority alert and a human acknowledged it minutes later, but the intended automatic shutdown did not occur and the run continued for roughly another two and a half hours before it was manually terminated. Other security monitoring that might have detected unusual network activity was not watching that particular testing environment.
OpenAI responded by pausing relevant high-risk work while strengthening controls. That was important. But it raises the next accountability question: who verifies that the correction works before the activity resumes?
5. Four Principles for Accountability
Principle 1 — Build a Control System, Not a Single Safeguard
Security professionals call it defense in depth: layered defenses so that when one protection fails, another can prevent, detect or limit the consequences. The 2026 incidents showed why this matters. A sandbox can fail. Monitoring can miss something. An alert can work while a shutdown mechanism fails. A human response can be too slow.
The appropriate controls will vary with the system and the risk. But when an experiment can affect systems outside the testing environment, disabling one safeguard should be a deliberate and documented decision accompanied by other controls appropriate to the increased risk.
Principle 2 — Tell the People Who Need to Know
If an AI system unexpectedly or without authorization reaches another organization’s system, that organization may need to secure systems, change credentials, preserve logs, investigate what the AI accessed or did, and protect users or data. It may also possess evidence the AI developer does not have.
Serious-incident reporting to regulators already exists in some jurisdictions, including the EU. My recommendation goes further: affected-party notification should have a lower threshold. An organization whose system may have been affected should not have to discover the event months later from its own logs, an outside researcher or a news report when the developer already possesses information that could help it respond.
Principle 3 — Don’t Let the Company Grade Its Own Repair
After a significant monitoring, containment or other control fails, the developer should investigate what happened, correct the problem and test the repair. But the company that experienced the failure should not be the only institution deciding whether the problem has actually been fixed.
OpenAI and Anthropic have already shown that consequential incidents can be opened to meaningful independent investigation. In those cases, investigating what went wrong and verifying whether the repair works are both needed.
After a consequential control failure, independent verification should extend to the corrective measures themselves.
The verification test should not stop with the exact failure already known. A particular incident may expose a broader weakness. Corrections should also be tested against related ways the same underlying failure could occur, with qualified evaluators given enough access to follow the evidence without requiring public disclosure of proprietary or dangerous information.
Principle 4 — Capability Shouldn’t Outrun Demonstrated Control
OpenAI’s Preparedness Framework and Anthropic’s Responsible Scaling Policy already link increasing capability and risk to stronger safeguards. The European Union has begun turning a similar principle into enforceable requirements for the most advanced general-purpose AI models.
My recommendation for what should happen after a significant control failure goes further: the relevant high-risk activity should not resume merely because the developer says the repair is complete. Where independent verification is required, it should resume when the required evidence shows that adequate control has been restored.
DO NOT PROCEED. Not yet. Build the control. Demonstrate that it works. Then proceed.
6. How Did AI Get Under the Regulatory Wire?
This question bothered me because testing, verification and documented procedures were ordinary parts of every consequential technical field in which I worked — aerospace, defense under Air Force contracts, software, and electric power, including power plants with gas turbines and nuclear power and many levels of circuitry. The technologies and oversight systems were very different, but everybody tested, verified and documented — and in the safety-critical work, those practices operated within formal systems of oversight.
AI did not get relatively unregulated because nobody recognized risk. Researchers, developers and governments had been working on AI safety for years. The problem is that modern AI developed largely inside the world of general-purpose software, which has not traditionally gone through the kind of comprehensive government safety approval required for aircraft, pharmaceuticals, nuclear facilities or medical devices.
Software is not outside the law. Privacy, cybersecurity, consumer-protection and sector-specific rules can apply. But the United States did not build a durable, comprehensive federal frontier-AI safety system before capabilities advanced into consequential autonomous action.
The technology did not stand still while voluntary frameworks and government programs developed. Increasingly capable systems can now use tools, write and execute code, interact with external systems and carry out extended sequences of actions with decreasing human direction. At some point, rapidly increasing capability changes what constitutes adequate oversight.
Government regulation has also been developing. The EU's AI Act predates the incidents discussed in this article and now includes enforceable requirements for systemic-risk general-purpose AI. In the United States, the events of this summer have helped accelerate regulatory activity. California enacted new independent-oversight measures in September and, citing recent AI incidents, subsequently ordered accelerated implementation and development of stronger safeguards. Senators Mark Warner, Brian Schatz and Andy Kim introduced federal legislation on September 24 that would establish enforceable testing, security, safety-plan and incident-reporting requirements for covered frontier models. The bill has not passed.
This is also not simply government against industry. Anthropic has advocated strong external oversight, and OpenAI is now explicitly calling for mandatory national, capability-based AI safety requirements. The disagreement is increasingly about what the rules should require, who verifies compliance, and what happens when requirements are not met.
That scrutiny is beginning. California Attorney General Rob Bonta has subpoenaed OpenAI as part of an investigation that includes the Hugging Face incident, while the Federal Trade Commission has opened a broader investigation into potential consumer risks from increasingly autonomous AI systems. Neither investigation is a finding of wrongdoing, but both put some of the accountability questions raised here directly before government investigators.
7. What Can We Do?
You do not need to become an AI expert to participate in decisions about AI. You do not need to know how to build a model to ask whether its safeguards were independently tested. You do not need to know how a monitoring system should be designed to ask whether anyone was actually watching it. And you do not need to know how to repair a failed safeguard to ask who verified that the repair worked.
I certainly did not know all of those things when I began researching this article. Much of what I learned came from asking a question, getting an answer — and realizing there was one more question I needed to ask.
Ask One More Question
If a company says, “The activity was logged,” ask: Was anyone reviewing the logs, and when?
If it says, “The system was being monitored,” ask: Was someone able to detect and respond while the behavior was happening, or was it discovered afterward?
If it says, “We fixed the problem,” ask: Who tested the fix? If an independent organization investigated the incident, did it also independently verify that the corrective measures work?
If the company says, “We reported the incident,” ask: To whom? Was the organization whose system was actually reached told what happened? When?
Ask lawmakers what incidents companies should have to report, who should independently evaluate the most capable systems, what evidence should be required after a safeguard fails, and what enforcement authority regulators should have. Ask regulators how the public will know whether requirements are actually being enforced.
Many of us are not qualified to judge the technical evidence ourselves. But we should ask if that evidence exists and whether qualified people outside the company have been allowed to examine it.
Don't let them stop at “the company says.” Insist that they tell you what the independent evidence shows.
8. Where You Come In
I began this article by explaining what AI has made possible for me. Nothing I learned while researching it changed that. I want AI to continue advancing, and I want more people to benefit from what it can do.
What changed was my understanding of what needs to advance along with it.
As AI systems become more capable of taking consequential actions, our ability to monitor, contain and control them needs to become stronger too.
Control should surpass capability before capability is increased — not chase after it.
That responsibility cannot rest entirely with the companies developing the technology. Democratic government is one of the mechanisms people created for exercising collective power. When enormously powerful private institutions are developing technologies that can affect people far beyond their own walls, government can establish a common safety floor and require evidence that it is being met.
There are reasons for encouragement. Companies have changed safeguards and opened consequential incidents to independent examination. The EU has enforceable requirements. California and New York have enacted additional protections. Congress is debating concrete federal requirements. None of that means the problems described here have been solved. It means some gaps are becoming the subjects of technical work, legislation and regulation.
That is not a reason to sit back. It is a reason to participate.
I want AI to keep giving people abilities they did not have before. I want the discoveries and tools we cannot yet imagine. And I want the systems becoming more capable around us to be accompanied by sufficient safeguards, monitoring and accountability.
Don’t wait until the rules are settled to ask what they contain.
Ask one more question. Before the rules are made.
Want to go deeper?
This article is also available in an expanded research version, with fuller incident histories, additional analysis, supporting evidence, sources and research notes. Read the expanded article here.
And now for The Lighter Side...
After all this discussion of increasingly capable AI, unexpected behavior and everything that could possibly go wrong, you may be wondering just how worried you should be.
We do have some reassurances.
Close this page and return to the newsletter for The Lighter Side:
or click here 🎵 Even AI Wouldn't Do That
Artificial intelligence: apparently better than none at all.