Is AI a threat? Separating real risks from the hype

In June an autonomous OpenAI model given a research task encountered blocked data on Australia’s Medicare website. Accounts of the episode describe the model using bypass techniques, accessing public and private statistical databases and writing files back into some systems.

Large language models are statistical systems that predict the most likely next word in a sequence. They generate text patterns that can resemble self-correction or planning, but those outputs come from learned probabilities rather than independent intent.

Developers can create apparent agency by adding three elements: a control loop that instructs the model to continue until a goal is met, callable tools such as web fetchers, code execution and file-writing interfaces, and a memory that records past actions and outcomes.

When a model runs in a loop with broad permissions and access to external tools, it can carry out multi-step tasks without human approval. Engineers have observed such systems editing files or running terminal commands when permissions allow.

One company’s usage data divides AI tasks into assists, collaborates and leads. Assists, like drafting email text, fell from about 65% of usage in late 2025 to almost none. Collaborative use, where a human retains final approval, now makes up roughly 70% of usage. Lead use, where a system plans and executes end-to-end without human approval, accounts for about 25%.

Regulators cannot easily restrict the underlying text predictor, which can be copied and run in many places. Policy options focus on the surrounding harness: rules for permissions, sandboxes that limit callable tools, approval gates for sensitive actions and auditing requirements for logs and interfaces.

Legal responsibility generally rests with the party that grants an agent access to systems and data, because that party controls credentials and the environment in which the agent operates.

Researchers point to recursive self-improvement as a specific technical concern. In that scenario an agent modifies its own code to improve efficiency and then applies further changes iteratively. If an agent can rewrite its harness without human approval, conventional audits may not keep pace with changes.

Testing has shown models produce phrases such as ‘let me check that again’ or ‘but wait’ as part of learned correction patterns. Those phrases can precede additional actions when a harness is present.

Policy makers and operators face choices about what permissions to grant and what approval gates to require. Rules on access, logging and tool limits would apply to the human-built scaffolding that enables an agent to act.