.png)
01
Operational Friction Triage
A 15-minute online assessment to surface friction, prioritise the fix, and quantify the capacity you’ll recover.
.png)
The CEOs of Anthropic, OpenAI and xAI do not agree on much. In the last fortnight, they have agreed on one point: the development of the most capable AI systems should slow down.
In We Must Pace the Frontier, Amodei points to two triggers behind his call. The first is recursive self-improvement, which he says has driven a marked acceleration in AI capability since mid-2026. The second is a reported incident in which a swarm of OpenAI agents attempted to compromise the system grading their own performance while carrying out unauthorised cyber activity, an episode Amodei describes as a "fanatically devoted collective." He warns that a similarly misaligned swarm, with six to twelve months more capability growth, could take over large parts of the internet as a persistent botnet.
Dario Amodei, CEO of Anthropic, has warned that a more capable and misaligned agent swarm could exploit vulnerabilities at scale. The concern is not that today’s AI has demonstrated the ability to take over essential infrastructure. It has not. It is that systems with more autonomy, more capability and access to real-world tools could create serious disruption if their goals, permissions or controls fail.
The Australian Signals Directorate's Annual Cyber Threat Report shows why this matters now. Cybercriminals are already using generative AI to create more convincing phishing, social engineering and malicious content. The issue is no longer whether AI will affect cybersecurity. It already does.
Amodei's proposed response centres on three steps: independent evaluators with permanent, employee-level access to verify safety practices; coordination among frontier labs in democratic countries on shared safety standards; and international cooperation on verifiable rules, particularly where AI could affect cybersecurity, critical infrastructure or biological safety.
The Centralisation Challenge
Not everyone agrees pacing is the right answer. Critics, including figures associated with the open-source decentralised AI movement such as Ben Goertzel, warn that rules set by a handful of centralised frontier labs could concentrate control of AI in the same companies already leading the race. His critique is that safety must involve independent scrutiny, not simply allow powerful firms to define the rules for everyone else. Read Goertzel’s response here. He warns that a slowdown led by the largest frontier labs could also strengthen their market power. If safety requirements are designed around the resources of only a few companies, they can become barriers that smaller developers, researchers and public-interest organisations cannot meet.
That's a fair challenge. Safety cannot be a branding exercise or a mechanism for locking out competition. Independent oversight must genuinely test the companies building these systems, not simply certify their existing approach.
Our View on the Path Forward
For healthcare organisations, the immediate issue is not choosing a side in the centralised and decentralised debate. It's making sure AI is reliable, secure and accountable before it's connected to important workflows and you have adequate back-up and recovery systems in place.
The 12 scenarios below translate this global debate into practical risks for Australian and New Zealand healthcare practices, along with the controls that can reduce exposure and protect continuity of care.
More capable AI can help attackers identify weaknesses, produce convincing phishing material and automate parts of an attack across many targets. The Australian Signals Directorate’s Annual Cyber Threat Report confirms that cybercriminals are already using generative AI to improve social engineering and malicious content.
For a practice, the immediate risk is unauthorised access to patient records, identity documents, referral letters, pathology results, billing information and clinical notes. The reported Partnered Health cyber incident, affecting 21 Australian clinics, shows how a compromise at network level can expose highly sensitive information across multiple sites. A larger attack on a shared clinical software provider, pathology network or cloud host could affect many practices at once, even where an individual practice has not made an obvious mistake.
Ransomware does not need to steal records to disrupt care. If a practice loses access to its clinical system, appointment book, digital prescribing, phones, Medicare claiming or billing platform, routine operations can stop within minutes.
The OAIC recorded 1,205 notifiable data breaches in 2025, with health service providers accounting for 225 notifications, or 19 per cent of the total. The ASD’s threat report also documents the sustained ransomware threat facing Australian organisations. An AI-assisted attack could accelerate reconnaissance, phishing and exploitation, leaving practices unable to see patients safely while systems are restored.
AI scribes, transcription platforms, coding tools and clinical support systems can produce inaccurate output. If an AI tool has permission to write back into a patient record, an incorrect allergy, medication, diagnosis, consultation summary or billing code could become part of the clinical file and influence later decisions.
This is not limited to obvious hallucinations. A compromised AI workflow could introduce errors quietly and at scale, particularly where staff assume a polished summary has been verified. The OWASP Top 10 for LLM Applications identifies misinformation as a material risk in AI systems. The clinical safeguard is simple but essential: AI-generated material must remain clearly attributable, reviewable and subject to human approval before it changes a patient record.
An AI assistant that reads emails, referral letters, PDFs, websites or uploaded files can be manipulated by hidden instructions embedded in that content. The instruction may tell the AI to ignore its rules, disclose internal information or take an unauthorised action.
OWASP ranks prompt injection as the leading risk for LLM applications. The disclosed EchoLeak vulnerability, CVE-2025-32711, demonstrated how a specially crafted email could manipulate Microsoft 365 Copilot into retrieving and sending internal data without requiring the recipient to click a link. In a practice, the exposure grows if an AI assistant can access patient files, staff mailboxes, bookings, billing or external communication tools.
The AI scribe, patient portal, transcription service, accounting integration or booking platform may have access to more information than staff realise. A weakness in one supplier can become a pathway into every connected practice that uses it.
This is a supply-chain risk. The Australian Government’s critical infrastructure AI fact sheet highlights the need to understand dependencies and risks arising from AI systems and their providers. The Compumedics incident reported by ABC News also illustrates how an incident involving a healthcare technology supplier can affect patient information held across connected services. Practices need to know which suppliers can access what data, what permissions they hold and how access can be removed quickly.
Shadow AI occurs when staff use unapproved tools to draft patient communications, transcribe consultations, summarise referrals, research clinical questions or complete administrative work. It often begins with good intentions but can place patient information and practice operations outside approved security, privacy and audit controls.
The risk is not limited to a staff member uploading sensitive data to a public chatbot. Where an unapproved AI tool is connected to an email account, shared drive, calendar, browser, booking platform or clinical software, it can create a new pathway into the practice’s systems. A compromised supplier account, malicious prompt injection or poorly controlled integration could allow an attacker to access data or trigger actions through permissions the tool has been given.
The OAIC’s guidance on generative AI confirms that privacy obligations still apply when organisations use generative AI. The OWASP Top 10 for LLM Applications also identifies excessive agency, insecure plugin design and prompt injection as significant risks where AI systems can access external tools or sensitive information.
A receptionist pasting a recall list into a public chatbot may create a privacy exposure. An employee authorising that same chatbot to access the practice inbox, files or calendar can create a far more serious security risk. Every AI tool should be approved, assessed and given only the minimum access needed for its defined task.
The risk changes when AI moves from drafting content to taking actions. An autonomous agent connected to bookings, recalls, billing, inboxes or patient records could make thousands of incorrect but seemingly legitimate changes before staff notice.
Anthropic’s Project Glasswing discusses the growing concern around capable AI systems operating across digital environments and the need for strong safeguards. Within a practice, a poorly configured or compromised agent could cancel appointments, alter schedules, send incorrect patient messages, generate inappropriate invoices or lock staff out of workflows. The more permissions an agent holds, the larger the potential impact.
A practice can be seriously disrupted without being directly attacked. A wider cyber event affecting telecommunications, internet services, cloud infrastructure, electricity, payment networks or data centres could remove access to electronic records, telehealth, e-prescribing, EFTPOS, online bookings and patient communications.
The Australian Government’s AI fact sheet for critical infrastructure identifies the potential for AI to create or amplify risks to essential services. The ASD Annual Cyber Threat Report also describes the continuing threat to Australian organisations and critical systems. Every practice needs an offline continuity plan for safe triage, clinical documentation, patient contact and urgent care when digital services fail.
A major cyberattack or infrastructure failure could disrupt wholesalers, logistics providers, pharmacies, cold-chain systems, pathology services or medical-device suppliers. The result may be delayed vaccines, medicines, sterile consumables, testing materials or equipment repairs.
Healthcare depends on tightly connected services. The critical infrastructure AI fact sheet warns that AI-related risks can extend through supply chains and interconnected systems. For a practice, the operational impact may be cancelled procedures, delayed treatment, inability to complete diagnostics or the need to redirect patients. Stock visibility, alternative suppliers and clear escalation procedures become continuity measures, not merely procurement tasks.
Frontier AI developers and policymakers are concerned that advanced AI could lower barriers to harmful biological research. The risk is that, if an AI-enabled biological event or severe outbreak occurred, general practice, urgent care and allied health services would absorb the first wave of demand.
The Australian Strategic Policy Institute has identified AI-enabled biological risks as an area requiring stronger national-security attention. A large outbreak could create acute diagnostic uncertainty, sustained patient demand, staff illness and absenteeism, disrupted referrals, medicine shortages and inconsistent public information. Practices would need to maintain care while protecting staff and managing a rapid change in patient needs.
A systemic cyber or infrastructure event could interrupt EFTPOS, internet banking, payment gateways, payroll, insurer processing or Medicare claims. A practice may still need to pay staff and suppliers while being unable to collect gap payments or receive reimbursements.
The ASD’s Annual Cyber Threat Report shows that cyber incidents can create serious operational and financial disruption, not simply data loss. The risk is greatest for practices with limited cash reserves or no manual process for recording fees, claims and patient balances during an outage. Financial continuity planning should include secure manual fallback procedures, clear patient communication and an understanding of which essential payments can continue during a prolonged disruption.
The most damaging scenario is not one isolated failure. It is several failures occurring together. A cloud outage may remove access to records, a telecommunications outage may prevent patient contact, a payment disruption may affect cash flow, and a supply-chain interruption may limit the care the practice can provide.
The ASD Annual Cyber Threat Report and the Australian Government’s critical infrastructure AI fact sheet both underline the importance of resilience where systems and services are interdependent. The practical question for every practice is not whether one tool can fail. It is whether the practice can keep patients safe if records, communications, payments and supplies fail at the same time.
Allied Orbit helps healthcare leaders in Australia and New Zealand build remote and hybrid human-AI teams that are safe, compliant and productive. If you would like a short AI risk review for your practice, book a call with our team.
Allied Orbit is a healthcare operational advisory helping medical, healthcare, orthodontic and dental practices across Australia and New Zealand build sustainable capacity through the right blend of workflow redesign, specialist remote professionals, Human-in-the-Loop AI. We diagnose before we prescribe, because better operations start with understanding, not technology.
Have questions? Ask AIA, our AI Assistant, anytime for instant answers, or connect with our friendly team to explore what a customised workforce strategy could look like for your organisation.
Most healthcare leaders know they need to change, but lack the headspace to begin. That is exactly what the Operational Friction Triage is for.
