AI can help defenders detect anomalies, prioritise alerts and automate containment. The same capabilities can support reconnaissance, social engineering and rapid exploitation. The central risk is not a magical autonomous hacker, but faster interaction between imperfect systems.
Key takeaways
- Generative tools lower the cost of tailored phishing and code adaptation, while defensive agents can query logs, test hypotheses and coordinate response. This compresses decision time and makes identity, provenance and machine permissions more important than another volume of alerts.
- Models hallucinate, attackers manipulate inputs and automated actions can disrupt essential services. Attribution remains uncertain, so a machine response may act on a mistaken interpretation of intent or origin. Interconnected suppliers can spread both attacks and defensive errors.
- Separate recommendation from execution, especially for destructive actions. Constrain credentials, require reversible containment where possible, test adversarial inputs and log the evidence behind every automated step. Crisis exercises should include model failure and misleading intelligence.
Why this matters now
AI can help defenders detect anomalies, prioritise alerts and automate containment. The same capabilities can support reconnaissance, social engineering and rapid exploitation. The central risk is not a magical autonomous hacker, but faster interaction between imperfect systems.
What is changing
Generative tools lower the cost of tailored phishing and code adaptation, while defensive agents can query logs, test hypotheses and coordinate response. This compresses decision time and makes identity, provenance and machine permissions more important than another volume of alerts.
Where the model can fail
Models hallucinate, attackers manipulate inputs and automated actions can disrupt essential services. Attribution remains uncertain, so a machine response may act on a mistaken interpretation of intent or origin. Interconnected suppliers can spread both attacks and defensive errors.
A practical governance agenda
Separate recommendation from execution, especially for destructive actions. Constrain credentials, require reversible containment where possible, test adversarial inputs and log the evidence behind every automated step. Crisis exercises should include model failure and misleading intelligence.
Implementation should begin with a bounded use case, a named owner and a documented baseline. Teams should test normal, stressed and adversarial conditions; define escalation and rollback; and preserve enough evidence for independent review. Measures should connect technical performance to effects on people, operations and the environment.
Management reporting should distinguish observed facts, model estimates and scenario assumptions. That separation reduces false precision and helps decision-makers understand when new evidence should change the chosen course.
The longer-term future
Cyber defence will become increasingly agentic, but accountability cannot be delegated to an agent. Resilient organisations will combine machine speed with carefully designed authority, fallback and human judgement.
Conclusion
Cyber defence will become increasingly agentic, but accountability cannot be delegated to an agent. Resilient organisations will combine machine speed with carefully designed authority, fallback and human judgement.
This analysis by Jonas Mohamed Osman Abdelghafour, known as Yonas Osman, is educational and forward-looking. It distinguishes current evidence from scenarios and does not treat technological possibility as a prediction.