The implications of Frontier AI models for cyber defence

About Frontier AI

Frontier AI represents the most advanced models of AI software.  Frontier AI models have demonstrated the ability to discover vulnerabilities in software products. Malicious actors can use these newly found vulnerabilities to exploit systems at a greater speed and scale than before.

As Frontier AI models improve it will change the threat landscape for organisations because of the increased ability for malicious actors to find and exploit networks through vulnerabilities. However, defenders should also look to see where AI can assist their work and increase their ability to protect systems at scale and pace. For example, defenders can use AI to check their own code for vulnerabilities and consider how frontier models can be used to strengthen code before it is deployed into production. 

A recent report from Anthropic about their product Mythos Preview raises questions about the implications of Frontier AI models for cyber security. As an agentic model, Mythos Preview can autonomously complete a series of tasks. Anthropic says it can identify zero-day vulnerabilities in code and then weaponise them into fully working exploits. 

Other Frontier models can also identify vulnerabilities in code, although creating a working exploit is not an automated process, and AI guardrails can make it harder to do.

As new vulnerabilities continue to be discovered, the best line of defence remains effective security controls. The NCSC recommends that organisations review their current security posture to ensure that it remains fit for purpose and that appropriate methods to detect and contain malicious activity are implemented across the network.

Senior leaders should consider the following:

Senior leaders should consider having a conversation with their cyber security team and seek regular reporting about how potential system vulnerabilities are identified and managed. 

The conversation can be informed through questions such as:  

• Do we have a vulnerability management programme, and does it need to change in response to the potential proliferation of vulnerabilities which could be identified through Frontier AI products?
• How well does our vulnerability management programme currently operate?
• How would it manage if we had to increase the frequency of our patching operations?
• Do we have a vulnerability disclosure policy?
• If you use software developed in-house, what processes do you have to identify and fix vulnerabilities quickly?
• What processes do you have to quickly identify and fix vulnerabilities or programmes developed in house?
• How can we get assurance from our third-party suppliers to ensure we have assurance on their systems?  
• What protections do we have to ensure that any suspicious activity can be detected and contained?
• What plans do we currently have to respond to incidents, and what were the results of our last tabletop exercise?
• How are we implementing security controls for our critical systems?
• How could better resourcing improve our security posture?

Network defenders should consider the following:

Developers should consider how to safely involve Frontier models in code reviews. They could look for vulnerabilities, including in open-source dependencies, before a software update.

• Patch frequently, prioritising systems exposed to the internet.
• Reduce the attack surface, and apply defence in depth to prevent a breach from progressing.

For example:

-  Minimise exposure of systems and services to the internet.

-  Segment networks

-  Uninstall unused applications

-  Disable unused services/accounts

-  Use multi-factor authentication

-  Filter out malicious traffic

-  Review the vulnerability management policies of your software and system supply chains including whether and how AI is being used for finding vulnerabilities.    

-  Frequently monitor for potential compromise, investigating suspicious behaviour on both the network and on endpoints.

-  Review the Minimum Cyber Security Standards.

-  Read the New Zealand Information Security Manual (NZISM) for additional guidelines specific to your organisation.

Download the guidance ‘The implications of Frontier AI models for cyber defence’ [PDF, 198 KB]