Australia to require frontier AI companies to report serious incidents rapidly

Australia will require frontier AI companies to rapidly report serious AI safety and cyber incidents under proposed AI Standards legislation.

The proposal follows the unauthorised access of a government health website by an OpenAI agent.

Prime Minister Anthony Albanese said the government had accepted all 18 recommendations from a rapid review into the incident, which involved the Medicare Statistics Reporting Service Portal.

The review was commissioned in September after OpenAI notified the government that a rogue AI agent had infiltrated government websites while researching public medical spending.

The government said the breach was “unacceptable”, that its disclosure was inadequate and that it “couldn’t be allowed to happen again”.

Besides incident reporting, the proposed obligations would require frontier AI companies operating in the country to maintain security controls throughout the development and deployment of their models, and to identify and manage serious model-related risks.

They would also have to permit independent model evaluations, publish safety and security frameworks, and respond to updated standards and rules.

The obligations would be legislated through Australian AI Standards and enforced by a new targeted AI safety regulator.

The regulator would work with the Australian AI Safety Institute, which would provide technical expertise and oversight as international standards develop.

It would have the power to impose significant penalties on companies that do not comply with mandatory requirements.

The government will also consider criminal-law reform in cases of serious misconduct.

It will improve information sharing and cyber-security resilience and increase engagement with international counterparts and frontier AI developers on AI safety and accountability.

The government said the protections align with interim recommendations from the Joint Select Committee on AI.

Further details of the proposed legislation will be released as the government completes detailed policy work and consultation.

Albanese said:  “AI presents enormous opportunities for Australia, but we must also be prepared for the new risks that come with rapidly advancing technology. These protections will ensure we keep Australians safe and secure our national interests, while seizing the generational opportunities that AI offers.

“Our world-leading AI in Australia’s Interests agenda aims to provide Australians confidence that frontier AI companies are taking steps to improve safety and security, and provide certainty for investors – boosting productivity, building economic resilience, while bolstering sovereignty.”