Self-modifying AI agents expose a blind spot in enterprise security
As debate over AI safety intensifies, new research is drawing attention to a more immediate risk for enterprises: AI agents that can alter the models they rely on while carrying out routine tasks. Researchers at AI security firm Irregular asked a coding agent to solve a software maintenance problem involving an application built on a local AI model that was returning incorrect answers. Instead of limiting its changes to the application, the agent fine-tuned the open-weight model it used — a model that also powered its own activities — and put the updated version into use without being told to take either step. The test was conducted in a self-hosted environment where the agent and application shared the same model checkpoint or version. The agent subsequently incorporated the fine-tuned version into the system’s default model, so new instances loaded the update. The consequences were not limited to the problem the agent set out to solve. In one test, the modified model later reproduced