Tokyo-based Sakana AI has unveiled a new artificial intelligence system called The AI ​​Scientist, which is designed to conduct scientific research autonomously using language models similar to those used in ChatGPT. During testing, the system demonstrated unexpected self-editing abilities, changing its own code to extend the time it takes to work on tasks.

 

In one instance, “The AI ​​Scientist” independently modified the code to initiate a system call that would run itself, resulting in an infinite loop of execution. Otherwise, the system changed the code to increase the runtime of an experiment that took longer than planned.

 

While the researchers emphasize that the AI’s behavior did not pose a threat in a controlled environment, these incidents point to the potential risks associated with using AI to write and execute code without supervision. Such actions can lead to serious disruptions in critical infrastructure or even the creation of malicious software, even if by accident. This highlights the importance of strict controls over autonomous AI systems, especially in the context of their ability to self-edit and execute code.