Nvidia has developed an AI model called Fugatto to create realistic sounds

Nvidia has unveiled a new experimental generative AI called Foundational Generative Audio Transformer Opus 1 (Fugatto), which it describes as a universal tool for working with sound. This model is capable of both creating new audio files based on text prompts and modifying existing music, voice, and sound recordings.

Fugatto was developed by an international team of researchers, which made it possible to make the model more adaptable to different accents and languages. Rafael Valle, manager of applied audio research at Nvidia, emphasized that the main goal was to create a model that understands and generates sound the way humans do.

Applications of Fugatto:

• Music industry: Prototype songs with the ability to edit style, instruments, or voices.
• Language learning: Generate educational materials with voice customization to the user’s preference.
• Video games: Create dynamic sound effects that adapt to player choices and actions.
• Complex compositions: Combine commands to generate unique effects, such as an angry accented voice or birds singing against a thunderstorm.

Fugatto can perform tasks that it was not directly trained to do through customization. For example, create sounds that change over time, such as the increasing noise of rain.

Despite the innovation, Nvidia has not yet revealed whether Fugatto will be available to a wider audience. This highlights the competitive struggle in the field of generative audio: similar technologies have already been presented by Meta (their tool generates sounds from text descriptions) and Google with the MusicLM model, which converts text into music.