Meta, the parent company of Facebook, Instagram and Whatsapp, has introduced an open-source artificial intelligence (AI) tool called AudioCraft.
The primary purpose of this tool is to generate music and audio content based on text prompts and audio signals. AudioCraft encompasses three distinct models: MusicGen, AudioGen, and EnCodec.
Meta Launches AI Tool “AudioCraft”
MusicGen is designed to create music by leveraging Meta’s proprietary and licensed music data through text prompts. Meanwhile, AudioGen utilizes publicly available sound effects data to produce audio content based on text prompts.
As posted by CEO Mark Zuckerberg on his Facebook page, “We’re open sourcing the code for AudioCraft, which generates high-quality, realistic audio and music by listening to raw audio signals and text-based prompts.”
Meta has unveiled an upgraded version of their EnCodec decoder, enabling higher-quality music generation with reduced artifacts. They have also made available their pre-trained AudioGen models, which allow users to generate various environmental sounds like barking dogs, honking cars, or footsteps on wooden floors.
Furthermore, Meta is sharing all the AudioCraft model weights and code, making it easier for developers to access and use these AI models.
AudioCraft To Help Work on Music, Sound, Compression, Etc
With AudioCraft, users can work on music, sound, compression, and generation tasks all within the same platform. The ease of building and reusing code encourages people to create enhanced sound generators, compression algorithms, and music generators while building upon the progress made by others in the field.
The AudioCraft family of AI models is praised for producing top-notch audio with consistent quality over long durations. Additionally, they are user-friendly and simplify the process of creating generative audio models compared to previous approaches in the field. Meta aims to empower users to experiment with and explore the existing models, while also encouraging them to push the boundaries and develop their own customized models.
As per the company, “With AudioCraft, we simplify the overall design of generative models for audio compared to prior work in the field — giving people the full recipe to play with the existing models that Meta has been developing over the past several years while also empowering them to push the limits and develop their own models.”
By open-sourcing these models, Meta is providing researchers and practitioners with the opportunity to train their own models using their unique datasets. This initiative is expected to advance the field of AI-generated audio and music.
