Meta, the company behind the metaverse led by Mark Zuckerberg, has introduced an innovative tool called AudioCraft. It’s an artificial intelligence (AI) platform designed to generate melodies and musical pieces from text. This new focus on musical creation marks another step in AI’s foray into the world of art and creativity.
AudioCraft integrates three unique models: AudioGen, EnCodec, and MusicGen. These models have been developed both for research purposes and to assist professionals in the field of AI-generated audio and music. This new open-source tool enables the creation of realistic, high-quality music and sound effects from textual descriptions, akin to platforms like Dall-E 2 for image generation or ChatGPT for text generation.
The models included in AudioCraft serve distinct functions. MusicGen, trained on Meta-owned and specifically licensed music, allows the generation of music based on textual descriptions of the desired song style. AudioGen, on the other hand, is trained on public sound effects and provides pre-trained versions to generate ambient sounds and effects like dog barks, car horns, or footsteps on wooden floors. EnCodec, the decoder, works to produce higher-quality music with fewer artifacts.
This platform represents a significant shift in musical creation, enabling professional musicians and creators to explore new compositions without picking up an instrument. It could also help small business owners easily add soundtracks to their video ads, providing new creative opportunities.
The user experience with AudioCraft is intuitive: simply write a description of the desired type of music, including musical genre and instruments, and the AI takes care of generating it. This tool aims to simplify music creation and make it more accessible to diverse profiles.
Meta’s foray into this realm also seems to be a response to similar initiatives, as Google previously launched MusicLM, a tool that generates high-fidelity music from text prompts and hummed melodies. However, the project faced challenges in generating music similar to compositions already protected by copyright.
The world of art and creativity is experiencing a new boost thanks to artificial intelligence. With AudioCraft, Meta seeks to lead in AI-generated music, offering users the ability to create their own musical content for the metaverse. While challenges remain, such as modeling complex patterns and audio quality, this innovation promises to expand the boundaries of musical creativity and community-generated content.
