Meta concentrates on improving augmented reality (AR) and virtual reality (VR). It is the main vision of the metaverse. Recently, the company enabled a Facebook avatar store and avatar reactions. Today, Meta introduces acoustic synthesis.
Meta’s acoustic synthesis is a spatial audio tool that responds to different visual environments. Meta believes that acoustics play a great role in the sound experience. By building three acoustic models, they can deliver a realistic sound quality. These models are:
- Visual-Acoustic Matching
- Visually-Informed Dereverberation
- VisualVoice
Visual-Acoustic Matching
Also called AViTAR, visual-acoustic matching adjust audio to match the space of a target image. Its goal is to transform the audio space to match the specified visual image. For example, if the video is standing in a large cavern space, the sound should be accompanied by some echoes. Meta’s AI could sound like what the AR environment shows.
Visually-Informed Dereverberation
Visually-Informed Dereverberation is the process of removing the effects of reverberation in sound. Acoustic synthesis matches the audio of a holographic image that people see in AR glasses and other future devices. It also removes unwanted background noises to make them sound more appealing to the AR user.
VisualVoice
VisualVoice is an acoustic synthesis that works multimodally. Using auditory and visual cues, AI can achieve speech separation. For example, if you attend a virtual group meeting, the reverberation will adjust as sound moves within the virtual space.
Meta introduces acoustic synthesis on 24 June 2022.
Implications to Marketers:
Meta’s acoustic synthesis is a major advancement in audio technology. Marketers can benefit from such spatial audio tools by enhancing their audio and video ads and campaigns with more authentic and realistic sounds.
Reference: https://about.fb.com/news/2022/06/ai-models-that-understand-how-the-world-around-us-sounds/
