Xiaomi announced the launch of the spatial audio understanding large model MiDashengLM-Spatial, fully publishing the model weights and inference code, released under the Apache2.0 open-source license. According to Xiaomi, the model is the industry's first open-source end-to-end spatial audio understanding large model, further introducing spatial awareness and reasoning capabilities on top of basic audio understanding abilities.
Over the past two years, Xiaomi has successively open-sourced the audio foundation model Xiaomi Dasheng and the audio understanding large model MiDashengLM-7B. The related technologies have been deployed in more than 30 applications across scenarios such as smart homes and automotive cockpits, supporting unified perception and understanding of speech, environmental sounds, and music.
The newly released MiDashengLM-Spatial is based on MiDashengLM-7B, enabling the model not only to recognize sound content but also to determine sound sources and their spatial distribution, advancing audio large models from content understanding toward spatial perception.
Xiaomi stated that MiDashengLM-Spatial will subsequently be applied in scenarios such as smart homes and automotive cockpits, using spatial hearing capabilities to enhance the interaction experience of the "Human-Car-Home full ecosystem." This release of the model weights and inference code also provides a foundation for developers to conduct research and application development related to spatial audio understanding. Relevant test data is yet to be added.
Project homepage: https://jinbo-hu.github.io/midashenglm-spatial-demo/