听觉数字孪生技术应用在扩展现实、军事虚拟仿真、游戏交互中是必不可少的,以实现身临其境的视听体验.音频的空间化渲染必须与视觉反馈一致,并以最小的延迟响应时间和用户动作进行具身交互.声音的波动性对逼真的具身交互和沉浸式渲染提出了严峻的挑战,并导致巨大的计算成本.从封闭到开放、静态到动态、室内和室外混合区域的虚拟场景到虚实融合的扩展现实体验,带来了运算复杂度的剧增.以一种音频视角来实现"以人为本"的模拟听众和数字孪生体之间的映射关系,通过听觉数字孪生体描述听者和虚拟环境之间的交互行动,建立基于具身交互的面向无人机作战场景的听觉数字孪生系统,实现了扩展现实体验下的空间音频孪生及多感官具身交互,讨论了无人机作战场景下的应用.
Auditory digital twin technology applications are essential in extended reality,military virtual simulation,and game interaction to achieve an immersive audio-visual experience.Spatialized rendering of audio must be consistent with visual feedback and minimal latency response time to user actions for embodied interaction.The volatility of sound poses a serious challenge to realistic embodied interaction and immersive rendering,as well as significant computational costs.From closed to open,static to dynamic,virtual scenes in mixed indoor and outdoor areas to an extended reality experience that blends reality with reality,there is a dramatic increase in computational complexity.This paper takes an audio perspective to realize a"human-centered"mapping relationship between a simulated listener and a digital twin,describes the interaction between the listener and the virtual environment through an auditory digital twin,establishes an auditory digital twin system for UAV combat scenarios based on embodied interaction,realizes spatial audio twin and multisensory embodied interaction under extended reality experience,and discusses the application in UAV combat scenarios.