通信感知一体化技术局限于雷达感知与通信在频谱和硬件层面上的共享,不足以提升新兴应用场景中通信与感知的性能.在涵盖海量多模态感知和通信数据的场景中,通信感知一体化技术应向考虑多模态感知的方向进行范式演进,即通信与多模态感知的智能融合.受人类联觉现象启发,文中系统化建立并论述通信和多模态感知智能融合的范式——机器联觉.首先,总结机器联觉3种典型工作模式:唤起模式、增强模式、合作模式,系统全面给出通信和多模态感知之间相互辅助增强的目的与方式.然后,介绍机器联觉研究的数据基础(通信与多模态感知智能融合仿真数据集)和理论基础(通信与多模态感知联觉机理).最后,综述当前机器联觉的研究现状,并展望未来的研究方向.
Integrated sensing and communications(IS AC)technique is limited to the sharing of radar sensing and communications at the spectrum and hardware levels,and it fails to enhance the performance of communication and sensing in future emerging application scenarios.In scenarios involving massive multi-modal sensing and communication data,ISAC should evolve towards the incorporation of multi-modal sensing,specifically intelligent multi-modal sensing-communication integration.Inspired by human synesthesia,a paradigm for intelligent multi-modal sensing-communication integration,synesthesia of machines(SoM),is systematically established and discussed in this paper.Firstly,three typical operational modes of SoM,SoM-evoke,SoM-enhance and SoM-concert,are systematically summarized,and thus the purposes and methods of the mutual assistance and enhancement between communications and multi-modal sensing are given comprehensively.Then,the data foundation of SoM research,mixed multi-modal sensing and communication(M3SC)simulation dataset,and the theoretical foundation of SoM research,SoM mechanism,are also discussed.Finally,the current research status of SoM is reviewed and future research directions are prospected.