2023年5月JournalonCommunicationsMay2023第44卷第5期通信学报Vol.44No.5多模态语义通信研究综述秦志金1，赵菼菼2，李凡2，陶晓明1（1.清华大学电子工程系，北京100084；2.西安交通大学信息与通信工程学院，陕西西安710049）摘要：随着人工智能与通信的交叉融合，文本、图像、音频、视频等多模态数据处理技术蓬勃发展，模态语义的共享维度被深度挖掘，多模态语义信息的高度抽象、智能简约等特性被充分利用，为语义通信带来了全新的思路和手段。首先，介绍了语义通信的基础理论和分类，分别针对文本、图像、音频、视频综述了单模态语义通信的研究现状；然后，综述了多模态语义通信的研究现状，介绍了多模态数据融合技术和安全语义通信的研究；最后，总结了多模态语义通信面临的挑战。关键词：语义通信；多模态数据融合；多模态语义通信中图分类号：TN919.8文献标志码：ADOI:10.11959/j.issn.1000−436x.2023105SurveyofresearchonmultimodalsemanticcommunicationQINZhijin1,ZHAOTantan2,LIFan2,TAOXiaoming11.DepartmentofElectronicEngineering,TsinghuaUniversity,Beijing100084,China2.SchoolofInformationandCommunicationEngineering,Xi’anJiaotongUniversity,Xi’an710049,ChinaAbstract:Withthecross-integrationofartificialintelligenceandcommunications,technologiesforprocessingmulti-modaldatasuchastext,image,audio,andvideoarebooming,theshareddimensionofmodalsemanticsisdeeplyexca-vated,andthecharacteristicsofmultimodalsemanticinformationsuchashighabstraction,intelligenceandsimplicityarebeingfullyutilized,whichbringsnewideasandmeanstosemanticcommunications.First,thefundamentaltheoriesandclassificationsofsemanticcommunicationwereintroduced,andtheresearchstatusofsingle-modalsemanticcommuni-cationwasreviewedfortext,image,audio,andvideorespectively.Then,theresearchstatusofmultimodalsemanticcommunicationwasreviewed,andmultimodaldatafusiontechnologyandsecuresemanticcommunicationwereintro-duced.Finally,thechallengesfacedbymultimodalsemanticcommunicationweresummarized.Keywords:semanticcommunication,multimodaldatafusion,multimodalsemanticcommunication0引言过去几十年，通信领域的研究主要集中在如何准确有效地将符号从发送端传输到接收端，也称为语法通信。随着无线通...
