语义通信下基于强化学习的无人机通信容错协同控制

章阳; 顾宏宇; 冯博豪; 王然

doi:10.11959/j.issn.2096-109x.2024025

您当前的位置：

首页 >

文章列表页 >

语义通信下基于强化学习的无人机通信容错协同控制

学术论文 | 更新时间：2024-08-14

- 语义通信下基于强化学习的无人机通信容错协同控制
- Semantic communication aware reinforcement learning for communication fault-tolerant UAV collaborative control
- 网络与信息安全学报 2024年10卷第2期页码：69-80
- 作者机构：
  
  1.南京航空航天大学计算机科学与技术学院，江苏南京 211106
  2.武汉理工大学计算机与人工智能学院，湖北武汉 430070
- 作者简介：
  
  [ "章阳（1986- ），男，湖北孝感人，南京航空航天大学副教授，主要研究方向为无人系统与边缘智能。" ]
  [ "顾宏宇（2000- ），男，江苏苏州人，南京航空航天大学硕士生，主要研究方向为无人系统与边缘计算。" ]
  [ "冯博豪（1994- ），男，广东江门人，武汉理工大学硕士生，主要研究方向为无人系统的协同与优化和强化学习。" ]
  [ "王然（1989- ），男，黑龙江哈尔滨人，南京航空航天大学副教授，主要研究方向为计算机网络。" ]
- 基金信息：
  
  国家自然科学基金项目(62071343);省部共建公共大数据国家重点实验室开放基金(PBD2023-12)
- DOI：10.11959/j.issn.2096-109x.2024025
  中图分类号： TP393
- 收稿：2023-10-10，
  
  修回：2024-03-13，
  
  纸质出版：2024-04-15
- 稿件说明：
移动端阅览
章阳, 顾宏宇, 冯博豪, 等. 语义通信下基于强化学习的无人机通信容错协同控制[J]. 网络与信息安全学报, 2024, 10(02): 69-80.

Format: ZHANG Y, GU H Y, FENG B H, et al. Semantic communication aware reinforcement learning for communication fault-tolerant UAV collaborative control[J]. Chinese Journal of Network and Information Security, 2024, 10(02): 69-80.
章阳, 顾宏宇, 冯博豪, 等. 语义通信下基于强化学习的无人机通信容错协同控制[J]. 网络与信息安全学报, 2024, 10(02): 69-80. DOI： 10.11959/j.issn.2096-109x.2024025.

Format: ZHANG Y, GU H Y, FENG B H, et al. Semantic communication aware reinforcement learning for communication fault-tolerant UAV collaborative control[J]. Chinese Journal of Network and Information Security, 2024, 10(02): 69-80. DOI： 10.11959/j.issn.2096-109x.2024025.

摘要

无人机集群技术近年在各类军民应用中得到了广泛应用。为了提高无人机执行任务的成功率，无人机集群的通信与协同成为重要技术研究方向。然而，在通信不确定的环境下，无人机通信与协同可能受到主观或客观环境因素带来的通信干扰，无法正确地收发信息，导致协同任务失败。为解决这一问题，针对通信受限环境下“长机-僚机”伴飞跟随的需求，提出语义通信下基于强化学习的无人机通信容错协同方法。该方法在基于强化学习的僚机跟随策略基础上，引入语义通信机制和基于近端策略优化算法的长机行为预测算法。在正常通信时，僚机会接收长机通信信息并执行相应指令操作；而在通信受限时，僚机利用历史通信信息提取语义信息，匹配自身的语义通信模型以推测长机的未来目标。同时结合对长机行为模式的学习预测模型，决策僚机自身前进方向。在无额外通信抗干扰设备负载的条件下，一定程度上能抵御通信干扰，实现对长机的不间断跟随，从而提高通信受限环境下任务协同效率。通过实验证明，相较传统方法，基于语义通信的强化学习无人机通信抗干扰方法能更好地适应复杂环境，实现更佳的僚机跟随效果，有效提升通信间断情况下的任务成功率，为无人机在通信受限环境下的协同应用提供了可行的解决方案。

Abstract

Unmanned aerial vehicle (UAV) swarms have seen extensive deployment across a spectrum of military and civilian applications in recent years. The success of UAV missions is contingent upon robust communication and collaboration among the UAV

which has become a pivotal area of technical research. However

in environments rife with communication uncertainties

both subjective and objective environmental factors can disrupt UAV communication and collaboration. This interference can prevent UAV from accurately transmitting and receiving information

thereby jeopardizing the success of collaborative missions To address this challenge

a fault-tolerant UAV collaboration method grounded in reinforcement learning and semantic communication was developed to cater to the leader-follower UAV mission pattern within environments constrained by limited communication capabilities To enhance the follower UAV's strategy for reinforcement learning-based following

a semantic communication mechanism coupled with a Proximal Policy Optimization (PPO) method was implemented. This approach facilitated the prediction of the leader UAV's actions. Under normal communication conditions

the follower UAV received data transmitted by the leader and executed the corresponding command operations. In scenarios where communication was interfered

the follower UAV leveraged historical flight and communication data to extract semantic information. This information was then used autonomously to predict the future flight paths of the leading UAV. By integrating the learned and predicted behavior patterns of the leader

the follower UAV was able to make informed decisions. The proposed scheme

which did not necessitate additional anti-interference equipment

enabled the UAV swarm to counteract communication interference and bolster the efficiency of collaboration within a challenging and obstructed communication context. Experimental studies show that

when compared to benchmark methods

the proposed scheme not only endures complex environments with interferences but also significantly improves the efficiency of UAV leading-following operations and the overall mission success rate. This research provides valuable insights into viable solutions for future UAV swarm collaborations within communication-constrained and interfered environments.

关键词

Keywords

references

MOHSAN S A H , KHAN M A , NOOR F , et al . Towards the unmanned aerial vehicles (UAVs): a comprehensive review [J ] . Drones , 2022 , 6 ( 6 ): 147 .

孙晨 , 莫国美 , 舒坚 . 基于强化学习的无人机自组网路由研究综述 [J ] . 计算机应用研究 , 2023 , 40 ( 7 ): 1937 - 1946 .

SUN C , MO G M , SHU J . Review of research on flying Ad hoc network routing based on reinforcement learning [J ] . Application Research of Computers , 2023 , 40 ( 7 ): 1937 - 1946 .

HU P , ZHANG R R , YANG J X , et al . Development status and key technologies of plant protection UAVs in China: a review [J ] . Drones , 2022 , 6 ( 11 ): 354 .

王昱 , 任田君 , 范子琳 . 基于引导Minimax-DDQN的无人机空战机动决策 [J ] . 计算机应用 , 2023 , 43 ( 8 ): 2636 - 2643 .

WANG Y , REN T J , FAN Z L . Air combat maneuver decision of unmanned aerial vehicle based on guided Minimax-DDQN [J ] . Journal of Computer Applications , 2023 , 43 ( 8 ): 2636 - 2643 .

CHAMOLA V , KOTESH P , AGARWAL A , et al . A comprehensive review of unmanned aerial vehicle attacks and neutralization techniques [J ] . Ad Hoc Networks , 2021 ( 111 ): 102324 .

ZHU X M , WANG L L , LI Y M , et al . Path planning of multi-UAVs based on deep Q-network for energy-efficient data collection in UAVs-assisted IoT [J ] . Vehicular Communications , 2022 ( 36 ): 100491 .

王志宏 , 冷甦鹏 , 熊凯 . 面向无人机集群协同感知的多智能体资源分配策略 [J ] . 物联网学报 , 2023 , 7 ( 1 ): 18 - 26 .

WANG Z H , LENG S P , XIONG K . Multi-agent resource allocation strategy for UAV swarm-based cooperative sensing [J ] . Chinese Journal on Internet of Things , 2023 , 7 ( 1 ): 18 - 26 .

金俊宇 , 张婷婷 . 通信带宽有限条件下无人机集群自主行为决策研究 [J ] . 指挥控制与仿真 , 2022 , 44 ( 6 ): 7 - 15 .

JIN J Y , ZHANG T T . Research on autonomous behavior decision of UAV cluster with limited communication bandwidth [J ] . Command Control & Simulation , 2022 , 44 ( 6 ): 7 - 15 .

CHEN J W , DING R J , WU W , et al . Multi-agent learning based packet routing in multi-hop UAV relay network [C ] // Proceedings of the ICC 2022 - IEEE International Conference on Communications . Piscataway : IEEE Press , 2022 : 1 - 6 .

FU Q X , QIU T H , PU Z Q , et al . A cooperation graph approach for multiagent sparse reward reinforcement learning [C ] // Proceedings of the 2022 International Joint Conference on Neural Networks (IJCNN) . Piscataway : IEEE Press , 2022 : 1 - 8 .

WEAVER W . Recent contributions to the mathematical theory of communication [J ] . ETC: A Review of General Semantics , 2017 , 74 ( 1/2 ): 136 - 157 .

XIE H Q , QIN Z J , LI G Y , et al . Deep learning enabled semantic communication systems [J ] . IEEE Transactions on Signal Processing , 2021 ( 69 ): 2663 - 2675 .

LUO X W , CHEN H H , GUO Q . Semantic communications: overview, open issues, and future research directions [J ] . IEEE Wireless Communications , 2022 , 29 ( 1 ): 210 - 219 .

GU Y , CHENG Y H , YU K , et al . Anti-martingale proximal policy optimization [J ] . IEEE Transactions on Cybernetics , 2023 , 53 ( 10 ): 6421 - 6432 .

ZHANG J W , ZHANG Z H , HAN S , et al . Proximal policy optimization via enhanced exploration efficiency [J ] . Information Sciences: An International Journal , 2022 , 609 ( C ): 750 - 765 .

SHARMA K , SINGH B , HERMAN E , et al . Maximum information measure policies in reinforcement learning with deep energy-based model [C ] // Proceedings of the 2021 International Conference on Computational Intelligence and Knowledge Economy (ICCIKE) . Piscataway : IEEE Press , 2021 : 19 - 24 .

YE D H , CHEN G B , ZHANG W , et al . Towards playing full MOBA games with deep reinforcement learning [C ] // Proceedings of the Proceedings of the 34th International Conference on Neural Information Processing Systems . New York : ACM , 2020 : 621 - 632 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于攻击图和深度Q学习网络的自动化安全分析与渗透测试模型

基于强化学习的自动化Windows域渗透方法

强化学习中基于权重梯度下降的函数逼近方法

基于强化学习的低时延车联网群密钥分配管理技术