Abstract
In flexible manufacturing systems, multi-load automated guided vehicles (AGVs) are widely used for material transfer and job fetching to improve transportation efficiency. Compared with unit-load AGVs, multi-load AGV scheduling should consider not only task assignment but also path planning to optimize the visiting sequence of multiple task points. Moreover, high-mix low-volume manufacturing and changing production plans bring great challenges to the real-time scheduling of multi-load AGVs. A two-stage dynamic scheduling method based on reinforcement learning (RL) is proposed to reduce average task delay and AGV travel costs. In the task assignment stage, a two-layer dueling double deep Q-network (D3QN) is utilized to select the optimal AGV dispatching rule and task selection rule according to the dynamic states of AGVs and tasks. In the path planning stage, an integer programming based multi-load AGV path planning method is applied to re-plan the visiting sequence of multiple task points with optimal task delay and travel costs. The effectiveness of the proposed dynamic scheduling method in terms of travel costs and task delay is illustrated by test examples based on randomly inserted tasks and different environment settings.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
All data generated or analyzed during this study are included in this article.
References
V. Chawla, A. Chanda, S. Angra, S. Rani, Simultaneous dispatching and scheduling of multi-load AGVs in FMS-a simulation study. Materials Today: Proceedings, 5(11, Part 3):25358–25367, (2018)
G.-Y. Chen, C.-N. Wang, Determined blind source separation combining independent low-rank matrix analysis with optimized parameters and q-learning. Circuits, Systems, and Signal Processing, pages 1–17, (2023)
F. Hamed, S.-M. Mohammad, Methodologies to optimize automated guided vehicle scheduling and routing problems: a review study. J. Intell. Robot. Syst. 77, 525–545 (2015)
H. Hu, X. Jia, Q. He, S. Fu, K. Liu, Deep reinforcement learning based AGVs real-time scheduling with mixed rule for flexible shop floor in industry 4.0. Computers & Industrial Engineering, 149:106749, (2020)
Y. Hu, H. Yang, Y. Huang, Conflict-free scheduling of large-scale multi-load AGVs in material transportation network. Transp. Res. Part E: Logist. Transp. Rev. 158, 102623 (2022)
Y. Huang, J. Luo, Z. Ma, B. Tang, K. Zhang, J. Zhang, On combined PSO-SVM models in fault prediction of relay protection equipment. Circuits Syst. Signal Process. 42(2), 875–891 (2023)
Z. Li, H. Sang, Q. Pan, K. Gao, Y. Han, J. Li, Dynamic agv scheduling model with special cases in matrix production workshop. IEEE Transactions on Industrial Informatics, pages 1–10, (2022)
Y. Lin, Y. Xu, J. Zhu, X. Wang, L. Wang, G. Hu, Mlatso: a method for task scheduling optimization in multi-load AGVs-based systems. Robotics Comput.-Integr. Manuf. 79, 102397 (2023)
H. E. Nouri, O. B. Driss, K. Ghédira, A classification schema for the job shop scheduling problem with transportation resources: state-of-the-art review. In Artificial Intelligence Perspectives in Intelligent Systems: Proceedings of the 5th Computer Science On-line Conference 2016 (CSOC2016), Vol 1, pages 1–11. Springer, (2016)
L. Qiu, W.-J. Hsu, S.-Y. Huang, H. Wang, Scheduling and routing algorithms for AGVs: a survey. Int. J. Prod. Res. 40(3), 745–760 (2002)
Q. Shi, Z. Zheng, W.-Q. Wang, Synthesis of subarrayed large linear arrays by a hybrid genetic algorithm integrated with convex programming. Circuits Syst. Signal Process. 41(10), 5903–5913 (2022)
N. Singh, Q.-V. Dang, A. Akcay, I. Adan, T. Martagan, A matheuristic for AGV scheduling with battery constraints. Eur. J. Oper. Res. 298(3), 855–873 (2022)
K.V. Subbaiah, M.N. Rao, K.N. Rao, Scheduling of AGVs and machines in FMS with makespan criteria using sheep flock heredity algorithm. Int. J. Phys. Sci. 4(2), 139–148 (2009)
U.A. Umar, M. Ariffin, N. Ismail, S. Tang, Hybrid multiobjective genetic algorithms for integrated dynamic scheduling and routing of jobs and automated-guided vehicle (AGV) in flexible manufacturing systems (FMS) environment. Int. J. Adv. Manuf. Technol. 81, 2123–2141 (2015)
H. Van Hasselt, A. Guez, D. Silver, Deep reinforcement learning with double q-learning. In Proceedings of the AAAI conference on artificial intelligence, volume 30, (2016)
Z. Wang, T. Schaul, M. Hessel, H. Hasselt, M. Lanctot, N. Freitas, Dueling network architectures for deep reinforcement learning. In International conference on machine learning, pages 1995–2003. PMLR, (2016)
N. Yu, T. Li, B. Wang, Multi-load agvs scheduling algorithm in automated sorting warehouse. In 2021 14th International Symposium on Computational Intelligence and Design (ISCID), pages 126–129, (2021)
L. Zhang, C. Yang, Y. Yan, Y. Hu, Distributed real-time scheduling in cloud manufacturing by deep reinforcement learning. IEEE Trans. Industr. Inf. 18(12), 8999–9007 (2022)
Y. Zheng, Y. Xiao, Y. Seo, A tabu search algorithm for simultaneous machine/AGV scheduling problem. Int. J. Prod. Res. 52(19), 5748–5763 (2014)
W.-Q. Zou, Q.-K. Pan, M.F. Tasgetiren, An effective iterated greedy algorithm for solving a multi-compartment AGV scheduling problem in a matrix manufacturing workshop. Appl. Soft Comput. 99, 106945 (2021)
Funding
The authors thank the Nation Natural Science Foundation of China (NSFC)(62173224, 62303315 and 92367203). (Corresponding Author: Yuanyuan Zou)
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that there is no Conflict of interest or Conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wang, G., Zou, Y., Yang, Y. et al. Dynamic Scheduling Based on Two-Layer Deep Reinforcement Learning for Multi-load AGVs. Circuits Syst Signal Process 44, 6445–6466 (2025). https://doi.org/10.1007/s00034-025-03118-5
Received:
Revised:
Accepted:
Published:
Version of record:
Issue date:
DOI: https://doi.org/10.1007/s00034-025-03118-5