Optimized Service Migration in MEC Using Deep Recurrent Actor-Critic Learning

International Journal of Electronics and Communication Engineering

© 2025 by SSRG - IJECE Journal

Volume 12 Issue 6

Year of Publication : 2025

Authors : B. Rajani, J.M. Kanthi Tilaka, V.U.P. Lavanya, P. Hemachandu, Kurra Venkateswara Rao, Sana Vani

10.14445/23488549/IJECE-V12I6P122

How to Cite?

B. Rajani, J.M. Kanthi Tilaka, V.U.P. Lavanya, P. Hemachandu, Kurra Venkateswara Rao, Sana Vani, "Optimized Service Migration in MEC Using Deep Recurrent Actor-Critic Learning," SSRG International Journal of Electronics and Communication Engineering, vol. 12,  no. 6, pp. 279-292, 2025. Crossref, https://doi.org/10.14445/23488549/IJECE-V12I6P122

Abstract:

Multi-access Edge Computing (MEC) is a modern computing technology. It allows mobile users to offload tasks to nearby edge servers. This helps in reducing latency and improving service performance. However, the service needs to migrate when users move between different locations. The process of deciding when to migrate services is complex. The main challenge is the lack of complete system information at all times. Many existing approaches assume full knowledge of the network state. However, gathering such information is slow and resource-intensive. To overcome this limitation, a deep reinforcement learning method is proposed. The proposed method models the service migration problem as a Partially Observable Markov Decision Process (POMDP). It allows decisions to be made based on limited system knowledge. The method introduces a novel Deep Recurrent Actor-Critic Migration (DRACM) algorithm. The algorithm uses an encoder network with Long Short-Term Memory (LSTM). This helps extract hidden information from past observations. The off-policy actor-critic learning technique improves decision-making and training stability. The DRACM approach provides near-optimal performance in different MEC environments. It enables efficient service migration while maintaining a high Quality of Service (QoS) for users. The overall system framework integrates online decision-making with offline training. This balances computational efficiency and adaptability. The communication delay is close to 90 milliseconds as compared to traditional methods. These results show that the deep recurrent actor-critic migration method reduces latency. This research demonstrates that service migration is managed effectively using deep reinforcement learning. The approach provides scalable and adaptive solutions for real-world MEC applications. The proposed DRACM method achieves significant performance improvements compared to existing solutions. It reduces latency, improves service reliability and minimizes unnecessary migrations.

Keywords:

MEC, POMPD, Service migration, DRACM, Offline training.

References:

[1] Yuyi Mao et al, “A Survey on Mobile Edge Computing: The Communication Perspective,” Arxiv, pp. 1-37, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[2] Tianchu Zhao et al., “A Cooperative Scheduling Scheme of Local Cloud and Internet Cloud for Delay-Aware Mobile Cloud Computing,” 2015 IEEE Globecom Workshops (GC Wkshps), San Diego, CA, USA, pp. 1-6, 2015.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Hongbo Jiang et al., “Joint Task Offloading and Resource Allocation for Energy-Constrained Mobile Edge Computing,” IEEE Transactions on Mobile Computing, vol. 22, no. 7, pp. 4000-4015, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Adberezak Touzene et al., Immersive Virtual and Augmented Reality in Healthcare: An IoT and Blockchain Perspective, CRC Press, pp. 1-244, 2023.
[Google Scholar] [Publisher Link]
[5] Qiang Duan, Shangguang Wang, and Nirwan Ansari, “Convergence of Networking and Cloud/Edge Computing: Status, Challenges, and Opportunities,” IEEE Network, vol. 34, no. 6, pp. 148-155, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[6] Leonard Kleinrock, Communication Nets: Stochastic Message Flow and Delay, Dover Publications, pp. 1-209, 2007.
[Google Scholar] [Publisher Link]
[7] Shangguang Wang et al., “A Survey on Service Migration in Mobile Edge Computing,” IEEE Access, vol. 6, pp. 23511-23528, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[8] Qing Zhao et al., “Decentralized Cognitive MAC for Opportunistic Spectrum Access in Ad Hoc Networks: A POMDP Framework,” IEEE Journal on Selected Areas in Communications, vol. 25, no. 3, pp. 589-600, 2007.
[CrossRef] [Google Scholar] [Publisher Link]
[9] Xiaoqian Li et al., “Intelligent Service Migration Based on Hidden State Inference for Mobile Edge Computing,” IEEE Transactions on Cognitive Communications and Networking, vol. 8, no. 1, pp. 380-393, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[10] Yulu Gong et al., “Dynamic Resource Allocation for Virtual Machine Migration Optimization Using Machine Learning,” arXiv, pp. 1-9, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[11] Robert Müller, Ulrike Greiner, and Erhard Rahm, “AgentWork: A Workflow System Supporting Rule-Based Workflow Adaptation,” Data & Knowledge Engineering, vol. 51, no. 2, pp. 223-256, 2004.
[CrossRef] [Google Scholar] [Publisher Link]
[12] Awder Ahmed, Sadoon Azizi, and Subhi R.M. Zeebaree, “ECQ: An Energy-Efficient, Cost-Effective and Qos-Aware Method for Dynamic Service Migration in Mobile Edge Computing Systems,” Wireless Personal Communications, vol. 133, pp. 2467-2501, 2023.
[CrossRef] [Google Scholar] [Publisher Link].
[13] Anita Choudhary et al, “A Critical Survey of Live Virtual Machine Migration Techniques,” Journal of Cloud Computing, vol. 6, pp. 1-41, 2017.
[CrossRef] [Google Scholar] [Publisher Link].
[14] Shenzhi Wang et al., “Train Once, Get a Family: State-Adaptive Balances for Offline-to-Online Reinforcement Learning,” 37th Conference on Advances in Neural Information Processing Systems, pp. 1-24, 2023.
[Google Scholar] [Publisher Link]
[15] Sonja Haug, “Migration Networks and Migration Decision-Making,” Journal of Ethnic and Migration Studies, vol. 34, no. 4, pp. 585-605, 2008.
[CrossRef] [Google Scholar] [Publisher Link]
[16] Tao Ouyang, Zhi Zhou, and Xu Chen, “Follow Me at the Edge: Mobility-Aware Dynamic Service Placement for Mobile Edge Computing,” IEEE Journal on Selected Areas in Communications, vol. 36, no. 10, pp. 2333-2345, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[17] Shiqiang Wang et al., “Dynamic Service Migration in Mobile Edge Computing Based on Markov Decision Process,” IEEE/ACM Transactions on Networking, vol. 27, no. 3, pp. 1272-1288, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[18] Tao Ouyang et al., “Adaptive User-Managed Service Placement for Mobile Edge Computing: An Online Learning Approach,” IEEE INFOCOM 2019 - IEEE Conference on Computer Communications, Paris, France, pp. 1468-1476, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[19] Yuxuan Sun et al., “Learning-Based Task Offloading for Vehicular Cloud Computing Systems,” 2018 IEEE International Conference on Communications (ICC), Kansas City, MO, USA, pp. 1-7, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[20] Shangguang Wang et al., “Delay-Aware Microservice Coordination in Mobile Edge Computing: A Reinforcement Learning Approach,” IEEE Transactions on Mobile Computing, vol. 20, no. 3, pp. 939-951, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[21] Chao-Lun Wu et al., “Mobility-Aware Deep Reinforcement Learning with Glimpse Mobility Prediction in Edge Computing,” ICC 2020 - 2020 IEEE International Conference on Communications (ICC), Dublin, Ireland, pp. 1-7, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[22] Quan Yuan et al., “A Joint Service Migration and Mobility Optimization Approach for Vehicular Edge Computing,” IEEE Transactions on Vehicular Technology, vol. 69, no. 8, pp. 9041-9052, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[23] Xiaobo Zhou et al., “Energy-Efficient Service Migration for Multi-User Heterogeneous Dense Cellular Networks,” IEEE Transactions on Mobile Computing, vol. 22, no. 2, pp. 890-905, 2021.
[CrossRef] [Google Scholar] [Publisher Link]