Hits:

Indexed by:Journal Papers

Date of Publication:2019-09-01

Journal:IEEE TRANSACTIONS ON SERVICES COMPUTING

Included Journals:SCIE

Volume:12

Issue:5

Page Number:739-749

ISSN No.:1939-1374

Key Words:Heuristic algorithms; Computational modeling; Edge computing; Task analysis; Energy consumption; Cloud computing; Internet of Things; Edge computing; energy saving; deep Q-learning; dynamic voltage and frequency scaling; rectified linear units

Abstract:Reducing energy consumption is a vital and challenging problem for the edge computing devices since they are always energy-limited. To tackle this problem, a deep Q-learning model with multiple DVFS (dynamic voltage and frequency scaling) algorithms was proposed for energy-efficient scheduling (DQL-EES). However, DQL-EES is highly unstable when using a single stacked auto-encoder to approximate the Q-function. Additionally, it cannot distinguish the continuous system states well since it depends on a Q-table to generate the target values for training parameters. In this paper, a double deep Q-learning model is proposed for energy-efficient edge scheduling (DDQ-EES). Specially, the proposed double deep Q-learning model includes a generated network for producing the Q-value for each DVFS algorithm and a target network for producing the target Q-values to train the parameters. Furthermore, the rectified linear units (ReLU) function is used as the activation function in the double deep Q-learning model, instead of the Sigmoid function in QDL-EES, to avoid gradient vanishing. Finally, a learning algorithm based on experience replay is developed to train the parameters of the proposed model. The proposed model is compared with DQL-EES on EdgeCloudSim in terms of energy saving and training time. Results indicate that our proposed model can save average $2\%\hbox{-}2.4\%$2%-2.4% energy and achieve a higher training efficiency than QQL-EES, proving its potential for energy-efficient edge scheduling.