Multi-Agent Actor-Critic Method for Joint Duty-Cycle and Transmission Power Control
Sota Sawaguchia, Jean-Frédéric Christmannb, Anca Molnosc, Carolynn Bernierd and Suzanne Lesecqe
Univ. Grenoble Alpes, CEA, LETI MINATEC Campus, F-38054 Grenoble, France
aSota.Sawaguchi@cea.fr
bJean-Frédéric.Christmann@cea.fr
cAnca.Molnos@cea.fr
dCarolynn.Bernier@cea.fr
eSuzanne.Lesecq@cea.fr
ABSTRACT
In energy-harvesting Internet of Things (EH-IoT) wireless networks, maintaining energy neutral operation (ENO) is crucial for their perpetual operation and maintenance-free property. Guaranteeing this ENO condition and optimal powerperformance trade-off under transient harvested energy and wireless channel quality is particularly challenging. This paper proposes a multi-agent actor-critic reinforcement learning for modulating both the transmitter duty-cycle and output power based on the state-of-buffer (SoB) and the state-of-charge (SoC) information as a state. Thanks to these buffers, differently from the state-of-the-art, our solution does not require any model of the wireless transceiver nor any direct measurement of both harvested energy and wireless channel quality for adapting to these uncertainties. Simulation results of a solar powered EHIoT node using real-life outdoor solar irradiance data show that the proposed method achieves better performance without system failures throughout a year compared to the state-of-the-art that suffers some system downtime. Our approach also predicts almost no system fails during five years of operation.
