Multi-Agent Learning in Repeated Double-side Auctions for Peer-to-peer Energy Trading

Abstract

Distributed energy resources (DERs), such as rooftop solar panels, are growing rapidly and are reshaping power systems. To promote DERs, feed-in-tariff is usually adopted by utilities to pay DER owners certain fixed rates for supplying energy to the grid. Such a non-market based approach may increase electricity rates and create inefficiency. An alternative is a market based approach; i.e., consumers and DER owners trade energy in a peer-to-peer (P2P) market, in which electricity prices are determined by real-time market supply and demand. A prevailing approach to realize a P2P marketplace is through double-side auctions. However, the auction complexity in an energy market and the participants’ bounded rationality may invalidate many well-established results in auction theory and hence, cast difficulties for market design and implementation. To address such issues, we propose an automated bidding framework based on multi-agent, multi-armed bandit learning through repeated auctions, which is aimed to minimize each bidder’s cumulative regret. Numerical results suggest the potential convergence of such a multi-agent learning game to a steady-state. We also apply the framework to three different auction designs (including uniform-price versus Vickrey-type auctions) for a P2P market to study the impacts of the different designs on market outcomes.