A Bayesian Bandit Approach to Adaptive Delay-based Congestion Control

Adapting the transmission rate of video telephony Internet applications in order to guarantee the maximal communication quality is still an open and extremely challenging problem. The congestion control algorithm, which is the algorithm responsible for adjusting the transmission rate according to the network conditions, should typically be able to reach the largest possible rate, in order to achieve a high video quality, at the minimum possible delay, in order to guarantee a good interactivity. At the same time, it should also guarantee a fair share of the network resources when competing with other communication protocols, in particular loss-based congestion protocols. These two objectives actually conflict with each other: whereas, in order to achieve the largest rate with the minimum delay, the delay-based congestion control should be extremely sensitive to delay variations, it should also be ideally immune to delay variations to have perfect coexistence with loss-based protocols. In order to achieve this double objective we propose a learning-based adaptive controller that tunes the delay sensitivity of an underlying delay-based congestion control according to the estimated network conditions. We first define a simple low-dimensional model for the network response. We then formulate a bayesian bandit problem for the selection of the delay sensitivity of the congestion control algorithm. By solving the bandit problem using an optimal learning method we are able to maximize effectively the long term utility provided to the user. Finally, we provide simulation results to demonstrate the operation of the proposed method and its effective ability to adapt to different network scenarios in order to maximize the communication quality.