Abstract
In access networks, the radio resource management is designed to deal with the system capacity maximization while the quality of service (QoS) requirements need be satisfied for different types of applications. In particular, the radio resource scheduling aims to allocate users' data packets in frequency domain at each predefined transmission time intervals (TTIs), time windows used to trigger the user requests and to respond them accordingly. At each TTI, the scheduling procedure is conducted based on a scheduling rule that aims to focus only on particular scheduling objective such as fairness, delay, packet loss, or throughput requirements. The purpose of this chapter is to formulate and solve an aggregate optimization problem that selects at each TTI the most convenient scheduling rule in order to maximize the satisfaction of all scheduling objectives concomitantly TTI-by-TTI. The use of reinforcement learning is proposed to solve such complex multi-objective optimization problem and to ease the decision making on which scheduling rule should be applied at each TTI.
TopIntroduction
The continuous growth of mobile data usage and the increased interest for immersive video applications are pushing network operators to find suitable solutions to accommodate these services with very stringent Quality of Service (QoS) demands (Cisco, 2017). According to Trestian, Comsa, and Tuysuz (2018), the Quality of Experience provisioning will become the main differentiator between network operators, in which, the satisfaction of heterogeneous QoS requirements is playing a crucial role. In this context, the 5th Generation (5G) of mobile networks comes up with the promise of very low end-to-end latencies and much higher system capacity while implementing some important features such as (G. Andrews et al., 2014): new waveforms, densification of access networks, higher frequency bands, mass scale antennas and millimeter-wave communications. An issue to be addressed when delivering these bandwidth-hungry applications in multi-user scenario refers to the management of radio resources that can strongly affects the overall performance of QoS provisioning (Li et al., 2017). The responsible entity is the Radio Resource Management (RRM) that aims to ensure an efficient allocation of the disposable system bandwidth in order to maximize the QoS satisfaction while implementing advanced technologies as provided by Olwal, Djouani, and Kurien (2017) able to: save the energy, control the mobility and power allocation, mitigate interference, schedule users’ packets in frequency domain at each TTI.
According to the performance of the scheduling process, the operator is struggling to provide the requested services while using the disposable radio infrastructure, regardless of the spatial/time positions of mobile terminals, user preferences, devices’ types and application requirements (Comşa, 2014a). A major concern is to increase the system capacity or data rates of all active users while satisfying the application requirements. Users located in the proximity of base stations experience better channel quality and consequently can get higher data rates than those users with poorer channel quality located farer away from any available base station (Trestian, Muntean, & Ormond, 2009; Trestian, Ormond, & Muntean, 2012; Vien, Akinbote, Nguyen, Trestian, & Gemikonakli, 2015). By providing the disposable spectrum to those users with better channel conditions, other users are starved in receiving the requested data for longer time (Vien, Nguyen, Trestian, Shah, & Gemikonakli, 2016). Then, the fairness measure between different users with the same QoS profiles is impaired. Certain tradeoff measures between system throughput and user fairness can be adopted (Jain, Chiu, and Hawe, 1984; Comşa, 2014a). Together with these aspects, the requested services should be provided under some predefined QoS requirements as imposed by 3GPP (2012) in terms of Guaranteed Bit Rate (GBR), Head of Line (HoL) packet delay and Packet Loss Rate (PLR). The QoS requirements become more restrictive with the evolution of cellular standards, system architectures and applications. By encompassing the above discussed aspects, the packet scheduler is dealing with an optimization problem aiming to maximize the system throughput and being constrained by fairness and QoS requirements. If we further consider that the constraints’ satisfaction refers to fairness and QoS objectives, then the Multi-Objective Optimization (MOO) is addressed as stated by Comşa, (2014a).
Key Terms in this Chapter
Radio Resource Management: Function of a mobile communication systems used for establishing, maintaining, and releasing of radio resources.
Reinforcement Learning: Training/learning method aiming to automatically determine the ideal behavior within a specific context based on rewarding desired behaviors and/or punishing undesired one.
Machine Learning: Category of algorithm that uses statistical techniques to analyze data and make predictions.
Multi-Objective Optimization: Mathematical optimization problems involving more than one objective function to be optimized simultaneously.
Unsupervised Learning: Type of machine learning algorithm that is capable of learning a function that best represents a model from datasets consisting of unlabeled input data.