A DSPL and Reinforcement Learning Approach for Context-Aware IoT Systems Development

Introduction

Internet of Things, introduced by Kevin Ashton in 1999, has emerged and evolved in recent years (Perera et al., 2014). IoT refers to a networked interconnection of things or objects – Radio-Frequency IDentification (RFID) tags, sensors, informational processors, and actuators- interacting with each other to reach specific goals (Atzori et al., 2010) (Khoo, 2010) (Díaz et al., 2016). It includes dimensions from anytime, anyplace connectivity from anyone, and for anything (Khoo, 2010).

IoT can be seen as an internet extension to real-world, embracing physical items, that can be controlled remotely and act as physical access points to internet services (Mattern & Floerkemeier, 2010). It defines connections based on smart physical objects (“things”) as devices and equipment, that collaborate autonomously, without human involvement, to access data and services remotely, and perform actions on behalf of users (Puliafito et al., 2010) (Al-Fuqaha et al., 2015) (Geraldi et al., 2020).

A thing can be any individual object equipped with a sensor tracking useful information about its state and other properties. Such things can sense the environment and communicate (Aggarwal et al., 2013). IoT objects not only sense information from their environment, and interact with other objects, but also provide services from information transmission, analytics, and applications (Gubbi et al., 2013).

In this regard, context-aware computing has been identified as an important IoT research prospect and a key enabler for IoT. Context-awareness leverages IoT systems, by giving them the ability to capture variations in the state of the environment and adapt accordingly, which makes them more interesting to users (Afzal et al., 2017) (de Matos et al., 2015).

Many approaches have been proposed to deal with context-aware IoT systems. Among those approaches, Software Product Line (SPL) and Dynamic Software Product Line (DSPL) were proposed. DSPL is an extension of the traditional SPL, that enables the identification of the reusable features at runtime, by adapting them to the environment changes (Pessoa et al., 2017) (Santos & Machado, 2018).

Using DSPL to develop IoT systems is usually based on the MAPE-K model (Pessoa et al., 2017), which comprises Monitoring, Analysis, Planning, and Execution.

In this paper, the proposition is to combine DSPL with a Reinforcement Learning algorithm, which is a promising way to solve sequential decision-making tasks, since it doesn’t need any supervision (Chen et al., 2021) (Zhang et al., 2020).

The remainder of this paper is structured as follows. The next section presents, firstly, a background about context-aware IoT, DSPL, and Reinforcement Learning. Secondly, it discusses some related work. The fourth section presents a motivating scenario that concerns a context-aware Smart home system. The fifth section describes the proposed approach and its different steps, illustrating it using the smart home system example. The last section is dedicated to conclusions and plans for future work.