A Methodological Approach to Assessment and Reporting of the Model Adequacy in Simulation Studies

A Methodological Approach to Assessment and Reporting of the Model Adequacy in Simulation Studies

Michael Sony, V. Marriapan
DOI: 10.4018/IJORIS.20211001.oa2
Article PDF Download
Open access articles are freely available for download

Abstract

Assessment and Reporting of Model Adequacy is an important step in the simulation modelling process. It stipulates the level of precision and accuracy, which are important features of the model predictions. In an academic research activity, an important step for model development is the process of the identification or accepting whether the model is wrong. The evaluation of the adequacy of developed models is not possible through a single statistical test. This paper delineates a technique to implement model adequacy. A live case is demonstrated on the proposed methodology by evaluating a simulation model which was designed by us to simulate a well-established mathematical model. A step by step methodological approach is delineated in this paper along with a case study of investigation of a simulation model with a mathematical model is used to demonstrate this methodology. The paper concludes with an Algorithm and a flow chart for performing model adequacy for assessing the adequacy of the developed model with existing models.
Article Preview
Top

1.0 Introduction

Simulation Models are used to represent various mechanisms of natural phenomenon or system behaviour or an event under consideration. Models are representations of reality. These simulation models can be used as an important tool for a decision support system for various stakeholders such as the policymakers, managers, researchers, engineers and others (S Michael, Mariappan, Amonkar, & Telang, 2009; Atamturktur, Stevens, & Brown, 2017; Miao, Xie, Yang, Tai, & Hu, 2017; Fritzsche, 2018). The tasks such as validation of precision, the accuracy of proposed simulation models form an important aspect. A wrong model can lead to erroneous conclusions, which can cost the stakeholders a large sum of money(M Sony & Mariappan, 2019). Besides simulation models can be used to communicate and advance the scientific knowledge, which in turn can lead to discoveries or challenging old discoveries or theories(Mac Nally, Duncan, Thomson, & Yen, 2018). The simulation Modeling process is a methodological process. The process begins with clearly identifying the statement of objectives. The other steps in are 1) exploring assumptions about the proposed model boundaries 2) how appropriate are the available data 3) the design of the model structure 4) evaluating the models results from studies like simulation and 5) recommending solutions(Tedeschi, 2006; S Michael, Amonkar, Mariappan, & Kamat, 2009; Michael Sony & Naik, 2011; Sony Michael, Mariappan, & Kamat, 2011; Montgomery, 2017). A rigorous model testing of simulation methodology is recommended to prove the appropriateness of a model(Khorsan & Crawford, 2014), as no single tests to confirm its appropriateness. The tests are performed on the models to gather evidence as to its acceptance and usability(Sterman, 2002). Improved model validity enhances the credibility of the research and acceptance among other researchers. The simulation modelling process is further strengthened by the process of by identifying wrongness of the model (Melhart, Sfikas, Giannakakis, Yannakakis, & Liapis, 2018). This delineation process of identifying the weakness strengthens the process of model development in all its phases. There has been strong criticism of the concept of model validation and verification(Addiscott, Smith, & Bradbury, 1995). It is partly due to the philosophy that it is impossible to validate all components of the model(Oreskes, Shrader-Frechette, & Belitz, 1994; Saltelli, Tarantola, Campolongo, & Ratto, 2004; Foures, Albert, & Nketsa, 2016). There are formal approaches for model verification and validation. One of the most widely used methods is quantitative methods for verification and validation. However, a recent study suggests that to explore all possible behaviours of models one should take into account both qualitative and quantitative approach (Foures et al., 2016). Another point to be considered is that model is a representation of process or phenomenon. In other words, it approximates reality or phenomenon. Thus, testing a model in an absolute sense is not advisable. Nevertheless, it should be tested for the purpose for which it was designed. Before undertaking any experimentation or sensitivity analysis with the model, it should be thoroughly tested(Tedeschi, 2006; Denil, Klikovits, Mosterman, Vallecillo, & Vangheluwe, 2017). Many researchers have devoted considerable attention to the process of model verification and validation (Tedeschi, 2006; McCoach & Black, 2008; Foures et al., 2016; Denil et al., 2017; Montgomery, 2017). However, despite all these studies, there still exists a requirement of a testing procedure which considers both the quantitative and qualitative methodology(Foures et al., 2016; Denil et al., 2017). In this paper, the authors intend to develop a step by step procedure for model adequacy with the help of a practical case study on Model Adequacy for assessing the fit between the developed model using simulation with a well-established Mathematical Model (Markov model). Besides a reporting methodology is also reported in the study.

Complete Article List

Search this Journal:
Reset
Volume 15: 1 Issue (2024)
Volume 14: 1 Issue (2023)
Volume 13: 2 Issues (2022)
Volume 12: 4 Issues (2021)
Volume 11: 4 Issues (2020)
Volume 10: 4 Issues (2019)
Volume 9: 4 Issues (2018)
Volume 8: 4 Issues (2017)
Volume 7: 4 Issues (2016)
Volume 6: 4 Issues (2015)
Volume 5: 4 Issues (2014)
Volume 4: 4 Issues (2013)
Volume 3: 4 Issues (2012)
Volume 2: 4 Issues (2011)
Volume 1: 4 Issues (2010)
View Complete Journal Contents Listing