GAN-Based Medical Images Synthesis: A Review

Introduction

In modern medical diagnosis, the accurate diagnosis and evaluation of diseases mainly depend on the collection of medical images and image explanation. Nowadays, with the improvement of hardware equipment, we are able to obtain various modern high-definition images, and the problem of image acquisition has been greatly eased. However, most of the work of image interpretation and analysis is still performed by doctors, which is easily affected by individual subjectivity, cognitive differences and work fatigue. In conventional medical diagnosis, doctors mainly find pathogens and analyze the cause of the disease by observing the patient's imaging images, which largely relies on the doctor's subjective judgment and own experience. Actually, there are many kinds of modern imaging images, which are scanned from different devices, such as CT, MRI, PET and X-ray. Images of different modalities can provide different diagnostic information, but due to the limited conditions, such as money, time and equipment, doctors in many cases cannot get sufficient imaging information support. Relying only on the morphological information provided by limited images and then making a diagnosis based on one's own experience is one of the main reasons leading to diagnosis errors (Vinod et al.2016).

For the purpose of improving the accuracy of diagnosis and reducing the subjective differences caused by individuals, Computer-Aided Diagnosis (CAD) has always been one of the hot spots in modern medical imaging research (Hu et al. 2020). CAD mainly relies on various modern medical imaging images. According to the clinical imaging characteristics of different diseases, computer algorithms are used to quantify the characteristics of the lesions and then make judgments, so as to avoid errors caused by individual subjectivity. This is the concept of medical CAD. With the help of artificial intelligence algorithms, especially deep learning algorithms, to process and analyze medical images, the application of CAD has been greatly extended (Khan et al. 2020). For example, segmentation and extraction (Li et al. 2020; Ji et al.2020; Vrtovec et al. 2020), three-dimensional reconstruction and three-dimensional display (Cheng et al. 2020) of human organs and lesions can assist doctors in targeted qualitative and quantitative analysis of lesions or other regions of interest (Renard et al.2020). Synthesis of missing or contaminated images, and translation among images of different modalities can make up for the inherent defect of insufficient image information (Jiao et al. 2020; Kang et al. 2020). The deformation registration of images of different modalities or images obtained in different periods of the same modal can improve the efficiency and reliability of doctors' diagnosis (Yang et al. 2020; Tang et al. 2019). In addition, the application of deep learning technology to medical teaching, surgical planning, surgical simulation and other medical research tasks can also play an important auxiliary role.

Under the current artificial intelligence boom, the emergence of Convolutional Neural Network (CNN) has improved the efficiency of image analysis, and the emergence of Generative Adversarial Networks (GAN) (Goodfellow et al. 2017) has further enhanced the quality of medical image generation (Yi et al. 2018). It enables the improvement of image synthesis no longer only rely on the raw data collected by hardware equipment, but can be achieved via feature extraction and target reconstruction methods from some data sets with the similar data distribution. At present, it has been widely used in image synthesis, segmentation, registration, low-resolution images to high-resolution graphics (Ledig et al. 2016), etc. Compared with natural images, medical images have the characteristics of high resolution, high correlation, large size and low contrast, which require larger storage space and more computing resources. Correspondingly, there are higher requirements for image compression, deformation, segmentation and other preprocessing operations, as well as image analysis and understanding. In order to promote deep learning-based methods into the medical field, the joint efforts of the fields of artificial intelligence and medicine are all required. This article focuses on GAN-based medical image synthesis, briefly introduces the basic framework of GAN and its improvement methods, and finally makes a summary and discussion.