Unsupervised Bidirectional MR to CT Synthesis Based on Generative Adversarial Networks

Date of Award


Publication Type


Degree Name



Electrical and Computer Engineering


Magnetic resonance, Computer tomography, Synthetic images, Medical images, DC-cycleGAN


Q.M. Jonathan Wu



Creative Commons License

Creative Commons Attribution 4.0 International License
This work is licensed under a Creative Commons Attribution 4.0 International License.


Magnetic resonance (MR) and computer tomography (CT) images are two typical types of medical images that provide mutually-complementary information for accurate clinical diagnosis and treatment. However, obtaining both images may be limited due to some considerations such as cost, radiation dose, and modality missing. Recently, medical image synthesis has aroused gaining research interest to cope with this limitation. In this thesis, we proposed unsupervised bidirectional learning models based on generative adversarial networks (GANs) to synthesize medical images from unpaired data. The first model is dual contrast CycleGAN (DC-cycleGAN), where a dual contrast (DC) loss is introduced into the CycleGAN’s discriminators to indirectly build constraints between real source and synthetic images by taking advantage of samples from the source domain as negative samples and enforce the synthetic images to fall far away from the source domain. In addition, cross-entropy (CE) and structural similarity index (SSIM) are integrated into the DC-cycleGAN to respectively consider samples’ luminance and structure when synthesizing images. To further improve performance, an attention-based generative model, namely ADC-cycleGAN, is proposed. Specifically, an attention mechanism is integrated into the generators to extract informative features from the channel and spatial domains. Despite the fact that both DC-cycleGAN and ADC-cycleGAN are able to produce promising results as compared with other CycleGAN-based methods, they fail to synthesize high-quality images when the training set contains multiple slices for each modality. To alleviate this issue, the K-means algorithm is adopted to cluster the training set into k groups, and then each group by trained a model. Several ablation studies are conducted to show the effectiveness of the various components of the proposed models, i.e., DC-cycleGAN and ADC-cycleGAN. Moreover, the experimental results indicate that both models are able to produce promising results as compared with other CycleGAN-based medical image synthesis methods such as CycleGAN, RegGAN, DualGAN, UGATIT, and NiceGAN.