Domain and Content Adaptive Convolution based Multi-Source Domain Generalization for Medical Image Segmentation

Abstract

The domain gap caused mainly by variable medical image quality renders a major obstacle on the path between training a segmentation model in the lab and applying the trained model to unseen clinical data. To address this issue, domain generalization methods have been proposed, which however usually use static convolutions and are less flexible. In this paper, we propose a multi-source domain generalization model, namely domain and content adaptive convolution (DCAC), for medical image segmentation. Specifically, we design the domain adaptive convolution (DAC) module and content adaptive convolution (CAC) module and incorporate both into an encoder-decoder backbone. In the DAC module, a dynamic convolutional head is conditioned on the predicted domain code of the input to make our model adapt to the unseen target domain. In the CAC module, a dynamic convolutional head is conditioned on the global image features to make our model adapt to the test image. We evaluated the DCAC model against the baseline and four state-of-the-art domain generalization methods on the prostate segmentation, COVID-19 lesion segmentation, and optic cup/optic disc segmentation tasks. Our results indicate that the proposed DCAC model outperforms all competing methods on each segmentation task, and also demonstrate the effectiveness of the DAC and CAC modules.

Highlights

Publication

• Domain and Content Adaptive Convolution for Domain Generalization in Medical Image Segmentation
  Shishuai Hu, Zehui Liao, Jianpeng Zhang, Yong Xia
  arXiv Code

Method

The proposed DCAC model is an encoder-decoder structure equipped with a domain predictor, a domain-aware controller, a content-aware controller, and a series of domain-adaptive heads and content-adaptive heads. The workflow of this model consists of four steps. First, the feature map produced by each encoder layer is aggregated and fed to the domain predictor. Second, based on the generated domain code, the domain-aware controller predicts the parameters of the domain-adaptive head. Third, the content-aware controller uses the final output of the encoder as its input to generate the parameters of the content-adaptive head. Finally, according to the deep supervision strategy, the output of each decoder layer is fed sequentially to a domain-adaptive head and a content-adaptive head, which predict the segmentation result on a pixel-by-pixel basis. The diagram of our DCAC model is shown in Figure 2.

Figure 2: Architecture of the proposed method. The feature map in orange color represents GAP(f_E^N), i.e., the output of the N-th encoder block after global average pooling.

Datasets

Three datasets were used for this study. For prostate segmentation, the dataset contains 116 T2-weighted MRI cases from six domains. We preprocessed the MRI data and only preserved the slices with the prostate region for consistent and objective segmentation evaluation. For COVID-19 lesion segmentation, the dataset consists of 120 RT-PCR positive CT scans with pixel-level lesion annotations, collected from the first multi-institutional, multi-national expert annotated COVID-19 image database. For OC/OD segmentation, the dataset contains 789 cases for training and 281 cases for test, which are collected from four public fundus image datasets and have inconsistent statistical characteristics. The statistics of three datasets were summarized in Table 1.

Table 1: Statistics of three datasets used for this study.

Datasets Download Link

• Prostate segmentation and dataset details.

• COVID-19 lesion segmentation.

• OC/OD segmentation.

Main Results

We compared the proposed DCAC model with the ‘Intra-domain’ setting (i.e., training and testing on the data from the same domain), ‘DeepAll’ baseline (i.e., training on the data aggregated from all source domains and testing directly on the unseen target domain), and four DG methods, including (1) BigAug: a data-augmentation based method (Zhang et al. 2020), (2) SAML (Liu, Dou, and Heng 2020) and FedDG (Liu et al. 2021a): two meta-learning-based methods, and (3) DoFE: a domain-invariant feature learning approach (Wang et al. 2020).

Figure 3: Quantitative results.

Figure 4: Visualization results.

Related Publications

• DoFE: Domain-oriented Feature Embedding for Generalizable Fundus Image Segmentation on Unseen Datasets. Shujun Wang, Lequan Yu, Kang Li, Xin Yang, Chi-Wing Fu, Pheng-Ann Heng. IEEE-TMI 2020

• Shape-aware Meta-learning for Generalizing Prostate MRI Segmentation to Unseen Domains. Quande Liu, Qi Dou, Pheng-Ann Heng. MICCAI 2020

• FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space. Quande Liu, Cheng Chen, Jing Qin, Qi Dou, Pheng-Ann Heng.CVPR 2021

• DoDNet: Learning To Segment Multi-Organ and Tumors From Multiple Partially Labeled Datasets. Jianpeng Zhang, Yutong Xie, Yong Xia, Chunhua Shen.CVPR 2021

Citation

                    
@article{hu2022domain,
  title={Domain and Content Adaptive Convolution based Multi-Source Domain Generalization for Medical Image Segmentation},
  author={Hu, Shishuai and Liao, Zehui and Zhang, Jianpeng and Xia, Yong},
  journal={IEEE Transactions on Medical Imaging},
  year={2022},
  publisher={IEEE}
}