# Semi-supervised Brain Tumor Segmentation using Diffusion Models
We provide the official PyTorch implementation of the paper titled
[Semi-supervised Brain Tumor Segmentation Using Diffusion Models](https://link.springer.com/chapter/10.1007/978-3-031-34111-3_27)
by Ahmed Alshenoudy, Bertram Sabrowsky-Hirsch, Stefan Thumfart, Michael Giretzlehner and Erich Kobler.
Our implementation is based on [Label-Efficient Semantic Segmentation with Diffusion Models](https://github.com/yandex-research/ddpm-segmentation),
where we also employ [Improved Denoising Diffusion Probabilistic Models](https://github.com/openai/improved-diffusion).
Various core functions were heavily influenced from [nnUNet V1](https://github.com/MIC-DKFZ/nnUNet/tree/nnunetv1/nnunet/training/network_training) as well.
## Overview
In this paper, we leverage learned visual representations from diffusion models for the challenging task
of brain tumor segmentation. We compare the segmentation performance against a supervised baseline over a varying degree
of training samples. For the downstream segmentation task, we used pixel-level classifiers and additionally proposed the
fine-tuning of the noise predictor network of the diffusion model. Our results show that, with less than 20 training samples,
all methods outperform the supervised baseline across all tumor regions. We also provide a practical use-case where
we automatically annotate tumor regions across different axial slices within the same patient, with very limited supervision.
## Data
We evaluate the presented approach on the Brain Tumor Segmentation (BraTS) 2021 data. Axial slices were extracted from the
original 3D MR sequences, while stratifying longitudinal slicing locations to increase the proportion of slices containing
a segmented tumor. All slices were normalized and down-sampled to (128, 128). This resulted in a dataset consisting of 8,757
slices, of which 8,000 were used for testing and the 757 remaining slices were used as a training pool to sample various
training datasets from for our experiments.
## Results
- Our trained diffusion model and a small batch of generated samples can be found [here](https://www.dropbox.com/sh/78x8aamupsidxax/AABmtL06uEh6CGwWWvP3fcCMa?dl=0).
We visualize a batch of generated mpMRI samples from the fully trained model in the figure below. This is to visually assess
how well the model is able to approximate the dataset distribution.
<figure>
<img src="https://raw.githubusercontent.com/risc-mi/braintumor-ddpm/main/docs/assets/generated_samples.png"
alt="Generated 128 x 128 BraTS samples" style="width:100%">
<figcaption><b>Fig. 1: Generated (128, 128) BraTS samples.</b></figcaption>
</figure>
- Extracted visual representations for sample inputs are visualized in Fig. 2, our results align with the original paper, where earlier
layer capture high-level features and later layers capture more detailed features while also becoming noisier.
<figure>
<img src="https://raw.githubusercontent.com/risc-mi/braintumor-ddpm/main/docs/assets/representations.png"
alt="Generated 128 x 128 BraTS samples" style="width:100%">
<figcaption><b>Fig. 2: Extracted visual representations for different samples across different layers and time steps.</b></figcaption>
</figure>
- Sample predictions of different tumor regions are visualized in Fig. 3, a comparison between two pixel-level classifier
architectures, the fine-tuned noise predictor network and the nnU-Net supervised baseline.
<figure>
<img src="https://raw.githubusercontent.com/risc-mi/braintumor-ddpm/main/docs/assets/sample_predictions.png"
alt="Generated 128 x 128 BraTS samples" style="width:100%">
<figcaption><b>Fig. 3: Sample predictions for multiple input scans.</b></figcaption>
</figure>
- A sample practical use-case, where a few tumor containing axial slices from the same patient are used to train the downstream
pixel-level classifier to produce segmentation maps for the remaining slices within the same volume. Performance is very
good for slices that contain larger sections of the tumor, while a significant drop in performance is observed for out of
distribution slices that were not used for training the downstream model or the diffusion model itself.
<figure>
<img src="https://raw.githubusercontent.com/risc-mi/braintumor-ddpm/main/docs/assets/usecase.png"
alt="Generated 128 x 128 BraTS samples" style="width:100%">
<figcaption><b>Fig. 4: Practical use-case for patient-level segmentation.</b></figcaption>
</figure>
### Citation
If you find this codebase useful for your research, we would appreciate citing the following conference paper:
```
@InProceedings{braintumor_ddpm2023,
author={Alshenoudy, Ahmed and Sabrowsky-Hirsch, Bertram and Thumfart, Stefan and Giretzlehner, Michael and Kobler, Erich},
editor={Maglogiannis, Ilias and Iliadis, Lazaros and MacIntyre, John and Dominguez, Manuel},
title={Semi-supervised Brain Tumor Segmentation Using Diffusion Models},
booktitle={Artificial Intelligence Applications and Innovations},
year={2023},
publisher={Springer Nature Switzerland},
address={Cham},
pages={314--325},
isbn={978-3-031-34111-3}.
doi={10.1007/978-3-031-34111-3_27}
}
```
### Acknowledgements
This project is financed by research subsidies granted by the government of Upper Austria.
RISC Software GmbH is Member of UAR (Upper Austrian Research) Innovation Network.
### TODO:
- [ ] Tutorial notebook on how to use the pipeline for segmentation
- [ ] Addition of Deep/Wide MLP architecture and normal initialization
- [ ] Requirements file notes:
- [ ] Numpy (< 1.24.0) due to conflict with MedPy (used for HD95 metric)
- [ ] Introduced anti-alias parameter in torchvision
- [ ] MPI, used for `dist_utils` in improved diffusion
没有合适的资源?快使用搜索试试~ 我知道了~
温馨提示
本项目致力于基于扩散模型的半监督脑肿瘤分割,旨在提高医学影像在脑肿瘤诊断中的应用。项目利用扩散模型对脑部MRI影像中的肿瘤区域进行精确分割。 在数据集方面,我们使用了公开的脑部MRI影像数据集,如BRATS、Medical Segmentation Decathlon等,并进行了预处理,包括格式转换、裁剪、翻转等操作。在环境搭建方面,我们使用Python编程语言,基于TensorFlow和PyTorch深度学习框架进行开发。为了提高计算效率,我们还使用了GPU加速计算。此外,我们还采用了Docker容器技术,确保实验结果的可重复性。 项目完成后,将实现对脑部MRI影像中肿瘤区域的精确分割,为脑肿瘤的诊断和治疗提供有力支持。同时,项目成果也可应用于其他医学影像分析领域,如肿瘤检测、病变分割等。
资源推荐
资源详情
资源评论
收起资源包目录
基于扩散模型的半监督脑肿瘤分割内含数据集和环境说明.zip (50个子文件)
braintumor_ddpm
__init__.py 0B
data
__init__.py 0B
online_transforms.py 6KB
brats.py 8KB
transforms.py 5KB
datasets.py 8KB
utils
__init__.py 0B
helpers.py 2KB
convert_data.py 13KB
palette.py 893B
data.py 5KB
core
__init__.py 0B
networks
__init__.py 0B
base_network.py 2KB
denoise_network.py 2KB
pixel_classifier.py 5KB
training
__init__.py 0B
PixelRepresentationsTrainer.py 14KB
SimpleNetworkTrainer.py 16KB
DenoiseNetworkTrainer.py 13KB
insights
evaluator.py 9KB
__init__.py 0B
plots.py 6KB
cm_metrics.py 8KB
representations.py 2KB
confusion_matrix.py 3KB
diffusion
__init__.py 0B
improved_ddpm
__init__.py 0B
losses.py 2KB
dist_util.py 2KB
fp16_util.py 8KB
gaussian_diffusion.py 33KB
resample.py 6KB
unet.py 24KB
nn_utils.py 5KB
logger.py 14KB
train.py 11KB
respace.py 5KB
config.py 1KB
scripts
__init__.py 0B
script_utils.py 9KB
docs
Visualize Representations.ipynb 1.4MB
Train a Pixel-level Calssifier.ipynb 4KB
assets
usecase.png 469KB
sample_predictions.png 355KB
data_splits.png 243KB
representations.png 4.46MB
generated_samples.png 1.37MB
Dataset Extraction and Data Splits.ipynb 8KB
README.md 6KB
共 50 条
- 1
资源评论
小码蚁.
- 粉丝: 2525
- 资源: 4090
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功