Knowledge Distillation in Lightweight U-Net Transformer Architectures for Brain Tumor Segmentation

Brain tumor segmentation from Magnetic Resonance Imaging was an essential step for therapy planning, prognosis evaluation, and treatment monitoring in glioma patients. Manual delineation required substantial time and was prone to inter-observer variability. Although deep learning models achieved high segmentation accuracy, performance improvements were often accompanied by increased computational complexity, limiting their applicability in resource-constrained clinical environments. To address this issue, an efficiency-oriented segmentation framework was developed based on a lightweight three-dimensional U-Net enhanced with a shallow Transformer module and guided by knowledge distillation. The main contribution of this study was the integration of logit-level and feature-level distillation to improve segmentation capability while maintaining low inference complexity. The framework emphasized a balanced trade-off between segmentation accuracy and computational efficiency rather than benchmark maximization. Experiments were conducted using the BraTS 2020 and BraTS 2021 datasets. The official training sets were internally split into eighty percent for training and twenty percent for validation. The student network was trained using a hybrid segmentation loss combined with temperature-scaled logit distillation and bottleneck feature alignment from a higher-capacity teacher model. Model performance was evaluated using Dice score, Intersection over Union, and computational complexity measured in floating-point operations. On the BraTS 2021 dataset, the proposed model achieved Dice scores of 0.6538 for Whole Tumor, 0.5382 for Tumor Core, and 0.5304 for Enhancing Tumor. Per-class Dice values were 0.9706 for background, 0.3028 for necrotic or non-enhancing tumor core, 0.4938 for edema, and 0.4936 for enhancing tumor. The corresponding Intersection over Union values followed similar trends. The model maintained an inference complexity of approximately 54.38 gigafloating-point operations for input patches of size 64 × 64 × 64. These findings indicated that the proposed framework achieved a stable balance between segmentation performance and computational efficiency, supporting practical deployment under limited computational resources