2/3/2025 | 3:30 PM - 5:30 PM | Regency A

Synthetic dataset pre-training for precision medical segmentation using vision transformers

Author(s)

Edgar Josafat Martinez-Noriega | AIST
Rio Yokota | Tokyo Institute of Technology Blog About
Peng Chen | AIST
Thao Nguyen Truong | AIST

Abstract

In medical segmentation, the acquisition of high-quality labeled data remains a significant challenge due to the substantial cost and time required for expert annotations. Variability in imaging conditions, patient diversity, and the use of different imaging devices further complicate model training. The high dimensionality of medical images also imposes considerable computational demands, while small lesions or abnormalities can create class imbalance, thus hindering segmentation accuracy. Pre-training on synthetic datasets in medical imaging may enable Vision Transformers (ViTs) to develop robust feature representations, even in the fine-tuning phase, when high-quality labeled data is limited. In this work, we propose integrating Formula-Driven Supervised Learning (FDSL) synthetic datasets with medical imaging to enhance pre-training for segmentation tasks. We implemented a custom Fractal dataset capable of generating high-resolution images, including those measuring 64K x 64K pixels or larger. Preliminary results indicate improved performance when using the SAM model for segmentation in conjunction with robust augmentation techniques, followed by fine-tuning on the PAIP dataset, a high-resolution, real-world pathology dataset focused on liver cancer. Additionally, we present results using another synthetic dataset, SegRCDB, for comparative analysis.