Autoregressive-Conditioned Diffusion for Semi-Supervised Thyroid Ultrasound Segmentation with Optical Flow-Based Pseudo Labels

Autoregressive-Conditioned Diffusion for Semi-Supervised Thyroid Ultrasound Segmentation with Optical Flow-Based Pseudo Labels Xiaorui Liu1, Yijun Yang1, Yingjing Xu2, Lei Zhu1,3,* 1The Hong Kong University of Science and Technology (Guangzhou) 2Zhejiang University 3The Hong Kong University of Science and Technology Abstract Ultrasound video segmentation plays a vital role in clinical diagnosis and treatment planning, yet the process is severely hindered by the high cost of manual annotation. Semi-supervised learning leverages limited labels effectively, but suffers from noisy pseudo-labels in lowcontrast frames and lacks global modeling, leading to blurred boundaries and temporal inconsistencies in video segmentation. To this end, we propose a novel semisupervised framework. We begin with a bidirectional optical flow-based pseudo-labeling strategy that fuses forward and backward warped masks using task-aware confidence weighting. To enhance spatial coherence, a diffusion model is employed to denoise the pseudo-labels in the latent space. Most importantly, we introduce a Visual Autoregressive (VAR) Encoder, a Transformer-based temporal modeling module that extracts cross-frame contextual priors from image latents. These priors are injected into every block of a DiT-based diffusion backbone via cross-attention, leveraging DiT’s strong global modeling capacity to refine spatial details while enforcing temporal consistency. Additionally, a YOLO-based post-processing step removes background artifacts by isolating the region of interest (ROI). Experimental results on a newly collected dataset of 80 patients and public datasets demonstrate that our framework achieves superior segmentation accuracy and temporal stability compared to existing semi-supervised methods. 1. Introduction Ultrasound video segmentation is essential for computeraided diagnosis and surgical planning [1,2,30,58]. However, the intrinsic characteristics of ultrasound imaging—low tissue contrast, speckle noise, and motion artifacts—pose significant challenges to achieving robust and accurate segmentation [26]. Fully supervised mod- *Corresponding author. els achieve high performance but require dense pixel-wise annotations, which are costly and time-consuming, limiting their scalability in clinical settings. Semi-supervised learning has emerged as a practical alternative by leveraging sparse annotations, where various strategies—such as pseudo-labeling, consistency regularization, or hybrid approaches—propagate supervision to unlabeled frames. However, in ultrasound videos with texture ambiguity and indistinct boundaries, many existing methods still struggle to maintain both spatial accuracy and temporal consistency. To facilitate clinically viable semi-supervised segmentation, we construct a thyroid ultrasound video dataset with expert-verified annotations. As shown in Fig. 1, each case comprises consecutive frames and corresponding masks, capturing the temporal evolution of thyroid nodules. This dataset provides a reliable benchmark for developing temporally-aware models under limited supervision. Despite recent advances in semi-supervised segmentation [9,40,44,48], three key challenges persist. First, within the pseudo-labeling paradigm, most optical-flowbased approaches rely on unidirectional propagation, making them vulnerable to cumulative drift: early-frame errors propagate and amplify over time, especially in low-contrast or fast-motion regions [32,52,59]. While bidirectional flow has been explored in other domains, it remains underutilized in ultrasound, where alignment is unstable and temporal artifacts are common. Second, although Transformer architectures such as ViViT [3] and certain 3D CNNs can capture temporal cues, many practical ultrasound segmentation pipelines still employ encoders with limited effective receptive fields or adopt per-frame prediction heads, which reduces robustness under speckle noise and hinders modeling of long-range semantic continuity. Third, frameindependent inference stages can lead to flickering or abrupt segmentation changes, which are unacceptable in clinical practice [43,56]. To overcome the limitations of existing semi-supervised segmentation methods, we propose a unified framework that tightly integrates spatial-temporal modeling to address la- Figure 1. Representative cases from our dataset. Each column pair represents an ultrasound frame and its corresponding segmentation mask. The rows illustrate temporal progression within each case. All videos are acquired from patients with thyroid nodules by experienced ultrasound clinicians. Annotations are manually verified by three radiologists with over three years of experience. bel noise, spatial instability, and temporal inconsistency. The framework begins with a bidirectional optical flowbased pseudo-labeling strategy, which propagates annotations across frames using task-aware weighting informed by grayscale intensity and boundary confidence. This effectively reduces label drift and improves temporal alignment. Building on this, a latent-space diffusion module progressively refines the pseudo-label representations, denoising them to suppress speckle artifacts and recover fine anatomical structures in low-contrast ultrasound images. To further enhance temporal modeling, a Visual Autoregressive (VAR) Encoder captures long-range dependencies across frames and injects this contextual information into a DiT-style latent diffusion backbone through cross-attention, promoting spatial coherence and inter-frame consistency. Additionally, a YOLO-based post-processing module isolates relevant anatomical regions, and a grayscale histogram-guided Task Token adaptively adjusts pseudo-label fusion according to frame quality. Together, these components enable robust and temporally stable segmentation under sparse supervision. Experiments on an in-house dataset of 80 patients demonstrate that our method consistently outperforms existing baselines in both segmentation accuracy and temporal consistency, underscoring its clinical potential. Our main contributions can be summarized as three-fold: • We propose a unified semi-supervised framework that integrates bidirectional pseudo-label propagation, latentspace diffusion refinement, and a Visual Autoregressive (VAR) encoder to jointly address label noise, spatial instability, and temporal inconsistency in ultrasound video segmentation. • We construct a new in-house ultrasound video dataset of 80 patients with frame-level annotations, providing a valuable benchmark for evaluating spatiotemporal consistency under sparse supervision. • Extensive experiments demonstrate that our method outperforms state-of-the-art semi-supervised approaches in both segmentation accuracy and temporal stability, highlighting its practical effectiveness in real-world clinical scenarios. 2. Related Work 2.1. Ultrasound Video Segmentation Recent approaches have introduced innovative hybrid transformer-based algorithms that fuse transformative and convolutional layer techniques for medical image segmentation (e.g., breast lesion, polyp) [5,19,20,36,45,51,54]. For thyroid segmentation in ultrasound images, Ma et al. [31] employed a region proposal network (RPN) with spatial pyramid RoIAlign to capture global and local features in ultrasound images, while Chi et al. [11] proposed a 2D Transformer-UNet that fuses multi-level features via a multiscale cross-attention transformer for thyroid gland segmentation. These algorithms skillfully manage the representations derived from high-definition medical images, however, they grapple with computational difficulties owing to complexity issues. Additionally, the direct application of such image segmentation methods may inadvertently overlook critical temporal context, thereby inducing temporal inconsistencies. In order to address temporal modeling in video-level segmentation, the innovative method of Space-Time Memory Networks (STM) [33] and its variants [10,25] are introduced, employing a memory network to extract vital information from a time-based buffer composed of all previous video sequences. Building upon this methodology, DPSTT [23] integrates a memory bank with decoupled transformers to track temporal lesion movement in medical ultrasound videos. MemSAM [12] proposes a novel ultrasound video segmentation model by incorporating space-time memory into SAM and carrying temporal cues. The challenge in ultrasound video segmentation revolves around efficiently harnessing the wealth of temporal data available. 2.2. Medical Semi-supervised Learning Semi-supervised learning has become increasingly important in medical image analysis, especially for segmentation tasks with limited annotations. Existing methods can be broadly categorized into pseudo-labeling [22,24], consistency regularization [17,49], contrastive learning [14,47, 48], and hybrid strategies [34]. POPCORN [22] pioneers progressive pseudo-labeling by constructing a curriculum learning framework based on feature similarity between labeled and unlabeled samples, further enhanced by enforcing consistency across different feature scales. Self-Loop Uncertainty [24] pretrains encoders through self-supervised tasks and fuses multi-view predictions to sharpen boundaries. AMVLM [34] introduces multiplicity-aware visionlanguage alignment, generating text-guided initial masks with uncertainty-weighted iterative refinement for complex multi-target segmentation. MemSAM [12] establishes a memory-driven pseudo-label generation paradigm, where a prototypical memory bank storing anatomical priors is used to generate prompt-guided segmentation masks for unlabeled images, eliminating manual annotation. 2.3. Diffusion Models Recently, the Diffusion model is well-known as a novel generative modeling approach for its superior achievements in image synthesis and generation tasks [13,21,27,39,55, 60]. In essence, Denoising Diffusion Probabilistic Models (DDPM) [21] adopted a parameterized Markov chain to optimize the lower variational bound on the likelihood function, which can simulate a diffusion process to iteratively improve the quality of target distribution than other generative models. DDIM [42] develops iterative implicit probabilistic models based on DDPM and introduces a deterministic sampling process for the trade-off between computational costs and sample quality. Very recently, a few pioneering works tried to adopt diffusion models for high-level perceptual tasks, such as image classification, segmentation and object detection [4,6,16,18,29,46,50,53,55,57]. Han et al. [18] combine a denoising diffusion-based conditional generative model and a pre-trained conditional mean estimator for natural image classification and regression. Diff-UNet [50] utilizes the diffusion models to solve 3D medical image segmentation problems and designs a StepUncertainty Fusion (SUF) module during inference for the robustness of the diffusion model’s predictions. Their potential for high-level vision in medical videos has yet to be fully explored. 3. Method 3.1. Overview Our framework addresses the challenges of semi-supervised ultrasound video segmentation by integrating bidirectional pseudo-label propagation, latent-space diffusion refinement, and temporal-contextual modeling. Specifically, we first generate pseudo-labels using a bidirectional optical flow strategy that fuses forward and backward warped masks with contrast-aware weighting. These labels are then denoised in the latent space via a diffusion model to improve spatial consistency and suppress speckle noise. To further enforce temporal coherence, we introduce a Visual Autoregressive (VAR) Encoder that extracts crossframe priors, which are injected into a DiT-based diffusion backbone via cross-attention. Additionally, a grayscale histogram-guided Task Token adaptively adjusts pseudolabel fusion according to frame quality. Finally, a YOLObased post-processing module eliminates background artifacts by refining the predicted masks based on detected anatomical regions. The overall pipeline ensures accurate, stable segmentation under limited supervision. Section 3.2 details the pseudo-labeling strategy, and Section 3.3 elaborates on the VAR-enhanced latent diffusion module and temporal consistency objective. 3.2. Bidirectional Optical Flow Pseudo-Labeling High-quality pseudo-labels are critical for reliable semisupervised video segmentation, particularly in the presence of noise and temporal variation. To ensure temporal alignment and spatial coherence, our framework adopts a three- Figure 2. Overview of the proposed framework. The framework consists of two main components: (1) Bidirectional Optical Flow-Based Pseudo-Labeling: RAFT optical flow propagates keyframe masks bidirectionally to generate pseudo-labels, and a contrast-aware mask fusion strategy is applied to reduce error accumulation and enhance boundary sharpness; (2) VAR-Enhanced Diffusion Model: The diffusion model refines pseudo-labels by denoising latent representations, improving segmentation consistency. A Visual Autoregressive (VAR) Encoder extracts temporal-aware latent tokens as additional conditional input to guide segmentation. A YOLO-based post-processing step removes background artifacts to enhance segmentation precision. stage strategy. RAFT [43] estimates bidirectional optical flow to propagate ground truth masks forward and backward. These warped masks are fused using a contrast-aware weighting scheme, followed by a refinement step that suppresses noise and enhances boundary sharpness. RAFT Optical Flow. We adopt RAFT [43] to compute dense optical flow between consecutive frames. In this work, RAFT is used in a frozen state without finetuning, initialized from weights pretrained on FlyingChairs and FlyingThings3D, which provide general motion priors applicable to ultrasound despite domain differences. Given two input frames Itand It+kat timestamps tand t+k, the forward flow Ft→t+k∈RH×W×2, which represents the pixel-wise displacement from Itto It+k, is defined as: Ft→t+k=Fθ(It, It+k),(1) where Fθ(·,·)denotes the RAFT model parameterized by θ. This flow is then used to warp the segmentation mask Mtat time tto the target frame t+k: ˜ Mt+k=W(Mt,Ft→t+k),(2) where W(·,·)is a differentiable warping function implemented via bilinear interpolation. The resulting warped mask ˜ Mt+kserves as the initial pseudo-label for frame t+k. This forward warping enables label propagation, but may introduce cumulative drift or occlusion-induced artifacts. To address this, we employ a bidirectional fusion strategy discussed below. Warped Mask Fusion. To enhance pseudo-label accuracy near object boundaries, we compute forward flow Ft→t+k and backward flow Ft+k→tbetween frames Itand It+k. Using the differentiable warping function W(·,·), we apply these flows to Mtand Mt+kto obtain two warped masks. The final pseudo-label at frame t+k, denoted as ˜ Mfinal t+k, is given by a weighted fusion: ˜ Mfinal t+k=λW(Mt,Ft→t+k) + (1 −λ)W(Mt+k,Ft+k→t), (3) where λ∈[0,1] balances the contributions of forward and backward propagation. This bidirectional fusion aggregates complementary spatial cues and mitigates errors from flow inaccuracy and occlusion, which often degrade singledirection warping. Here, λis only used in the fusion stage, computed from the grayscale histogram mean intensity via a linear mapping f(h)to the range [0.3,0.7], which we found empirically to yield stable results. Pseudo-label Refinement. After fusion, we further im- prove boundary quality via a refinement operator G(·): ˜ Mrefined t+k=G(˜ Mfinal t+k),(4) where G(·)applies morphological filtering to remove small noisy regions and smooth edges, followed by binarization to enforce spatial consistency. Unlike the fusion step, this refinement does not reuse λand operates directly on the fused mask. Its role is to ensure that the pseudo-labels used for training have clean, well-defined boundaries before entering the segmentation network. This step complements the latent diffusion module: refinement is performed offline during pseudo-label generation to provide high-quality supervision, while latent diffusion performs online denoising in the latent space during model prediction. Algorithm 1 Bidirectional Optical Flow-Based PseudoLabeling Require: Keyframe masks MK, video frames I, pretrained RAFT model Fθ Ensure: Pseudo-labels MP 1: for each keyframe tin MKdo 2: for each frame t+kdo 3: Compute flows: Ft→t+k,Ft+k→t 4: Warp masks: ˜ Mf t+k,˜ Mb t+k 5: Fuse: ˜ Mfinal t+k←λ˜ Mf t+k+ (1 −λ)˜ Mb t+k 6: Refine: ˜ Mrefined t+k← G(˜ Mfinal t+k) 7: Store: MP[t+k]←˜ Mrefined t+k 8: end for 9: end for 10: return pseudo-labels MP To enhance fusion robustness under varying visual conditions, we adopt a contrast-aware weighting scheme. For each frame, we extract a 16-bin grayscale histogram h∈ R16 and compute λ=f(h)based on the histogram’s mean intensity. The final pseudo-label is computed as: ˆ Mt=λ·Mf+ (1 −λ)·Mb,(5) where Mfand Mbdenote the forwardand backwardwarped masks. High-contrast frames yield larger λ, promoting confident fusion; low contrast encourages conservative averaging to suppress drift. This histogram-guided scheme adaptively adjusts fusion strength based on visual quality, improving pseudo-label reliability in challenging ultrasound scenes with blurred or ambiguous boundaries. The grayscale histogram statistics are also used to generate a Task Token that encodes frame quality, which is later provided as an additional conditioning signal to the diffusion model. 3.3. Diffusion Model Enhancement Diffusion models demonstrate strong denoising capabilities, making them particularly suitable for ultrasound video segmentation under speckle noise and low contrast. We enhance a latent-space diffusion framework by integrating spatial features from a Variational Autoencoder (VAE) [7], temporal priors from a Visual Autoregressive (VAR) Encoder, and multi-branch conditioning within a DiT backbone. Fig. 2(right) presents an overview of the proposed architecture. Visual Autoregressive Encoder Conditioning. To incorporate temporal context into the denoising process, we design a Visual Autoregressive (VAR) Encoder. Its input consists of a temporal window of spatial latents {x(t−k) cond , ..., x(t+k) cond }centered at frame t, where each x(t) cond ∈ RH×W×Cis derived from the VAE encoder [7] and encodes semantic visual features. These tensors are concatenated along the temporal axis and reshaped into a sequence of tokens: xseq =Concat(x(t−k) cond , ..., x(t+k) cond ), resulting in xseq ∈RN′×d′, where N′denotes the number of tokens and d′the feature dimension. The sequence is processed by a lightweight Transformer with LVAR selfattention layers to capture long-range temporal dependencies. Subsequently, a linear projection maps the output to N compact temporal tokens: VAR(xseq)→ {ci}N i=1, ci∈Rd, where each cirepresents motion-invariant semantic priors across frames. These tokens are incorporated into each DiT block via a dedicated cross-attention branch, alongside spatial features xcond, noisy latents zt, and the Task Token from histogram statistics, enabling joint modeling of spatial appearance, temporal dynamics, frame quality, and denoising signals. Latent Representation Refinement. To facilitate semantic-level denoising, we perform diffusion in the latent space rather than in the pixel domain. Each pseudo-mask Mtis encoded into a latent representation x0∈R64×64×4 via the VAE. Gaussian noise ϵ∼ N(0, I)is added to simulate the forward diffusion process: zt=√αtx0+√1−αtϵ, where αtis the noise schedule coefficient at timestep t. The denoising model ϵθ(zt, t), implemented via DiT, is trained to estimate the added noise: Ldiff =Ezt,t h∥ϵ−ϵθ(zt, t)∥2i.(6) The timestep tis encoded and injected into each DiT block using Adaptive LayerNorm (AdaLN) to provide explicit diffusion-step conditioning. In parallel, the VAE decoder reconstructs the mask ˆ Mtfrom x0, supervised by a reconstruction loss and a Kullback-Leibler (KL) divergence: LVAE =E  Mt−ˆ Mt   2,LKL =DKL (q(x0|Mt)∥p(x0)) , (7) where q(x0|Mt)denotes the approximate posterior, and p(x0)is the standard Gaussian prior. The total segmentation loss is formulated as: Lseg =λdiff Ldiff +λVAE LVAE +LKL,(8) with λdiff and λVAE as weighting coefficients. Diffusion Transformer Backbone. We employ DiT [35], a Transformer-based diffusion architecture operating in the latent domain. Each frame Itand pseudo-mask Mtis encoded into xcond, x0∈R64×64×4by the VAE. The corrupted latent ztis generated from x0as previously described. Each DiT block, among a total of L= 12, comprises four attention branches: (1) self-attention over ztto model internal structure, (2) cross-attention to xcond for spatial context, (3) cross-attention to VAR-derived tokens {ci}for temporal context, (4) cross-attention to the Task Token for frame quality context. All branches are modulated by AdaLN and combined through residual connections, enabling joint reasoning over noise, appearance, motion priors, and frame reliability. Unlike the morphological refinement applied to pseudo-labels before training, the diffusion module operates during inference to suppress residual noise and recover fine structures. Temporal Consistency in Semi-supervised Learning. To promote temporal coherence, we incorporate a consistency loss that aligns predictions across adjacent frames. Let Mt and Mt+1 be the predicted masks, and Ft→t+1 the optical flow: Ltemp = T−1 X t=1 ∥Mt−W(Mt+1,Ft→t+1)∥2,(9) where W(·,·)denotes a differentiable warping operator. The overall objective is: Ltotal =Lseg +Ltemp.(10) YOLO-based Mask Refinement. During inference, we apply a YOLO-based post-processing module [41] to detect anatomical regions of interest (ROIs). Predicted masks are cropped according to the detected bounding boxes, reducing false positives and improving spatial accuracy without affecting training. 4. Experiments 4.1. Dataset and Implementation Dataset Analysis. We analyze the thyroid ultrasound dataset for temporal and spatial diversity (Fig. 3). Most sequences contain 50–100 frames, with a few longer cases (Fig. 3(a)). Masks typically occupy under 5% of a frame (Fig. 3(b)), consistent with small nodules, though larger lesions appear occasionally. Aspect ratios are usually 1.0–1.5, with some up to 2.4 (Fig. 3(c)). Such variations, together with low contrast and motion blur, motivate our adaptive pseudo-labeling and latent refinement. The dataset includes 80 anonymized sequences totaling 5,334 frames, each annotated by experienced radiologists. The median length is 67 frames (IQR: 52–84). To preserve temporal continuity, we adopt a sequence-aware split, assigning all frames from the same video to a single partition, with training, validation, and test sets in a 7:1:2 ratio, adjusted to respect video boundaries. For preprocessing, we apply contrast-limited adaptive histogram equalization (CLAHE) [37] to enhance local contrast. Frames are padded for RAFT compatibility and unpadded post-inference. Explicit intensity normalization is omitted, as the VAE inherently normalizes latent features for diffusion input. 4.2. Implementation Details All experiments use PyTorch on an NVIDIA GPU (24GB VRAM). We train for 50 epochs with AdamW [28] (lr=1× 10−4, cosine annealing, batch size 8) and gradient clipping (max norm 1.0). The diffusion module follows PolypDDPM [15], encoding each frame via a VAE before latentspace denoising, with a linear noise schedule from βstart = 10−4to βend = 0.02 over 1000 steps. Pseudo-labels are produced by a frozen RAFT model via bidirectional keyframe propagation, with flow fusion weight λadaptively determined from keyframe histograms. Task Tokens are injected in each DiT block through crossattention to incorporate temporal context. A sequenceaware split avoids data leakage. At inference, the diffusion model outputs refined masks, and YOLO-based post-processing suppresses background noise and sharpens boundaries. 4.3. Model Performance Comparison We benchmark our method against a comprehensive set of state-of-the-art segmentation models, covering both convolutional and Transformer-based architectures. The evaluated baselines include UNet [38], UNet++ [62], TransUNet [5], SETR [61], and DAF [45], as well as videolevel models such as ViViT [3], STM [33], MemSAM [12], and Vivim [56]. To evaluate generalization, we conduct experiments on both our curated thyroid dataset and the pub- Figure 3. Statistical analysis of the Thyroid Ultrasound Dataset. (a) Frame count distribution across videos; (b) Distribution of mask area ratio (mask pixel count divided by image size); (c) Distribution of mask aspect ratio (width/height of mask bounding box). Red curves indicate Gaussian fits; gray dashed lines denote medians. Table 1. Quantitative comparison with state-of-the-art methods under the semi-supervised setting using RAFT-based pseudo-labels. Dice, Jaccard, Precision, and Recall are reported on our dataset and the VTUS dataset [56]. Best scores are highlighted in bold. Method Venue Type Ours VTUS Jaccard Dice Precision Recall Jaccard Dice Precision Recall UNet [38] MICCAI15 image 0.5256 0.6890 0.5919 0.8296 0.6585 0.7733 0.8145 0.8009 UNet++ [62] DLMIA18 image 0.5561 0.7147 0.6217 0.8653 0.6838 0.7962 0.8317 0.8792 TransUNet [5] arXiv21 image 0.6123 0.7595 0.7123 0.8894 0.6846 0.8093 0.8561 0.8961 SETR [61] CVPR21 image 0.6234 0.7683 0.7212 0.8915 0.6835 0.7988 0.8352 0.8764 DAF [45] MICCAI18 image 0.6048 0.7236 0.6599 0.8223 0.6766 0.7458 0.7974 0.8130 ViViT [3] ICCV21 video 0.6328 0.7728 0.7321 0.8278 0.6715 0.7996 0.8324 0.8231 STM [33] ICCV19 video 0.6296 0.7713 0.7187 0.8561 0.6701 0.8022 0.8253 0.8475 MemSAM [12] CVPR24 video 0.6393 0.7814 0.7516 0.8567 0.6972 0.8153 0.8593 0.8659 Vivim [56] TCSVT’25 video 0.6347 0.7741 0.7284 0.8721 0.7140 0.8114 0.7965 0.8838 VAR-DiT (Ours) – video 0.6472 0.7950 0.7688 0.8304 0.7162 0.8364 0.8827 0.8291 lic VTUS benchmark [56], which presents diverse clinical scenarios. All models are trained under a unified semisupervised setup using RAFT-based pseudo-labels and consistent data splits. Table 1reports the results. Our model, VAR-DiT, consistently outperforms all baselines. It achieves the best Jaccard (0.6472) and Dice (0.7950) on the internal dataset, and establishes new VTUS records in Jaccard (0.7162), Dice (0.8364), and Precision (0.8827), demonstrating strong cross-domain robustness. Compared to recent video-level methods such as MemSAM [12], ViViT [3], and Vivim [56], VAR-DiT delivers more stable and accurate segmentations, attributed to its latent denoising and autoregressive priors. Although SETR [61] and TransUNet [5] exhibit high recall, their lower precision indicates oversegmentation. VAR-DiT maintains better recall-precision balance, leading to more localized outputs. Figure 4shows representative predictions, including the ultrasound image, ground truth, and outputs from VAR-DiT and five strong baselines. Our model produces smoother, more coherent masks with sharper boundaries, especially under low contrast or morphological ambiguity, matching the observed Dice gains. This alignment between qualitative and quantitative results supports its robustness in clinically challenging conditions. 4.4. Ablation Study Model Component. To evaluate key components, we conduct an ablation on the YOLO-based post-processing module and the VAR encoder (Table 3). YOLO post-processing alone improves Jaccard and Dice by 1.0% and 1.1% by suppressing background noise and refining lesion boundaries. Adding the VAR encoder yields a further 1.3% and 1.2% improvement by capturing long-range temporal dependencies. These results confirm that spatial refinement and temporal context jointly enhance segmentation accuracy. Pseudo-label Generation. As shown in Table 2, our RAFT-based method achieves the highest performance (Jaccard: 92.73%, Dice: 96.02%) with balanced precision and recall. This benefit comes from dense, frame-aware Figure 4. Qualitative comparison of segmentation results across representative frames. Each row shows an ultrasound frame with its corresponding ground truth (GT) and predictions from six methods: our approach (VAR-DiT), DAF [45], MedSAM [30], TransUNet [5], SETR [61], and ViViT [3]. Our method demonstrates superior boundary adherence and robustness to shape variations and background noise, particularly in low-contrast or ambiguous regions, matching the observed improvements in Dice and Jaccard scores. Table 2. Comparison of different pseudo-label generation methods. The best-performing method is highlighted in bold. Pseudo-label Method Jaccard (%) Dice (%) Precision (%) Recall (%) MedicalSAM (Pre-trained) 78.74 85.14 80.32 90.28 Fine-tuned MedicalSAM 80.61 86.49 81.77 91.42 Classical CV (Two-side Flow Warp) 91.69 95.65 94.88 96.42 Distance-based Flow Fusion 87.95 93.52 90.65 96.61 Cross Pseudo Supervision (CPS) [8] 89.83 94.31 91.03 97.94 RAFT-based Optical Flow (Ours) 92.73 96.02 95.16 96.88 Table 3. Ablation study of key modules. The full model uses both YOLO post-processing and VAR encoder. Configuration Jaccard Dice Precision Recall Baseline 0.6238 0.7724 0.7450 0.8020 YOLO only 0.6341 0.7831 0.7551 0.8150 YOLO+VAR 0.6472 0.7950 0.7688 0.8304 flow and histogram-guided fusion, which preserve structural alignment even under low SNR and irregular motion. In contrast, prompt-driven MedicalSAM variants yield coarse masks, CPS [8] improves recall but blurs boundaries, and classical warping remains sensitive to motion artifacts. Overall, robust pseudo-labeling with dual-branch conditioning delivers outputs that are both quantitatively superior and temporally coherent, which is essential for real-world clinical ultrasound analysis. 5. Conclusion We propose a semi-supervised segmentation framework integrating bidirectional pseudo-label propagation, latentspace diffusion, and a Visual Autoregressive (VAR) encoder. The VAR extracts temporal context and injects autoregressive priors into Transformer blocks via crossattention, improving boundary precision and temporal consistency. By combining spatial denoising, temporal modeling, and adaptive supervision, our method tackles noise, scarce annotations, and motion-induced inconsistencies. Extensive experiments show consistent superiority over baselines across datasets and metrics, validating the effectiveness of the joint design. Acknowledgments This work is supported by the Guangdong Science and Technology Department (No. 2024ZDZX2004). References [1] Sharib Ali, Yamid Espinel, Yueming Jin, Peng Liu, Bianca Güttner, Xukun Zhang, Lihua Zhang, Tom Dowrick, Matthew J Clarkson, Shiting Xiao, et al. An objective comparison of methods for augmented reality in laparoscopic liver resection by preoperative-to-intraoperative image fusion. arXiv preprint arXiv:2401.15753, 2024. 1 [2] Sharib Ali, Yamid Espinel, Yueming Jin, Peng Liu, Bianca Güttner, Xukun Zhang, Lihua Zhang, Tom Dowrick, Matthew J Clarkson, Shiting Xiao, et al. An objective comparison of methods for augmented reality in laparoscopic liver resection by preoperative-to-intraoperative image fusion from the miccai2022 challenge. Medical image analysis, 99:103371, 2025. 1 [3] Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Luˇ ci´ c, and Cordelia Schmid. Vivit: A video vision transformer. In IEEE/CVF International Conference on Computer Vision, pages 6836–6846, 2021. 1,6,7,8 [4] Huanran Chen, Yinpeng Dong, Zhengyi Wang, Xiao Yang, Chengqi Duan, Hang Su, and Jun Zhu. Robust classification via a single diffusion model. arXiv preprint arXiv:2305.15241, 2023. 3 [5] Jieneng Chen, Yongyi Lu, Qihang Yu, Xiangde Luo, Ehsan Adeli, Yan Wang, Le Lu, Alan L Yuille, and Yuyin Zhou. Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306, 2021. 2,6,7,8 [6] Shoufa Chen, Peize Sun, Yibing Song, and Ping Luo. Diffusiondet: Diffusion model for object detection. arXiv preprint arXiv:2211.09788, 2022. 3 [7] Xi Chen, Diederik P Kingma, Tim Salimans, Yan Duan, Prafulla Dhariwal, John Schulman, Ilya Sutskever, and Pieter Abbeel. Variational lossy autoencoder. arXiv preprint arXiv:1611.02731, 2016. 5 [8] Xiaokang Chen, Yuhui Yuan, Gang Zeng, and Jingdong Wang. Semi-supervised semantic segmentation with cross pseudo supervision. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2613–2622, 2021. 8 [9] Yifei Chen, Chenyan Zhang, Yifan Ke, Yiyu Huang, Xuezhou Dai, Feiwei Qin, Yongquan Zhang, Xiaodong Zhang, and Changmiao Wang. Semi-supervised medical image segmentation method based on cross-pseudo labeling leveraging strong and weak data augmentation strategies. In 2024 IEEE International Symposium on Biomedical Imaging (ISBI), pages 1–5. IEEE, 2024. 1 [10] Ho Kei Cheng, Yu-Wing Tai, and Chi-Keung Tang. Rethinking space-time networks with improved memory coverage for efficient video object segmentation. Advances in Neural Information Processing Systems, 34:11781–11794, 2021. 3 [11] Jianning Chi, Zelan Li, Zhiyi Sun, Xiaosheng Yu, and Huan Wang. Hybrid transformer unet for thyroid segmentation from ultrasound scans. Computers in Biology and Medicine, 153:106453, 2023. 3 [12] Xiaolong Deng, Huisi Wu, Runhao Zeng, and Jing Qin. Memsam: Taming segment anything model for echocardiography video segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9622–9631, 2024. 3,6,7 [13] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780–8794, 2021. 3 [14] Min Dong, Ating Yang, Zhenhang Wang, Dezhen Li, Jing Yang, and Rongchang Zhao. Uncertainty-aware consistency learning for semi-supervised medical image segmentation. Knowledge-Based Systems, 309:112890, 2025. 3 [15] Zolnamar Dorjsembe, Hsing-Kuo Pao, and Furen Xiao. Polyp-ddpm: Diffusion-based semantic polyp synthesis for enhanced segmentation. In 2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pages 1–7. IEEE, 2024. 6 [16] Junkai Fan, Jiangwei Weng, Kun Wang, Yijun Yang, Jianjun Qian, Jun Li, and Jian Yang. Driving-video dehazing with non-aligned regularization for safety assistance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26109–26119, 2024. 3 [17] Yunqi Gu, Tao Zhou, Yizhe Zhang, Yi Zhou, Kelei He, Chen Gong, and Huazhu Fu. Dual-scale enhanced and crossgenerative consistency learning for semi-supervised medical image segmentation. Pattern Recognition, 158:110962, 2025. 3 [18] Xizewen Han, Huangjie Zheng, and Mingyuan Zhou. Card: Classification and regression diffusion models. arXiv preprint arXiv:2206.07275, 2022. 3 [19] Ali Hatamizadeh, Yucheng Tang, Vishwesh Nath, Dong Yang, Andriy Myronenko, Bennett Landman, Holger R Roth, and Daguang Xu. Unetr: Transformers for 3d medical image segmentation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 574–584, 2022. 2 [20] Yufan He, Vishwesh Nath, Dong Yang, Yucheng Tang, Andriy Myronenko, and Daguang Xu. Swinunetr-v2: Stronger swin transformers with stagewise convolutions for 3d medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 416–426. Springer, 2023. 2 [21] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020. 3 [22] Reda Abdellah Kamraoui, Vinh-Thong Ta, Nicolas Papadakis, Fanny Compaire, José V Manjon, and Pierrick Coupé. Popcorn: Progressive pseudo-labeling with consistency regularization and neighboring. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part II 24, pages 373–382. Springer, 2021. 3 [23] Jialu Li, Qingqing Zheng, Mingshuang Li, Ping Liu, Qiong Wang, Litao Sun, and Lei Zhu. Rethinking breast lesion segmentation in ultrasound: A new video dataset and a baseline network. In MICCAI, pages 391–400. Springer, 2022. 3 [24] Yuexiang Li, Jiawei Chen, Xinpeng Xie, Kai Ma, and Yefeng Zheng. Self-loop uncertainty: A novel pseudo-label for semi-supervised medical image segmentation. In Medi-