2/5/2025 | 8:50 AM - 10:30 AM | Grand Peninsula A

Optimizing frame selection for improved video quality assessment through embedding similarity

Author(s)

Abderrezzaq Sendjasni | CNRS, Univ. Poitiers, XLIM, UMR 7252
Mohamed-Chaker Larabi | CNRS, Univ. Poitiers, XLIM, UMR 7252
Seif-Eddine Benkabou | LIAS, Univ. Poitiers

Abstract

No-reference (NR) perceptual video quality assessment (VQA) is a complex, unsolved, and important problem for social and streaming media applications. Efficient and accurate video quality predictors are needed to monitor and guide the processing of billions of shared, often imperfect, videos. Accurate VQA requires a strategic selection of frames that effectively represent overall perceptual quality without compromising computational efficiency. Traditional VQA methods, which often rely on uniform or random sampling, risk neglecting critical temporal features affecting perceived quality. In this work, we propose a similarity-preserving technique for frame selection, designed to maintain frame-level consistency and preserve perceptually relevant features throughout video sequences. The proposed selection preserves structural and semantic similarities within selected frames by analyzing the similarity among frames in the embedding space.

To this end, frame embeddings are extracted using ResNet-50, capturing high-level visual representations. The selection algorithm then evaluates similarity among embeddings based on distance metrics and residual learning to identify perceptually important frames. Key frames are ranked and retained according to their contribution to the quality assessment task, with emphasis on those that exhibit significant variations in perceptual features.