Procedural activity videos often exhibit a long-tailed action distribution due to varying action frequencies and durations. However, state-of-the-art temporal action segmentation methods overlook the long tail and fail to recognize tail actions. Existing long-tail methods make class-independent assumptions and struggle to identify tail classes when applied to temporal segmentation frameworks. This work proposes a novel group-wise temporal logit adjustment (G-TLA) framework that combines a group-wise softmax formulation while leveraging activity information and action ordering for logit adjustment. The proposed framework significantly improves in segmenting tail actions without any performance loss on head actions.
Our proposed Group-wise Temporal Logit Adjustment. (1) Group-wise classification, leveraging class co-occurrence priors to mitigate activity irrelevent false positives. (2) Temporal logit adjustment, leveraging action ordering priors to mitigate temporal illogical false positives.
Our method is evaluated on several datasets and metrics. Our work shows good performance on (1) tail class performance without sacrificing head performance. (2) per class performance as well as global performance. (3) good head-tail and frame-segment performance trade-offs
@article{pang2024long,
title={Long-Tail Temporal Action Segmentation with Group-wise Temporal Logit Adjustment},
author={Zhanzhong Pang and Fadime Sener and Shrinivas Ramasubramanian and Angela Yao},
journal={arXiv preprint arXiv:2408.09919},
year={2024}}