Gaussian Video Transformer advances video representation with 2DGS tokenization

Researchers have developed the Gaussian Video Transformer (GVT), a novel framework for video representation that utilizes a feed-forward 2D Gaussian Splatting (2DGS) tokenization scheme. This approach enhances spatial adaptability by dynamically assigning rendering weights based on information content and improves generalization by avoiding per-video optimization. The GVT also incorporates a Gaussian Set Partitioning strategy to separate static and dynamic content, enabling more compact representations. Evaluations across video reconstruction, action recognition, compression, and generation tasks show state-of-the-art performance in reconstruction and compression, with competitive results in other areas. AI

IMPACT Introduces a novel tokenization method for video representation that improves efficiency and performance across multiple video tasks.

RANK_REASON Research paper detailing a new method for video representation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• Gaussian Set Partitioning
• Gaussian Video Transformer
• kinetics
• MAGVIT-v2
• Spatio-Temporal Gaussian Embedding
• Ucf101
• University of California, Davis
• Zhenghao Chen

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →