Header logo is

Bilinear Spatiotemporal Basis Models




A variety of dynamic objects, such as faces, bodies, and cloth, are represented in computer graphics as a collection of moving spatial landmarks. Spatiotemporal data is inherent in a number of graphics applications including animation, simulation, and object and camera tracking. The principal modes of variation in the spatial geometry of objects are typically modeled using dimensionality reduction techniques, while concurrently, trajectory representations like splines and autoregressive models are widely used to exploit the temporal regularity of deformation. In this article, we present the bilinear spatiotemporal basis as a model that simultaneously exploits spatial and temporal regularity while maintaining the ability to generalize well to new sequences. This factorization allows the use of analytical, predefined functions to represent temporal variation (e.g., B-Splines or the Discrete Cosine Transform) resulting in efficient model representation and estimation. The model can be interpreted as representing the data as a linear combination of spatiotemporal sequences consisting of shape modes oscillating over time at key frequencies. We apply the bilinear model to natural spatiotemporal phenomena, including face, body, and cloth motion data, and compare it in terms of compaction, generalization ability, predictive precision, and efficiency to existing models. We demonstrate the application of the model to a number of graphics tasks including labeling, gap-filling, denoising, and motion touch-up.

Author(s): Akhter, Ijaz and Simon, Tomas and Khan, Sohaib and Matthews, Iain and Sheikh, Yaser
Journal: ACM Transactions on Graphics (TOG)
Volume: 31
Number (issue): 2
Pages: 17
Year: 2012
Month: April
Publisher: ACM

Department(s): Perceiving Systems
Bibtex Type: Article (article)
Paper Type: Journal

URL: http://dl.acm.org/citation.cfm?id=2159523

Links: pdf
project page


  title = {Bilinear Spatiotemporal Basis Models},
  author = {Akhter, Ijaz and Simon, Tomas and Khan, Sohaib and Matthews, Iain and Sheikh, Yaser},
  journal = {ACM Transactions on Graphics (TOG)},
  volume = {31},
  number = {2},
  pages = {17},
  publisher = {ACM},
  month = apr,
  year = {2012},
  url = {http://dl.acm.org/citation.cfm?id=2159523},
  month_numeric = {4}