Gaze-driven Video Re-editing

Given the current profusion of devices for viewing media, video content created at one aspect ratio is often viewed on displays with different aspect ratios. Many previous solutions address this problem by retargeting or resizing the video, but a more general solution would re-edit the video for the new display. Our method employs the three primary editing operations: pan, cut, and zoom. We let viewers implicitly reveal what is important in a video by tracking their gaze as they watch the video. We present an algorithm that optimizes the path of a cropping window based on the collected eyetracking data, finds places to cut, and computes the size of the cropping window. We present results on a variety of video clips. We conduct two experiments to evaluate our results. First, we eyetrack viewers on the result videos generated by our algorithm, and second, we perform a subjective assessment of viewer preference. These experiments show that viewer gaze patterns are similar on our result videos and on the original videos, and that viewers prefer our results to a state-of-the-art algorithm.

People

• Eakta Jain (Carnegie Mellon University, Disney Research Pittsburgh, University of Florida)
• Yaser Sheikh (Carnegie Mellon University)
• Ariel Shamir (Interdisciplinary Center Israel, Disney Research Pittsburgh)
• Jessica Hodgins (Carnegie Mellon University, Disney Research Pittsburgh)

Publications

"Gaze-driven Video Re-editing", Eakta Jain, Yaser Sheikh, Ariel Shamir, Jessica Hodgins. ACM Transactions on Graphics (TOG), 2014.

• Paper (preprint) (PDF)
• Bibtex entry:
  

  @article{jain:2014,
  author = {Jain, Eakta and Sheikh, Yaser and Shamir, Ariel and Hodgins, Jessica},
  title = {Gaze-driven Video Re-editing},
  journal = {ACM Transactions on Graphics},
  year = {2014}
  }
  

More downloads

We provide our eyetracking data to enable the community to further research in this area. However, we do not own the copyright for the videos, so we can only provide the timestamps of the video clips here. We are happy to answer questions.

• Eyetracking data as text files (4.4MB)
• Timestamps

Acknowledgements

We thank The Walt Disney Company for permission to use the video clips in this project.

Copyright notice

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Comments, questions to Eakta Jain.