Here you can find some of my work grouped according to topics. Use the publications link for a complete list of papers.

Immersive Video, Cinematic Virtual Reality

Keywords: VR/AR, Omnidirectional Video, Immersive Experience

Historically, head-mounted displays (HMDs) are usually associated with gaming applications and computer-generated content. However, recent developments in video capture and processing enable us to create media-based immersive experiences involving real-world scenes. We refer to such applications as ‘‘Cinematic VR.’’ To achieve high levels of immersion, we need high resolution, wide field-of-view, high frame rate, adjustable stereo baseline, head motion parallax, etc. This opens up interesting questions on content representation and compression, and may need fundamental redesign of the media pipeline in the context of VR/AR.

Immersive video applications aim to revolutionize the viewing experience from passive watching to active engagement of the audience by choosing what or where they want to view within the scene. We are at the confluence of various tech trends in acquisition (high-res sensors, camera arrays, etc.) and display devices (Oculus Rift, Go, Vive, etc.) where such immersive experiences may become realizable in the near future.

We focus on novel video processing algorithms for different stages of the VR/AR pipeline, such as, capture & postproduction of content, compression & delivery to a remote user, and rendering on user’s device.

• M. Yu, H. Lakshman, B. Girod,
  Content Adaptive Representations of Omnidirectional Videos for Cinematic Virtual Reality [pdf]
  ACM Multimedia Workshop on Immersive Media Experiences, Brisbane, Australia, November 2015.
  

• M. Yu, H. Lakshman, B. Girod,
  A Framework to Evaluate Omnidirectional Video Coding Schemes [pdf]
  IEEE International Symposium on Mixed and Augmented Reality, Fukuoka, Japan, September 2015.
  

• J. Thatte, J.-B. Boin, H. Lakshman, B. Girod,
  Depth Augmented Stereo Panorama for Cinematic Virtual Reality with Head Motion Parallax [pdf]
  IEEE International Conference on Multimedia and Expo, Seattle, USA, July 2016.
  

• J. Thatte, J.-B. Boin, H. Lakshman, G. Wetzstein, B. Girod,
  Depth Augmented Stereo Panorama for Cinematic Virtual Reality with Focus Cues [pdf]
  IEEE International Conference on Image Processing, Phoenix, USA, September 2016.
  

Image/Video processing

• H. Lakshman, W.-Q Lim, H. Schwarz, D. Marpe, G. Kutyniok, T. Wiegand,
  Image Interpolation using Shearlet based Sparsity Priors [pdf]
  IEEE International Conference on Image Processing, Melbourne, Australia, September 2013.
  

• H. Lakshman, M. Koeppel, P. Ndjiki-Nya, and T. Wiegand,
  Image Recovery using Sparse Reconstruction based Texture Refinement [pdf]
  IEEE International Conference on Acoustics, Speech, and Signal Processing, Dallas, TX, USA, March 2010.

• H. Lakshman, P. Ndjiki-Nya, M. Koeppel, D. Doshkov, and T. Wiegand,
  An Automatic Structure-Aware Image Extrapolation Applied to Error Concealment [pdf]
  IEEE International Conference on Image Processing, Cairo, Egypt, November 2009.

3D and Multi-view video

• M. Farre, H. Lakshman, P. Helle, H. Schwarz, K. Mueller, D. Marpe and T. Wiegand,
  Edge aware disparity estimation for intermediate view synthesis [pdf]
  Picture Coding Symposium, San Jose, December 2013.

• K. Mueller, H. Schwarz, D. Marpe, C. Bartnik, S. Bosse, H. Brust, T. Hinz, H. Lakshman, P. Merkle, H. F. Rhee, G. Tech, M. Winken, and T. Wiegand,
  3D High Efficiency Video Coding for Multi-View Video and Depth Data [pdf]
  IEEE Transactions on Image Processing, Special Issue on 3D Video Representation, vol. 22, no. 9, pp. 3366-3378, September 2013.

• P. Ndjiki-Nya, M. Koeppel, H. Lakshman, P. Merkle, K. Mueller, T. Wiegand,
  Depth Image-based Rendering with Advanced Texture Synthesis for 3D Video [pdf]
  Transactions on Multimedia, June 2011.
  2013 IEEE Communications Society MMTC Best Journal Paper Award

Video coding, Scalable coding

• P. Helle, H. Lakshman, M. Siekmann, J. Stegemann, T. Hinz, H. Schwarz, D. Marpe, and T. Wiegand,
  A Scalable Video Coding Extension of HEVC [pdf]
  Data Compression Conference, Snowbird, Utah, March 2013.

• H. Lakshman, H. Schwarz, T. Wiegand,
  Generalized Interpolation Based Fractional Sample Motion Compensation [pdf]
  IEEE Transactions on Circuits and Systems for Video Technology, vol. 23, no. 3, pp. 455 - 466, March 2013.

• H. Lakshman, H. Schwarz, T. Blu, and T. Wiegand,
  Generalized Interpolation for Motion Compensated Prediction [pdf]
  IEEE International Conference on Image Processing, September 2011.
  Invited Paper

• H. Lakshman, B. Bross, H. Schwarz, and T. Wiegand,
  Fractional-Sample Motion Compensation using Generalized Interpolation [pdf]
  Picture Coding Symposium, Nagoya, Japan, December 2010.

• H. Lakshman, H. Schwarz, and T. Wiegand,
  Adaptive Motion Model Selection using a Cubic Spline based Estimation Framework [pdf]
  IEEE International Conference on Image Processing, Hong Kong, September 2010.
  Best student paper award finalist

Image retrieval

• A. F. de Araujo, F. Silveira, H. Lakshman, J. Zepeda, A. Sheth, P. Perez and B. Girod,
  The Stanford/Technicolor/Fraunhofer HHI Video Semantic Indexing [pdf]
  TRECVID2012 Workshop, November 2012.

• M. Makar, H. Lakshman, V. Chandrasekhar and B. Girod,
  Gradient Preserving Quantization [pdf]
  IEEE International Conference on Image Processing, Orlando, USA, September 2012.
  Best student paper award finalist

Invited Talks

• “Quality Aspects in Media-based VR Experiences,” ET@2018, Google Event Center, Sunnyvale, Jun. 2018

• “Towards Full-Parallax Immersive Video Experiences,” Electronic Imaging, Burlingame, Jan. 2018

• “Panel Discussion on Cinematic VR,” ETCA, Mountain View, May 2017

• Workshop on Cinematic VR and Immersive Storytelling, May 2016

• “Data Representations for Cinematic Virtual Reality,” SCIEN Talk, Stanford, USA, Mar. 2016

• “Immersive Video Formats for Cinematic VR,” AhG on Light Fields, San Diego, USA, Feb. 2016

• “Compact Video Representations for Cinematic Virtual Reality,” MPEG L3.1, Netflix, Los Gatos, Dec. 2015

• “Cinematic Virtual Reality: Creating Immersive Visual Experiences,” University of California, Santa Barbara, USA, Oct. 2015

• “Interpolation for Coding and Upsampling of Video Signals,” Peking University-Stanford Workshop, Sep. 2014

• “Camera Image Processing for Rescaling,” ARRI Tech seminar, Jul. 2012

• “Recent Advances in Video Coding,” Indian Institute of Science, May 2012

• “Spline based Representation for Video Coding,” Max Planck Center seminar, Feb. 2012

• “Fraunhofer HHI's contribution to the ongoing High Efficiency Video Coding standardization - Prediction Methods,” ITG Germany, Jul. 2010

Awards & Honors

• Conference Best Paper Award for Industry, IEEE Signal Processing Society, 2016

• Visual Computing and Communication Fellowship, Max Planck Center, 2014

• Best Journal Paper Award, IEEE Communications Society, 2013

• Best Student Paper Award Finalist, IEEE ICIP, 2012

• Best Student Paper Award Finalist, IEEE ICIP, 2010

• Siemens Fellowship, 2007