A Multiview Synthesis Core in 65 nm CMOS
In order to overcome limitations of current three dimensional video systems, such as the necessity to wear glasses, next generation 3D relies on autostereoscopic displays. Simple autostereoscopic displays force the user to a very specific position in front of the display. To overcome this problem of strong spatial confinement and to allow several persons to simultaneously use the same display, modern displays – so called multiview autostereoscopic displays (MADs) – use multiple views of the same scene. However, a larger number of views (currently 8-9) renders content creation an increasingly difficult process, and results in higher transmission bandwith and storage requirements. Moreover, the number of views and the post processing for MADs are highly display-type dependent. Therefore, so called multiview synthesis techniques have gained increased interest over the last years. The idea is to generate the large number of views from a small subset, e.g. from stereo 3D footage.
The ASIC developed in this project implements an image domain warping (IDW) based rendering stage compliant with the multiview synthesis framework developed at Disney Research Zurich. The chip receives two 1080p images plus two-dimensional transformation data (image warps), generates the required views and interleaves them appropriately into one 1080p output image which can directly be displayed on a MAD. It supports a maximum number of 9 views and has a throughput of 44.2 interleaved frames per second with 8 enabled views. The rendering core operates at 400 MHz and the IOs at 100 MHz. Parameters, such as the view positions, the display interleaving pattern and the view-to-display sub-pixel allocation, are fully programmable. Assumptions on the vertical displacement of image pixels allowed to design a hardware architecture that does not require an off-chip memory. The complexity of the chip is 6.75 MGE (including 4.36 MBit SRAM macros).
- M. Schaffner, P. Greisen, S. Heinzle, F. K. Gürkaynak, H. Kaeslin, A. Smolic, "MADmax: A 1080p Stereo-to-Multiview Rendering ASIC in 65 nm CMOS based on Image Domain Warping.", Proceedings of the European Solid-State Circuits Conference (ESSCIRC), Bucharest, Romania, 2013
- M. Schaffner, P. Greisen, S. Heinzle, A. Smolic, "Efficient image resampling for multiview displays.", Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Vancouver, Canada 2013