Light Field Superresolution
 |
 |
 |
 |
[Light field image] |
[Light field (left) rearranged as multiple views] |
[Central view extracted from the light field] |
[Central view superresolved] |
Abstract
Light field cameras have been recently shown to be very
effective in applications such as digital refocusing and 3D
reconstruction. In a single snapshot these cameras provide
a sample of the light field of a scene by trading off spatial
resolution with angular resolution. Current methods produce
images at a resolution that is much lower than that
of traditional imaging devices. However, by explicitly modeling
the image formation process and incorporating priors
such as Lambertianity and texture statistics, these types
of images can be reconstructed at a higher resolution. We
formulate this method in a variational Bayesian framework
and perform the reconstruction of both the surface of the
scene and the (superresolved) light field. The method is
demonstrated on both synthetic and real images captured
with our light-field camera prototype.
Download
Related work
-
E. H. Adelson and J. Y. Wang, "Single lens stereo with a plenoptic camera", IEEE J. PAMI, 14(2):99-106, Feb 1992.
-
M. Ben-Ezra, A. Zomet, and S. Nayar, "Jitter camera: high resolution video from a low resolution detector", In Proc. CVPR 2004, volume 2, pages 135-142, 2004.
-
T. E. Bishop, "Blind Image Deconvolution: Nonstationary Bayesian approaches to restoring blurred photos", PhD thesis, University of Edinburgh, 2008.
-
T. E. Bishop, R. Molina, and J. R. Hopgood, "Blind restoration of blurred photographs via AR modelling and MCMC", In IEEE International Conference on Image Processing (ICIP), 2008.
-
S. Borman and R. Stevenson, "Super-resolution from image sequences - a review", In Midwest Symposium on Circuits and Systems, pages 374-378., 1999.
-
M. Born and E. Wolf, "Principles of Optics", Pergamon, 1986.
-
J.-X. Chai, S.-C. Chan, H.-Y. Shum, and X. Tong, "Plenoptic sampling", In SIGGRAPH '00: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, pages 307-318, New York, NY, USA, 2000. ACM Press/Addison-Wesley Publishing Co.
-
W.-S. Chan, E. Lam, M. Ng, and G. Mak, "Superresolution reconstruction in a computational compound-eye imaging system", Multidimensional Systems and Signal Processing, 18(2):83-101, Sept. 2007.
-
M. Elad and A. Feuer, "Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images", IEEE transactions on image processing, 6:1646-1658, 1997.
-
S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in super-resolution", International Journal of Imaging Systems and Technology, 14:47-57, 2004.
-
K. Fife, A. El Gamal, and H.-S. Wong, "A 3d multi-aperture image sensor architecture", In Custom Integrated Circuits Conference, 2006. CICC '06. IEEE, pages 281-284, 2006.
-
T. Georgeiv and C. Intwala, "Light field camera design for integral view photography", Technical report, Adobe Systems Incorporated 345 Park Ave, San Jose, CA 95110, 2006.
-
B. Ha and Y. Li, "Scaling laws for lens systems", Applied Optics, 28:4996-4998, 1989.
-
B. R. Hunt, "Super-resolution of images: Algorithms, principles, performance", International Journal of Imaging Systems and Technology, 6 Issue 4:297-304, 2005.
-
A. Isaksen, L. McMillan, and S. J. Gortler, "Dynamically reparameterized light fields", In K. Akeley, editor, Proc. SIGGRAPH '00, pages 297-306. ACM Press / ACM SIGGRAPH / Addison Wesley Longman, 2000.
-
A. Katsaggelos, R. Molina, and J. Mateos, "Super resolution of images and video", Synthesis Lectures on Image, Video, and Multimedia Processing. Morgan & Claypool, 2007.
-
A. Levin, W. T. Freeman, and F. Durand, "Understanding camera trade-offs through a Bayesian analysis of light field projections", In European Conference on Computer Vision, number 14, pages 619-624, 2008.
-
M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, "Light field microscopy", ACM Trans. Graph., 25(3):924-934, 2006.
-
C.-K. Liang, G. Liu, and H. H. Chen, "Light field acquisition using programmable aperture camera", In IEEE International Conference on Image Processing (ICIP), 2007.
-
G. Lippmann, "Epreuves reversibles donnant la sensation du relief", Journal of Physics, 7(4):821-825, 1908.
-
A. Lumsdaine and T. Georgiev, "Full resolution lightfield rendering", Technical report, Indiana University and Adobe Systems, 2008.
-
M. K. Ng and A. C. Yau, "Super-resolution image restoration from blurred low-resolution images", In Journal of Mathematical Imaging and Vision, volume 23, pages 367-378, Norwell, MA, USA, 2005. Kluwer Academic Publishers.
-
R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, "Light field photography with a hand-held plenoptic camera", Technical Report CSTR 2005-02, Stanford University, April 2005.
-
S. C. Park, M. K. Park, and M. G. Kang, "Super-resolution image reconstruction: A technical overview", IEEE Signal Processing Magazine, 20(3):21-36, 2003.
-
E. P. Simoncelli, "Statistical modeling of photographic images", In A. Bovik, editor, Handbook of Image and Video Processing, 2nd Edition, chapter 4.7. Academic Press, 2 edition, Jan 2005.
-
J. Stewart, J. Yu, S. J. Gortler, and L. McMillan, "A new reconstruction filter for undersampled light fields", In EGRW '03: Proceedings of the 14th Eurographics workshop on Rendering, pages 150-156,
2003.
-
V. Vaish, M. Levoy, R. Szeliski, C. Zitnick, and S. B. Kang, "Reconstructing occluded surfaces using synthetic apertures: Stereo, focus and robust measures", Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 2, pages 2331-2338, 2006.
-
A. Veeraraghavan, R. Raskar, A. K. Agrawal, A. Mohan, and J. Tumblin, "Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing", ACM Trans. Graph. (Proc SIGGRAPH 2007), 26(3):69, 2007.
-
G. J. Ward, "Measuring and modeling anisotropic reflection", SIGGRAPH '92: Proceedings of the 19th annual conference on Computer graphics and interactive techniques, pages 265-272, New York, NY, USA, 1992. ACM.
Acknowledgments
We wish to thank Mohammad Taghizadeh and the
diffractive optics group at Heriot-Watt University for providing
us with the microlens arrays and for stimulating discussions,
and Mark Stewart for designing and building our
microlens array interface. This work has been supported by
EPSRC grant EP/F023073/1(P).
Last update: January 26 2009