Speaker
Ilya is a third year PhD Student in the Princeton University Computational Imaging Lab, advised by Professor Felix Heide, and is an NSF graduate research fellow. His work is at the intersection of computational photography, depth estimation, and designed optics research, modelling the image acquisition pipeline from signal collection to scene reconstruction.
Abstract
Modern smartphones can continuously stream 12-megapixel 14-bit RAW images, LiDAR-driven low-resolution depth maps, accelerometer, and gyroscope measurements, estimated lens focal lengths, and a wide array of other camera metadata. Yet, most research treats smartphone images as just two-dimensional 8-bit RGB arrays and burst photography pipelines only looks at a couple of extra frames, still treating pixel motion as a 2D alignment problem. In our work we look at what we can extract from a “long-burst”, forty-two frames captured in a two-second sequence. We find there is enough parallax information from natural hand tremor alone to recover high-quality scene depth. To this end we propose fitting a simple neural RGB-D scene model directly to this long-burst data to jointly estimate depth and camera motion, with no disjoint pre-processing steps, no feature extraction, and no cost volume estimation.