August 30, 2006 - Following this month’s 33rd Annual Siggraph Conference in Boston, MA, a research team at Mitsubishi Electric is catching the attention of camera manufacturers for their photo motion deblurring technology, called a flutter shutter camera.
The flutter shutter camera is a modified camera that can capture moving objects at an exposure time of over 50 milliseconds, like high speed motion cameras. Using a coded exposure sequence, the new flutter shutter camera could recover text from a speeding car and sharpen images, according to the researchers.
Introduced in early August, three Mitsubishi Electric researchers presented the abstract, “Coded Exposure Photography: Motion Deblurring using Fluttered Shutter” at the largest computer and graphics conference, Siggraph. After one year of research development, Mitsubishi Electric Research Lab (MERL) senior researcher Ramesh Raskar, MERL visiting researcher Amit Agrawal, and Northwestern University computer science assistant professor Jack Tumblin launched the new prototype with the goal of deblurring photos.
The prototype is made with an 8 megapixel Canon PowerShot Pro1, although it could be applied to any camera. Instead of leaving the shutter open during one exposure duration, the camera’s attached lens filter flutters the shutter multiple times during a single exposure, based on a carefully chosen binary sequence.
Raskar, who celebrates his sixth year at Mitsubishi this month, woke up one day with the idea, according to the researcher. Raskar said the fluttered shutter method was “so simple” and wondered if it could work.
Above: Mitsubishi Electric Research Lab (MERL) senior researcher Ramesh Raskar with a flutter shutter prototype |
“We have UV filters. We have polarizing filters. What about time filters?” said senior research scientist Ramesh Raskar at Mitsubishi Electric. Just a few weeks ago, Raskar and the Mitsubishi team saw the dream actualized with its official introduction. This time filter is made up of an external ferro-electric shutter that flutters based on a rapid binary sequence.
Traditional cameras typically have a single shuttered exposure in which moving subjects result in blurry images. On standard cameras, object motion can be described by convolution of a sharp image with a temporal box filter, according to researchers, thus destroying high frequency spatial details in the image. The coded exposure camera, on the other hand, uses a method called deconvolution, in which the convolution filter changes to a broadband filter and preserves spatial frequencies and image detail.
A post-capture linear system algorithm is then applied to recover image sharpness. The algorithm is “fundamentally different from other deblurring algorithms,” stated authors of the abstract. The algorithm, Ax=b (a Matlab Code), is the simplest deconvolution algorithm possible, according to researchers. Unlike Photoshop or other deblurring methods, this deconvolution does not result in ringing or deconvolution artifacts, halo-like distortions on the image.
In light of a deblurring technology released by Massachusetts Institute of Technology at Siggraph, Mitsubishi has spoken with the group led by principal scientist Rob Fergus of MIT. [For more information, refer to
http://www.digitalcamerainfo.com/content/Photo-deblurring-Research-Debuts-at-Siggraph-Conference-.htm.] The “best solution,” as Raskar states, is if the two Cambridge, MA research teams could combine anti-shake and motion deblurring methods.
Based on the abstract “Removing Camera Shake from a Single Photograph,” the MIT method handles hand-camera shake. The coded exposure camera method, on the other hand, can correct extreme amounts of motion blur from moving objects, according to Raskar. “There has not been a photo in the world that has that much blur and has been brought back to recognition,” said Raskar about the resulting images from the fluttered shutter camera.
The fluttered shutter camera and algorithm can handle such extreme blurs that it can retrieve details from moving cars. With possible applications in law enforcement, the police could record a plate number from a speeding vehicle using the fluttered shutter camera.
The camera can even capture images from a flying plane, according to Raskar. With large-scale aerial photography applications, such as Google Maps, the camera can use an aperture ten times smaller than currently used on the expensive aerial cameras. For planes mapping the entire earth, this would reduce time and camera costs significantly.
Some may wonder why not use a short shutter speed, such as those on action and sport modes for shooting fast-moving objects. The difference, as Raskar explains, the action-shot
shutter speed needs a large aperture, whereas the fluttered shutter camera does not.
The fluttered shutter could be implemented on any “off-the-shelf camera” with only a slight modification, according to authors of the abstract. The universal fluttered shutter also could be installed internally within the camera, making the external lens filter unnecessary. Although the Mitsubishi team is consulting with camera manufacturers about future plans, Raskar anticipates it will take a few years before photo enthusiasts will see the technology in consumer cameras. While the prototype costs roughly $500, the researcher predicts the equipment for commercial cameras would only cost tens of dollars when it does become commercialized.
“It always takes time to go into consumer photography,” said Raskar.
The Mitsubishi Electric team is also working on a project for correcting out-of-focus blur, although Raskar could not disclose details.
For additional information, go to:
- http://www.cfar.umd.edu/~aagrawal/sig06/sig06.html
- http://www.merl.com/people/raskar/deblur/
