Read about the 13-gigapixel panorama of Harlem, created by Gerard Maynard and Autopano Pro.
May 14, 2007 - One sunny summer afternoon in Harlem, the pulse of the city came to a standstill 2,045 times. A camera perched on the roof of a building at 7th Avenue and 110th Street in New York City panned and tilted, capturing the skyline from 4:43 p.m. to 6:53 p.m.
The man behind the camera was Gerard Maynard, a Harlem, New York-based artist. It is apparent from his online portfolio that Maynard is an ideas man. It features a collection of large panoramas (ranging in size from 317 megapixels to 2.157 gigapixels), abstract paintings, and an eye-cross-inducing abstract video of one of his paintings taking off from the canvas and swirling across the room. The 13-gigapixel panorama evolved from an idea.
"I have been constructing large photo images for the past 4-5 years. I was working on large 200+ images, when Doug Lyons made the first stitched gigapixel image," Maynard said in an interview with DigitalCameraInfo.com. "Everything is experimental in all the images, the photos may work or may not work. What is most important is that I am pushing myself and technology with each image. The point is to do a visual experiment and share the results as a completed photo… [I just wanted to] try to push the envelope. It could have been 11 or 14 gigapixels."
Armed with his idea and images, Maynard took to the Internet and Googled around for a software company that might be willing to take on the challenge of stitching together his 2,045 photos. He found Alexandre Jenny of Kolor Company, a France-based software development company that specializes in panorama stitching tools. Its main product is the Autopano Pro software.
"In early October 2006 Alexandre was asking for images to improve his software, so I submitted 600 photos to his FTP server," Maynard said. "Soon after, I told him that there were another 1,400 images. By the end of October we had decided to stitch the image, render it and share it on the Internet."
Almost 9 months after taking capturing the Harlem skyline, Maynard's idea came to fruition when Harlem-13-Gigapixels.org, a website that displays the panorama, launched on May 10th.
**For Maynard to turn his camera on Harlem was timely and logical. For many years Harlem was considered a slum of New York, but in recent years, it has experienced a revolution.
"I live in Harlem. I have made quite a few images of the Harlem skyline over the past 4 years. I lived in the building on 110th Street, where the photos were taken from. The Harlem skyline is constantly changing," Maynard said. "The image is about the present, the past and the possibilities of the future. The current Harlem skyline is ragged but full of promise. It has a deep history that can be lost if not remembered."
On August 13, 2006, Maynard headed to the roof armed with a Nikon D2X, a 300mm Nikkor lens and a modified version of Peace River Studio’s PixOrb.
PixOrb is a motorized, programmable camera head used for panoramic and spherical photography. Maynard had worked with Peace River Studios in the past to modify equipment to curtail it to the needs of his projects.
"For this capture, the first issue was to optically center the 300mm lens. I used a 30-pound counter weight to compensate for the forward position of the lens," Maynard said. "There were about 4 seconds between each shutter release. The counter weight also helped minimize vibration. I had to reinforce the extruded aluminum structure of PixOrb to further dampen vibration."
Maynard noted the most significant modification was to the PixOrb software, allowing it to pan 2 degrees and tilt 3 degrees to capture each 12-megapixel image.
"Phil Lockwood and I developed [the software] to allow the capture of different types of arrays. Left to right, top to bottom and its inverse. Top to bottom left to right and its inverse," Maynard said. "Phil even came up with spiral format. These modifications can make some very interesting images. It allows me to control time in relation to the capture of the panorama."
Maynard noted that the view of Harlem’s surrounding skyline has changed even thought it’s only been nine months since he took the photos.
"There are 12-story buildings in Central Park North. [The photo] is a snapshot of Harlem was that day," he said.
After capturing the images, Maynard burned three dual layer DVDs of the images and shipped them to France.
**Autopano Pro’s workflow is two-fold. It detects and optimizes the image, and then it renders the image. The user begins by setting the software loose on a folder of images. Using SIFT patented algorithm, it digs through and analyzes the images, keeping only those necessary for the panorama. The program then automatically stitches the images together.
"Autopano Pro is the only software that can find which pictures belong to a panorama and which ones don't belong to a panorama. You can analyze a folder with hundred photos in one click," said Jenny. "It's the only software that corrects color variation between shots by solving the real photometric equation. And it was the first one to do HDR."
Users also have the option to manually select images to be analyzed.
After the panorama has been stitched together, the software applies color correction to adjust for color and exposure variances among the images. Color corrections can be made automatically and manually.
The Kolor team worked with Dr. David Lowe and Matthew Brown of the University of British Columbia, Vancouver to modify the company’s Autopano Pro software to handle the size of the panorama’s vast size. This addresses two challenges. First, the blue sky’s lack of focus and details made it difficult for the software to establish features.
"A custom algorithm [had to be] designed to solve this: it analyzes how the shooting was done and finds a good average location of unlinked pictures," Jenny said. "There are 2,045 pictures in the panorama. Around 1,400 pictures have features, and 600 don't. Using the custom algorithm, only 15 pictures had to be moved manually to the right position instead of 600."
It took one day for the software to detect and optimize the 2,045 images.
The second challenge was the color correction engine.
"We have improved our color correction engine to be able to support such a large picture set, and it's now working perfectly even on such a large panorama," stated a company press release.
Before [the modification], it could take 1 hour for the software to solve color correction for 300 pictures. It now takes 5 minutes for the software to color correct 2,000 pictures. This is a huge improvement, explained Jenny.
The second step of the workflow is rendering. The team employed souped-up hardware to render the panorama.
"The rendering was done with a 64bit Linux version of the software. We set up special hardware to do the rendering: dual xeon quad-core processor with 8GB memory and 2 high-speed 150GB hard drives," according to a company press release.
It took 46 hours to render the 13-Gigapixel panorama. During the two days, Jenny and his team would eagerly check its status.
"…When we arrived at work, we all went into the server room to look at the progress," said Jenny. "Before we [left for] the evening, same story. Normally, [we don’t spend time in there] it was just noisy ... Of course, we were really curious!"
The final image is exported as a PSB file.
Though the team made improvements to the software, they are still working to address some issues with stitching and rendering large panoramas.
"There are still some part we could not manage to get to work with such a panorama. For example, the Smart Blend technology that removes moving objects in the panorama," Jenny said. "That's why you can see some ghosts in this picture. We're working to make such technology work on [larger panoramas], too."
All improvements made to Autopano Pro will be included in the next release of the software (version 1.4), which according to Jenny, will be in beta stage beginning next month. The program retails for about $119.
**After the 13-gigapixel panorama was complete, the team had to find a way to effectively display it on the Web. They chose to build a website and use the Zoomify tool, which allows users to zoom and pan across large images displayed on the Web.
"Zoomify seems to be the only user friendly cross-platform answer at the moment. So that was a given," Maynard said.
The site also gives the option to view the image in HDView.
"HDView is another technology for displaying large panoramas on the Web. It is brand new and the tool currently provided with this package cannot read PSB files either. So we also had to design an internal tool to allow direct export of a single PSB to HDView," according to a company press release.
Now that the panorama is displayed on the Web, Maynard is in search of a printer that, like Autopano, is willing to experiment.
"I have some contacts – forward thinking people – who would be interested in giving it a shot," he said. "Some printers with expensive equipment would not want to experiment with it. They don’t want to jam a printer that’s half a million dollars."