Color (9.05)
Never mind all the gimmicks manufacturers add to cameras – what everybody really cares about is how the pictures look, and the most striking aspect of picture quality is color. We run a standardized color performance test on every camera we review, photographing a GretagMacbeth color chart under controlled lighting, and analyzing the image with Imatest software, a leading program for evaluating digital camera performance. The image below was created by Imatest software to show the difference between the R1's image and the ideal rendition of the chart. For each color tile: the large outer square is the color the R1 captured, while the vertical rectangle in each square shows the original ideal color. The small inner square shows the ideal color, corrected for luminance.
The Sony R1 did very well in our color tests. 105.5 percent saturation is typical of many entry-level DSLRs (though it’s a bit high for most prosumer or professional-level designs), it’s still lower than the great majority of high-end all-in-one cameras. The ideal, perfectly accurate score is 100 percent, but most manufacturers boost saturation at least a little, knowing that most users prefer bright colors. We prefer accurate color straight from the camera – it's easy to boost an image's saturation on a computer, but it's impossible to recover the detail that can be lost when a camera over-saturates.
The graph below, also produced by Imatest, illustrates the R1’s color reproduction capabilities in a more quantitative manner. The circles illustrate the R1’s rendering of the color, while the corresponding squares are the ideal. The line joining the two shapes displays the degree of error for that particular tone. The longer the line, the more inaccurate that particular color is.
The R1's 5.64 mean color error is an outstanding score, which puts the camera in the same league as current DSLRs. On this test, the R1 performed slightly better than Nikon's hot new DSLR, the D200.
Still Life Scene
Below is a shot of our classic still life assemblage, captured with the Sony Cyber-shot DSC-R1.
Click on the image above to view the full resolution file.
Resolution (5.6)
We test cameras' resolution, or ability to record fine detail, by shooting a standard resolution test chart under controlled lighting and feeding a section of the chart into Imatest software. Imatest delivers results in “line widths per picture height,” which indicates how many distinct individual lines the camera could theoretically show in an image. A camera's resolution depends not only on the number of megapixels on the chip, but also on the camera lens and the image processor, which digitally sharpens the image as it is saved.
Click on the chart above to view the full resolution image
The R1 delivers 1924 lw/ph horizontally with 9.91% oversharpening, and 1914 lw/ph vertically with 11.7% oversharpening. That's a bit more oversharpening than most DSLRs, but less than many all-in-one super zooms which sometimes hit 20% in-camera oversharpening.
We shoot our resolution test at several apertures and focal lengths and report our best results. The R1 did best at an aperture of f/8 and a focal length of 56mm. We set sharpening to zero in the camera's menu, but as is typical, the R1 performs some sharpening anyway.
Noise Auto (7.29)
Our auto noise test is just like our manual noise test, but we set the camera to Auto ISO. Since we shoot the auto test in bright light, the result should look just like the result for manual noise at the lowest ISO setting. The R1 gave us just that – in auto, it performed the same way it did when we manually set the ISO at 160.
Noise Manual (9.38)
Image noise is the variation in color and brightness in an image file that doesn't come from the subject photographed. Like static in a radio signal, image noise interferes with the information that the processor tries to communicate. In digital photos, noise looks like speckles of light and dark or off colors, somewhat like the grain in film photographs, although much less visually pleasing. Noise increases at higher ISO settings.
We tested the R1’s noise suppression at each available ISO setting. The results are plotted on the graph below. The horizontal axis shows the ISO setting, while the vertical axis indicates the resulting noise.
We use Imatest software to evaluate cameras' noise performance. The R1 performed very well in our tests, showing a steady rise in noise as ISO increased from 160 all the way to 800. There's a notable jump in noise from 800 to 1600, though 1600 still looks fairly good. The jump from 1600 to 3200 is even more pronounced, and we expect that most users will use 3200 only when it's the sole way of getting the shot. However, across the ISO range, the R1 handled noise remarkably well, particularly for a fixed lens design.
Low Light Performance (7.0)
We shoot the GretagMacbeth color chart for our low light test as well, again using Imatest to evaluate images for color accuracy, saturation, and nosie. We photograph the chart at four light levels, measured at 60, 30, 15 and 5 lux. 60 lux is typical room lighting, less than most people would choose to read by. 5 lux is very dim. We set up the test to allow the camera to take long exposures, because most digital cameras show increasing noise and decreasing color accuracy at longer exposure times.
|
 |
 |
|
60 Lux
|
30 Lux
|
 |
 |
|
15 Lux
|
5 Lux
|
The R1 does not defeat the laws of physics. Its image noise increases and color accuracy wanes as exposures increase from 1 second to 5 seconds. Its saturation increases with longer exposures, which is odd. We expect that it is not the imaging chip that produces this effect, but must be the R1's image processing. We haven't seen this before.
The graph below displays the R1's long exposure performance. The horizontal axis indicates the duration of the exposure (in seconds), while the vertical axis is the corresponding noise. These shots were taken at the camera's ISO 3200 setting without the use of the flash.
|
Lux
|
60
|
30
|
15
|
5
|
|
Exposure (sec.)
|
0.4
|
0.77
|
5
|
8
|
|
Saturation
|
98.00%
|
98.56%
|
100.80%
|
103.60%
|
|
Noise
|
4.43
|
4.7
|
4.96
|
5.04
|
Dynamic Range (7.25)
Dynamic range measures the difference between the brightest and darkest values a camera can record in a single image. It's easy to find a scene that includes subjects that can't be photographed at the same exposure value. The most obvious is the sun and anything else – expose for the sun, and everything else goes black. Less extreme contrast can still overwhelm digital cameras – snow, white clothing, stage lighting and so on.
To test dynamic range, we photograph a Stouffer step chart, a calibrated target that shows more than 13 stops of exposure range, and run the results through Imatest software. We shoot at each ISO setting on the camera, bracketing the exposure and taking the best result. The results are useful for comparing cameras, but because they are taken under testing conditions, they show the maximum range in which the camera is capable. It is unlikely that users will achieve as much dynamic range shooting normal scenes in a natural-looking way.
Our chart shows two lines: Low, for low-quality dynamic range, and High, for high-quality dynamic range. Low shows the range of stops the camera picked up with a tolerance of 1 stop of image noise. High shows the range of stops picked up within 1/10 of a stop of image noise.
The Sony R1 shows DSLR-like dynamic range, which is very good, but not really surprising – its sensor is APS-sized, just as the ones in most DSLRs. Larger imaging chips mean larger individual sensor sites for each pixel, linking dynamic range to sensor size. The R1 compares well with the Nikon D200, which also has a 10 megapixel, APS-size sensor. The D200 looks significantly better with low-quality dynamic range at ISO 400, 800 and 3200, but the R1 is very close in high-quality dynamic range. The R1 is more than half a stop worse at ISO 400 and 3200, but otherwise, the differences are slight.
Speed / Timing
Start-up to First Shot (7.9)
Our best time from start-up to first shot with the Sony R1 was 2.1 seconds. This is significantly slower than typical DSLRs, which tend to fire up in well under a second. Users should keep the R1 switched on when shooting opportunities are likely to arise suddenly.
Shot to Shot Time (7.38)
The R1 got off 3 shots in about 0.62 seconds. We really can't call that 4.8 frames per second, because the R1's maximum burst in JPEG is only 3 frames. It takes more than 7 seconds to clear the buffer. Worse, its maximum burst in RAW mode is... well, one shot; basically, it won't burst in RAW.
Shutter to Shot Time (8.3)
The main cause of the delay produced between pressing the shutter and getting the shot on the R1 is focusing. When autofocus is engaged, the R1 took 0.35 seconds to get a shot. In manual focus, we couldn't measure a delay – it was too short to catch with our testing mechanism.
