D5 Dx0Mark scores

[dslater]

I did a quick comparison of the D5,D4S, & D4 at Dx0Mark.
http://www.dxomark.com/Cameras/Compare/Side-by-side/Nikon-D5-versus-Nikon-D4s-versus-Nikon-D4___1062_945_767
Seems like m out of the reason you're buying a D5 is for the improved AF system and framerate. Interesting, they give the D4 & D4S better high ISO marks too.
  What do people here generally think of Dx0Mark and their testing?

[armando_m]

I checked over the weekend

dxo lists the D800 with better high iso than the D5

The images shared here and in other sites have shown good quality to ISO levels never seen before

so... whatever dxo did this time I do not agree with their results about the D5

[Akira]

Sorry to be off-topic, but I couldn't help laughing when I saw the ISO sensitivity chart.   

[abergon]

Everyone would want a unique, objective measurement of image quality. But I am not sure there is such a thing. Dx0 has become some kind of a reference but:
- For cameras, they only test sensors, and Bayer sensors at that. They are honest about it, don't look for measures about Fuji or Sigma cameras.
- I don't know if it is by design, but in practice they fuel the mega-pixel race. A Nikon D810 is rated higher than a D5 (or D4 or Df), a Sony A7RII is rated higher than a Sony A7SII, a Canon 5DS than a 1Dx.

I am not saying that they are biased, they offer a lot a information about their methodology, I am sure highly qualified scientists work there. But I am less convinced about the value of their data for me as an amateur photographer. This is perfectly all right of course, I am not paying to access their data.

[MFloyd]

Apparently, it has all to do with the methodology: to measure the "sport / high ISO performance" they rely on the following: "An SNR value of 30dB means excellent image quality. Thus low-light ISO is the highest ISO setting for a camera that allows it to achieve an SNR of 30dB while keeping a good dynamic range of 9 EVs and a color depth of 18bits."  I have no idea where the D5 sensor "failed": dynamic range, or color depth.

I have no reasons to doubt about the seriousness of the measurements. The field results of the D5 simply demonstrate that DXO's methodology is not in line (anymore ?) with the "reality".

In general, top of the line pro bodies, be it Nikon or Canon, are not good performers in the little DXO world of sensor performance measurements.

I have far more trust in the tiny Senscore / Lenscore organization http://www.senscore.org Unfortunately D5 / D500 measurements are not yet available.

[Bjørn Rørslett]

DXO faces the same issues as map makers. One tries to project something into a space of lower dimensionality. This is not possible without loss of information.

The DXO rating of super-ISO cameras suffers badly because the ISO performance dimension is not well represented in their model. A parallel problem to maps using Mercator projection and showing Africa comparatively small compared to Greenland.

[Frank Fremerey]

True words, Bjørn. Let alone the things that are not measured at all. For fifth gen:

AF accuracy
AF speed
AF coverage
tracking capability
Color Consistency vs ISO
Color Consistency vs Light Sources

[Les Olson]

DXO faces the same issues as map makers. One tries to project something into a space of lower dimensionality. This is not possible without loss of information.

The DXO rating of super-ISO cameras suffers badly because the ISO performance dimension is not well represented in their model. A parallel problem to maps using Mercator projection and showing Africa comparatively small compared to Greenland.

Yes, but Mercator did not design the projection in order to make Greenland look big.   

DxO make money selling the software that generates the measurement index they promote.  The more people use their index the more widely it is accepted as the gold standard and more software they sell - that is why they give the data away. 

It is a universal human trait: people whose business is measuring things over-value measurement compared to qualitative results and among measurements over-value those they can make easily (SNR, eg) relative to those they cannot make easily (as Frank says, AF accuracy, eg).  This happens everywhere, even in serious science.  It only becomes a serious problem when a metric is promoted to identify a problem by someone who wants to sell you the cure (disease-mongering if it is people or "back-focus" and "ETTR" if it is a camera).

It is like the drunk looking for his keys under the street lamp because that is where the light is.  The opposite principle is called Sutton's Law, after a bank robber named Willie Sutton, who allegedly replied to a reporter who asked him why he kept robbing banks "Because that is where the money is". 

[Ilkka Nissilä]

In my opinion dxomark offer good measurement data on sensors but their scores are largely arbitrary and of not much value.  They weigh heavily the base ISO performance in their overall score, which is fine if you mostly use base ISO. Their method of coming up with the "Sports" score is a bit ridiculous as it is based on thresholding (which is set abitrarily) and the method disregards a lot of data and differences in the high ISO performance between cameras. And finally, to talk about a "sports" score with complete disregard of the camera's autofocus performance is just silly and highly misleading to a casual reader who has not invested the time to study their methodology.

But, as I said I think the graphs are, correctly interpreted, useful, as one source of information.

[Bjørn Rørslett]

Yes, but Mercator did not design the projection in order to make Greenland look big.   

I used this as an example that a projection into lower dimensionality always creates loss of information and thereby, skewed relationships. Being European I'm sure he didn't mind Europe being shown as 'larger than life'.

.....finally, to talk about a "sports" score with complete disregard of the camera's autofocus performance is just silly and highly misleading to a casual reader who has not invested the time to study their methodology.

But, as I said I think the graphs are, correctly interpreted, useful, as one source of information.

Wholeheartedly agree. Flooding the readers with information that requires additional insights and analysis in order to be meaningful, if at all, is bad presentation and misleading practice. Even the people at DXO must realise that their ISO rating is a joke and should either be done differently or removed from the compilation.

Presenting actual images taken under conditions of interest to the readers (and potential buyers) may be more informative, but of course any standardisation now becomes a major headache. However, better leave the headaches to the analysts and lab people than foist them upon the lay audience.

[abergon]

...
Presenting actual images taken under conditions of interest to the readers (and potential buyers) may be more informative, but of course any standardisation now becomes a major headache. However, better leave the headaches to the analysts and lab people than foist them upon the lay audience.

The strange thing is that they do present images when they test smartphones, but never for cameras, even cameras with fixed lenses.

[MFloyd]

you find a much more interesting analysis, about the why and the how, with DPReview http://www.dpreview.com/news/9402203921/nikon-d5-shows-drop-in-dynamic-range