Multivariate Analysis on Image Sensor Classification and Variability

[MFloyd]

Multivariate Analysis on Image Sensor Classification and Variability (a case study on Canon, Nikon, Sony), worthwhile reading 

https://multianalytics.wordpress.com/blog/multivariate-analysis-on-sensor-classification-and-variability/

[chambeshi]

Yes a most interesting report - much to digest.  62+24+10 = 80% is a credible result for a PCA using 9 variables on 92 samples (Table 2). It would be surprising if these data have not been sifted already to identify trends, but perhaps the manufacturers have chosen not to highlight them.

EDIT - Thom Hogan has pointed out the obvious trend, which stands in Fig 8, and would be more informative if the recent models are labelled on Fig 8 and/or coloured coded...
http://www.dslrbodies.com/newsviews/an-intriguing-analysis.html

It is strange that I cannot see how the Factor Loadings decompose into each variable i.e. where are the equations that quantify respective contributions of individual variables in each of the 3 PCs? Leaving these out of a submitted manuscript would not get past an editor as the reviewers would kick it straight into touch. The first aspect I check in any PCA of morphometric or ecological data. But as I understand Fig 2, DSNU and PRNU appear to be dominant variables in explaining the differences across the domain of sensor design space.

There are several intriguing patterns, and expressing at different scales. The role of sensor area punches out stridently in the 3D plot - Fig 1b.There's no lack of more intricate signals in the graphs, including the distinctiveness between the Df and D4. And also the convergence in the Df and D5. Both the PCA and the dendrograms recover congruent relationships for these 3 FX Nikons. The tight sister relationship between the D500 and D7500 are no surprise, and what one would expect from the same core sensor. This underscores how the sensors in the Df and D4 exhibit distinctly different designs (contrary to the marketed specs)

Much to dissect here :-)

Many thanks for sharing this report

kind regards Woody

[longzoom]

Great job, but it is unclear what methodology was in use. Not all of cameras allow use direct port/cable connections. So, in some cases, different lenses and cards must be involved, what may ruined final result. Even different cables from different manufactures will create unacceptable situation. Extremely difficult obstacles, I'd say. Of course, scientists who performed such the great test, knew that way better, so, couple words about methods involved, would be welcome! Thank you!  LZ   

[chambeshi]

The raw data are raw sensor readings of measured Heatmap attributes from PhotonstoPhotons, with minor additions of descriptive categories by the author of this paper. See some labels I've taken the liberty of adding to his Fig 8 for more recent Nikon sensors. Besides the general shift "to the right" as DR etc has increased, there have also been "leaps" in specs between iterations of a sensor - e.g. how iterations of the D3 models raised Noise in derived models. And again this plot points to the predilection of so many photographers for the Nikon Df.

The design challenges for Sony et al are to push the technology down and to the bottom right-hand corner, increasing Dynamic Range etc and also Resolution BUT reducing noise. It will indeed be interesting to see how this space not only fills up but fills out (?)

[longzoom]

Multivariate Analysis on Image Sensor Classification and Variability (a case study on Canon, Nikon, Sony), worthwhile reading 

https://multianalytics.wordpress.com/blog/multivariate-analysis-on-sensor-classification-and-variability/

     Without the explanation of the criteria and technical abilities to compare ancient 2002 technology to the one of 2017, as well as clearness of such the experiment, so let me, personally, be a little bit skeptical. Would you forgive me my nastiness/naivety, but HOW they perform this extremely difficult comparison? Without the base nobody is able to build his/hers conclusion, I somehow believe. THX!   LZ

[chambeshi]

the usual way we see raw data compiled in photonstophoton is as bivariate graphs. eg DR vs ISO. And these patterns are discussed at length and too often with undeserved reverence. There are several such variables that can be projected in this 2D format, to gain insights into sensor performance in comparisons. Each is a proxy of how the image is rendered.... But it is challenging to try and consider what more than 3 variables tell us - in a more composite map - ie multivariate.

PCA is a standard technique to collapse the variation in the several measurements (typically 5-20) into a reduced set of standard units by calculating how much each variable of the total contributes to a n-dimensional map of data variation (9 variables in this case of sensor performance). And this can be collapsed into the minimum set of  components that bring together the most similar - ie redundant - variables, which group in their convergence / overlap in what they are measuring. In this study, of Principal Component 1 represents differences in how sensors record and digitize tones in the photograph. Component 2 represents the variables most influenced by Noise and also Resolution etc. Together these 2 components account for 80% of the differences in performance of these 92 sensors. This is a solid result for a PCA, in contrast to some I've seen in biology and geology where the first 2 PCs account for a paltry ~30 % of the total variation, and the 3rd and 4th etc even less [say 20%]. Basically in the latter case, there are little if any strong patterns in the collective dataset ,and the variables are strongly independent.

The several observations in the final PCA that point to further credibility - eg the D500 and D7500 (basically the same sensor) almost overlap points to further confidence we can place in this PCA. It is a realistic map of sensor space as represented by R&D and DSLR production since 2003. But as a pattern recognition system - to quote the author of this study - a PCA cannot resolve causal relationships with high confidence, if any.

Nevertheless, I would still like to see hard percentages of what each of these variables (and which) contributes to the ordination of PC1 and PC2 in these plots.....

As I see it - bottom line - these analyses put numbers on more subjective observations, and they further show up the inadequacy of any single rating that attempts to quantify the respective performance of a device as complex as a modern DSLR sensor (let alone all the other factors). As I posted earlier in a parallel thread in DPR: This situation in modern marketing and all the cottage industry in Camera reviews recalls one of the timeless observations by the late Stephen Jay Gould: "I wonder if we will ever get past the worst legacy of IQ theory in its unilinear and hereditarian interpretation -- the idea that intelligence can be captured by a single number and that people can be arrayed in a simple sequence from idiot to Einstein" [Gould 1989. Wonderful Life. Penguin Books]

kind regards

woody

[Netr]

Thanks, Woody. Your explanation was a big help to me. Ross

[Frank Fremerey]

Multivariate Analysis on Image Sensor Classification and Variability (a case study on Canon, Nikon, Sony), worthwhile reading 
https://multianalytics.wordpress.com/blog/multivariate-analysis-on-sensor-classification-and-variability/

Insightful article of no practical value to me.

I made the decision for a camera system, keep investing into Nikon glass, try & buy the best cameras that carry my lenses natively and serve my artistic & my customers commercial interests. I am happily married, why care much for other women? I got work to do, shiploads of it.

[Jack Dahlgren]

There are several intriguing patterns, and expressing at different scales. The role of sensor area punches out stridently in the 3D plot - Fig 1b.

Not very surprisingly though. PDR is proportional to sensor size by definition. This conflating of dynamic range and sensor size is one of the main problems with it as a measure in my mind, and is exposed here.