Discussion of 'Equivalence'

[simsurace]

Over time and in various topics, discussions over the meaning and validity of 'Equivalence' have appeared and continue to appear.
Let's discuss these issues here without risking to derail other topics.

I will open with the following question:

Which of the arguments in http://www.josephjamesphotography.com/equivalence/ do you think is circular, and why?

I have yet to see these models take into account the basic fact that by comparing two formats printed to a final fixed and similar size, the magnification of detail along the image chain will be different and thus also will the effective aperture.

'Equivalent' apertures are defined in order to take this into account.
http://www.josephjamesphotography.com/equivalence/#equivalence

[Bjørn Rørslett]

For the models under discussion to have any meaning, the native performance of the sensor itself has to be compared. Later massaging data to fit to say a fixed print size or what have you should not enter the discussion at all.

Thus, can a real difference in sensor performance of a FX camera set to DX mode be manifested and documented? If not, all  conclusions following this first, critical step is just artefacts due to model parameters and implied constraints.

So, I challenge anyone to document that cropping off the RAW file from an FX camera to match the DX format is significantly different from setting the DX option directly in the same camera. We need apples-to-apples comparisons here. I have put forward a testable hypothesis. The null hypothesis is that there is no difference.

[simsurace]

So, I challenge anyone to document that cropping off the RAW file from an FX camera to match the DX format is significantly different from setting the DX option directly in the same camera. We need apples-to-apples comparisons here. I have put forward a testable hypothesis. The null hypothesis is that there is no difference.

You mean just cropping the image and nothing else?
This will not produce an equivalent image because you will get a different framing. Thus the uncropped and the cropped image cannot be compared side-by-side as images of the same scene. One image will display only a portion of the other image, and if the final display size is proportional to the size of the used portion of the sensor, there will be no other differences. Since you could have obtained the same result by cutting the print of the FX frame after the fact, this is totally obvious and does not require any more experiments as far as I'm concerned.

The predictions of 'equivalence' would have to be tested in a setting where we try to obtain two images, one with the FX sensor and one with the FX sensor cropped to DX, that look the same in terms of framing, perspective, DOF, diffraction, motion blur, noise, and where we compare the two images at identical display sizes (thus at sizes not uniformly scaled from the capture sensor).

[Bjørn Rørslett]

The point here is that the shown graphs (Hasselblad X1D thread) says DX is inferior to FX in terms of dynamic range.

If this statement assumes a lot of manipulation going on after the RAW data has been captured, it cannot lend support to the claim above. We are left with consequences of model parameters and constraints, NOT a true sensor difference. All the "equivalence" thinking of the world won't change that fundamental fact. "Equivalence" has built-in circularity.

Please provide hard evidence refuting or corroborating the null hypothesis I put forward (reply #1). Quoting "authorities" will not lead anywhere - hard facts are required.

[simsurace]

The point here is that the shown graphs says DX is inferior to FX in terms of dynamic range.

(for anyone coming freshly to this topic, we are talking about the photographic dynamic range (PDR) charts on http://www.photonstophotos.net/Charts/PDR.htm)

Well, the graphs are making this statement with regards to a very specific measure, PDR, which is defined in a way that takes into account the different secondary magnifications that are required to produce a standardized output from different sensor sizes. 

It should be reiterated that anyone reading these graphs and using them to make decisions should read the definition of PDR and check whether it matches his or her idea of what is being displayed, and whether these criteria for comparing images suits them.

PDR is a derived quantity, but the data that underlies the graphs is still measured. The fact that they are measured shows up in the following way:

The graphs show slight deviations from equivalence theory when you compare different sensor models. So the idea that they are just predictions from a model is not consistent with the displayed values. The data being displayed in units of PDR, however, facilitates the comparison with the predictions of equivalence theory.

[simsurace]

Please provide hard evidence refuting or corroborating the null hypothesis I put forward (reply #1). Quoting "authorities" will not lead anywhere - hard facts are required.

I will not defend a hypothesis that I never claimed was different from null. I already explained why that null hypothesis is tautologically true.
Mere cropping is not what is being talked about in Bill Claff's graphs (see also my previous reply).

[Bjørn Rørslett]

Well, the fact remains that there has been asserted a relation that has to be supported by hard evidence. There is a frequently held view, quite prominently displayed on threads here on NG and seen elsewhere that once one "crops" to DX format in camera, or equivalently, on the sampled file, the DX will have a reduced DR. Can such a view be supported - yes or no? If it cannot be supported please stop talking about "equivalence" as anything else than a fancy label for overall magnifcation.

Here is an example of claimed DR difference using DX to FX from the same camera. Fact or fiction?

[simsurace]

I did not introduce the term. However, the term is about more than just magnification and describes the parameters that are required to produce very similar-looking images (when viewed at a standardized size).
I think that comparing images at a standardized output size, and showing the same framing, perspective, etc., is really the only technically sound way to compare them, but that is just me.

Neither Joseph James, nor Bill Claff or me have put forward the hypothesis that you are asking to be tested.

The rest, e.g. whether something is a property of the sensor or not, is a matter of definitions/terminology. A digital image sensor can be viewed as a fixed unit, with a certain size, pixel pitch, fill factor, quantum efficiency etc., or as a medium with variable size (similar to film). I tend to favor the former, and think that the size of a sensor as the property with the biggest impact on the photographic process. Perhaps you find the latter view more natural. That difference might explain some of the disagreement.

[simsurace]

Here is an example of claimed DR difference using DX to FX from the same camera. Fact or fiction?

The graph is not fictitious insofar as applying the definition of PDR (for the respective formats) to the SNR measurements of the D5 produces these data points.
I cannot personally verify the SNR measurements myself, so if you contest the accuracy of those measurements, this is a totally different matter.

As for the definition of PDR, I think we already discussed this earlier.

[Jack Dahlgren]

I did not introduce the term. However, the term is about more than just magnification and describes the parameters that are required to produce very similar-looking images (when viewed at a standardized size).

The rest, e.g. whether something is a property of the sensor or not, is a matter of definitions/terminology. A digital image sensor can be viewed as a fixed unit, with a certain size, pixel pitch, fill factor, quantum efficiency etc., or as a medium with variable size (similar to film). I tend to favor the former, and think that the size of a sensor as the property with the biggest impact on the photographic process. Perhaps you find the latter view more natural. That difference might explain some of the disagreement.

If we move this to a different domain, say automotive, then this "only care about the end result" approach starts to fall apart. Imagine the sensor is the engine and the "print" is the body of the car. If you change the body size to be larger (and heavier) certainly the accleration will change, but claiming that the same engine has a different "equivalent torque" in a different body makes no sense. Comparisons based on that equivalent torque would be meaningless.

Good practice is to separate elements so you do not attribute qualities of the system to the wrong element.

In this discussion, and in my analogy, magnification is the key element, not dynamic range of the sensor.

As you state, larger sensor size is important because it allows lower magnification, not because it has better/different dynamic range.

[Bjørn Rørslett]

The problem with the 'equivalence' thinking is that putting forward alternative approaches to reach a final result (be it print size or whatever) is masked by all the muddle of confused terminology. Will, for example, increasing the sheer number of pixels in the first step mitigate some of the problems caused later by large magnification? This again introduces another constraint, namely also the need for making the print not only fixed size but also with a fixed dpi setting of course (just to show how confused any line of thought ends up with so tongue in cheek is required). Will downscaling an FX sensor result in "increased DR" if the downscaling is done by binning photo sites? and so on. A maze of likely unusable roads can be followed for those having plenty of spare time upon their hands. The wiser ones would pick up the nearest camera and go shooting instead.

[JohnMM]

Well, the fact remains that there has been asserted a relation that has to be supported by hard evidence. There is a frequently held view, quite prominently displayed on threads here on NG and seen elsewhere that once one "crops" to DX format in camera, or equivalently, on the sampled file, the DX will have a reduced DR. Can such a view be supported - yes or no?

Yes - it's a matter of definition.

Let's take a pixel with a dynamic range equal to DRPIX.
We use the full well capacity (FWC) for the upper limit and the read noise (RN) for the lower limit.
DRPIX=FWC/RN

If we combine a number, N, of these pixels to make a DX sensor we get, using an obvious notation, DRDX = sqrt(N)*DRPIX
This follows since the upper limits are added arithmetically and the lower lmits are added in quadrature.

For an FX sensor which is linearly 1.5 times bigger we need 2.25*N pixels.
DRFX=1.5*sqrt(N)*DRPIX

DRFX=1.5*DRDX

Claff uses a different lower limit and talks about "photographic dynamic range" - but similar ideas apply.

[simsurace]

The problem with the 'equivalence' thinking is that putting forward alternative approaches to reach a final result (be it print size or whatever) is masked by all the muddle of confused terminology. Will, for example, increasing the sheer number of pixels in the first step mitigate some of the problems caused later by large magnification? This again introduces another constraint, namely also the need for making the print not only fixed size but also with a fixed dpi setting of course (just to show how confused any line of thought ends up with so tongue in cheek is required). Will downscaling an FX sensor result in "increased DR" if the downscaling is done by binning photo sites? and so on. A maze of likely unusable roads can be followed for those having plenty of spare time upon their hands. The wiser ones would pick up the nearest camera and go shooting instead.

Yes, I think these are valid concerns, but unsurprisingly they all concern second-order (smaller) effects (I hope that I mentioned that we are dealing with an approximation, i.e. as pointed out by someone 'a model that is wrong but useful').

For instance, increasing the number of pixels of the sensor has a rather small effect. If the used lenses are sufficiently high-resolving, the higher number of pixels can reveal some additional fine details, but only up to the limits imposed by diffraction. Regarding noise, on a pixel-by-pixel level there will be more noise because each pixel records a smaller number of pixels photons, but when viewed on the final print, unless we are dealing with very low resolution the additional noise will be largely invisible (printing is also a noisy process).

As said, the limitation of the model is that the number of pixels and other details of the sensor design are not included, because they are way too difficult to model with such a simple framework. However, the differences we see between formats that have vastly different image sensor areas, are well captured by the model. E.g. going between m4/3 and FX, or between a smartphone camera and FX, etc.

EDIT: To clarify the remark about photons: smaller pixels require a lower FWC if the base ISO is maintained because they will be struck by fewer photons for a certain exposure. If the read noise does not change, the DR of each pixel is reduced (lower FWC, same read noise ---> lower DR).

[Bjørn Rørslett]

Yes - it's a matter of definition.

Let's take a pixel with a dynamic range equal to DRPIX.
We use the full well capacity (FWC) for the upper limit and the read noise (RN) for the lower limit.
DRPIX=FWC/RN

If we combine a number, N, of these pixels to make a DX sensor we get, using an obvious notation, DRDX = sqrt(N)*DRPIX
This follows since the upper limits are added arithmetically and the lower lmits are added in quadrature.

For an FX sensor which is linearly 1.5 times bigger we need 2.25*N pixels.
DRFX=1.5*sqrt(N)*DRPIX

DRFX=1.5*DRDX

Claff uses a different lower limit and talks about "photographic dynamic range" - but similar ideas apply.

Yes?? and no. The claimed increase is a direct function of N, the number of pixels. Thus if the density of pixels is increased on SAME sensor, the value will increase. The higher number of pixels per sensor the higher the dynamic range all things else considered the same. This is not borne out in practice with existing cameras.

Check your definitions applied on a subset of the FX file, cut to proportion of the DX format. The sampling of data and their conversion into digital domain is already performed. Thus *any* change has to be attributed to the ensuing operations on the initial data, not related to the sensor performance at all. Same can be said of a scheme in which every 2 out of 3 pixels are sampled and combined to a new frame (FX>DX incidentally). In the latter case one can hardly assume a lower well capacity as the conversion to digital is completed at the time of the subsampling.

The old adage of 'garbage in, garbage out' is as sound today as the day it was coined.

[simsurace]

If we move this to a different domain, say automotive, then this "only care about the end result" approach starts to fall apart. Imagine the sensor is the engine and the "print" is the body of the car. If you change the body size to be larger (and heavier) certainly the accleration will change, but claiming that the same engine has a different "equivalent torque" in a different body makes no sense. Comparisons based on that equivalent torque would be meaningless.

Using your analogy, you already have the 'equivalent' measure to compare cars: acceleration. This is what matters, let's say, on a test track. A more massive body has to be compensated by a stronger engine.
In engineering and physics, the reformulation of relationships in terms of dimensionless quantities usually represents a step towards a deeper understanding, since stuff that is irrelevant for the qualitative behavior of the system gets moved out of the way.
Of course 'qualitative behavior' might not be so straightforward to define in photography. Still, I think that the methods of Joseph James are not too far off the mark.