That there is so much resistance here and elsewhere makes this an interesting topic for me. There must be reasons for the controversy. One of them is that many people always conflate technical discussions with art, and demand that any discussion has a net result in terms of artistic growth. You don't get that from this discussion. You might walk away with a better sense of how to evaluate your tools.
No one has mentioned the dynamic range (or is that engineering dynamic range?) of our own little eyeballs. I need to go look that one up because I'm wondering if the camera sees more zones than I do or is it the opposite?
Of course the discussion has to come back to actual photographs. Otherwise we are like doctors evaluating a drug without asking whether the patients get better, or cooks talking about emulsion stability while ignoring the taste of the hollandaise.
The issue is not whether evaluating cameras would be useful but whether the metrics we are being presented with are, in fact, useful for evaluation. People like to feel that they understand and are in control, and that creates a powerful urge to evaluate and rank, and a metric is the tool you need to achieve that. The urge to evaluate and rank is
so strong that people will use metrics
knowing they are misleading because that is all they have, like the drunk looking for his keys under the street-lamp because that is where the light is, or a photographer comparing Imatest results across systems. As a result, the world is full of metrics, and people are so relieved by the feeling of understanding and control that they ignore the fact that very few metrics are adequately tested, and of those that are very few perform adequately. Many metrics widely used for evaluation and ranking do not, in fact, predict with any accuracy the outcome we are really interested in: BMI as a metric of obesity-related illness and exam results as a metric of school performance, eg.
To assess the performance of a metric we need a gold standard: "the outcome we are really interested in". In some cases "the outcome we are really interested in" is clear - morbidity and mortality in the case of BMI, eg. So what is "the outcome we are really interested in" when we use a metric to evaluate cameras? If it is not the appearance of the photographs, what is it, and what is the basis for that choice? What we have at present is circularity: "Dynamic range is the relevant outcome because that is what we measure to evaluate cameras", or "Equivalence at identical framing and output size is the relevant outcome because that is how we calculate equivalence".
There is also the issue that metrics are promoted and criticised for commercial reasons.
Of course someone who wants to sell you equipment to measure MTF50 or perceptual megapixels will tell you that is the metric you need to evaluate your equipment
and that looking at your prints is inadequate.
Asking for the dynamic range of the eye is the wrong question: we never see the image formed in the eye, only a
heavily post-processed version. The dynamic range of the
human visual system is far higher than any photographic system, but we cheat. If we look at a high contrast scene - a person sitting in deep shade and a background in bright sun, eg - we have the impression we see both at once because we switch between the two areas without being aware of it. It is the same as depth of field, which we are not aware of because we switch from near to far without noticing, so everything appears to be in focus all the time.