Author Topic: Pixel-Shifting Vs. Larger Sensors  (Read 30991 times)

Les Olson

  • NG Member
  • *
  • Posts: 502
  • You ARE NikonGear
Re: Pixel-Shifting Vs. Larger Sensors
« Reply #90 on: November 03, 2017, 18:20:36 »
+1

The irony is, this is a Nikon forum, and Nikon doesn't have pixel shift, so how many here are going to be well-experienced in its usage?

Till now (as far as I am aware) only Pentax carried the technology ...

There is more experience that you might think, though not with consumer cameras.

The principle underlying pixel shift is not complicated.  I have a feeling not everyone gets how it does what it does, so here is how it does what it does.


You have a 4 x 4 array, with a detail in the top right corner that cannot be resolved: that pixel will be pale grey and all the rest white. If you move the pixel half a pixel across, down, and across again in the other direction, the small black detail has moved to all four corners of its pixel, and when you combine the four images that pixel will be black and the rest will be white. Detail resolved. If you move a third of a pixel you can resolve even smaller detail.  Of course, it isn't real resolution: the detail is now four times its true size, and it was always there, you just could not be sure it wasn't noise.  Unless it was noise, of course.

I am showing this so people can convince themselves that if you move a whole pixel, as in the Pentax and, as far as we know, the Sony, you lose contrast resolution - all four pixels are now pale grey.

The pixel shift principle is used in some scanners, which have staggered arrays - two lines of pixels offset half a pixel.  It doesn't make much difference, if any.

The principle is also used in sensors for machine-vision cameras because it allows you to use a lower MP and therefore cheaper sensor.  In that application the sensor is cooled to -15 degrees so dark noise is (almost) eliminated so you are less likely to be resolving noise, and the lenses used can be highly corrected because they only have to work at a single object distance.  Even under those circumstances the effect is hard to see with four images - you have to take 32 or 64 to get a convincing improvement.

Whole pixel shift means that each pixel-sized area of the image has a measured RGB value (not "each pixel has a measured RGB value").  The idea that this will improve resolution because the Bayer filter, being 2 x 2 pixels, means you are sampling at half the rate, is wrong.  There are good theoretical reasons it should be, but we don't have to discuss them.  We can just look at the effect of removing the Bayer filter, which has been done for the P45 Phase One sensor (the AA and IR filters were removed as well).  The effect on resolution was imperceptible.

Les Olson

  • NG Member
  • *
  • Posts: 502
  • You ARE NikonGear
Re: Pixel-Shifting Vs. Larger Sensors
« Reply #91 on: November 03, 2017, 20:55:16 »
If two adjacent pixels have very different RGB values when they should be roughly the same (e.g. white), we may speak of color artifacts.
------------------------
 he claims that there is no rigorous way in which the potential benefits of pixel-shift may be measured. I'm trying to understand his claim but so far I fail.

But how do you know what they "should be"?  The only way you can know is if you have eliminated picture detail - by photographing a white screen, eg.

There is a perfectly rigorous way to measure the effects of pixel shift: compare the resulting image to the original. However, if by "rigorous" you mean something not involving human subjectivity, there is no such method as far as colour is concerned because colour is fundamentally subjective.

To make this point practical let me ask you to look at the Mars images at https://www.jpl.nasa.gov/news/news.php?feature=6989  Do you think the colours are correct?  Obviously, the question is nonsense.  But it is no more nonsense than if you are looking at one of Michael's flower images: you have no idea what the "correct" colour is. 

Edit: If you mean is there a rigorous way to define the effects - ie, leaving out the claim that they are good - of pixel shift, of course there is: look at the RGB values.  I pointed out a while ago that anyone who cares can download the files from DPR and see what the RGB values for the K1 or the K3 with and without pixel shift are. 

Erik Lund

  • Global Moderator
  • **
  • Posts: 6529
  • Copenhagen
    • ErikLund.com
Re: Pixel-Shifting Vs. Larger Sensors
« Reply #92 on: November 03, 2017, 21:59:09 »
Steady on.  I will not be put in the wrong. 

Michael's original post said: "I am wondering [...] whether the pixel-shift technology of the A7R3 may give me the color (most important to me) and the enhanced resolution (however that works), so that [...] I might (at least for a time) be happy with what I have (or will soon have with the A7R3)?"

How is that a practical question?  This is a man asking us to predict his state of mind when he owns a camera that will not be on sale for a month and that for the feature he is fixated on requires software that as of now is at "pre-beta" (https://www.sonyalpharumors.com/sony-also-announced-new-imaging-edge-software-suite/).  But we can't have any theoretical discussion or talk about how it might be this or it might be that?! 

He went on: "I am sure some of you here will have more technical thoughts about this conundrum I am in, either agreeing with me or pointing out something I have not thought of."

That is, straightforwardly, an invitation to discuss technical issues.  It is not OK to whine and snarl because the invitation was taken up.

 


Put you wrong ?


I have no intention of such, Re read what I wrote.
Then calm down or this will end in moderation
Thanks
Erik Lund

Erik Lund

  • Global Moderator
  • **
  • Posts: 6529
  • Copenhagen
    • ErikLund.com
Re: Pixel-Shifting Vs. Larger Sensors
« Reply #93 on: November 03, 2017, 22:01:13 »
+1

The irony is, this is a Nikon forum, and Nikon doesn't have pixel shift, so how many here are going to be well-experienced in its usage?

Till now (as far as I am aware) only Pentax carried the technology ...

Even more ironic, Michael has used the technology, and is the very "person with experience" (in precisely his own style of shooting) ... and therefore has the answers to his own questions.

The only remaining questions are, "What will the pixel-shift tech of the Sony AR7rIII be like, and will it render better color than the Nikon D850?" Michael already owns and is happy with.

Since no one owns the Sony yet, how can anyone provide an in-depth, practical response?

Theoretical discussion is the only option, other than not to respond at all.

If we attempt to discuss theory, perhaps a re-read of Bjørn's suggestion, Also worth keeping in mind is the Law of Diminishing Returns, is the most relevant.

How many photos must we take, stack/shift, and combine ... with how many different camera/lens combinations ... before we can be happy with what we do?

Certainly, Michael has produced some exquisitely-rendered images ... so is the Sony pixel-shift/stack + adapter really going to make a difference over a D850 stack?

I honestly doubt it; in fact, already the D850 has shown to have better Base ISO DR.

At some point, it pays just to be happy, rather than forever chasing a rainbow of perfection ... that can never be caught.


This is not a NIKON forum, all brands welcome!
Erik Lund

Erik Lund

  • Global Moderator
  • **
  • Posts: 6529
  • Copenhagen
    • ErikLund.com
Re: Pixel-Shifting Vs. Larger Sensors
« Reply #94 on: November 03, 2017, 22:08:37 »
A moderation comment to all, no one in particular;


You don’t have to convince everybody to agree with you.


To disagree is perfectly ok,,,


Here on NikonGear we are here to learn and have fun!
Erik Lund

OCD

  • Obsessive Corgi Disorder
  • NG Member
  • *
  • Posts: 97
Re: Pixel-Shifting Vs. Larger Sensors
« Reply #95 on: November 03, 2017, 23:08:05 »
Thank you Erik, your moderation is appreciated.


JKoerner007

  • Guest
Re: Pixel-Shifting Vs. Larger Sensors
« Reply #96 on: November 03, 2017, 23:33:39 »
This is not a NIKON forum, all brands welcome!

Perhaps a re-name to "All Brands Gear"?  ;)

Though I've seen a couple of Sony-folks ... can't say I've seen a single Canon user post.

Pretty sure the theme title is a filter toward Nikon aficionados ...

Bjørn Rørslett

  • Fierce Bear of the North
  • Administrator
  • ***
  • Posts: 8252
  • Oslo, Norway
Re: Pixel-Shifting Vs. Larger Sensors
« Reply #97 on: November 03, 2017, 23:56:14 »
The site name is for historical reasons. We have no intention of changing it.

Hugh_3170

  • NG Supporter
  • **
  • Posts: 2128
  • Back in Melbourne!
Re: Pixel-Shifting Vs. Larger Sensors
« Reply #98 on: November 04, 2017, 02:18:31 »
Well the Olympus OMD E-M5 MkII certainly has had pixel shift high resolution on offer for some time now, as do/have certain Pentax and Sony models (as already noted in the various posts herein), so it is not just restricted to the one manufacturer any more.  I suspect that those manufacturers that use in body sensor shift image stabilisation should be better placed to introduce Pixel-Shift higher resolution options.

EDIT:  The newer Olympus OMD E-M1 Mk II also has pixel shift high resolution modes;  Olympus first introduced the feature on the E-M5 MkII.

EDIT:  Those interested in the outcomes obtained fromthe Olympus pixel shift high resolution implementations should Google on Robin Wong and Ming Thein and take a look at their reviews and field tests of the Olympus E-M5 MkII and the E-M1 MkII.  I cannot add further as my E-M1 is the MkI version that doesn't have the pixel shift feature.


+1

The irony is, this is a Nikon forum, and Nikon doesn't have pixel shift, so how many here are going to be well-experienced in its usage?

Till now (as far as I am aware) only Pentax carried the technology ...

............................................................................................
............................................................................................
Hugh Gunn

JKoerner007

  • Guest
Re: Pixel-Shifting Vs. Larger Sensors
« Reply #99 on: November 04, 2017, 14:35:32 »
Well the Olympus OMD E-M5 MkII certainly has had pixel shift high resolution on offer for some time now, as do/have certain Pentax and Sony models (as already noted in the various posts herein), so it is not just restricted to the one manufacturer any more.  I suspect that those manufacturers that use in body sensor shift image stabilisation should be better placed to introduce Pixel-Shift higher resolution options.

EDIT:  The newer Olympus OMD E-M1 Mk II also has pixel shift high resolution modes;  Olympus first introduced the feature on the E-M5 MkII.

Thanks for the correction.

Honestly never paid attention to Olympus, so the Pentax K1 was my first awareness of pixel shift. Seems to have extremely limited application, regardless, and it's not an exciting enough difference for me to switch brands (or even buy a single camera) in order to achieve it.

Here is an example of pixel-shift limitations in real world use (look at the flags).

Even under the best conditions, I have never seen a K1/Olympus image I didn't think I could take with my D810; but there are many images I could take with my D810 that I could not take (due to lens selection limitation, as well as others) with a K1.

Now that Sony has it, I still don't like their lens limitations either. And, even where the glass choices are comparable, you could never take a landscape shot (with running water, moving trees/leaves) using pixel shift, so it still seems more academic than practical, for me at least.

For those who do absolutely static shots, it may produce a slightly-better effect, I don't know, but I think Nikon's in-body-stacking has more real-world use than pixel-shift. Not for extreme macro, but definitely for those happy with AF lenses, from close to landscape, in-body stacking seems a more useful overall function than pixel-shift.

Michael Erlewine

  • Close-Up Photographer
  • NG Supporter
  • **
  • Posts: 2067
  • Close-Up with APO
    • Spirit Grooves
Re: Pixel-Shifting Vs. Larger Sensors
« Reply #100 on: November 04, 2017, 17:58:32 »
JKoerner writes:

"For those who do absolutely static shots, it may produce a slightly-better effect, I don't know, but I think Nikon's in-body-stacking has more real-world use than pixel-shift. Not for extreme macro, but definitely for those happy with AF lenses, from close to landscape, in-body stacking seems a more useful overall function than pixel-shift."

For your use, Jack, quick stacks in the field make great sense. You can capture what otherwise you could not and have it all in focus. I have done decades of field work, so I understand. And the motion of the wind, not to mention, critters is a real handicap that the new D850 can help to overcome.

Partially because of age, but also because of inclination (they may be related! LOL.), I no longer chase after or travel to remote areas to capture photos. I have morphed into a “found” photographer, photographing whatever is around in summer and in winter, dragging whatever flowers, plants, etc. into my little studio and photographing them.

For me, the built-in Nikon focusing does not look very interesting, although should Nikon come out with a fast, sharp, highly corrected macro lens with autofocus (or someone else does), I could be perhaps persuaded.

Pixel-shift to me looks very usable, jast as the Nikon stacking feature does to you.

The problem with the new internal focus-stacking from Nikon is the same thing we stub our toe on all the time, the fact that Nikon autofocus lenses tend to be not as well corrected, etc. as some manual lenses, Nikon or other.

For your use, Jack, quick stacks in the field make great sense. You can capture what otherwise you could not and have it all in focus. I have done decades of field work, so I understand. And the motion of the wind, not to mention, critters is a real handicap that the new D850 can help to overcome.

Partially because of age, but also because of inclination (they may be related! LOL.), I no longer chase after or travel to remote areas to capture photos. I have morphed into a “found” photographer, photographing whatever is around in summer and in winter, dragging whatever flowers, plants, etc. into my little studio and photographing them.

For me, the built-in Nikon focusing does not look very interesting, although should Nikon come out with a fast, sharp, highly corrected macro lens with autofocus (or someone else does), I could be perhaps persuaded.

Pixel-shift to me looks very usable, jast as the Nikon stacking feature does to you.
MichaelErlewine.smugmug.com, Daily Blog at https://www.facebook.com/MichaelErlewine. main site: SpiritGrooves.net, https://www.youtube.com/user/merlewine, Founder: MacroStop.com, All-Music Guide, All-Movie Guide, Classic Posters.com, Matrix Software, DharmaGrooves.com

David H. Hartman

  • NG Member
  • *
  • Posts: 2790
  • I Doctor Photographs... :)
Re: Pixel-Shifting Vs. Larger Sensors
« Reply #101 on: November 05, 2017, 01:11:02 »
After a positive PM from Michael I'll venture a question to those with pixel shift experience.

From time to time I've had a problem with either the red or blue channel maxing out where the other channel and the green channel is fine. The Bayer array has one red, two green and one blue sensors. I don't recall this problem with the green channel.

I'm thinking of a photograph of a garden with small in the photo pink flowers. They were an almost solid one tone pink (255R, xxxG,  xxxB) in the photo. I had a struggle in Nikon Capture NX-D and Photoshop to get an acceptable but not really up to my standard photo. Can pixel shifting help with this problem or can pixel shifting and other post processing help.

Dave Hartman
Beatniks are out to make it rich
Oh no, must be the season of the witch!

JKoerner007

  • Guest
Re: Pixel-Shifting Vs. Larger Sensors
« Reply #102 on: November 05, 2017, 02:25:03 »
For your use, Jack, quick stacks in the field make great sense. You can capture what otherwise you could not and have it all in focus. I have done decades of field work, so I understand. And the motion of the wind, not to mention, critters is a real handicap that the new D850 can help to overcome.

Agreed, Michael.



Partially because of age, but also because of inclination (they may be related! LOL.), I no longer chase after or travel to remote areas to capture photos. I have morphed into a “found” photographer, photographing whatever is around in summer and in winter, dragging whatever flowers, plants, etc. into my little studio and photographing them.

Lol, understood.

My own studio use is limited to 3:1 macro, and above, which cannot really be done in the field ... esp. without a flash.

For me, neither the Nikon in-body stacking, nor pixel-shift, are answers to ultra-macro (my only studio work).

The in-body stacking (based on what I read) is too crude, and not fine enough, for high-mag macros ... so I have bought a WeMacro rail for this ultra-precise purpose ... as it is adjustable in 1 μm units in either direction.

Pixel-shift seems like it's just a superfluous consideration to add to a high-mag stack.



For me, the built-in Nikon focusing does not look very interesting, although should Nikon come out with a fast, sharp, highly corrected macro lens with autofocus (or someone else does), I could be perhaps persuaded.

Understood. That said (and back to an earlier Olympus reference), I've seen some superb in-body-stacked field macro images from Olympus cameras ... that, while not using 'highly-corrected' lenses ... still used pretty darned good lenses that rendered fantastic field-derived stacks, that would likely not have been possible with a highly-corrected lens. (Or even with a StackShot/WeMacro devise fitted for the field.) The subjects captured weren't really 1:1, more like 1:2, but the quick in-body stacks really improved the presentation.

Right now, I am satisfied with my D810 and CV 125, as the subtle focus ring adjustments allow me to achieve a 25 to 50-stack image, fairly quickly, and with acceptable precision, just due to the enormous and precise focus throw.



Pixel-shift to me looks very usable, jast as the Nikon stacking feature does to you.

I can see why: for you and your specialty, delving into pixel-shift would make far more sense. Am interested to see what you think, when all is said and done.



The problem with the new internal focus-stacking from Nikon is the same thing we stub our toe on all the time, the fact that Nikon autofocus lenses tend to be not as well corrected, etc. as some manual lenses, Nikon or other.

I think Nikon will come out with a 200mm FL-Micro one of these days ... and that may be pretty nice. Pure speculation, though.

Also, on long-Nikkor super-teles (which are about as well-corrected as any Zeiss Otus), there may be stacking options with the D850 to where (as you like to do) a wildlife photographer can set a 200, 300, 400, or 500 mm at f/2.8 (to f/4), and multi-stack a fairly-stationary telephoto shot, quickly and effectively, in-body... and thereby achieve tremendous DOF detail on the subject ... while yet still enjoying totally creamy background bokeh due to the comparatively-fast aperture at that focal length. Obviously, this couldn't be done with a bird in flight, but perhaps a perfectly-stationary one (or a bobcat 'freezing' on the hunt, a lion, buffalo, etc.). Lots of room, and applications, to apply this feature to ... again, including landscape.



You can capture what otherwise you could not and have it all in focus. I have done decades of field work, so I understand. And the motion of the wind, not to mention, critters is a real handicap that the new D850 can help to overcome.

Precisely 8)



Partially because of age, but also because of inclination (they may be related! LOL.), I no longer chase after or travel to remote areas to capture photos. I have morphed into a “found” photographer, photographing whatever is around in summer and in winter, dragging whatever flowers, plants, etc. into my little studio and photographing them.

For me, the built-in Nikon focusing does not look very interesting, although should Nikon come out with a fast, sharp, highly corrected macro lens with autofocus (or someone else does), I could be perhaps persuaded.

Pixel-shift to me looks very usable, jast as the Nikon stacking feature does to you.

Hey, we all have our interests, as well it should be.

And different features will make more, or less, sense accordingly.

Cheers.

Roland Vink

  • NG Supporter
  • **
  • Posts: 1537
  • Nikon Nerd from New Zealand
    • Nikon Database
Re: Pixel-Shifting Vs. Larger Sensors
« Reply #103 on: November 05, 2017, 20:54:25 »
From time to time I've had a problem with either the red or blue channel maxing out where the other channel and the green channel is fine. The Bayer array has one red, two green and one blue sensors. I don't recall this problem with the green channel.

I'm thinking of a photograph of a garden with small in the photo pink flowers. They were an almost solid one tone pink (255R, xxxG,  xxxB) in the photo. I had a struggle in Nikon Capture NX-D and Photoshop to get an acceptable but not really up to my standard photo. Can pixel shifting help with this problem or can pixel shifting and other post processing help.

Dave Hartman
If the red channel is maxed out on a single Bayer-array image, wouldn't it still be maxed out on a pixel shifted RGBG image?  Of course, the red channel is no longer interpolated, you have red information at every pixel, but would still be saturated, so surely the problem would remain (this is only my educated guess ... I don't have experience with pixel shifting...)
The only solution is to lower the ISO to increase your dynamic range, or reduce your exposure to prevent the channel from saturating. But there is no free lunch, lower ISOs mean longer shutter speeds which could result in camera shake or motion blur, underexposing risks more noise in the shadows.


David H. Hartman

  • NG Member
  • *
  • Posts: 2790
  • I Doctor Photographs... :)
Re: Pixel-Shifting Vs. Larger Sensors
« Reply #104 on: November 05, 2017, 21:29:08 »
If the red channel is maxed out on a single Bayer-array image, wouldn't it still be maxed out on a pixel shifted RGBG image?  Of course, the red channel is no longer interpolated, you have red information at every pixel, but would still be saturated, so surely the problem would remain (this is only my educated guess ... I don't have experience with pixel shifting...)
The only solution is to lower the ISO to increase your dynamic range, or reduce your exposure to prevent the channel from saturating. But there is no free lunch, lower ISOs mean longer shutter speeds which could result in camera shake or motion blur, underexposing risks more noise in the shadows.

No free lunch! Oh dear, one can hope.

I don't think of this single channel being max for the green channel as a problem. I wondered if this had anything to do with two green cells in the array. I wondered interpolation of the red or blue channel increased the problem.

I've taken to keeping the blinkies set to Red and switching them to Blue if I suspect there might be a problem. I suspect ISO is often the problem even at 800 or 1000. The D800 isn't a great high ISO camera. I'm pressed for time but found the photo I was thinking of was shot at ISO 1000. It looks overcast so color saturation would be high while shadows close to non-existent.

Thanks for the reply,

Dave

Anyone else?
Beatniks are out to make it rich
Oh no, must be the season of the witch!