Knut, I know exactly what you are talking about.
Back in film days I used a Nikon FM2 but added an MD-12 to add some bulk. Especially at night this added bulk helped me keep the camera steady at 1/15th of a second at up to 50mm. It also balanced well with the 80-200 f4.
My first experience with digital was with a Nikon D100 and that was still ok, but my first digital Nikon was a D70s instead, which was just too small and the controls were too limited in use to make me ever really like th ecamera. When I bought a D200, not too much later, I started enjoying digital a lot more and started using the FM2/MD12 less and less.
Then mirrorless reared it's head and I thought about the days where I went out with a bare FM, 50mm f2 and 17mm f3,5 in a coat pocket and jumped on the mirrorless bandwagon. I was duly disappointed because the ergonomics I knew from my D200 and D300 was gone. The optical viewfinder was gone. I just could not adapt, even though the image quality was good.
So I sold the Sony and bought a D3. In short, I have never been happier with a camera!
Everything is in the right place. The camera is perfect with bigger AF lenses, but also feels perfectly aligned with the small yet dense manual focus lenses I like using so much.
Same as you, I'm outdoors a lot and under any conditions, so the ability to handle the camera naturally when wearing gloves is not just a bonus. It is a must!
Size and weight? Yes please! Size really is directly related to ergonomics. Too small a camera is no longer easy to handle when you rely on external controls and need to change certain variables often and quickly to keep up with the situation.
As far as color goes, I really love the 12 Mp D3 sensor and it's output. I almost exclusively shoot NEF, but I shot an 80 picture portfolio yesterday selected from 400 pictures and I converted about half of the files from NEF to JPG without editing. The rest needed WB adjustments and some shadow recovery because the light kept changing. I have never used a camera with output that much to my liking.