NikonGear'23
Gear Talk => Processing & Publication => Topic started by: Ron Scubadiver on February 09, 2016, 17:40:07
-
With digital photography it is kind of hard to get by without a computer. I suppose one could shoot from a phone and upload directly to the internet, but that is kind of limiting.
The D8xx series of cameras poses a bit of a problem as the 36mp raw files take a bit of horsepower to process. My 2.4 ghz Haswell powered Macbook Pro bogs down unless I stick to lower resolution previews. Even with the low res previews there are occasional delays when using a brush in ACR. My other computer is a 5 year old sandy bridge I5-2500 overclocked to 4.5 ghz running Windows 10. It is acceptably fast.
I wouldn't mind something faster, but a $425 I7-6700k (local price) might offer a 20% improvement clock for clock and probably would not allow the low fan speeds I use with my current machine. Add to that a minimum of $220 for a new motherboard and 16 gigs of DDR4 memory assuming I reused as much stuff from the old machine as possible. It's really kind of discouraging. The good old days of each new generation of hardware being a lot faster than the last are gone. Its going to get worse:
http://www.extremetech.com/extreme/222590-an-end-to-scaling-intels-next-generation-chips-will-sacrifice-speed-to-reduce-power (http://www.extremetech.com/extreme/222590-an-end-to-scaling-intels-next-generation-chips-will-sacrifice-speed-to-reduce-power)
New ultra low power chips will be slower although in data centers they could be packed far more tightly than they are now. This leveling off of technology seems to be with everything. Each new generation of digital cameras used to be a lot better than the last, but now improvements are incremental and in small increments at that. My D800 is starting to look like it has been in a war but I can't get motivated to replace it with a D810. I suppose if the D800 broke I would be on the phone with B&H to order an 810 before even asking Nikon about a repair. Unfortunately, Houston, the fourth largest city in the USA does not have an authorized Nikon repair shop.
Did I mention Windows 10? I have updated several Win 7 machines and like it. There is a significant performance improvement in file copy operations, particularly across external drives which I use a lot. The update clears out the sluggishness that an old Win 7 system has from the endless updates and no SP2. MS appears to be ready to deliver new builds of Win 10 with feature updates as entire OS builds rather than simple updates or service packs. They already had one November. Yeah, I know some people will say they hate this, but is it that different from Apple's annual OS X refresh?
Probably the main reason to get a Mac is they just work. There is no crapware. Notebook displays are better than what is found in most popular Windows notebooks. If one checks the owners forum for the popular Dell XPS15 there are endless software glitch problems solved by reinstalling Windows from scratch. Sounds like crapware and dated drivers to me. There is one big problem with Macs right now. Apple has been delaying with hardware updates. It might be that slow incremental improvements in hardware is discouraging them too:
http://buyersguide.macrumors.com/#Mac (http://buyersguide.macrumors.com/#Mac)
Meanwhile as computer performance more or less stagnates, new versions of Lightroom and ACR manage to get slower. If you think Adobe is slow try Photo Ninja or any other raw developing program based on that same open source image processing package whose name I can't remember right now.
Perhaps someone around here has some brilliant ideas regarding a way around these problems, I hope.
-
I'm running on a ThinkPad W540 with win10 and my files from D3X is flying, Adobe CS6 and stitching 3, 4 or 5 images in PtGui is super fast, no lag or annoying hiccups,,, I use an 24" Apple Cinema Display via the mini DVI port, colour calibration was challenging, but mostly due to Win10 and the user ::)
I'm a long time MacBook Pro user 15" but switching between Win and IOS user interface was tiresome,,, :o
-
I'm still on Win7. Tried Win8 on someones machine and nearly gagged when i saw the new start menu, have not touched it further. Does Win10 still have that wannabe-tablet UI?
-
Eric, with only 12 mp those files should fly. It was when I switched from a D700 to a D800 that computer horsepower became an issue.
Tristan, Windows 10 has returned the start menu. There is still a remnant of the tablet interface attached to the right side of the start menu. Non tablet programs are slowly becoming able to utilize the tablet interface and place an icon there.
Somehow I am able to switch between Windows and Mac without too much fumbling, but I will probably go with windows for my next notebook.
-
Ron,
imho, nothing from the foundational technologies in IT (CPU , storage size, memory, ..) grow slower than the file size of D-SLRs. The growth in NEF file size from the D1 (1999) to the D800 (2012) was about 10x. Basically, over time, the performance issue rather shrinks than grow :)
I do have a decent set of fast single socket, dual and even quad socket machines at home, but for photo editing, one of the lower performing machines is more than enough - for what I do.
Usually larger main memory is more useful than a faster CPU.
Unfortunately, quite a few photo editing applications are not leveraging the available parallelism in modern CPUs. For instance Lightroom has its sweetspot with 4 cores. I tested LR on 2 socket machines, but the performance dimishes further as the application can't cope with NUMA architectures at all. LR on a 2 socket 24-core workstation is often slower than on a fast 4-core machine.
As you mentioned notebooks, they are most often limited by thermal design limits, not the max performance a certain CPU architecture could provide. Hence I avoid using notebooks as primary photo editing machines.
To minimize time when selecting keeper images, I use Fastpictureviewer. A properly configured machine is able to display 5-6 D800 RAW photos per second. Basically it is as fast as you can press the keyboard to advance to the next image. Very convenient for the selection process.
My current photo editing machine is a i7-3570K with 32 GB memory and 2x Samsung SSDs for temp space. I don't store files locally, all data is residing in a home server (600k RAW files, 900k JPEGs, approx 7 TB data volume for photos).
I try to minimize my photo editing time, so the workflow is very simple. As example: When returning back from a trip with 5.000 D800 photos (lossless RAW), the selection process to get down to 30-50 keepers and processing in CNX takes about 2-3 hrs in total.
rgds,
Andy
-
You can get a 6600k for half the price and it is only 10% difference in performance. Measurable. Not relevant.
my next machine will be Intel 6way or 8way with 32GB RAM.
i am still on a January 2009 core2quad and it does all I need.
Most important in my book is stability, so I get the best motherboard, power supply and RAM and always Intel.
plus I never overclock.
have been working as an editor in the hardware department of a renowned computer magazine earlier in life.
-
Andy, when you say 4 cores do you mean 4 threads or 8?
Frank the I7 is $390+8.25% tax added at checkout locally. The I5 is $230 plus tax. Both will overclock to about 4.5 ghz reliably although the stock speed on an I7 is considerably higher. On Photoshop the difference clock for clock is probably less than 10%, but I have not been able to find a good benchmark. Adobe says only some functions use hyperthreading.
-
Andy, when you say 4 cores do you mean 4 threads or 8?
Ron,
with "core" I referred to the physical resource on the chip, so it is 4.
Threads are execution paths and Intel introduced a while ago SMT (https://en.wikipedia.org/wiki/Simultaneous_multithreading) in their product line-up. (branded as Hyperthread in Intel language)
Please note, that for high performance computing, SMT (Hyperthreads) are quite often turned off, to improve overall performance.
You might try it with your setup as well. (Many applications aren't scaling well with too many threads, so it might help to reduce the number of threads.)
rgds, Andy
-
Just upgraded my main machine from Win7 Core i7 (I don't remember the code name) to Win10 Core i5 6500T (Skylake), both with 16GB RAM and the latter with 256GB Sundisk SSD. I opted for a quieter desktop machine, and now the only moving parts are the twin CPU cooling fans rotating above the CPU whose TDP is 35W. I haven't done any extensive processing with my main (and currently only) editor CC2015. I don't do batch processing, and my current camera yields humble losslessly compressed 12bit 20MP RAW files. So, I would be content with the processing power.
At this moment, my only major complaint is Edge and Mail. Edge freezes when I start playing movies posted in Facebook. The UI of Mail is nothing but crap. I switched to Firefox (now 64bit version is available) and Thunderbird.
Upgrading from a 2.5inch 7200rpm HDD to an SSD made a significant difference in speed.
-
I'm still on Win7. Tried Win8 on someones machine and nearly gagged when i saw the new start menu, have not touched it further. Does Win10 still have that wannabe-tablet UI?
Agree'd, win8 was terrible. I like Win10, it installs with an over inflated start menu that has a bunch of useless apps in it but they can all be removed so you are left with a clean and simple start menu. From what I've read if you do move to Win10 it is better to do a clean install of it as opposed to upgrading through Win7.
-
Frank the I7 is $390+8.25% tax added at checkout locally. The I5 is $230 plus tax. Both will overclock to about 4.5 ghz reliably although the stock speed on an I7 is considerably higher. On Photoshop the difference clock for clock is probably less than 10%, but I have not been able to find a good benchmark. Adobe says only some functions use hyperthreading.
In Germany the 6600K sells for under 215€ incl. Tax plus shipping. We tested in our lab, that a synthetic performance difference of 30% or more translates into a perceptible difference. I guess that holds true till today.
I see you are stitching, so doubling your RAM will increase performance more than a 10% increase in measurable performance.
PS: I like Win 10 very much, upgraded all my systems from win7 and win8 and with the current version you can do a clean install using your old Win7 or Win8 code.
PPS: If you want to sacrifice system life and stability for overclocking (very rarely brings the necessary 30% performance increase) I recommend to dive into water cooling.
-
I do much of my processing on a couple of Thinkpad W520 machines, each with Intel Quad i7 CPUs, 32 GB RAM, 512 GB SSD + 1 TB HDD. Both run Win7/64 and are darned fast. Their monitors are quite good and suffice for my modest requirements. I usually put them into docking stations to provide support to some more monitors for extra work space.
-
Ron the D3X is 24 MP
16GB Ram and 520 GB Samsung SSD PRO
-
Ron, I realise from your postings on this and other sites that you travel a fair bit, so I am guessing that you need an airline friendly computer and that you are therefore probably not interested in a machine that isn't portabe.
If I have guessed wrongly, then a custom built desk top can deliver huge performance advantages at affordable prices, particularly if you are into heavy image stitching or stacking. PM me if you are interested and I can give you the name of another photographer that has recently gone down this path. I am sure that he would be prepared to share with you the configuration of his purpose built machine. He has been known to routinely stitch many hundreds of input images into a single output.
-
Andy, I don't have to turn off hyperthreading because my I5 doesn't have that feature.
Clock for clock the latest Skylake chips are about 20% faster than Sandy Bridge, on highly threaded applications, less on most stuff. The old Sandy Bridge chips overclock easily so both would be running around 4.5 ghz.
Eric, forgive me for my little mistake. Still 24 mp processing should be about 50% faster than 36 mp processing.
Having a lot of memory definitely makes a big difference when stitching. With 8 GB on the Macbook pro things get really slow if there are a lot of frames. I mean two cups of coffee slow. With 16gb on my desktop 16 frames are stitched pretty quick. 32 GB should handle anything I will ever do.
Local prices for Skylake chips dropped a bit today, $370 for the I-7, $220 for the I-5, add 8.25% tax to each at the register. Just love living in the USA...
For the money, a 32 GB I5 should do better on stitching than a 16 GB I7.
Hugh, I have an airline friendly notebook and a custom built desktop. I am considering upgrading my desktop at this time. I can't get by with a single premium notebook with the amount of travel I do. There is too much risk something could happen to it.
-
Having a lot of memory definitely makes a big difference when stitching. With 8 GB on the Macbook pro things get really slow if there are a lot of frames. I mean two cups of coffee slow. With 16gb on my desktop 16 frames are stitched pretty quick. 32 GB should handle anything I will ever do.
I don't know what you stitch with, but the latest PTGui version added GPU support, now stitching is much faster. I didn't time it, but I don't think you could even make half a coffee when you have less than 30-40 images to stitch. I'm using a late 2014 MBP with SSD and 8Gb of RAM.
-
Have been stitching with ACR. Time to look at ptgui
-
PTGui can indeed be surprisingly fast when you stitch a large number of frames. Having massive amounts of RAM and SSD disk(s) helps too.
-
a custom built desk top can deliver huge performance advantages at affordable prices, particularly if you are into heavy image stitching or stacking.
I stack and I stack 36MP layers, sometimes 150 at a shot. I use Zerene Stacker, but there are others that work too.
As for computers, I stopped buying premade computers systems years ago and began having them built for the specs we need to process stacks, panos, video, etc. I am on my third custom-built computer and each has been increasingly expensive, because time is worth life, and I am getting older.
It is one thing to have a fast computer, but you need software that can access that speed as well, especially with graphics. IMO, forget about building your own. Even forget about having the local computer guru build one for you. Spend some money and have it tuned to what you actually need. I had one built by Boxx. It worked, but the staff there tried too much to upsell me with B.S.. Even tried to tell them to desist, but they were just rude.
The best company I know of is Puget Systems, out of Auburn Washington.
https://www.pugetsystems.com/
This company and the folks who work there are beyond what we can expect. They even call and re-tune your system, as well as provide a complete notebook with backup disks, photos of your particular system, and every last part on it. I cannot recommend them enough. That’s the good news.
The bad news is no different than any of these companies. These fast computers cost money and a lot of it. They have to be budgeted just as we budget money for cameras and lenses. At this point I have a very fast system, but it is still too slow sometimes. I have several Macs, but I only use PCs for photography because I find they are faster, less expensive, and more easily interfaced to peripherals.
Right now I am using a water-cooled system based on:
Asus X99 Deluxe Motherboard
Intel Core i7 5960X 3.0 GHs eight-core 20MN 140W (overclocked) CPU
128 GB DDR-4 2133 REG ECC RAM
Dual PNY Quandro K5200 PCI-E 8GB GPU video cards
And a variety of internal SSD and SATA drives, connections, etc.
The bad news is that it cost more than $10K....
This machine does well with still and video photography, but is IMO still held up by Adobe’s software, as far as I can tell. Here are some photos.
-
I checked on the Ptgui site. GPU computing does not support INTEL IGP's which is what I have on my notebook. My desktop is pretty fast with ACR except on the largest ones I have done where the thing runs out of memory.
As for home brew computers, I have done many. Puget Systems is a good source of information regarding the parts they use. Lovely system, Michael, especially the dual Quadro gpu's and tons of memory.
-
Yes, I forgot to say that I use a dedicated graphics card.
-
Ptgui?
Try Kolor.fr
Evalutate.
Smokes them all.
-
Ptgui?
Try Kolor.fr
Evalutate.
Smokes them all.
LOL! This is amazin, Frank!
-
PtGui is about precision, calculated re-positioning of pixels, works for my stitched architecture superbly.
-
M. Erlewine. That is a serious sytem. Based on a Lian Li case and a great power supply.
That looks pretty much like my 2009 system. Apart from the fact that I use only one
Graphics Board and a slow one. I do not play much and I am not sure
If the computing power of the GPUs can be fully utilized for number
crunching. Maybe it is possible today.
-
Just for fun I looked up a few of the components in Michael's workstation;
CPU $1,050
GPU, around $2,000 each
Memory is around $1100, but I could not find quickly find a quote for a 16x8 ecc kit.
Other components are not super expensive, but a lot of the cost is in having someone getting all of it to work together.
-
Memory is around $1100, but I could not find quickly find a quote for a 16x8 ecc kit.
Ron,
the CPU doesn't support ECC memory (only Xeon's do). No need to pay a premium for ECC memory. Not sure, why there is 128 GB memory in the system, as the CPU supports 64 GB max.
http://ark.intel.com/products/82930/Intel-Core-i7-5960X-Processor-Extreme-Edition-20M-Cache-up-to-3_50-GHz
If you want to speed up your system, avoid connecting your fast drives on the SATA/SAS ports located on the motherboard (starting with 3 SSDs the DMI is the bottleneck), use a cheap LSI host bus adapter on PCI Express for your SATA SSDs or connect your SSD directly to the PCI Express bus. Don't spend money on RAID controllers, not needed.
In case you want to build a fast system without too much financial investment:
The dual Quadro Setup is about as fast as a single Titan Black (approx 1000$; FYI: the Titan has no ECC graphics memory). If you don't need high double precision performance, use the GTX 980 Ti instead. This card is faster than the Titan in graphics and costs about 600-700$
If you want lots of cores, look up Intel's Xeon E5-2670 (v1) on eBay. They are well below 100$ / piece and you get 8c/16t per socket. Imho, currently the best bang for the buck if you intend to run parallel aware applications. Original SRP was 1500$/pc then. (look for C2 stepping - SR0KX)
http://ark.intel.com/products/64595/Intel-Xeon-Processor-E5-2670-20M-Cache-2_60-GHz-8_00-GTs-Intel-QPI
If you have really computationally intensive workloads where CPUs get hot, use watercoolers for approx 60$ each. Performance increase is usually in the 30% range. But only if the load is for longer time. Short spikes don't make much difference.
Combine it with a dual socket workstation MB - ca. 300$. The architectural limit of the Xeons is 768 GB main memory - in case memory prices go down and your needs up.
Use cheaper (but more) 1600 MHz ECC memory. Usually, the perf difference to 2133 MHz memory is in normal apps in the single digit range. 64 GB is most often enough (unless you have a stitching app, which can really use more memory). Be conscious about your memory needs. Too much memory in the system often slows down the startup times of apps. (The OS need to clean the memory for security reasons before the app can use the allocated memory. As example: Cleaning 256 GB takes about 60 seconds, unless a special API is used. Most apps don't use it. With the optimized API it takes about 2 seconds)
Building a balanced system is always fun - enjoy :)
Andy
-
Andy, I appreciate your analysis, but you are talking about things that are way beyond my needs. Someone who needs cutting edge performance will read your post and be grateful. My main gripe is 5 years of technology has not produced much of an improvement in photo editing performance owing to Intel's quest for energy efficiency, larger MP files and Adobe releasing consecutively slower versions of ACR and Lightroom. I may have to let the desktop upgrade go for a month or two because I have some other more pressing challenges .
-
That is a cool Idea: Intel's Xeon E5-2670.
On German Ebay you get two of these for ~260 Euros incl. deductible VAT. Intel's original asking price was 1550 US$ per piece.
16 physical Processors & 32 Threads for less than 600 Euros including new motherboard? WOW.
Thank you.
Question: Which Xeon-Motherboard (two way) do you recommend?
Is this Xeon fit for Two Way?
I remember in olden days Tyan and Supermicro were the weapons of choice.
-
Andy that's serious good advise here! Thanks ;)
-
I ran across a review of the i5-6600k and i7-6700k. The slower i5 ran Ps as fast as the i7, or faster in some cases.
http://www.tomshardware.com/reviews/skylake-intel-core-i7-6700k-core-i5-6600k,4252-5.html (http://www.tomshardware.com/reviews/skylake-intel-core-i7-6700k-core-i5-6600k,4252-5.html)
-
My primary computer is a ASRock Z97 Extreme4 with an Intel Core i7-4790 running at 3.6GHz. I have a 250GB Samsung SSD for the OS and Programs and a mid tower case with 9TB in four hard drives. Currently I have 16GB or RAM in two sticks leaving room for two more. I'll soon upgrade the RAM to 32 or 48GB. The OS is Windows 7, when I figure out how to get them both registered I'll dual boot Windows 10.
I didn't buy a ready made box as they frequently come with a 300 watt power supply. I opted for a 600 watt power supply as I was pretty sure a 300watt supply would be stressed buy a full load of HD(s). I'm currently running with the integrated display provided by the i7-4790 as 2D work doesn't need a separate video card so much and I needed to save money somewhere. I figured the somewhere to save money was not the system board, CPU or power supply. I scavenged 1TB HD(s) from my previous computer. They were SATA 2 drives and I'm slowly replacing them with 3 and 4TB SATA 3 drives. I've been buying Western Digital Black HD(s) with a 5 year warranty.
That's my primary computer. It's a good, fast, not outstanding in anyway computer that's been totally stable.
My second computer is a Mid 2007, 24" iMAC with an 2.4GHz Intel Core 2 Duo and 4GB of RAM. It has a 0.5TB SATA 2 HD and a 1TB external SATA 2 HD running on Firewire 800. I've networked the iMAC's external HD so I can echo my uploaded photos to it as soon as I upload photos to my primary computer. The iMAC sleeps and wakes perfectly for this use. Within minutes of uploading new photos I have them stored on 3 HD. I don't use a Raid on the primary computer as that slows performance while working. Since I never but never vary from this it's like a Raid with one extra on a separate computer.
I have additional external HD(s) to backup photos, documents and OS backups. That's my humble system.
Dave
-
David, I could definitely do something to improve disk performance. Right now I transfer everything to an external 2tb USB3 drive which is backed up periodically to another 2tb USB3 drive. The plan is to replace both of those when they are full. I probably should keep the raw files on my much faster SSD while working on them, back them up to the external drives, eventually deleting the files on the SSD. There are no spinning hard drives in the case of my desktop. I thought if Apple could do that on their fancy Mac Pro, I could do it on my home brew computer.
-
not quite sure i follow ron.
is the issue your desktop, or laptop?
if the former, perhaps it's just time to get a new system. for somewhere in the $1000 to $1500 range you can get a system with an ssd, 32gb ram, core i7, etc, that ingest as many 36MP files as you'd like to throw at it and in record time. if you're using a $3000 camera, a bunch of $1000 lenses, and traveling the world to take photos a $1250 computer can't be out of reach! you'll thank yourself, even if your wallet might not. ;)
as for laptops, expectations have to be lowered unless you're willing to spend a lot of money and carry around something very heavy. i'm not willing to do the latter, so i use my lowly 12" macbook with ACR and Photoshop on D810 files all the time, processing 30 or 40 files in an hour or two without too much frustration. however, i don't do super complex stitching on that machine. i save that kind of thing for big monitors and big processors. for those who don't like the form factor of the 12" macbook, a 13" air is only $1249 with core i7 and 8gb ram. again, not such a big investment compared to the rest of the hobby.
the other machine i use on the go is a thinkpad yoga. i can't get by without a true stylus for my work machine, and again i haven't had any problems processing my d810 files on that machine. it's one generation older than the current one, which is again less than $1500 for a very ACR/PS capable configuration - skylake i7, 16gb ram, ssd, etc. not a great screen though, but again it wasn't meant for photographers...
-
Agree'd, win8 was terrible. I like Win10, it installs with an over inflated start menu that has a bunch of useless apps in it but they can all be removed so you are left with a clean and simple start menu. From what I've read if you do move to Win10 it is better to do a clean install of it as opposed to upgrading through Win7.
Slightly off topic, but : I use every day Win7 (office, for ages), Win8 (home, for 2 1/2 years) and Win10 (M$ Surface pro laptop, since I upgraded, which was stupid). Win 7 is legacy stuff and that's its only merit; I pay respects to ancestor. Win 8 is an attempt to redesign from ground up, only unsatisfactory when it hesitated from departing from the legacy stuff (example: the obnoxious, messy configuration panel of Win 7 was partly reshaped). Win 10 is a step backward, tearing every enjoyable or useful innovation away from Win 8, as if dementors had been at work.
Note : I'm a "power user" (Python & delphi development, Adobe CC incl. InDesign and Dreamweaver, Visual Studio, R, etc.) and to me, an OS is definitely not a toy.
Win 8 was a fundamental re-design. Bad market uptake is essentially a proof that habits are hard to change, no matter if good or bad, and no matter what the redesign has to offer.
Win 8 is the only version with a convenient integration of the cloud (OneDrive in that case - a feature RUINED in Win 10, and of course absent from Win 7) and good accessibility to seldom used apps.
Win 8 was the first Windows version able to restore a fully crashed PC in two hours, complete with OS, apps and user files, while on a biz trip, with zero assistance, zero backup devices (except of course the cloud), and totally unattended: it actually happened to me. While the restoration went on in my hotel room, I had dinner. And Win 8 does not crash more often than others, rather less (the only crash I experienced was due to a buggy OS auto-update).
In Win 10 you get the start menu back (so what? I'm not married with it), no placeholders for files only available in the cloud (essentially making the cloud useless, as you do not even know what it has to offer beyond your local storage), and the usual (totally outdated - Norton Commander was better anyway) file explorer that cannot conveniently be used without a mouse and spectacles if you are over 50. Duh. Plus only one stupid alphabetic list of apps, offered by the "beloved" start menu, quite inadequate if you have dozens of them - with Win 8 you had these displayed on full screen (multiple columns) and sorted according to app types or editors or recent installations, as you liked it. A much, much better overview. Duh, duh, duh. And so on.
Ok, now back to photography.
-
It is safe to assume that there are divergent opinions on these questions ....
-
Facts however are facts. Rash reactions to a change in the surface level of the UI are no facts, you may call them opinions ;)
-
No need to discuss that further. Disagreement is no problem unless one choose to make it so.
-
My issue is after 5 years the best I can do is roughly a 25% increase in CPU/memory performance. I could go up to 32 gigs of ram to help the occasional large pano, and the best M2 SSD's are faster than the best SATA SSD's, but that small increment in CPU performance has me scratching my head.
-
Meanwhile as computer performance more or less stagnates, ....
.... but that small increment in CPU performance has me scratching my head.
Ron,
as you keep on raising this point about "CPU performance", let me add some thoughts on it.
The overall "CPU performance" did increase over the last 5 years by 2x to 2.5x vs. the i5-2500k. What you are referring to is not CPU performance. More accurately, it is "single thread-performance with legacy code" (legacy code, which is not taking advantage of architectural improvements - even for single threaded applications, nevertheless for concurrency). This is not to be mixed up with "CPU performance".
Even if people don't believe it, don't want it and don't like it, the free lunch is over (http://www.gotw.ca/publications/concurrency-ddj.htm) since 2004. While it was a good decision in the past, to check for frequency and cache size of the CPU's under consideration, going forward the focus should be on the application software. If the SW package is able to demonstrate besides the functional requirements good "concurrency", then you have a good base that the performance experience of the overall solution for the user will probably leverage the contemporary and future reality of digital hardware.
If your objective is to increase the overall system performance, your M2 comment is a good intermediate step. Later this year, a new memory technology will land in the market, which has the potential to fundamentally transform the memory hierarchy in computers, moving persistent data much closer to the CPU. Speed increase vs. SSD: 1000x
My recommendation:
Wait until 3D X-Point lands later this year, sit back and relax. Wait until the dust settles and defer your upgrade decision to 2017. At minimum, SSD prices will go down for competitive reasons.
BTW,
the NVM Express interface used in M2, was from the outset optimized for 3D X-Point as migration technology for existing systems (Video (https://www.youtube.com/watch?list=PLinuRwpnsHae596PxDr71JQbehBg0pzjo&v=tgDq0CAVxjY), see 11:50)
2004 was fun in the CPU space, 2016/17 will be fun in the memory/storage space .....
rgds,
Andy
-
Andy, the way I feel lately, "sit back and relax" sounds great. I almost pulled the trigger a few days ago, but the motherboard I wanted suddenly was out of stock. An I7-6700k is down to $349 plus 8.25% locally. Nothing is going anywhere, and as you say improvements are always coming. Most benchmarks I see show a 25% improvement, so that must be the legacy code you refer to. The Tom's Hardware test showed large improvements running Ps under Windows 10, which leaves me wondering.
We are not talking big bucks here, but the difference is mainly convenience. It isn't the same thing as upgrading a lens to one with better IQ, or getting a lens with an FOV you did not have before.
I have read a little bit about moving to very fast storage and its implications for data centers.
There is a lot that could be done to improve application software, but right now Adobe seems more focused on adding features than improving performance. Hence we see ever slower versions of Lightroom and ACR.
-
my new double heart to build a 16core/32thread/64G RAM machine ... arrived!!!
-
Frank
Which motherboard?
JJ
-
Andy recommended an ASUS ATX-Board he has used before for a two-hearted workstation.
http://www.asus.com/Commercial-Servers-Workstations/Z9PAD8C/
I ordered one today plus 8 x 8GB of 1600 DDR3 ECC Registered RAM
-
Good luck with the project Frank and don't fry the motherboard :D
-
Good luck with the project Frank and don't fry the motherboard :D
Thank you!
I got a very experienced professional screw driver to build that system for me. He also build my current system which runs now for 7 years without flaws of any kind.
He also ordered a BeQuiet 650W Professional Power Source that has two 8-Pin CPU-connectors.
We had to stand back from the idea to use Water coolong though, because the 140mm-Fans are too large for my Lian Li PC-A7110 Servercase I love & want to keep. It features space for two rear fans with 120mm.
We will use heat pipe standard air coolers by Arctic specified for CPU heat loss up to 320 Watt...
-
Use Noctua fans. They are great and very quiet in operation. I have them in all my boxes.
-
Use Noctua fans. They are great and very quiet in operation. I have them in all my boxes.
Fans of the Arctic Heat pipe system can be replaced at will. Last time I bought an extremly expensive Zalmann-cooler. Arctic was much quieter although half the price.
The ones I chose are made for 320 Watts, but my Xeons have a TDP of only 115 W and I will not drive tham at 100% all the time.
-
The system runs, soft wind blows, 9 slowly spinning fans keep the system cool. Now I have to try rendering to get my 32 logic cores spinning.
Does someone have a reliable tool that can correctly read data from Asus Server boards?
I'd like to keep an eye on temperatures while rendering.
-
Thanks to Andy I have built my silent Workstation 16core/32thread/64GB ECC RAM ... silent in the sense that it is as loud as the old one although the performance is much higher.
Andy asked whether the Software really makes use of 32 threads???
I can confirm that for Photoshop CC and for Autopano Giga alike. Not so sure about Photo Ninja.
Yesterday Photoshop crashed on me for the first time ever! ... just gone, not even any error message ... it was just loading a HUGE panorama into a Google Nik filter pack and "pooof". gone.
I might want to reinstall windows 10 Pro 1511, which is today possible to do without the penalty of having to reinstall the applications.
Here are two screen shots that show 32 threads at up to 100% CPU usage:
-
Andy asked whether the Software really makes use of 32 threads???
Bibble 5 has been tested with 32 real cores (not just 16 cores+hyperthreading).
Aftershot therefore should do it as well and unless Phase One did something weird, it would use them as well.
cheers
afx
-
Hey, Andreas, you really run 32 cores?
-
Try setting Ps affinity to one processor and see what happens.
-
Hey, Andreas, you really run 32 cores?
Nope, I only have a Hex i7, which pretends to have 12 cores.
But Bibblelabs had a 32 Core AMD machine for testing and demoing. Played with it on Photokina. Really a beast.
cheers
afx
-
Hey, Andreas, you really run 32 cores?
Yes. (but not for photo stuff)
Congrats & enjoy your new machine Frank :)
rgds,
Andy
-
But Bibblelabs had a 32 Core AMD machine for testing and demoing. Played with it on Photokina. Really a beast.
cheers
afx
If I remember correctly you showed me Bibble on that machine at Photokina :)
-
Yes. (but not for photo stuff)
Congrats & enjoy your new machine Frank :)
rgds,
Andy
Thank you very much for you idea and help with component choice. The whole thing set me back less than 1500€...
I Run 16core Intel Xeon ... AMD? I am sorry, not for me. I had AMD boxes for years running in parallel with my Intels.
Rule of thumb: For the same amount of number crunching AMD uses at least double the amount of Electricity, resulting in heat, resulting in noise.
AMD might be cheaper and it is nice to have competition in the market, but sorry, not AMD for me anymore.
Even the newly built hardcore gaming box I buiolt for my son is silent, is Intel, is Nvidia.
I am cured.
-
Sounds like a very powerfull machine in a nicely quiret setup without busting the bank - congratulations
-
So, this is what happened...
After doing a lot of research, I concluded getting an I7 over and I5 would produce little benefit in Ps, the only application I use that is CPU hungry. For the time being it appears that 16gb is enough for the way I work. Big panos are rare and complex Ps projects are almost nonexistent. The motherboard I selected is inexpensive but get great reviews. The local Microcenter gets a few in every two weeks and they sell out immediately. What it lacks is a fancy power supply for extreme overclocking and USB 3.1. Right now I have no USB 3.1 peripherals and don't expect to see any for a while. USB 3 is more than fast enough for the external spinning drives I use a lot.
So the I5-6600k, Asrock Z170M Pro 4s and 16gigs of DDR4-3000 totaled $379 at my local Microcenter, including sales tax. I did not need 3000 speed memory, but it was less expensive than the slower stuff. Everything went together quickly. My case is so old that I had to modify one of the headers as the standard had changed a bit. The CPU overclocked to 4.4 ghz without a lot of voltage increase and runs reasonably cool. It will run at 4.5, but it isn't worth it in terms of voltage increase and heat. In order to save time I used my existing Win 10 setup although it had to be reactivated with M$.
Although it is nothing like years ago when I upgraded from a Lynnfield to Sandy Bridge, the increase in speed is a bit better than I anticipated. When I get around to it, I can install an M2 SSD which is twice as fast as the already fast Samsung SSD I have now. The old MB will go in a case that now houses the slow 7 year old Lynnfield machine.
-
After running some tests I wound up taking back the 16 gigs of memory for 32. If you are building a Skylake/Z170 system, avoid any memory that uses more than 1.2v. It will make your system run hot with very little improvement in performance. A lot of the memory marketed for high speeds will only reach it on X99 boards.
-
X99 is sill quite an interesting option. I chose parallel execution performace over single thread performace. 2 x 4 memory channels are really nice.
-
X99 is the current high end platform. It excels when high memory bandwidth is necessary. The drawback is the CPU's for this board are based on Haswell and not the latest Skylake. As for single thread vs multi, an I-5 still has 4 cores and 4 threads.
-
Yes. The Skylakes with 4 channel memory are up in late summer.
That is why I got a 2011setup.
-
Yes. The Skylakes with 4 channel memory are up in late summer.
That is why I got a 2011setup.
The new Skylake chips with 8 or more cores and HT will be hot stuff. Meanwhile I took back the 32 gigs and am back to the ddr 3000 16 gig chips. Some tests showed me the extra memory did not do that much on the size panos I do, and the 32 gig sticks had some serious compatibility issues that limited their performance.
-
I built a Win10 machine based on H170 chipset and Skylake Core i5 6500T a couple of months ago. When I tried to stitch eleven 24MP NEFs using Photomerge in CC2015, the 16GB ram was almost fully used. I thought more ram would be used if my machine had 32GB.
The initial specification of the barebone case I used indicated that it can recognize the RAM up to 16GB, which turned out to be wrong. It actually recognizes up to 32GB. I regretted having installed only 16GB, but, apparently 16GB is enough according to the info shared here.
-
Photoshop's stitching algorithms seem to be far less efficient than panorama tools'. A few days ago I stitched 47 24mp 16bit tifs without a problem on my 8gb MacBook pro while having about 10 other applications running. I didn't time it, but it was so fast that I couldn't even develop another picture before the stitch was done and exported to .psb. The program really took off wrt. speed since they introduced GPU acceleration.
The same panorama, but with resized 1024px wide jpegs took Photoshop CS6 so long that I aborted it, I would have been curious how well it would be stitched given tha vast amount of OOF areas. Maybe the CC version is better.
-
What I noticed was Widows would start swapping when it ran out of memory on a stitch. It was slower, but not dramatically slower. There are probably better ways to stitch very large panos than Ps. The nice part about Ps is it is easy as you don't have to make tiff's first. I need to look at some alternative for my MBP as with only 8 gigs of memory it is seriously slow.
-
The key to a stable windows system is enough memory.
-
What I noticed was Widows would start swapping when it ran out of memory on a stitch.
Whenever an OS (Windows, Linux, OSX) runs out of main memory because of application software's requirements and need to start swapping, the 10.000x slower access speed of SSD vs. main memory will start to be noticeable. Depending on the % of needed swap and the memory architecture of the application software, all is possible. A bit slower on the good end and a seemingly stand still at the other end. Without access to the source code of the application, the easiest and cheapest fix is to add main memory.
If you are interested in this wonderful world of memory for software, this paper is highly recommend. :)
What every programmer should know about memory (https://lwn.net/Articles/250967/) or here the 114-page PDF (http://people.redhat.com/drepper/cpumemory.pdf)
rgds, Andy
-
Andy, your link returns a 404 Error.
-
Andy, your link returns a 404 Error.
Sorry, fixed now. Andy
-
CS6 stitching is in Ps. In CC the stitching is in ACR. When the swapping started it was slower than without swapping, but only a little slower. I think the only delay was for part of the data that already been worked on to swap out. Remember, my test is based on a 16 shot D800 data set. There are stability issues running 4 sticks of memory. Right now 16x2 32 GB memory kits are pricey. Memory keeps getting cheaper and faster. Who knows what I will be able to snag in a year. It's fine to have 32 gb of memory but for the occasional large pano it doesn't make sense for me. Forget about 8 gb, that's for gaming and MS office.
I am not saying 16 gb is enough for everyone or even most photographers, but it is enough for the way I work now, and probably for while.
If you have a desktop with 4 sticks of 8 gb you can try some tests. On a notebook, it might not work the same as there are only 2 sticks and going to single channel is a hit.
Let me mention this, Intel makes a tiny barebones computer, the Nuc. I have one of the lower powered models in use as a dedicated home theater PC. Their latest offerings have I-5 and I-7 notebook Skylake chips. Add memory, SSD, operating system and peripherals and you have a computer. They go together with fewer hitches than an ATX/mATX system.
-
Running 32 GB as 4x8 configuration on my Thinkpads. They fly along.
-
Running 32 GB as 4x8 configuration on my Thinkpads. They fly along.
It is nice that your Thinkpads have room for 4 sticks of memory. That is a high end feature.
-
My PC has only two slots for DDR3L-1600 SODIMMs, but there is no 16G SODIMM on the market. :(
-
It is nice that your Thinkpads have room for 4 sticks of memory. That is a high end feature.
That applies to the W-series Thinkpads, which I use as moveable graphical workstations through a docking station to allow additional displays. The smaller X-series (used for travels) "only" handle 16 GB.
Lenovo only specifies the Wxxx Thinkpads to use 16 GB, though. However that specification must date from an earlier time in history where suitable 8 GB sticks didn't exist. The additional 16 GB come in handy for big stitching projects or for running PhotoNinja as the latter easily devours 10-15 GB on its own.
-
That applies to the W-series Thinkpads, which I use as moveable graphical workstations through a docking station to allow additional displays. The smaller X-series (used for travels) "only" handle 16 GB.
Lenovo only specifies the Wxxx Thinkpads to use 16 GB, though. However that specification must date from an earlier time in history where suitable 8 GB sticks didn't exist. The additional 16 GB come in handy for big stitching projects or for running PhotoNinja as the latter easily devours 10-15 GB on its own.
There is no one size fits all solution. It is likely this was my last investment in traditional desktop computing. Notebook performance has improved a lot because reductions in power consumption allow more powerful chips in these small enclosures. Meanwhile, desktop performance moves ahead very slowly. Workstations benefit from more cores, but only on certain applications.
-
The new Skylake chips with 8 or more cores and HT will be hot stuff. Meanwhile I took back the 32 gigs and am back to the ddr 3000 16 gig chips. Some tests showed me the extra memory did not do that much on the size panos I do, and the 32 gig sticks had some serious compatibility issues that limited their performance.
maybe the cheapo motherboard was not the wisest decision?
I have 2x4 memory channels meaning 8x8GB = 64GB.
The memory was cheap. Kingston sealed all channel tested packages. ECC. Set me back 325€ all 8 of them.
-
I stitched 500 6MP files back in the old days with the D70.
Computer was slow. Next to no memory. Autostitch. Ran more tgan 24 hours. Possible. But who wants that now?
-
maybe the cheapo motherboard was not the wisest decision?
I have 2x4 memory channels meaning 8x8GB = 64GB.
The memory was cheap. Kingston sealed all channel tested packages. ECC. Set me back 325€ all 8 of them.
All Z170 motherboards have the same chipset and follow reference designs. The more expensive boards have more features and better power supplies to support higher overclocks. I found a long thread where users of expensive Asus motherboards were getting worse results with the 16 gigs of DDR-3000 I have. It will run at full speed on my machine, but the .3% improvement in performance over 2800 speed (with the same timings) is not worth the extra heat. The problem with the 32 gig set has been experienced on many Z170 systems while it works on nearly all X99 systems. A few Z170 systems recently received bios updates to fix the problem. My experience, and that of many others, is overclocked systems are more stable and run at lower voltages with 2 sticks of memory rather than 4. Having few features makes for a less confusing bios. This machine went together with about a third of the effort of the last one, and would have been a real breeze had I not become indecisive about the first 16 gig memory kit, brought it back for 32 and then returned to the same spec 16 gig kit. Performance is better than I expected with Ps.
-
I would not have bought the ASUS if Andy had not told me he tested it and it was a good one.
I usually use the Gigabyte Ultra durable series. They trun really cool and are really very well made.
I never overclock. I buy overclocking hardware though for bigger headroom & increased stability