some numbers on PCI-E to USB3 adapters

[Frank Fremerey]

Hey!

You got the fastest XQD-card, a faster even SSD and the best card reader, but you are still far from reaching the transfer rates announced?

Currently I got three USB3-adapters in my systems: one onboard, two PCI-E to USB3.

My fastest is this model: https://www.amazon.co.uk/dp/B01M7UG7A3/ref=psdc_430539031_t1_B01M0A5HSP that can be found with many different names and different prices on ebay & amazon. watch the numbers on the card & the layout of the microchips.

SSD (550MB/s) & XQD (440MB/s):

210MB/s to SSD (avg over 67GB data)

120MB/s to the XQD (avg over 67GB data)

120MB/s seems the max write speed I seem to get to XQD, independent of the USB3-path.

62MB/s is the speed with which I write to the SSD with the other two USB3-paths.

[Frank Fremerey]

Comment

I guess 120MB/s is the max. write speed for the Lexar 2933x series of cards.

440MB/s is the rating of the XQD, I guess the rating is a "read speed".

The limitation to 210MB/s might be the real world write speed of the Samsung EVO SSD

OR

the real world speed of current USB3-path implementations.

I will see to test these guesstimations as soon as I have a good idea for a test method and some time

[Seapy]

Remarkable you should post this Frank, yesterday I was up to my armpits in PCIe.

Last week while browsing my favourite 'hock shop' (pawnbrokers) I spotted a Samsung M2 128Gb drive card for £28.  I photographed it, then checked it out on-line, sound like it was exactly what I needed so next day I bought it.  I browsed online and found a PCIe Wolftech card for the M2 which was heavily discounted, which I bought, together with a PCIe 2.5" SSD card, also heavily discounted, they arrived yesterday, I mounted a spare 275Gb Crucial SSD on the PCIe card and the M2 drive on it's card.  The SSD worked perfectly,  unfortunately  the M2 drive didn't work at all and was returned in exchange for another Crucial SSD.

I had been concerned about the slow offloading of my images from my CF cards via Firewire 800, it seems to be getting slower, so I have moved my current year image files to the PCIe mounted SSD which is about five times faster than the main, OS drive in my Mac Pro 5.1.  I also moved the Lightroom catalogue and preview files onto the PCIe SSD.  That move has completely transformed Lightroom, it's a different animal entirely.  It had been bogging down and very lethargic, now I can sweep through large folders fast as I can flick them with the trackpad. Almost zero delay going to 100% zoom, it's now a pleasure to work with again.

I have just run X Bench speed tests of the SSD's and the WD image file drive, the differences in speeds are quite amazing.  This is quite an old computer, about 2008 I think.  I have upgraded the CPU and hacked the firmware so it can take later OS's.

[Frank Fremerey]

Remarkable you should post this Frank, yesterday I was up to my armpits in PCIe.

Last week while browsing my favourite 'hock shop' (pawnbrokers) I spotted a Samsung M2 128Gb drive card for £28.  I photographed it, then checked it out on-line, sound like it was exactly what I needed so next day I bought it.  I browsed online and found a PCIe Wolftech card for the M2 which was heavily discounted, which I bought, together with a PCIe 2.5" SSD card, also heavily discounted, they arrived yesterday, I mounted a spare 275Gb Crucial SSD on the PCIe card and the M2 drive on it's card.  The SSD worked perfectly,  unfortunately  the M2 drive didn't work at all and was returned in exchange for another Crucial SSD.

I had been concerned about the slow offloading of my images from my CF cards via Firewire 800, it seems to be getting slower, so I have moved my current year image files to the PCIe mounted SSD which is about five times faster than the main, OS drive in my Mac Pro 5.1.  I also moved the Lightroom catalogue and preview files onto the PCIe SSD.  That move has completely transformed Lightroom, it's a different animal entirely.  It had been bogging down and very lethargic, now I can sweep through large folders fast as I can flick them with the trackpad. Almost zero delay going to 100% zoom, it's now a pleasure to work with again.

I have just run X Bench speed tests of the SSD's and the WD image file drive, the differences in speeds are quite amazing.  This is quite an old computer, about 2008 I think.  I have upgraded the CPU and hacked the firmware so it can take later OS's.

These are generic tests that do not really tell me about the performance in a real world situation.

It means a lot to me to hear that Lightroom "feels" much faster for you now.

What numbers do you get copying files between your Camera Cards and the different drives?

[Seapy]

These are generic tests that do not really tell me about the performance in a real world situation.

It means a lot to me to hear that Lightroom "feels" much faster for you now.

What numbers do you get copying files between your Camera Cards and the different drives?

I understand what you mean, raw data numbers are, just numbers.  The numbers I marked in red do correlate with my experience.  I was disappointed by the small difference between the slow SSD and my fastest spinning drive - 1Tb WD Black.

Next time I have a significant number of images on a card I will time the off loading process, to various storage media.

[Frank Fremerey]

the aim of the thread is to get a better idea of where the bottlenecks in my setup or your setup are. I guessed the USB3 adapters were the limit but now I see real world write speed of my drives seems to be an important factor as well.

You seem to have just found and fixed one of the bottle necks in your system. Congrats

[Seapy]

Yes, thanks it has made Lightroom much more fluid.

I have conducted one test, I loaded another 11.44Gb onto one of my Sandisk Extreme 32Gb 120MB/s CF cards resulting in a total load of 25.94Gb on the card.  All D3 full depth NEF's, something over 1400 files I think.

I used my Sandisk Firewire 800 CF card reader direct into my Mac Pro 5.1.

First I offloaded the entire folder from the CF card to the main OS SSD, a Crucial 250Gb SSD. It took 13 minutes exactly to transfer the folder.

Secondly I did the same to the Western Digital 'black' 1 Tb drive where I normally store my images. That took 5 minutes 57 seconds.

Thirdly I offloaded the entire folder to my new Crucial 275 fitted to a 'Velocity Solo' PCIe card which is in a 4x slot.  This took 6 minutes 10 seconds.

The third result was a surprise.

I think the slow speed on the first test is a result of it being an OS drive and sharing bandwidth with the computer OS which would be managing the operation.

My next move will be to test the import speed into Lightroom of a significant batch of image files. Given the data is on the same drive as the Lightroom catalogue it may speed things up.

I can't do that until I have a fresh batch of image files.

[Frank Fremerey]

I did use an older folder with pictures, 67GB altogether.

A limiting factor for these transfers is file size. Take bigger files and the transfer will be faster.

Another limiting factor is the path. You use firewire, I use several USB3 adapters. In the past I saw transfer over the onboard Intel USB3 was significantly slower than the same transfer done over my NEC PCI-E adapter and if I used the Hub it slowed down the process further. Since I installed the second PCI-E adapter the transfer on the first USB3 NEC adapter got as slow as the onboard adapter. I guess some shared ressources might be the problem. Have to check that

[Seapy]

That was why I used NEF's.

USB hubs are out for file transfers, think traffic lights at cross roads.  I have always regarded USB as something for keyboards and mice, back in the day I used SCSI for data transfer but while fast it was fussy about cables and the order of the chain of peripherals.

I only have USB 2, I have considered a PCIe - USB3 card but given I don't have any USB3 devices that seems pointless.  That may change if I get a D810...  But I don't see myself using a D810 for high volume photography, so my Firewire 800 Sandisk card reader CF still has some future.

Data bus sharing will definitely have a limiting/congestion effect, I would expect better performance with a PCIe - USB3 card, more direct wide path access to the CPU.  Choice of manufacturer could be another factor, I believe there are cheaper chips used in some components like card readers and PCIe cards, which can inhibit flow of data.  Checking out trustable reviews might help there.  Sonnet are a reliable source for Mac's but I don't know if they do PC versions.

Gaming forums can be a good place for reliable info on speed critical components.

[Frank Fremerey]

this test seems to be interesting: https://www.macworld.com/article/2039427/how-fast-is-usb-3-0-really-.html

[Seapy]

this test seems to be interesting

Yes, very interesting Frank, thanks for posting.

For my part I'm more concerned about the internal file handling and 'feel' of the software and to that end, as I have said the PCIe mounted SSD is a winner.  I am very keen to see how it affects the import to Lightroom of several hundred NEF's.  I imported 44 yesterday and it definitely felt faster, the creation of 100% previews seemed much quicker.  It will be more tangible with several hundred and I will time it, both from a spinning drive and the PCIe mounted SSD.

I am not overly concerned about the transfer from card to computer, I usually do some other task and come back to the computer later, once the files have transferred.  It's the import to Lightroom and the liveliness of Lightroom and Photoshop handling the images.  That is an issue and I think it would be a serious issue with very large files such as those from the D810 or D850, especially in a relatively old and slow computer such as my own Mac Pro 5.1, despite it's upgrades.

[arthurking83]

What computer specs Frank?
Mac? PC?
What mainboard, what memory .. etc.

USB3 is now slow, and 120Mb/s is top whack for USB3.
USB3.1 allows a usable speed upgrade over USB3.

If you look into USB3.1, a few things to be mindful of:
1/. ASM chipset only!! that is, AMS1142 is the normal Asmedia chipset used by most PCI-E to USB3.1 devices, but if you happen to locate any using the ASM2142 or ASM3142 chipsets even better.

Note on the topic of PCI-E to USB devices .. just because its says PCI-E to USB, doesn't make them all the same. The adapting chipset used is absolutely critical in terms of speed.
If you're not sure of the chipset used, and the adapter card maker doesn't specify then avoid. Chipsets brands I now avoid like a plauge/virus/demonic annoyance!! .. are VIA and JMicron

2/. If you plan to keep PC into the future, then having a USB type C connector can be handy.

SO!! if you want a bit more USB speed, look for USB3.1 devices, avoid non Asmedia chipsets(Intel are also good, but you won't find any on peripheral cards).

There is a brand of peripheral card I'm looking into ATM called Speed Dragon(google that name and browse their site).
The problem is, I can't find any retailers selling their products, and I think the reason is that Speed Dragon are wholesale manufacturers that may only make generic parts where other brands place their name on the product.

For USB3(and 3.1 too) check for updated drivers for your chipset(s).
My mainboard has both USB3 and 3.1 onboard.

USB3.1 is Asmedia chipset. USB3 is Via chipset. I don't have any adapter cards(but IU've been looking into them)
I have a Volans USB3.1 docking station(for HDDs) on the USB3.1 port(USBtype C connector).
I also have multiple USB3 devices, one of which is a USB3 HDD enclosure.

Both HDDs are WD blacks .. quite fast for mechanical drives.
I've done Crystal Diskmark testing. They can do approx 140-150MB/s write speeds, just a touch more read speeds.

On the VIA chipset USB3 line, both those drives struggle to do over 60-70MB/s for more than a quick burst, sustained they can only really achieve 50-ish MB/s
On the Asmedia chipset USB3.1, they easily do 125-140MB/s transfers(depending on file types).

Prior to fully setting up my PC, I quickly tested one of my SSD drives. I have two. 1 M.2 Samsung 950 and one Kingston SATA.
I plugged the Kingston into the USB3.1 docking station just to see, and it easily sustained 400+ MB/s doing a proper file transfer. files were two large 1+ Gb movie files just for testing.
Crystal Diskmark gave it a 480MB/s rating on that USB3.1 line! 

Note tho that it's not all roses, and again it all comes down to chipsets and devices used.
Some devices seem to be optimised for different chipsets or something.

I have a USB3 Lexar CF card reader.
I plug it into various ports for testing, but generally keep it in the USB3 port because:

I have one Lexar CF card for the D800E, 800x speed rating. In the USB3.1 port, it never reads over 100MB/s, writes at about 60-70MB/s
I have one Patriot CF card that only works in the D300(ie. this card won't work in the D800E) Lexar card works in the D300 too tho.

On the USB3 Lexar card reader: Lexar card writes at 50MB/s max. never seen more than this no matter what files I transfer to it.
Patriot card writes at 60-70MB/s depending on files used for transfer.

Because of that, there is no point trying the USB3 Lexar reader on the USB3.1 line .. no speed advantage in doing so, so the USB3.1 line (for now) is reserved for the USB3.1 docking station.

[Frank Fremerey]

Yes, very interesting Frank, thanks for posting.

For my part I'm more concerned about the internal file handling and 'feel' of the software and to that end, as I have said the PCIe mounted SSD is a winner.  I am very keen to see how it affects the import to Lightroom of several hundred NEF's.  I imported 44 yesterday and it definitely felt faster, the creation of 100% previews seemed much quicker.  It will be more tangible with several hundred and I will time it, both from a spinning drive and the PCIe mounted SSD.

I am not overly concerned about the transfer from card to computer, I usually do some other task and come back to the computer later, once the files have transferred.  It's the import to Lightroom and the liveliness of Lightroom and Photoshop handling the images.  That is an issue and I think it would be a serious issue with very large files such as those from the D810 or D850, especially in a relatively old and slow computer such as my own Mac Pro 5.1, despite it's upgrades.

I just extendded the test to my mini notebook: transfer to a very fast Samsung Pro+ SD Card that writes 90MB/s measured on another system was 26MB/s, while transfer to my internal m2.ssd was more that 200MB/s. In this case the USB3 interface seems to be very well implemented, while the internal card reader of the Notebook seems to be the limit.

The m2.ssd (Samsung 860 EVO 1TB) is hell fast and all file handling with my apps seems  instantaneous.

[Frank Fremerey]

King Arthur: very, very helpful information for everybody reading this thread. Thank you very much. I am already on the hunt for ASM. My latest insight was to use NEC Chipsets, because they were faster than Intel, but this is not current.

Here is the hardware information you requested: Water cooled in-spec-driven 2970 double Xeon on an ASUS server board with 64GB of unregistered DDR3 RAM and a set of Lexar Professional Workflow card readers for XQD and SSD. Windows 10 Pro Insider.

VIA? Gosh no, never again!

Some more useful background information: https://en.wikipedia.org/wiki/USB_3.0