Advice needed on extracting date and time from 15000 JPEGs

Gear Talk > What the Nerds Do

(1/2) > >>

Bruno Schroder:
I'm looking for advice on the easiest way to extract date and time of shot of about 15.000 jpegs. The images are stored in multiple directories of roughly a thousand files each.
There are two ways of accessing the info: at file level and in the exif. Both could work but the images are from an IR camera trap and also store temperature and moon phase. It is possible (likely) the data scientists might need these additional exif parameters in a second stage, so an exif based method is preferable.

Project background:
The participants at the Scottish meeting could note my interest in bats, they had to suffer a stroboscopic session when I flash photographed the emergence of about 100 bats from under the roof of the lodge. You might have noticed the composite image Birna made of a few of the shots.
I'm involved in a multiyear bat conservation project trying to measure the influence of microclimate conditions in occupation of bats shelters. I'm in charge of the data acquisition, meaning temperature and humidity sensors and IR recording of the presence of bats.
Roughly speaking, I'm getting 600K datapoints and 50K images a year. Images are reviewed by volunteers and when bats are present, images are tagged and a pointer is added to the database of the sensor data at the right time and date. In a second stage, data scientists will look for correlations between whatever interior or exterior climate data and the presence of the bats.

All in all, it means 15000 images to handle. Automatisation is very welcome :)

How would you do it?

Stephan:
On Mac I use "exiftool", don't know if it's available on Windows though.

Renamed and organized fully automatically >50.000 pics according to date/time shot, parsed through subdirectories and created new ones (Y/YMD/YMD.nef in my case)

unbelievable powerful which also means complex, needed a lot of advice from the net, but there is a lot available -> so highly recommended!

Bruno Schroder:
Thanks, Stephan. It is also available on Windows. I'm occasionally using it in drag and drop mode for 1 image. I'll dig deeper in the other functionalities.

Matthew Currie:
There is a program called "Exiftool GUI" which piggybacks on Exiftool, and runs in Windows. It allows you to export EXIF data to separate files, HTML, Text, etc., one data file for each image, and can do a whole folder in bulk. The new files will be added to the folder, so accompanying, say, DSC5000.JPG would be DSC5000.TXT. So far I see no way to have it export only selected data, so every accompanying file has the whole lot. But you might be able to do some kind of filtering or selecting from those resulting files afterwards.

You can probably do just about anything in the core Exiftool program if you can figure out how. But it's mighty complicated, and being DOS, it is pretty finicky about formatting.

I miss our bats (Vermont). The white nose die-off has made the little brown bats nearly extinct here.

e.t.a. there is also a program called "exif pilot," which allows the display of selected exif values, but it does not seem to support copying or transferring the view in bulk. The program is free but the bulk extension is sold, and I'm not sure how the export function works, as I so far haven't been able to make it actually export any information anywhere even on a single file, though if I read the instructions right it might 9with bulk extension added) be able to export to an excel file. Unfortunately, you cannot select the date displayed, so the only way I can see right away to get it from the screen would be to screen print it piece by piece. But on the computer screen it's pretty easy to read the selected data, with a little thumbnail optional for each picture.

mxbianco:
You can look into the XDIR program: works in Windows, in DOS mode, but then you'd have to work with BAT scripts (pun intended).

In order to use XDIR you have to be sure that the creation date/time, or modified date/time are equal to the date/time shot, Exiftool GUI has a batch function for equalizing (copying the date/time shot to Create date/time *and* Modified Date/time). For example, if you copy iPhone pics from the cloud to your local drive, they get a create date/time that's different from the date/time shot. You solve it by using the equalize function.
An example of an XDIR command :

XDIR *.JPG *.NEF *.TIF / "FORM=*d[7890]*d[345]*d[312] *t,`*F*E`,`*D*P`" >> myname.txt

Pretty cryptic, huh? You have to read the documentation! the output is like this (CSV format):
...
2022/10/06 11:17:48,"DSC_7982.JPG","J:\DCIM\DA_DF\"
2022/10/06 11:17:50,"DSC_7983.JPG","J:\DCIM\DA_DF\"
2022/10/06 11:17:50,"DSC_7984.JPG","J:\DCIM\DA_DF\"
...
that is, date/time shot, filename, folder

Exiftool has the ability to extract selected EXIF fields to a TXT file, there's a limit of 1024 characters to the command line you can create

As an example, from pic DSC_7983.JPG you can extract the make, model, Lens model, exposure date/time, focal length with this command:

exiftool -S -G -tag -exif:Make -exif:Model -exif:LensModel -exif:ExposureTime -exif:FocalLength -c "%.6fº" DSC_7983.JPG

output goes to standard console ( = SCREEN), but you can pipe it to a text file by adding at the end >myname.txt
or append it to a file by adding at the end >>myname.txt

This is the command that creates a new file "myname.txt"
exiftool -S -G -tag -exif:Make -exif:Model -exif:LensModel -exif:ExposureTime -exif:FocalLength -c "%.6fº" DSC_7983.JPG >myname.txt

And this is the command that appends information to existing file "myname.txt"
exiftool -S -G -tag -exif:Make -exif:Model -exif:LensModel -exif:ExposureTime -exif:FocalLength -c "%.6fº" DSC_7983.JPG >>myname.txt

This is an incomplete list of the exif fields you would use in your script:

-IFD0:ImageDescription
-exif:Make
-exif:Model
-exif:LensModel
-exif:ExposureTime
-exif:FNumber
-exif:ISO
-Nikon:FocusDistance
-exif:FocalLength
-ExifIFD:FocalLengthIn35mmFormat
-ExifIFD:ExposureMode
-Nikon:WhiteBalance
-Nikon:FocusMode
-Nikon:NikonMeteringMode
-Nikon:AfAreaMode
-exif:Flash
-Nikon:VibrationReduction
-Nikon:ActiveD-Lighting
-exif:Orientation
-Nikon:HDR
-exif:DateTimeOriginal
-exif:CreateDate
-Nikon:TimeZone
-Nikon:DaylightSavings
-exif:Artist
-exif:Copyright
-exif:UserComment
-exif:Software
-Gps:GPSLatitude
-Gps:GPSLongitude
-Nikon:Altitude
-nikon:ShutterCount
-Nikon:MechanicalShutterCount
-xmp-dc:Title
-xmp-dc:Subject
-nikon:SerialNumber
-ExifIFD:LensMake
-nikon:Lens
-ExifIFD:LensSerialNumber
-nikon:DirectoryNumber
-nikon:FileNumber
-exif:exifImageWidth
-exif:exifImageHeight

commands can be quite long, for example I use this one to enumerate most EXIF fields:

exiftool -S -G -tag -IFD0:ImageDescription -exif:Make -exif:Model -exif:LensModel -exif:ExposureTime -exif:FNumber -ExifIFD:ExposureCompensation -exif:ISO -Nikon:FocusDistance -exif:FocalLength -ExifIFD:FocalLengthIn35mmFormat -ExifIFD:ExposureMode -Nikon:WhiteBalance -Nikon:Quality -Nikon:FocusMode -Nikon:NikonMeteringMode -Nikon:AfAreaMode -exif:Flash -Nikon:VibrationReduction -Nikon:ActiveD-Lighting -exif:Orientation -Nikon:HDR -exif:DateTimeOriginal -exif:CreateDate -Nikon:TimeZone -Nikon:DaylightSavings -exif:Artist -exif:Copyright -exif:UserComment -exif:Software -Gps:GPSLatitude -Gps:GPSLongitude -Nikon:Altitude -nikon:ShutterCount -Nikon:MechanicalShutterCount -xmp-dc:Title -xmp-dc:Subject -nikon:SerialNumber -ExifIFD:LensMake -nikon:Lens -ExifIFD:LensSerialNumber -nikon:DirectoryNumber -nikon:FileNumber -exif:exifImageWidth -exif:exifImageHeight -c -n %1 >> C:\Programmi\EXIFTOOL5\ListaEXIF.TXT

and the output is something like this:

[EXIF] ImageDescription: Passeggiata nel vallone delle 7 Fontane, Altopiano des Baïsses 02-08-2021
[EXIF] Make: NIKON CORPORATION
[EXIF] Model: NIKON Df
[EXIF] LensModel: 70-180mm f/4.5-5.6 AF-D micro-Nikkor
[EXIF] ExposureTime: 1/1250
[EXIF] FNumber: 6.3
[EXIF] ExposureCompensation: -2/3
[EXIF] ISO: 100
[MakerNotes] FocusDistance: 1.50 m
[EXIF] FocalLength: 180.0 mm
[EXIF] FocalLengthIn35mmFormat: 180 mm
[EXIF] ExposureMode: Auto
[MakerNotes] WhiteBalance: Auto1
[MakerNotes] Quality: Normal
[MakerNotes] FocusMode: AF-C
[MakerNotes] AFAreaMode: Single Area
[EXIF] Flash: No Flash
[MakerNotes] VibrationReduction: Off
[MakerNotes] ActiveD-Lighting: Auto
[EXIF] Orientation: Horizontal (normal)
[MakerNotes] HDR: Off
[EXIF] DateTimeOriginal: 2021:08:02 09:37:06
[EXIF] CreateDate: 2021:08:02 09:37:06
[MakerNotes] TimeZone: +01:00
[MakerNotes] DaylightSavings: Yes
[EXIF] Artist: Massimo Bianco
[EXIF] Copyright: ©2021 Massimo Bianco
[EXIF] UserComment: ©2021 Massimo Bianco
[EXIF] Software: Ver.1.03
[MakerNotes] ShutterCount: 175821
[MakerNotes] SerialNumber: 8405658
[EXIF] LensMake: NIKON
[MakerNotes] Lens: 70-180mm f/4.5-5.6
[EXIF] LensSerialNumber: 200883
[MakerNotes] DirectoryNumber: 131
[MakerNotes] FileNumber: 1761
[EXIF] ExifImageWidth: 4928
[EXIF] ExifImageHeight: 3280

Ciao from Massimo

Navigation

[0] Message Index

[#] Next page

Go to full version