A script to find duplicate image files

Discussion regarding all scripting related questions
Please DO NOT post to this thread anything that is not directly related to scripting of Capture One.

Re: A script to find duplicate image files

Postby Eric Nepean » Tue Jan 15, 2019 1:15 am

NNN636734910749887695 wrote:Renewing this thread, doesn't seem to be running on C1-12. Been looking at the code trying to find a workaround but no luck.

I haven't used this script for a long time. Some of the code is pretty old. Seems to be working, except for the parts that intentionally check the version.

I've changed the parts that will stop if the Capture One version is not as expected, and replaced them with a warning message "Use at your own risk etc. etc.".

Perhps I should do that with other scripts. I've come to realise that forward compatibility (using a newer version of C1) is pretty good with Capture One's applescript, it's the backward compatibility that is likely to break scripts.

Other than that I didn't find anything (but I don't have duplicate images in my catalogs)

If you find problems using this with Capture One 12, please run it with debug set to 1, and post the log results here as code (use the "code" button on the reply menu to avoid re-formatting and character problems), I will check it out.

And do let me know how this works for you. I post these scripts but almost never hear from people using them, it's hard to know if that's because there are few users, or because they generally work.

Code: Select all
##  Applescript to search a COP 11 Catalog for Duplicate Images
-- Version 1.3 !! Best Effort Support !!  Eric Valk, Ottawa, Canada

-- ***To Initialise
-- Start Script Editor, open a new (blank) file, copy and paste all of this code into the file, compile (hammer symbol) and save.
-- Best if you make "Scripts" folder somewhere handy in your Douments or Desktop

-- *** Operation
-- Start Capture One Pro with the desired catalog
-- Open  the compiled and saved file in Script Editor
-- Open the Script Editor log window, and select the messages tab
-- Run the script
-- Results appear first in the AppleScript Log when the search  is completed
-- This script does not write or delete any information in the COP Catalog or Session or the image file

global debug, heavy_debug_image_limit
global image_date_list, image_count, image_name_list, no_date_info_D, start_date_D, end_date_D, image_count0, list_start, list_finish

log "Find Dupes Version 1.3"
set debug to 0
set heavy_debug_image_limit to 10

validateCOP()

## Set up date limits
## the date assigned if there is no date
set no_date_string to "01/01/70"
set no_date_info_D to (date no_date_string)
## The starting date for sorting (one year before the date assigned if there is no date)
copy no_date_info_D to start_date_D
set year of start_date_D to (year of start_date_D) - 1
## # The ending date for sorting (one year after the current date)
set end_date_D to current date
set year of end_date_D to (year of end_date_D)
set time of end_date_D to 0

if debug ≥ 1 then log "Getting image dates"
tell application "Capture One 12"
   set image_date_list to EXIF capture date of every image
end tell
set image_count0 to length of image_date_list
log "Retrieved " & image_count0 & " images"
if debug ≥ 2 then log {items 1 through 2 of image_date_list, class of image_date_list, "end", (item image_count0 of image_date_list)}

## Add the starting and ending  dates at the beginning of the date list
## This provide search endpoints so that the search routines will not run off the ends of the list
## All the pointers to image lists are now 2 higher than they would otherwise be
tell image_date_list
   set beginning of image_date_list to end_date_D
   set beginning of image_date_list to start_date_D
end tell
set image_count to length of image_date_list
set list_start to 1
set list_finish to 2

tell application "System Events" to set frontmost of process "Script Editor" to true

sort_imagedates()

if debug ≥ 2 then pointer_check()

cleanup_sort()

Duplicate_Search()

cleanup_dupe_search()

if debug ≥ 1 then log "Getting image names"
tell application "Capture One 12" to set image_name_list to name of every image
tell image_date_list -- to make the items in the name list sync with the items in the date list
   set beginning of image_name_list to "EndofList Dummy Image.bad"
   set beginning of image_name_list to "StartofList Dummy Image.bad"
end tell

Evaluate_display_dupe_list()

final_cleanup()

log "Done"

################# Handlers

on validateCOP()
   set minCOPversion to 10.0 as real
   
   tell application "System Events"
      set COPProcList to every process whose name contains "Capture One" and background only is false
      if debug ≥ 2 then log COPProcList
   end tell
   if (count of COPProcList) = 0 then error "COP is not running"
   if (count of COPProcList) ≠ 1 then error "Unexpected: >1 COP instances"
   set theAppRef to item 1 of COPProcList
   
   tell application "System Events" to set theAppName to ((get name of theAppRef) as text)
   
   using terms from application "Capture One 12"
      tell application theAppName to set copVersion to (get app version)
   end using terms from
   
   tell application "System Events" to set copDetailedVersion to get version of my application theAppName
   
   if debug ≥ 2 then
      --properties of application "Capture One 10"
      tell application "System Events"
         log (get every process whose background only is false)
      end tell
      log "theAppName: " & theAppName
      log "COP Version: " & copVersion
      log "COP Detailed Version: " & copDetailedVersion
   end if
   
   if the theAppName ≠ "Capture One 10" then
      display notification "Untested Capture One version, use at your own risk"
      log "Found " & theAppName & " The only verified version is Capture One 10"
   end if
   
   set numCOPversion to (item 1 of (extract_version_strings(".", (word -1 of copVersion)))) as integer
   if numCOPversion < minCOPversion then
      display notification "COP Version is too low"
      error "This COP Version is " & numCOPversion & " - the minimum supported COP version is " & minCOPversion
   end if
   
   tell application "System Events" to set frontmost of process theAppName to true
   
   set COPeveryDoc_L to name of every document of my application theAppName
   if debug ≥ 2 then log {length of COPeveryDoc_L, "Documents", COPeveryDoc_L}
   if (count of COPeveryDoc_L) = 0 then error theAppName & " has no open documents"
   
   set is_catalog_found to false
   repeat with COP_doc in COPeveryDoc_L -- now find the catalog
      using terms from application "Capture One 12"
         tell application theAppName to set thisDocKind to (get kind of document COP_doc) as text
      end using terms from
      if thisDocKind = "catalog" then
         set is_catalog_found to true
         exit repeat
      end if
   end repeat
   
   if not is_catalog_found then
      display notification ("This Applescript only works with Catalogs - Only found " & COPeveryDoc_L)
      error "No open catalog found"
   end if
   
   log "Capture One " & thisDocKind & ": " & COP_doc
   
   log "Capture One Version" & copDetailedVersion
   
end validateCOP

on extract_version_strings(d, t)
   set astid to AppleScript's text item delimiters
   try
      set AppleScript's text item delimiters to d
      set version_strings to text items of t
   on error
      set AppleScript's text item delimiters to astid
   end try
   set AppleScript's text item delimiters to astid
   
   return version_strings
end extract_version_strings

on sort_imagedates()
   global image_date_list, list_begin, pointer_dates_L_Len, list_end, up_pointer_dates_L, down_pointer_dates_L, this_image_date, this_image_index, debug, heavy_debug_image_limit, image_count, iteration_counter, list_start, list_finish, start_date_D, end_date_D, no_date_info_D
   global ymd_start_pointers, year_list, yearmonth_list, ymd_list
   if debug ≥ 1 then log "Starting Date Sort"
   
   set list_end to image_count + 1
   set list_begin to 0 as integer
   
   set pointer_dates_L_Len to 2 as integer
   set up_pointer_dates_L to {list_finish, list_end}
   set down_pointer_dates_L to {list_begin, list_start}
   
   set iteration_counter to 0 as integer
   set this_image_index to list_finish
   set this_image_date to item this_image_index of image_date_list
   
   ## Initialisation for the update_compare_index() handler
   set {year:start_image_date_year, month:start_image_date_month, day:start_image_date_day} to (item list_start of image_date_list)
   set {year:finish_image_date_year, month:finish_image_date_month, day:finish_image_date_day} to (item list_finish of image_date_list)
   set year_list to {{year:start_image_date_year, up_ptr:list_finish, down_ptr:list_begin, ym_ptr:list_start}}
   set yearmonth_list to {{year:start_image_date_year, month:start_image_date_month, up_ptr:list_finish, down_ptr:list_begin, ymd_ptr:list_start}}
   set ymd_list to {{year:start_image_date_year, month:start_image_date_month, day:start_image_date_day, up_ptr:list_finish, down_ptr:list_begin, dates_ptr:list_start}}
   set end of year_list to {year:finish_image_date_year, up_ptr:list_end, down_ptr:list_start, ym_ptr:list_finish}
   set end of yearmonth_list to {year:finish_image_date_year, month:finish_image_date_month, up_ptr:list_end, down_ptr:list_start, ymd_ptr:list_finish}
   set end of ymd_list to {year:finish_image_date_year, month:finish_image_date_month, day:finish_image_date_day, up_ptr:list_end, down_ptr:list_start, dates_ptr:list_finish}
   
   if debug ≥ 2 then log_status()
   set sort_start_time to current date
   repeat while this_image_index < image_count
      set compare_index to this_image_index
      set compare_date to this_image_date
      set this_image_index to this_image_index + 1
      set this_image_date to item this_image_index of image_date_list
      
      if debug ≥ 2 then log {"Now processing image ", this_image_index, this_image_date, "with Compare Image ", compare_index, compare_date}
      
      ## if the date info is dodgy, missing or out of range, then assign the date "no_date_info_D"
      if not (this_image_date > no_date_info_D and this_image_date < end_date_D and (class of this_image_date) = date) then -- this expression also catches the case image_date = missing value
         if debug ≥ 2 then log {"Date Error", this_image_index, this_image_date, (class of this_image_date)}
         set this_image_date to no_date_info_D
      end if
      
      ## Get a better search start if the previous image was on a different day --  speeds up searching by up to 4x
      if (get {year, month, day} of this_image_date) ≠ (get {year, month, day} of compare_date) then set {compare_index, compare_date} to update_compare_index()
      
      if this_image_date < compare_date then
         search_down(compare_index, compare_date)
      else
         search_up(compare_index, compare_date)
      end if
      
      if debug ≥ 3 then
         log {"Done", "up:", up_pointer_dates_L, "down:", down_pointer_dates_L}
         log "  "
         if debug ≥ 4 then
            log_status()
            --if this_image_index > 10 then error
            if this_image_index > heavy_debug_image_limit and debug ≥ 4 then exit repeat
         end if
      end if
   end repeat
   set sort_stop_time to current date
   if debug ≥ 1 then log {"number of Iterations: ", iteration_counter, "  Sort Duration: ", (sort_stop_time - sort_start_time), " Seconds"}
   --if debug ≥ 2 then log_status()
end sort_imagedates

on log_status()
   global image_date_list, list_begin, pointer_dates_L_Len, list_end, up_pointer_dates_L, down_pointer_dates_L, this_image_date, this_image_index, debug, image_count, list_start, list_finish
   global ymd_start_pointers, year_list, yearmonth_list, ymd_list
   log "*****"
   log "Status"
   log {"Up Pointer List", up_pointer_dates_L}
   log {"Down Pointer List", down_pointer_dates_L}
   log {"Image Date List:", (items 1 thru pointer_dates_L_Len of image_date_list)}
   log {"year_list", year_list}
   log {"yearmonth_list", yearmonth_list}
   log {"ymd_list", ymd_list}
   log "*****"
end log_status

on update_compare_index()
   -- This subroutine maintains a list of pointers to years, months and days used to increase the search speed
   -- It returns a pointer to an already sorted image in the nearest day (same day if there is one)
   -- there is a lot of code, but most of it does not execute often
   global image_date_list, list_begin, pointer_dates_L_Len, list_end, up_pointer_dates_L, down_pointer_dates_L, this_image_date, this_image_index, debug, heavy_debug_image_limit, image_count, iteration_counter, list_start, list_finish
   global ymd_start_pointers, year_list, yearmonth_list, ymd_list
   
   set {year:this_image_date_year, month:this_image_date_month, day:this_image_date_day} to this_image_date
   set indexing_entry_missing to false
   
   -- Search for Year entry
   -- Reference: set year_list to {{year:this_image_date_year, up_ptr:list_end, down_ptr:list_begin, ym_ptr:list_start}}
   -- Reference: set ymd_start_pointers to {year_up:list_start, yearmonth_up:list_start, ymd_up:list_start, year_down:list_start, yearmonth_down:list_start, ymd_down:list_start}
   set year_index_found to false
   set prev_year_index to list_start
   set compare_year_index to up_ptr of item prev_year_index of year_list
   repeat
      set iteration_counter to iteration_counter + 1
      if compare_year_index = list_finish then exit repeat -- hit the end of year_list and didn't find the entry
      if (get year of (item compare_year_index of year_list)) = this_image_date_year then
         if debug ≥ 3 then log "year found"
         set this_year_index to compare_year_index
         set year_index_found to true
         set compare_yearmonth_index to ym_ptr of item compare_year_index of year_list
         exit repeat
      else if (year of (item compare_year_index of year_list)) < this_image_date_year then
         set prev_year_index to compare_year_index
         set compare_year_index to up_ptr of item prev_year_index of year_list
      else -- (year of (item compare_year_index of year_list)) > this_image_date_year
         exit repeat -- didn't find the entry between already existing entries
      end if
   end repeat
   
   if not year_index_found then -- add index entry for the year
      set indexing_entry_missing to true
      if debug ≥ 2 then log "***"
      if debug ≥ 2 then log {"adding Year", this_image_date, "this_image_index", this_image_index}
      
      set end of year_list to {year:this_image_date_year, up_ptr:compare_year_index, down_ptr:prev_year_index, ym_ptr:(1 + (length of yearmonth_list))}
      set this_year_index to length of year_list
      set down_ptr of item compare_year_index of year_list to this_year_index
      set compare_yearmonth_index to ym_ptr of item compare_year_index of year_list
      set prev_yearmonth_index to down_ptr of item compare_yearmonth_index of yearmonth_list
      set up_ptr of item prev_year_index of year_list to this_year_index
      if debug ≥ 3 then log {"Added this_year_index", this_year_index, "compare_year_index", compare_year_index, "prev_year_index", prev_year_index}
      if debug ≥ 4 then log {year_list}
   end if
   
   -- Search for YearMonth entry
   -- Reference: set yearmonth_list to {{year:this_image_date_year, month:this_image_date_month, up_ptr:list_end, down_ptr:list_begin, ymd_ptr:list_start}}
   
   set yearmonth_index_found to false
   if not indexing_entry_missing then -- search for the Year:month
      set prev_yearmonth_index to down_ptr of item compare_yearmonth_index of yearmonth_list
      repeat
         set iteration_counter to iteration_counter + 1
         if compare_yearmonth_index = list_finish then exit repeat -- hit the end of yearmonth_list and didn't find the entry
         if (get {year, month} of (item compare_yearmonth_index of yearmonth_list)) = {this_image_date_year, this_image_date_month} then
            set this_yearmonth_index to compare_yearmonth_index
            if debug ≥ 3 then log "yearmonth found"
            set yearmonth_index_found to true
            set compare_ymd_index to ymd_ptr of item compare_yearmonth_index of yearmonth_list
            exit repeat
         else if (((get month of (item compare_yearmonth_index of yearmonth_list)) < this_image_date_month) and ((get year of (item compare_yearmonth_index of yearmonth_list)) = this_image_date_year)) then
            set prev_yearmonth_index to compare_yearmonth_index
            set compare_yearmonth_index to up_ptr of item prev_yearmonth_index of yearmonth_list
         else -- (month of (item compare_yearmonth_index of yearmonth_list)) > this_image_date_month  or (year of (item compare_yearmonth_index of yearmonth_list)) ≠ this_image_date_year)
            exit repeat -- didn't find the entry between already existing entries
         end if
      end repeat
   end if
   
   if not yearmonth_index_found then -- add index entry for the yearmonth
      set indexing_entry_missing to true
      if debug ≥ 2 then log "***"
      if debug ≥ 2 then log {"adding YearMonth", this_image_date, "this_image_index", this_image_index}
      
      set end of yearmonth_list to {year:this_image_date_year, month:this_image_date_month, up_ptr:compare_yearmonth_index, down_ptr:prev_yearmonth_index, ymd_ptr:(1 + (length of ymd_list))}
      set this_yearmonth_index to length of yearmonth_list
      set down_ptr of item compare_yearmonth_index of yearmonth_list to this_yearmonth_index
      set compare_ymd_index to ymd_ptr of item compare_yearmonth_index of yearmonth_list
      set prev_ymd_index to down_ptr of item compare_ymd_index of ymd_list
      set up_ptr of item prev_yearmonth_index of yearmonth_list to this_yearmonth_index
      
      if year_index_found then -- if year already existed then update its yearmonth pointer if this is the first yearmonth of the year
         if (year of item prev_yearmonth_index of yearmonth_list) ≠ this_image_date_year then set ym_ptr of item this_year_index of year_list to this_yearmonth_index
      end if
      
      if debug ≥ 3 then log {"Added this_yearmonth_index", this_yearmonth_index, "compare_yearmonth_index", compare_yearmonth_index, "prev_yearmonth_index", prev_yearmonth_index}
      if debug ≥ 4 then log {yearmonth_list}
   end if
   
   -- Search for YMD entry
   -- Reference: set ymd_list to {{year:this_image_date_year, month:this_image_date_month, day:this_image_date_day, up_ptr:list_end, down_ptr:list_begin, dates_ptr:list_start}}
   
   set ymd_index_found to false
   if not indexing_entry_missing then -- search for Year:month:day
      set prev_ymd_index to down_ptr of item compare_ymd_index of ymd_list
      repeat
         set iteration_counter to iteration_counter + 1
         if compare_ymd_index = list_finish then exit repeat -- hit the end of ymd_list and didn't find the entry
         if (get {year, month, day} of (item compare_ymd_index of ymd_list)) = {this_image_date_year, this_image_date_month, this_image_date_day} then
            if debug ≥ 3 then log "ymd found"
            set this_ymd_index to compare_ymd_index
            set ymd_index_found to true
            exit repeat
         else if (((get day of (item compare_ymd_index of ymd_list)) < this_image_date_day) and ((get {year, month} of (item compare_ymd_index of ymd_list)) = {this_image_date_year, this_image_date_month})) then
            set prev_ymd_index to compare_ymd_index
            set compare_ymd_index to up_ptr of item prev_ymd_index of ymd_list
         else -- (day of (item compare_ymd_index of ymd_list)) > this_image_date_month  or (month of (item compare_ymd_index of ymd_list)) ≠ this_image_date_month)
            exit repeat -- didn't find the entry between already existing entries
         end if
      end repeat
   end if
   
   if not ymd_index_found then -- add index entry for the yearmonthday
      set indexing_entry_missing to true
      if debug ≥ 2 then log "***"
      if debug ≥ 2 then log {"adding YearMonthDay", this_image_date, "this_image_index", this_image_index}
      
      set end of ymd_list to {year:this_image_date_year, month:this_image_date_month, day:this_image_date_day, up_ptr:compare_ymd_index, down_ptr:prev_ymd_index, dates_ptr:this_image_index}
      set this_ymd_index to length of ymd_list
      set down_ptr of item compare_ymd_index of ymd_list to this_ymd_index
      set up_ptr of item prev_ymd_index of ymd_list to this_ymd_index
      
      if yearmonth_index_found then -- if yearmonth already existed then update its ymd pointer if this is the first ymd of the month
         if (month of item prev_ymd_index of ymd_list) ≠ this_image_date_month then
            set ymd_ptr of item this_yearmonth_index of yearmonth_list to this_ymd_index
         end if
      end if
      
      if debug ≥ 3 then log {"Added this_ymd_index", this_ymd_index, "compare_ymd_index", compare_ymd_index, "prev_ymd_index", prev_ymd_index}
      if debug ≥ 4 then log {ymd_list}
   end if
   
   if indexing_entry_missing then
      set updated_compare_date_ptr to dates_ptr of (item compare_ymd_index of ymd_list) -- the pointer to the first image for the next higher date
   else -- indexing entry was not missing
      set updated_compare_date_ptr to dates_ptr of (item this_ymd_index of ymd_list) -- the pointer to the first image for this date
      --set updated_compare_date_ptr to dates_ptr of (item compare_ymd_index of ymd_list) -- (alternate) the pointer to the first image for the next higher date
   end if
   
   set updated_compare_date to item updated_compare_date_ptr of image_date_list
   return {updated_compare_date_ptr, updated_compare_date}
   
end update_compare_index

on search_up(prev_candidate_image_index)
   global image_date_list, pointer_dates_L_Len, up_pointer_dates_L, this_image_date, this_image_index, debug, iteration_counter
   if debug ≥ 3 then log "***"
   if debug ≥ 3 then log {"Search_Up", prev_candidate_image_index}
   set found to false
   repeat while not found --search for the first image which is newer than this image
      set iteration_counter to iteration_counter + 1
      set next_candidate_image_index to item prev_candidate_image_index of up_pointer_dates_L
      set next_candidate_image_date to item next_candidate_image_index of image_date_list
      if this_image_date > next_candidate_image_date then --search further up
         set prev_candidate_image_index to next_candidate_image_index
      else
         insert_image_at(next_candidate_image_index, prev_candidate_image_index)
         set found to true
      end if
   end repeat
end search_up

on search_down(prev_candidate_image_index)
   global image_date_list, pointer_dates_L_Len, down_pointer_dates_L, this_image_date, this_image_index, debug, iteration_counter
   if debug ≥ 3 then log "***"
   if debug ≥ 3 then log {"Search_Down", prev_candidate_image_index}
   set found to false
   repeat while not found --search for the first image which is older than this image
      set iteration_counter to iteration_counter + 1
      set next_candidate_image_index to item prev_candidate_image_index of down_pointer_dates_L
      set next_candidate_image_date to item next_candidate_image_index of image_date_list
      if this_image_date < next_candidate_image_date then -- search further down
         set prev_candidate_image_index to next_candidate_image_index
      else
         insert_image_at(prev_candidate_image_index, next_candidate_image_index)
         set found to true
      end if
   end repeat
end search_down

on insert_image_at(higher_image_index, lower_image_index)
   global pointer_dates_L_Len, up_pointer_dates_L, down_pointer_dates_L, this_image_date, this_image_index, debug
   if debug ≥ 3 then log {"Insertion", this_image_index, "between", higher_image_index, lower_image_index}
   set end of up_pointer_dates_L to higher_image_index
   set item lower_image_index of up_pointer_dates_L to this_image_index
   set end of down_pointer_dates_L to lower_image_index
   set item higher_image_index of down_pointer_dates_L to this_image_index
   set pointer_dates_L_Len to pointer_dates_L_Len + 1
end insert_image_at

on pointer_check()
   global image_date_list, pointer_dates_L_Len, list_begin, list_end, up_pointer_dates_L, down_pointer_dates_L, debug, list_start, list_finish
   global ymd_start_pointers, year_list, yearmonth_list, ymd_list
   log {"list_start", list_start, "list_finish", list_finish, "up_pointer_dates_L", (items 1 thru heavy_debug_image_limit of up_pointer_dates_L), "image_date_list", (items 1 thru heavy_debug_image_limit of image_date_list), "image_name_list"}
   
   if debug ≥ 4 then --  really long detailed listing  Only do this with the short list in debug level; 4.
      log "Listing Dates List Pointers"
      log {"unsorted: ", (items 1 thru pointer_dates_L_Len of image_date_list)}
      set next_pointer to list_start
      set sorted_image_date_list to {item next_pointer of image_date_list}
      set sorted_image_list to {next_pointer}
      
      repeat with counter from 1 to pointer_dates_L_Len - 1
         set next_pointer to item next_pointer of up_pointer_dates_L
         set end of sorted_image_date_list to item next_pointer of image_date_list
         set end of sorted_image_list to next_pointer
      end repeat
      log {"Sorted up: ", sorted_image_date_list}
      log {"Sorted up: ", sorted_image_list}
      
      set next_pointer to list_finish
      set sorted_image_date_list to {item next_pointer of image_date_list}
      set sorted_image_list to {next_pointer}
      repeat with counter from 1 to pointer_dates_L_Len - 1
         set next_pointer to item next_pointer of down_pointer_dates_L
         set end of sorted_image_date_list to item next_pointer of image_date_list
         set end of sorted_image_list to next_pointer
      end repeat
      log {"Sorted down: ", sorted_image_date_list}
      log {"Sorted down: ", sorted_image_list}
   end if
   
   if debug ≥ 2 then
      log "Checking Dates List Pointers"
      set next_pointer to list_start
      set this_date to item next_pointer of image_date_list
      set list_good to true
      repeat with counter from 1 to pointer_dates_L_Len - 1
         set prev_date to this_date
         set prev_pointer to next_pointer
         set next_pointer to item next_pointer of up_pointer_dates_L
         set this_date to item next_pointer of image_date_list
         if this_date < prev_date then
            set list_good to false
            log {"Up pointer error between Dates List items ", next_pointer, this_date, "and", prev_pointer, prev_date}
         end if
      end repeat
      
      if item next_pointer of up_pointer_dates_L ≠ list_end then
         log {"error in last Dates List up pointer", next_pointer, (item next_pointer of up_pointer_dates_L)}
         set list_good to false
      end if
      
      if list_good = true then log "Up Pointers for Dates List are OK"
      
      set next_pointer to list_finish
      set this_date to item next_pointer of image_date_list
      set list_good to true
      repeat with counter from 1 to pointer_dates_L_Len - 1
         set prev_date to this_date
         set prev_pointer to next_pointer
         set next_pointer to item next_pointer of down_pointer_dates_L
         set this_date to item next_pointer of image_date_list
         if this_date > prev_date then
            set list_good to false
            log {"Down pointer error between Dates List items ", next_pointer, this_date, "and", prev_pointer, prev_date}
         end if
      end repeat
      
      if item next_pointer of down_pointer_dates_L ≠ list_begin then
         log {"error in last Dates List down pointer", next_pointer, (item next_pointer of down_pointer_dates_L)}
         set list_good to false
      end if
      
      if list_good = true then log "Down Pointers for Dates List are OK"
   end if
   
end pointer_check

on cleanup_sort()
   global down_pointer_dates_L
   global ymd_start_pointers, year_list, yearmonth_list, ymd_list
   if debug ≥ 1 then log "Emptying Variables used only for Sorting" --  Avoids stack overflow problems
   set down_pointer_dates_L to {}
   set ymd_list to {}
   set yearmonth_list to {}
   set year_list to {}
   set ymd_start_pointers to {}
end cleanup_sort

on Duplicate_Search()
   global image_date_list, list_begin, pointer_dates_L_Len, list_end, up_pointer_dates_L, debug, heavy_debug_image_limit, image_count, duplicate_image_lists_L, list_start, list_finish
   if debug ≥ 1 then log "Starting Duplicate Search"
   set this_pointer to item list_start of up_pointer_dates_L
   set this_date to item this_pointer of image_date_list
   set first_dup_date to {}
   set dupe_found to false
   set duplicate_image_sublist to {}
   set duplicate_image_lists_L to {}
   repeat with counter from 1 to pointer_dates_L_Len - 2
      set prev_date to this_date
      set prev_pointer to this_pointer
      set this_pointer to item prev_pointer of up_pointer_dates_L
      set this_date to item this_pointer of image_date_list
      if this_date < prev_date then log {"error prev_pointer", prev_pointer, "prev_date", prev_date, "this_pointer", this_pointer, "this date", this_date, "counter", counter}
      if this_date = prev_date then
         if dupe_found = false then -- start of a series of duplicates -  found first two
            if debug ≥ 3 then log "Found Dupe"
            if debug ≥ 4 then log {"prev_pointer", prev_pointer, "prev_date", prev_date, "this_pointer", this_pointer, "this date", this_date, "counter", counter}
            set dupe_found to true
            set first_dup_date to prev_date
            set duplicate_image_sublist to {first_dup_date, prev_pointer, this_pointer}
         else -- adding other duplicates
            set end of duplicate_image_sublist to this_pointer
         end if
      else if dupe_found = true then -- end of a series of duplicates
         set dupe_found to false
         set end of duplicate_image_lists_L to duplicate_image_sublist
         set duplicate_image_sublist to {}
         set first_dup_date to {}
      end if
      if debug ≥ 4 and counter > heavy_debug_image_limit then error
   end repeat
end Duplicate_Search

on cleanup_dupe_search()
   global up_pointer_dates_L, image_date_list
   if debug ≥ 1 then log "Emptying Variables used only for Duplicate Searching" --  Avoids stack overflow problems
   set image_date_list to {}
   set up_pointer_dates_L to {}
end cleanup_dupe_search

on Evaluate_display_dupe_list()
   global debug, duplicate_image_lists_L, no_date_info_D
   -- the intent of this handler is to eleminate obvious non-duplicates
   ----Images with the same date but different image types (likely a RAW-JPG pair, but this is not checked)
   ----Image groups which are part of a (high speed) burst sequence
   -- Images which are reported are likely duplicates, but are NOT guaranteed duplicates
   
   if debug ≥ 3 then log "Evaluating  Duplicates"
   if debug ≥ 4 then log duplicate_image_lists_L
   
   
   
   repeat with sublist_index from 1 to (length of duplicate_image_lists_L)
      
      set {dupe_date, duplicate_image_list} to make_dupe_name_list(item sublist_index of duplicate_image_lists_L)
      
      if dupe_date ≠ no_date_info_D then
         repeat with sublist_index from 1 to count of duplicate_image_list --For each sublist
            
            if (count of item sublist_index of duplicate_image_list) > 2 then -- if there is more than one image name then determine if this could be a burst
               if debug ≥ 3 then log (item sublist_index of duplicate_image_list)
               set is_burst to true -- start by assuming its a burst, and then test it
               set image_seq_list to {}
               set image_name_ext to item 1 of item sublist_index of duplicate_image_list
               repeat with sublist_counter from 2 to length of item sublist_index of duplicate_image_list -- the first entry in the sublist is the extension
                  
                  set image_name to item sublist_counter of item sublist_index of duplicate_image_list
                  set image_num_list to ""
                  set image_char_list to ""
                  repeat with char_counter from 1 to count of image_name
                     set this_char to item char_counter of image_name
                     if this_char = "." then
                        exit repeat
                     else if "0123456789" contains this_char then
                        set image_num_list to image_num_list & this_char
                     else
                        set image_char_list to image_char_list & this_char
                     end if
                  end repeat
                  
                  set image_num_count to count of image_num_list
                  if image_num_count = 0 then -- if there are no numbers in the image name, this is not a burst
                     set is_burst to false
                     exit repeat
                  end if
                  set image_num to image_num_list as integer
                  set end of image_seq_list to image_num
                  
                  if sublist_counter = 2 then -- first item
                     if (count of image_char_list) > 0 then
                        set has_image_char to true
                        set image_char_ref to image_char_list
                     else
                        set has_image_char to false
                     end if
                     set image_num_count_ref to image_num_count
                     set image_seq_min to image_num
                  else -- case: sublist counter >2  (additional items)
                     
                     if has_image_char then -- check that the image characters match the first image
                        if image_char_list ≠ image_char_ref then
                           set is_burst to false
                           exit repeat
                        end if
                     else -- first image name has no characters
                        if (count of image_char_list) > 0 then
                           set is_burst to false
                           exit repeat
                        end if
                     end if
                     
                     if image_num_count ≠ image_num_count_ref then -- now check the numbers
                        set is_burst to false
                        exit repeat
                     end if
                     if image_num < image_seq_min then set image_seq_min to image_num -- find the starting sequence number
                  end if
                  if debug ≥ 4 then log image_num
                  if debug ≥ 4 then log image_char_list
               end repeat
               
               if is_burst then -- check that the image numbers are sequential with increments of 1
                  set image_seq_count to count of image_seq_list
                  set last_seq to image_seq_min
                  repeat with incr_counter from 2 to image_seq_count
                     set this_seq_found to false
                     repeat with seq_counter from 1 to image_seq_count
                        if item seq_counter of image_seq_list = last_seq + 1 then
                           set last_seq to (item seq_counter of image_seq_list)
                           set this_seq_found to true
                           exit repeat
                        end if
                     end repeat
                     if this_seq_found = false then
                        set is_burst to false
                        exit repeat
                     end if
                  end repeat
               end if
               if not is_burst then
                  log {"Possible Duplicates: ", (items 2 through (count of item sublist_index of duplicate_image_list) of item sublist_index of duplicate_image_list), "on", dupe_date}
               end if
            end if
         end repeat
      else
         log {"Images with no date info: ", dupe_date, duplicate_image_list}
         
      end if
      
   end repeat
end Evaluate_display_dupe_list

on make_dupe_name_list(date_ptr_sublist)
   global debug, image_name_list
   
   set dupe_date to item 1 of date_ptr_sublist --  the date and time of these duplicates
   
   set dupe_name_list to {}
   repeat with pointer_counter from 2 to length of date_ptr_sublist
      set image_name to item (item pointer_counter of date_ptr_sublist) of image_name_list -- get the name of the image
      
      ## extract the type of the image
      set image_type_found to false
      set image_type to text -1 of image_name
      repeat with char_counter from 2 to 5
         set this_char to item (-char_counter) of image_name
         if this_char = "." then
            set image_type_found to true
            exit repeat
         else
            set image_type to this_char & image_type
         end if
      end repeat
      
      if not image_type_found then
         log "WTF?? Image type not found in Image Name: " & image_name
         exit repeat
      end if
      
      ##  search for a sublist with this type of image already in the dupe_name_list
      set image_type_in_list to false
      repeat with sublist_index from 1 to count of dupe_name_list
         if item 1 of item sublist_index of dupe_name_list = image_type then
            set image_type_in_list to true
            exit repeat
         end if
      end repeat
      
      ## if not already in dupe_name_list add a sublist for this image type
      if not image_type_in_list then
         set end of dupe_name_list to {image_type}
         set sublist_index to count of dupe_name_list
      end if
      
      set end of item sublist_index of dupe_name_list to image_name --add the image name to the sublist
      
   end repeat
   return {dupe_date, dupe_name_list}
end make_dupe_name_list

on final_cleanup()
   global image_date_list, image_name_list, pointer_dates_L_Len, up_pointer_dates_L, up_start_pointer_dates
   global ymd_start_pointers, year_list, yearmonth_list, ymd_list
   if debug ≥ 1 then log "Emptying all  Variables" --  Avoids stack overflow problems
   set image_date_list to {}
   set image_name_list to {}
   set up_pointer_dates_L to {}
   set pointer_dates_L_Len to {}
   set up_start_pointer_dates to {}
end final_cleanup
Cheers, Eric
[late 2015 iMac, 4GHz i7, 24GB RAM, external SSDs. GX8, E-M1, GX7, GM5, GM1 ....]
Eric Nepean
 
Posts: 445
Joined: Sat Oct 25, 2014 8:02 am
Location: Ottawa

Previous

Return to Scripting



Who is online

Users browsing this forum: No registered users and 1 guest