Remove duplicates from database?

[nymous]

Hi, I've tried to add another version of a game to a platform, but both rescan and import weren't able to see it. So I did an import with forced "add duplicates for existing games" on platform's folder. I got every of my games added 2 or even 3 times. I'm aware of automatic merging option so I did it too. Now I have correct number of titles, but when I open version list, I have 2-3 versions which point to exact same file. Can I somehow automatically clean those dups without manual editing?

Edited July 6, 2021 by nymous

[nymous]

So I ended up with writing a simple Ruby script, which operates on Platform xmls and removes duplicates. Be sure to make a backup before using it!

#!/usr/bin/env ruby
require 'nokogiri'

file = ARGV[0]
unless File.exists?(file)
  puts "#{file} doesn't exists!"
  exit -1
end

map = {}
edited = 0

doc = Nokogiri::XML(File.open(file), nil, 'UTF-8') do |config|
  config.noblanks
end
doc.slop!

nodes = doc.xpath('//AdditionalApplication')
nodes.each do |node|
  k = node.GameID.content
  v = node.ApplicationPath.content
  map[k] = [] unless map.key?(k)
  if map[k].include?(v)
    #puts "Removing duplicate node! #{v}"
    node.remove
    edited += 1
  else
    map[k].push(v)
  end
end

if edited > 0
  bak_file = file+".bak"
  File.rename(file, bak_file)

  puts "#{file}: found #{edited} duplicates"
  File.write(file, doc.to_xml)
else
  #puts "#{file}: no duplicates found"
end