Better iTunes song deduping
Update 5/16/2010. When I am old and grey, I am going to tell my children about the most popular blog post I ever wrote, this one about iTunes deduping! I finally have to pass the baton of deduping on to TidySongs, a product from a company that coincidentally is in my home town of Charlottesville, Virginia!
I’m happy to share my hacked version of Doug Adams Corral iTunes Dupes AppleScript. You can grab my version from here.
His version uses some wicked fast PERL to find all the duplicate songs based on info in the Library, and then puts them all in a Dupes playlist. However, I wanted to filter by Name, Artist, and Album, and have a single playlist where I could delete in one fell swoop all the duplicates. So I tweaked his AppleScript to have another prompt that asks if you want all dupes to go in the playlist, or just copies 2…X to go in.
I run it once with “All” to make sure I have what I want, and then rerun it with “Just Dupes” and build the playlist of files to delete. Works great, and I’m mailing him a copy, maybe it’ll be in the 1.1 version!
I was also very happy with the application Seek and Destroy Music Duplicates. It is written in Java using WebStart, and gave some good results as well. It is better in that you can pick which copy of the duplications you want to keep, and color codes each set of duplicates.
The tool iDupe really didn’t do that much for me, and the demo is limited to just doing 3 deletes. Seek and Destroy is a better choice.
February 3rd, 2007 at 3:18 pm
where can I get your de-duping script? the link is broken. sounds like it’s exactly what everyone needs
February 5th, 2007 at 2:48 pm
I’ve updated the links in this blog post. Sorry about the confusion, this was caused by a revamp of our blog system and web site.
February 7th, 2007 at 4:33 pm
The script wants to run in Classic. Did I do something wrong?
Also, the mac os x folder is empty … ?
February 7th, 2007 at 5:32 pm
Aha — tried something other than Stuffit — alles gut/
April 25th, 2007 at 4:03 pm
Jim dropped me a note with this question:
_After running your program, it creates the
“dupes” playlist with all the dupes. Now that I have
these in a playlist, is there anyway that I can
actually have the dupes removed from the itunes
library? When I select all in the dupes playlist and
hit delete, it just deletes this playlist and i still
have the dupes in my library._
To actually delete the underlying files, you need to hold down the option (or is it control??) key. ITunes will ask you if you want to delete the actual files, and you click “Yes”. Drove me crazy for a while as well.
May 5th, 2007 at 8:05 pm
Excellent! Works perfectly, I had around 1600 duplicate files due to a transfer to a new computer, m3u files, and general forgetfulness. THE most important script I have on my computer.
May 15th, 2007 at 12:26 am
[...] Finally I came accross this applescript, and it does precicely what I want. (and what I imagine everyone with this problem wants): finds duplicate songs according to user-specified metadata dimmensions, and put “all but one” of each file in a playlist. Select all, option delete, DONE. [...]
May 17th, 2007 at 9:50 am
Do you have anything that will run on a pc?
July 10th, 2007 at 9:37 am
Late to the party here but this is EXACTLY what I needed! Thank you.
July 10th, 2007 at 4:20 pm
Ok so I’ve got the dupes playlist. How do I delete the FILES? not just the entires in the Library?
Cheers
c
July 26th, 2007 at 12:54 am
To answer Craig’s: the best way i think is to add a comment to all songs in the dupe playlist (select all -> show info), then sort by that column and delete all the songs with that particular comment. Example: ‘ * ‘ as a comment.
August 31st, 2007 at 8:54 am
got 6000 duplicates, so this script is heaven 4 me!!! THANKS
(ps: i just renamed all the artist names in the dupes list to ‘jsdhdgfsdjkjgvks’ and then deleted those in the library
December 24th, 2007 at 10:50 pm
Nice contribution. I’m puzzled though, when you’ve expanded on someone else’s work (and indeed contributed a patch to them), why did you not allow access to your script inside the app? I would much rather have just downloaded the script than an app, or an app where I can get at the script. I want to see the code before I run it, and possibly tweak it for my own needs. I think I’ll just get Doug’s script.
December 24th, 2007 at 11:01 pm
Ah, I see that Doug packaged the script in the same way, and that I can open these apps in the Script Editor. I’m new to Applescript – my bad, my apologies.
January 6th, 2008 at 7:22 am
[...] OpenSource Connections » Blog Archive » Better iTunes song deduping nice deduper (or cover song finder!) (tags: itunes applescript) [...]
January 20th, 2008 at 11:38 am
Totally amazing script, I found the best results where returned when I filtered the Dupes for ‘Size’ rather than name or albumn. That way you can make sure the ones that are the same size i.e. same song (regarless of albumn or name) are filtered into the Dupes playlist. Thanks to Jim for telling us how to delete tunes directly from a playlist.
January 22nd, 2008 at 5:52 am
So, im currently running your script, its been going for 2 days now, i wish it didnt take so long, but i do have 330 gigs of music it is going through… I’m hoping it ends soon, cause i cant use itunes to put any new music on my iphone or ipods…. Is there a way to end the script and remove the duplicates its found so far… im afraid to force quit itunes and I dont see any other way to quit the script… thanks and if anybody knows how to do that, post a replyon here… also was wondering if anybody tried phelix, and how that compares to removing dupes versus this script… btw thaks a ton for putting in your fine work on this script…
March 3rd, 2008 at 12:06 pm
Yeah – I believe the Size is the best attribute to sort by if you have an extensive collection (which I’m sure everyone does) and have 3 or 4 artists that play the same song name. Excellent Script!
April 20th, 2008 at 5:20 pm
I’ve run into a rather frustrating problem that maybe this script can help fix. I’m re-ripping my library with Apple Lossless and have discovered that iTunes isn’t actually replacing the songs, but making duplicates. Would there be a way to tweak the script to automatically choose the lower bitrate (or at least smaller file) file for the duplicate? I’m sure I could figure it out with a week or two of studying, but if you’re a Perl wiz (and really, the mod should stay in your excellent script), you could probably knock it out in an hour, ya?
May 28th, 2008 at 1:38 am
Months of looking for a solution have come to an end. Eric, this is the best thing that has happened to me in my MacLife so far. Cheers for that.
May 31st, 2008 at 6:19 pm
Thank you very much! This is exactly what i needed!
June 22nd, 2008 at 11:14 pm
Thanks!
Saving me LOTS of time!
In line with Keith’s comment, it would be great to have a cancel option while the script is running (versus force quit the script)
July 3rd, 2008 at 11:53 am
This looks compiled. iTunes does not recognize it as a script, nor does the script editor.
unzip corral-itunes-dupes-eric.zip
open in Script Editor:
unzip corral-itunes-dupes-eric.zip
˛Ì˙Œ
July 17th, 2008 at 7:10 pm
dude. you just saved me about 6 hours of mind numbing manual labour. thanks a million!
September 13th, 2008 at 6:19 pm
i have never used the scripts, so I have no clue how to use this… and i have million duplicates… please advise =)
October 8th, 2008 at 12:35 pm
I just got a Macbook and imported 18000+ songs onto it with iTunes. In my pc i had never found a good way to get rid of all the dupes. Now this is amazing. Thanks a million!!
November 2nd, 2008 at 9:33 pm
Hi,
This is a great tool! Not sure that I am using it correctly though? I have followed your instructions and have the dedupe folder populated, trouble is, it seems to have selected all of the songs. For example, where I have three songs, it moves all three to the dedupe folder, not just the duplicates? thus if I mass delete the folder I delete my entire itunes directory?
can you please help?
November 2nd, 2008 at 9:36 pm
Hi,
when I run this, it puts all songs into the dedupe list, not just the 3 or 3 dupes?
November 6th, 2008 at 7:03 am
can someone pleeease tell us challenged folk how to actually use it? i downloaded the script but don’t know what im doing after that.
November 16th, 2008 at 4:21 am
Eric
As an owner of a new (second) ipod and a new PC, I am shocked by how CRAPPY iTunes is when it comes to migrating and synching music. Like everyone here, I now have an iPod with 60 GB – half of which are duplicated (more than 8,000 files). I Googled and found your killer script. Unfortunately I am enslaved on Windows Vista. Is there any Win version of your most valuable utility?
It would be awesome if you gifted your script to Apple to make their core iTunes EXE more useful for serious music fans like us all. Any tips appreciated! Cheers, Peter
December 3rd, 2008 at 12:45 pm
I am just another of those “your script rocks” posts. Maybe you could mention that one can delete the files from any playlist by pressing “Command(Alt?)” + “Delete”.
I had to scroogle this first, but it works like a charm.
Thank you for your contribution!
December 4th, 2008 at 4:44 pm
man, what an amazing tool. found 722 duplicates (3+GB on my laptop) very nice.
for deleting the files, i just went into the “Dupes” playlist and Edit/Select All…Then held down CTRL-Option-Applekey and hit delete. I’m not sure which one does the trick, but using all 3 does it!
December 15th, 2008 at 2:42 pm
Hi,
Make sure that the XML file path is without caracters like ‘ and > and < and such… Otherwise! Great! Idiots like me usually use paths like those…. hehehe so a warning would be great here!
Ruben
December 16th, 2008 at 12:39 pm
I need one that works for windows. Do you know of one.
December 26th, 2008 at 2:57 am
[...] il vous faire quelques étape à la main. Commencez par télécharger le script Corral iTunes Dupes(Source). Décompressez le. Fermez iTunes (si celui-ci était lancé). Aller ensuite dans le dossier [...]
January 20th, 2009 at 12:10 pm
[...] Via OpenSource Connections [...]
February 28th, 2009 at 12:35 am
Need a windows alternative. I’m a Mac man but my music library is twice the size of the hard drive in my macbook pro. I have 11,000+ duplicates to get rid of (22,000+ when I show duplicates)
March 8th, 2009 at 12:48 pm
amazing. thank you a thousand times over for this script.
March 15th, 2009 at 11:06 am
Any version for PC, and Lovely Vista Sh…?
thanks
March 22nd, 2009 at 9:43 pm
Hello All,
Can someone describe how to install the files?
Thanks a bunch!
April 20th, 2009 at 8:26 pm
i got phelix, ran it, have a folder with all the files now…so to create room do i just delete the phelix folder? sorry….comp ill.
so now do i delete all the songs in itunes and make a new folder or what? help!
April 22nd, 2009 at 4:37 am
Thank you very much for this script! I’ve just bought a new mac recently and what I’ve found so far – the best community that makes and shares stuff! Keep up the good work. I think the 3-years lasting comments on this post just prove how good your script is!
June 2nd, 2009 at 7:33 pm
Thanks! It worked perfectly!
June 14th, 2009 at 1:13 am
Great script enhancement. Congrats.
Works flawlessly!!!
July 1st, 2009 at 6:10 am
You rock!!!
This is amazing!
Thanks so much!
July 3rd, 2009 at 4:42 pm
AWSOME!!!!
You saved me 15$ for Dupin!! I had soooo many duplicates.
July 6th, 2009 at 3:31 pm
Great script
thank you soooo much. Cleaned my library up!
July 8th, 2009 at 6:15 am
Thanks man! This rock! I luv u! =D
July 18th, 2009 at 12:44 pm
Excellent app! Appreciate your skils.
MAC users rock!
July 23rd, 2009 at 7:28 am
[...] 1) Download the script here. [...]
July 27th, 2009 at 6:46 pm
this is FANTASTIC! works so well.
thank you so much!!! i cleared out almost 3,000 songs in my library in just a few minutes :]
August 12th, 2009 at 1:09 am
Awesome Eric!
It removed all my 1210 duplicate files. It didn’t remove the files with same name and different albums, that was really what I wanted.
Thanks a lot.
September 1st, 2009 at 8:27 pm
[...] downloading and using this great Apple-Script file you can simply run it, and it will make a duplicated playlist for you when complete. You then can [...]
September 25th, 2009 at 8:32 pm
Ran it, deleted all the files, deleted all my music, fortunately I have it backed up in like 4 different places, must not have followed directions :/
October 1st, 2009 at 12:57 pm
hi there, thnx for the script. question: the script runs, but says ” no duplicates found” whilst my ” show doubles” in itunes shows about 2000 songs??
i just installed snow leopard and have all the last updates.
somebody knows why the script cant find anything?
October 29th, 2009 at 9:07 pm
Okay, so I’ve run the script, it puts the dupes in the playlist, and I click remove.
But every time I re-run the script, it finds the same dupes, and every time I delete them, it only deletes them from the playlist, even if I hit CTRL.
What am I doing wrong? Help!
October 31st, 2009 at 8:53 pm
doesnt work for itunes9. nothing is listed in the dupes-playlist
January 21st, 2010 at 10:46 am
Thanks dude. that was really helpful. Was looking for something like this for a while… Thanks for the tip on option-deleting too!
January 28th, 2010 at 3:30 pm
Hay i know im abit late for the party but can you tell me, when you select dupes only, how dose the script determine which dupe to put in the playlist? I have a few dupes in my list, about 1000, but some are broken mp3's that stop abruptly.
I think ill just have to go through th “all” list slowly and manually!
February 20th, 2010 at 6:38 pm
This looks perfect – but crashes when near the end of the process in Snow Leopard (10.6.2). Bummed!
March 28th, 2010 at 5:58 pm
anyone got a copy for the much hated windows?
April 14th, 2010 at 9:23 pm
Thank you so much. This works really well!
May 18th, 2010 at 8:16 pm
the link is not working. where can i get the program?
May 23rd, 2010 at 7:09 pm
the link goes to a dead page, where do I find Eric’s scripts?
July 7th, 2010 at 12:44 pm
We had to move our “oscshare” ftp directory, and I still need to port over the old documents. However, to be honest, the TidySongs is way better then what I had! Hopefully I will have stuff moved soon though…